Sun Netra CP3240 Switch Installation Guide Table Of ContentsPrevious ChapterNext ChapterBook Index

C H A P T E R  2
Overview

This chapter introduces the key features of the switch and RTMs. This chapter includes a product definition, a list of product features, and functional block diagrams with brief descriptions. This chapter can be used to compare the features of the switch against the needs of a specific application.

This chapter contains the following topics:

2.1 Features

Part of Sun’s ATCA platform, the Sun Netra CP3240 switch complies with PICMG 3.0 R2.0 ECN002 and PCIMG 3.1 Option 1 and Option 9. The Sun Netra CP3240 switch implements two separate switched networks on a single PCB. By separating the Base and Fabric networks, the Sun Netra CP3240 switch provides a separate control plane and data plane. It provides 1Gigabyte Ethernet switching on the 3.0 Base Fabric, and the 3.1 Expansion Fabric provides 1Gigabyte/10Gigabyte Ethernet switching. Both of these networks are fully managed with the robust FASTPATH management suite. Both networks support Layer 2 switching as well as Layer 3 routing.

The Sun Netra CP3240 switch can host three AdvancedMC mid-size modules. A variety of different AMC types can be used, including AMC.1, AMC.2, and AMC.3. Netra CP3240 switch supports connectivity to three AMCs and to RTMs with multiple 10Gigabyte-Ethernet links.

The XCP3240H-RTM-CU-Z (copper) and XCP3240H-RTM-OP-Z (fiber optic) are Sun’s RTMs that are paired with the Sun Netra CP3240 switch.

The following sections briefly outline the features of the Sun Netra CP3240 switch.

2.1.1 General

2.1.2 Base Interface

2.1.3 Fabric Gigabit Interface

2.1.4 AMC Sites

Site 1: AMC.0 Mid-size, AMC.2 Type E1
Site 2: AMC.0 Mid-size, AMC.2 Type 5 and E1
Site 3: AMC.0 Mid-size, AMC.2 Type E1

2.2 Switch Components

The following sections list and describe key components of the Sun Netra CP3240 switch.

2.2.1 Broadcom StrataXGS 3 BCM56503 Ethernet Switch

The Sun Netra CP3240 switch uses Broadcom StrataXGS 3 BCM56503 for Base Ethernet switching/routing. This chip is an Ethernet switch with 24 1Gigabyte ports and 3 10Gigabyte ports. It provides non-blocking, wire-speed switching and routing on all ports under 100% load. The BCM56503 features many advanced features presented to the user via the FASTPATH software. See the Section 4.10, FASTPATH for more information.

2.2.2 Broadcom StrataXGS 3 BCM56800 Ethernet Switch

The Sun Netra CP3240 switch uses Broadcom StrataXGS 3 BCM56800 for Fabric Ethernet switching/routing. This chip is an Ethernet switch with 20 10Gigabyte ports. It provides non-blocking, wire-speed switching and routing on all ports under 100% load. The BCM56800 features many advanced features presented to the user via the FASTPATH software. See Section 4.10, FASTPATH for more information.

2.2.3 Broadcom BCM5464R and BCM5461S 10/100/1000Base-T Ethernet PHY

Quad and single ports respectively, the Broadcom PHYs provide the physical interfacing for 10/100/1000Base-T. They are low-power devices and provide features such as jumbo frames support, auto-MDIX, and cable testing.

2.2.4 Freescale PowerQUICC II MPC8247 Communications Processor

The Freescale MPC8247 is a microprocessor designed for maximum flexibility. It features a dual core architecture with a PPC G2 LE core and a RISC core controlling the peripherals. Running at 400MHz with only 1 watt of power, the MPC8247 provides high performance with incredibly low power consumption. Paired with the 256MB PC100 SDRAM and 64MB flash, the CPU subsystem of the Sun Netra CP3240 switch is more than enough to handle the switching application and future upgrades.

2.2.5 Pigeon Point BMR-H8S-AMCc AdvancedTCA IPMI Subsystem

The Sun Netra CP3240 switch uses the near-industry standard Pigeon Point Systems BMR-H8S-AMCc for dedicated IPMI management. It controls all interaction between the Sun Netra CP3240 switch and the shelf management controller. It fully supports the PICMG 3.0 R2.0 ECN002 and the AMC.0 R2.0 specifications. The BMC-H8S-AMCc provides all required management for the AMC modules, in addition to the voltage monitoring, temperature monitoring, e-Keying, and other services it provides for the Sun Netra CP3240 switch. Pigeon Point Systems is the leading provider of IPMI firmware for AdvancedTCA, and the firmware has been thoroughly tested to be fully compliant with the specification.

2.3 Protocols, RFCs, and MIBs Support

The Sun Netra CP3240 switch features Level7 FastPath 2340 switching software, version 4.4.4 and newer. This software provides layer 2 switching, quality of service, IPv4 routing, IPv6 routing, and IP multicast.

2.3.1 FASTPATH Switching

2.3.1.1 Additional Layer 2 Functionality

2.3.1.2 System Facilities

2.3.1.3 Switching MIBs

2.3.1.4 Routing MIBs

2.3.2 FASTPATH Routing

2.3.3 FASTPATH Quality of Service

2.3.3.1 DiffServ

Optional policy attributes:

2.3.3.2 Access Control Lists (ACLs)

Permit/Deny actions for inbound or outbound IP traffic classification based on:

Permit/Deny actions for inbound or outbound Layer 2 traffic classification based on:

Optional rule attributes:

2.3.3.3 Class of Service (CoS)

Direct user configuration of the following:

2.3.3.4 Quality of Service MIBs

2.3.4 FASTPATH Multicast

2.3.4.1 Multicast MIBs

2.3.5 FASTPATH IPv6 Routing

2.3.5.1 IPv6 Routing MIBs

2.3.6 FASTPATH Management

2.3.6.1 SSL 3.0 and TLS 1.0

2.3.6.2 SSH 1.5 and 2.0

2.3.6.3 Additional Management Features

2.4 Functional Diagrams and Port Maps

2.4.1 Switch

FIGURE 2-1 Switch Functional Block Diagram


 


TABLE 2-1 Switch External Ports

Front Panel

Port Type

Fiber Optic AMC

Module

Logical Port

Comments

RJ-45 #1

Serial RS-232

RS-232

RJ-45

 

 

RJ-45 #2

Management

10/100 Base T

RJ-45

 

 

RJ-45 #3

Base 1GBE

10/100/1000Base T

RJ-45

Base - 18

Standard I/F is copper

RJ-45 #4

Base 1GBE

10/100/1000Base T

RJ-45

Base - 19

Standard I/F is copper


2.4.2 Rear Transition Modules (RTM)s

The Sun Netra CP3240 switch supports RTMs through ATCA Zone 3 connectors (see TABLE 2-2). The Sun Netra CP3240 switch was designed to support multiple RTM designs.

4 Fabric 10Gigabyte, 2 Base 10Gigabyte, 2 Base 1Gigabyte, AMC I/O, and management ports are run to the RTM. Several voltage rails are supplied, and all of the signals needed to design an ECN002 compliant RTM are present.


TABLE 2-2 Zone 3 RTM Ports

Base Ports 1G Infrastructure 10G Infrastructure

Base Port 20 1G SGMII None

Base Port 21 1G SGMII None

Base Port 26 None 10G XAUI

Base Port 27 None 10G XAUI

Fabirc Port 17 1G SGMII 10G XAUI

Fabirc Port 18 1G SGMII 10G XAUI

Fabirc Port 19 1G SGMII 10G XAUI

Fabirc Port 20 1G SGMII 10G XAUI


The Sun Netra CP3240 switch is paired with either a copper or fiber optic RTM. The following sections describe each RTM.

2.4.2.1 Copper RTM

The XCP3240H-RTM-CU-Z is the Sun Netra CP3240 switch’s copper RTM pairing. This RTM supports all the ports the Sun Netra CP3240 switch provides using 10GBase-CX4 for the 10Gigabyte and 10/100/1000Base-T for the 1Gigabyte ports.

FIGURE 2-2 Copper RTM Functional Block Diagram


 


TABLE 2-3 Copper RTM External Ports

Port Type

RTM

Module

Logical Port

Comments

Base 10GBE

XAUI

CX4

Base - 27

 

Base 10GBE

XAUI

CX4

Base - 26

 

Base 1GBE

10/100/1000Base T

RJ-45

Base - 20

Standard I/F is copper

Base 1GBE

10/100/1000Base T

RJ-45

Base - 21

Standard I/F is copper

Management

10/100 Base T

RJ-45

 

 

Serial RS-232

RS-232

RJ-45

 

 

Fabric 10GBE

XAUI

CX4

Fabric - 17

 

Fabric 10GBE

XAUI

CX4

Fabric - 18

 

Fabric 10GBE

XAUI

CX4

Fabric - 19

 

Fabric 10GBE

XAUI

CX4

Fabric - 20

 


 


TABLE 2-4 Copper RTM Port Restrictions

Port

1G Infrastructure

1G Port

10G Infrastructure

10G Port

Base Port 20

SERDES

1000Base-T RJ45

None

None

Base Port 21

SERDES

1000Base-T RJ45

None

None

Base Port 26

None

None

XAUI

10GBase-CX4

Base Port 27

None

None

XAUI

10GBase-CX4

Fabirc Port 17

None

None

XAUI

10GBase-CX4

Fabirc Port 18

None

None

XAUI

10GBase-CX4

Fabirc Port 19

None

None

XAUI

10GBase-CX4

Fabirc Port 19

None

None

XAUI

10GBase-CX4

Fabirc Port 20

None

None

XAUI

10GBase-CX4


2.4.2.2 Fiber Optic RTM

The XCP3240H-RTM-OP-Z is the Sun Netra CP3240 switch’s fiber optic RTM pairing. This RTM supports all the ports the Netra CP3240 switch provides using 10GBase-X for the 10Gigabyte and 1000Base-X for the 1Gigabyte ports.

FIGURE 2-3 Fiber Optic RTM Functional Block Diagram


 


TABLE 2-5 Fiber Optic External RTM Ports

Port Type

Fiber Optic RTM

Module

Logical Port

Comments

Management

10/100 Base T

RJ-45

 

Not a module, 10/100Base T

Base 10GBE

10GBase-X

SFP+

Base - 27

Cannot Support 1G Operation

Base 10GBE

10GBase-X

SFP+

Base - 26

Cannot Support 1G Operation

Base 1GBE

1000Base-X

SFP

Base - 20

Can Support 1G SFP Copper

Base 1GBE

1000Base-X

SFP

Base - 21

Can Support 1G SFP Copper

Serial RS-232

10GBase-X

SFP+

Fabric - 17

Can Support 1G SFP Copper

Fabric 10GBE

10GBase-X

SFP+

Fabric - 18

Can Support 1G SFP Copper

Fabric 10GBE

10GBase-X

SFP+

Fabric - 19

Can Support 1G SFP Copper

Fabric 10GBE

10GBase-X

SFP+

Fabric - 20

Can Support 1G SFP Copper

Serial RS-232

Serial RS232

RJ-45

 

Not a module, RS232 Serial



Note - For enabling 1G operation of Fabric ports 17-20, see Section 4.12, Fiber Optic RTM Configuration.

 


TABLE 2-6 Fiber Optic RTM Port Restrictions

Port

1G Infrastructure

1G Port

10G Infrastructure

10G Port

Base Port 20

SERDES

SFP LX SX

None

None

Base Port 21

SERDES

SFP LX SX

None

None

Base Port 26

None

None

XAUI

SFP+ LX SX LRM

Base Port 27

None

None

XAUI

SFP+ LX SX LRM

Fabric Port 17

SERDES

SFP LX SX

XAUI

SFP+ LX SX LRM

Fabric Port 18

SERDES

SFP LX SX

XAUI

SFP+ LX SX LRM

Fabric Port 19

SERDES

SFP LX SX

XAUI

SFP+ LX SX LRM

Fabric Port 20

SERDES

SFP LX SX

XAUI

SFP+ LX SX LRM


2.4.3 Base Fabric Switch Subsystem

FIGURE 2-4 Base Fabric Switch Subsystem


PICMG 3.0 AdvancedTCA defines 10/100/1000Base-T Ethernet over the Base Fabric, called “Base” in this guide. The Base is designed to be the control plane for the shelf. Scaling from 10Mbps to 1000Mbps, the Base interface can accommodate a wide variety of node boards. The Base interface on the Sun Netra CP3240 switch is based around three main components: the Broadcom BCM56503, the Broadcom BCM5464x, and the Freescale MPC8247.

2.4.4 Expansion Fabric Switch Subsystem

FIGURE 2-5 Expansion Fabric Switch Subsystem


PICMG 3.0 AdvancedTCA provides an agnostic mesh on the backplane called the Expansion Fabric. This interface is the data plane in the shelf. The Expansion Fabric is called “Fabric” in this guide. This fabric can be several different technologies defined by AdvancedTCA sub-specifications. The Sun Netra CP3240 switch is designed to comply with PICMG 3.1 Ethernet/Fibre Channel for AdvancedTCA Systems, options 1 and option 9. That means the Sun Netra CP3240 switch provides a single 1/10Gigabyte port to each node board.

The Fabric uses 10GBase-BX4 Ethernet to provide connectivity between boards though the backplane. The Fabric can scale down to 1Gigabyte and work with 1000Base-BX boards. The Fabric subsystem is based around two main components: the Broadcom BCM56800 and the Freescale MPC8247.

2.4.5 AdvancedMC Sites

PICMG AdvancedMC AMC.0 defines hot-swappable daughter cards to be used in PICMG architectures. The Sun Netra CP3240 switch supports three AMC sites so that the functionality of the Sun Netra CP3240 switch can be expanded. AMC.2 (Ethernet) AMCs are supported with direct connections to the Base and Fabric subsystems. Other types of AMCs, such as AMC.1 (PCI-Express) or AMC.3 (SAS/SATA), can be used as well. Nearly any type of AMC can be supported, because the Sun Netra CP3240 switch connects the AMCs sites directly together on certain ports.

2.4.5.1 AMC Port Maps

FIGURE 2-6 AMC Port Map Diagram


 


TABLE 2-7 AMC Ethernet Port Availability

Slot

Location

1G Infrastructure

10G Infrastructure

AMC 1

Top

1G Base SGMII

None

AMC 2

Middle

1G Base SGMII

XAUI from BCM56800 Fabric Port 16

AMC 3

Bottom

1G Base SGMII

None


 


TABLE 2-8 AMC Site 1: AMC.0 Mid-size, AMC.2 Type E1, AMC.2 Type 5 (Optional), LED Module Support

Port

Type

Connected

0

GB

Base (3.0) switch

1

Any

Not connected

2

Any

Port 3 of AMC #3, usually used for SAS/SATA

3

Any

Port 3 of AMC #2, usually used for SAS/SATA

4

Any

Ports 4-7 of AMC #2, usually used for PCIE

5

6

7

8

 

Not connected

 

9

10

11

12

 

Custom LED board support

13

 

Not connected

14

 

Not connected

15

 

Not connected

17

Any

RTM for future use

18

Any

RTM for future use

19

Any

RTM for future use

20

Any

RTM for future use

TCLK1

CLK IN

FPGA and Zarlink

TCLK2

CLK OUT

FPGA and Zarlink

TCLK3

CLK IN

FPGA and Zarlink

TCLK4

CLK OUT

FPGA and Zarlink

FCLK

100MHz

100MHz


 


TABLE 2-9 AMC Site #2 AMC.0 Mid-Size, AMC.2 Type E1, AMC.2 Type 5, Double module support, LED Module Support

Port

Type

Connected

0

GbE

Base (3.0) switch

1

Any

Port 1 of AMC #3, usually used for GbE

2

Any

Port 2 of AMC #3, usually used for SAS/SATA

3

Any

Port 3 of AMC #1, usually used for SAS/SATA

4

Any

Ports 4-7 of AMC #1, usually used for PCIE

5

6

7

8

10GbE

Fabric (3.1) switch.

9

10

11

12

 

custom LED board support

13

 

Not connected

14

 

Not connected

15

 

Not connected

17

Any

RTM for future use

18

Any

RTM for future use

19

Any

RTM for future use

20

Any

RTM for future use

TCLK1

CLK IN

FPGA & Zarlink

TCLK2

CLK OUT

FPGA & Zarlink

TCLK3

CLK IN

FPGA & Zarlink

TCLK4

CLK OUT

FPGA & Zarlink

FCLK

PCIE CLK

100MHz



TABLE 2-10 AMC Site 3: AMC.0 Mid-size, AMC.2 Type E1, Master Clock Generator Support, LED Support

Port

Type

Connected

0

GB

Base (3.0) switch

1

Any

Port 1 of AMC #2, usually used for GB

2

Any

Port 2 of AMC #2, usually used for SAS/SATA

3

Any

Port 2 of AMC #1, usually used for SAS/SATA

4

 

Not connected

5

 

Not connected

6

 

Not connected

7

 

Not connected

8

 

Not connected

9

 

Not connected

10

 

Not connected

11

 

Not connected

12

 

custom LED board support

13

 

Not connected

14

 

Not connected

15

Any

Update channel, usually used for clock sync

17

Any

RTM for future use

18

Any

RTM for future use

19

Any

RTM for future use

20

Any

RTM for future use

TCLK1

CLK IN

FPGA and Zarlink

TCLK2

CLK OUT

FPGA and Zarlink

TCLK3

CLK IN

FPGA and Zarlink

TCLK4

CLK OUT

FPGA and Zarlink

FCLK

100MHz

100MHz


2.4.5.2 AMC Module Support by Site


TABLE 2-11 AMC Modules by Site

AMC Module

Site 1 (top most when the board is vertical)

Site 2 (middle)

Site 3 (bottom most site when the board is vertical)

AMC10G-XFP

No*

Yes

No

AMC10G-CX4

No*

Yes

No

AMCLED001

Yes

Yes

Yes

PCI-Express AMC

Yes

Yes

No

1G Common Options Ethernet AMC

Yes

Yes

Yes

10G Fat Pipe Ethernet AMC

No*

Yes

No


2.4.5.3 AMC Port Restrictions for RTMs

The following tables list AMC port restrictions for RTMs


TABLE 2-12 AMC Port Restrictions for Copper RTMs

1G Infrastructure

1G Base Ports

10G Infrastructure

10G Fabric Ports

BCM5464R Copper PHY Port 22

10/100/1000 BaseT/RJ-45

XAUI Pass through

10G Fabric XAUI CX4

BCM5464R Copper PHY Port 23

10/100/1000 BaseT/RJ-45

None

None

BCM5464R Copper PHY Port 24

10/100/1000 BaseT/RJ-45

None

None

Front panel Base Port 18

J13 3rd

BCM5464R Copper PHY

 

Front panel Base Port 19

J13 Bottom

BCM5464R Copper PHY

 


 


TABLE 2-13 AMC Port Restrictions for Fiber Optic RTMs

1G Infrastructure

1G Base Ports

10G Infrastructure

10G Fabric Ports

SERDES

SFP - LX SX

 

SFP+ LRM

SERDES

SFP - LX SX

 

None

SERDES

SFP - LX SX

 

None

10/100/1000 BaseT/RJ-45

 

 

 

10/100/1000 BaseT/RJ-45

 

 

 




Sun Netra CP3240 Switch Installation Guide 820-3251-13 Table Of ContentsPrevious ChapterNext ChapterBook Index

Copyright © 2009 Sun Microsystems, Inc. All rights reserved.