| Netra CT 900 Server Product Overview |    
|
| C H A P T E R 5 |
|
Switch Description |
Two switches are available to use in the Netra CT 900 server: the Netra CP3140 and the Netra CP3240. This chapter describes the Netra CP3140 switch.
For documentation about the Netra CP3240 switch, refer to the following web site:
http://docs.sun.com/app/docs/prod/cp3240.switch?l=en#hic
The Netra CP3140 1-GbE switch is an AdvancedTCA 3.0 and 3.1 Option 1 switch. This means that the switch implements two separate switched networks on a single printed circuit board (PCB). By separating the Base and Extended Fabric networks, the switch provides a separate control plane and data plane. It provides 10/100/1000BASE-T Ethernet switching on the 3.0 Base Fabric interface and on the 3.1 Extended Fabric interface it provides 1000BASE-X Ethernet switching. Both of these networks are fully managed and work with the robust FASTPATH management suite. Both networks support Layer 2 switching as well as Layer 3 routing. The Netra CP3140 switch also supports a rear transition module to expand connectivity with additional uplink ports.
This chapter includes the following topics:
FIGURE 5-1 shows the functional block diagram for the Netra CP3140 switch, and FIGURE 5-2 shows the functional block diagram for the rear transition module for the switch. Refer to TABLE 5-1 for a key to the different colored blocks in the block diagrams.
FIGURE 5-1 Functional Block Diagram of the Netra CP3140 Switch
FIGURE 5-2 Functional Block Diagram for the Netra CP3140 Rear Transition Module
The switch can be divided into four portions:
The following sections provide overviews of the switch’s main components and subsystems.
PICMG 3.0 AdvancedTCA defines 10/100/1000BASE-T Ethernet over the Base Fabric interface, called “Base” in this manual. Pictured in gray in FIGURE 5-1, the Base is designed to be the control plane for the shelf. Scaling from 10 Mbps to 1000 Mbps, the Base interface can accommodate a wide variety of node boards.
PICMG 3.1 AdvancedTCA provides an agnostic mesh on the backplane called the Extended Fabric interface. This fabric can be several different technologies defined by AdvancedTCA subspecifications. The switch is designed to comply with the PICMG 3.1 Ethernet/Fibre Channel for AdvancedTCA Systems. Specifically, the switch supports option 1 of the PICMG 3.1 specification, which provides a single gigabit Ethernet port to the node board. The Extended Fabric gigabit Ethernet is depicted in green in FIGURE 5-1. The gigabit Ethernet portion of the Fabric interface is called “Fabric GbE” in this manual.
The Fabric GbE interface uses 1000BASE-BX to provide connectivity between boards through the backplane. This interface is the data plane in the shelf. Fabric GbE interface is a different type of Ethernet than Base interface. It is 1000BASE-BX, whereas Base interface is 10/100/1000BASE-T. 1000BASE-BX is digital and does not scale down to 10 Mbps or 100 Mbps. It only operates at 1000 Mbps. The Fabric GbE subsystem is based on the same components as the Base except BCM5464x (transceivers) are not needed for the backplane ports. Note that the Fabric GbE RJ-45 ingress and egress ports are 10/100/1000BASE-T, not 1000BASE-BX.
There is a difference in port shutdown behavior between the Base and Fabric interfaces. The Fabric ports are physically shutdown. This means that when a Fabric port is shutdown, the other side will not show a link. In contrast, Base ports are only logically shutdown. This means that when a Base port is shutdown, the other side will show a link. This difference is not a defect, but merely a difference in the Fabric and Base interfaces.
This difference occurs because Fabric ports are a different type of Ethernet than Base ports. Base ports need external PHYs to support 1000Base-T. Fabric ports do not need external PHYs, because the BCM5695 can support 1000Base-BX directly.
When a shutdown command is issued on both the Base and Fabric ports, the command goes to the BCM5695, because the BCM5695 connects directly to the backplane in case the Fabric link goes down. In the case of the Base, even when the BCM5695 port is shut down, the BCM5464R port is not shut down, so the link partner still sees a link.
The Netra CP3140 switch supports a rear transition module through ATCA Zone 3 connectors. Seven Base ports and eight Fabric ports go to the rear transition module. Rather than run 10/100/1000BASE-T to the rear transition module, SGMII signals are used for each port. This means the rear transition module can support 10/100/1000BASE-T, 1000BASE-CX, or 1000BASE-LX in any combination of these technologies. In addition to the uplink ports, serial management ports for both the Base and Fabric interfaces, as well as a 10/100 management port, are provided to the rear transition module.
The following sections provide a short description of key parts on the switch.
The switch uses Broadcom StrataXGS 2 BCM5695 for Ethernet switching and routing. This chip is an Ethernet switch with twelve 1-GbE ports and 1-HiGig+ (12 GbE) port. There are four total BCM5695s on the switch, two for the Base and two for the Fabric GbE interface. The two chips in each subsystem are connected to each other with their HiGig+ (12 Gps) ports. Therefore, these two chips are set up to act as a single 24-port non-blocking, wire-speed gigabit Ethernet switch and router. The BCM5695 features hardware support for line rate switching, a 16-Kbyte MAC address table, IP multicast, Rapid Spanning Tree Protocol (RSTP), jumbo frames, and packet processor for Quality of Service (QoS) among other features.
The quad (BCM5464R) and single (BCM5461S) Broadcom ports provide the physical interfacing for 10/100/1000BASE-T. They are low-power devices and provide features such as jumbo frames support, auto-MDIX, and cable testing.
The Freescale MPC8247 is a microprocessor designed for maximum flexibility. It features a dual core architecture with a PPC G2 LE core and a RISC core controlling the peripherals. Running at 400 MHz with only 1W power, the MPC8247 provides high performance with incredibly low power consumption. Paired with the 128-Mbyte PC100 SDRAM and 32-Mbyte flash memory, the CPU subsystem of the switch is only 20 percent utilized in worst case conditions. This provides plenty of room for customer applications and future upgrades.
The following sections briefly describe the basic system requirements and configurable features of the switch. Links are provided to other chapters and appendixes containing more detailed information.
The two switches must be installed in logical slots 1 and 2 (physical slots 7 and 8) in the Netra CT 900 server.
The Base interface is always routed in a Dual Star topology on the midplane. This means every node slot has a Base channel routed to each of the switch slots. Independent of how the Fabric interface is used, a switch is always needed for the Base interface. The Fabric interface is normally routed the same way, one Fabric channel per node per switch for a total of two Fabric channels per node. Shelves routed like this are called Dual Stars and they are the most common.
TABLE 5-2 describes the power requirements for the switch.
The -48 VDC has a tolerance of 0 VDC to -75 VDC without damage. The switch operates when -48 VDC is --40.5 VDC to -72 VDC, inclusive.
|
Caution - Any input voltage not in the range of 0 VDC to -75 VDC can damage the switch. |
The switch might contain materials that require regulation upon disposal. Please dispose of this product in accordance with local rules and regulations. For disposal or recycling information, please contact your local authorities or the Electronic Industries Alliance at http://www.eiae.org/.
FIGURE 5-3 shows the location of the ports and LEDs on the front of the switch, and FIGURE 5-4 shows the location of the ports and LEDs on the rear transition module for the switch.
FIGURE 5-3 Ports and LEDs on the Netra CP3140 Switch
FIGURE 5-4 Ports on the Netra CP3140 Rear Transition Module
|
Base and Extended Fabric Ethernet 10/100BASE-TX management port |
|
|
Extended Fabric (3.1) gigabit Ethernet serial management port |
|
|
Extended Fabric (3.1) gigabit Ethernet 10/100/1000BASE-T ports 17-20 |
|
|
Extended Fabric (3.1) gigabit Ethernet 10/100/1000BASE-T ports 21-24 |
The LED Select push button switches between 3.0 and 3.1 mode on the Port Status LEDs. The 3.0 mode is the Base Fabric which is a dual-star 10/100/1000BASE-T Ethernet configuration. The 3.1 mode is the Extended Fabric which is a mesh 10/100/1000BASE-T Ethernet configuration.
The numbering of the Port Status LEDs is shown in TABLE 5-5. Slots 7 and 8 are the switches. LED numbers 16 to 24 corresponds to the ports on the rear transition module. The port numbers are labeled on the rear transition module.
There is a single set of 24 LEDs on the faceplate of the switch. Each LED represents a port on one of the subsystems’ switch. They are numbered 1 through 24, and the number lights up when the corresponding port is linked.
There are three LED locations defined by AdvancedTCA to monitor board status.
|
Out of service. This LED lights on a critical switch error, such that the board should be removed. |
|||
Note that both the OOS and MINOR LEDs are lit when the board is powered on but not booted. This includes all hot-swap states M1 through M3. It remains on until the FASTPATH software has booted on both the Base and Fabric interface. See Hot-Swap LED for more information on the hot-swap states.
The Fabric gigabit Ethernet 10/100/1000BASE-T and Base 10/100/1000BASE-T Ethernet uplink ports on the switch use standard RJ-45 connectors.
The Base 10/100/1000BASE-T port is port number 17 on the Base network. The Base 10/100/1000BASE-T port is mutually exclusive with the second ShMC port. That is, if the ShMC cross-connection is being used, this port goes to the second ShMC and not the faceplate of the switch.
The Fabric gigabit Ethernet 10/100/1000BASE-T port is port number 16 on the Fabric network.
FIGURE 5-5 shows the pinouts for the 10/100/1000BASE-T port.
FIGURE 5-5 10/100/1000BASE-T Ports Connector Diagram
TABLE 5-8 gives the pinout information for the 10/100/1000BASE-T port.
The Base 10/100BASE-TX management port uses a standard RJ-45 connector. This port can be used to manage the Base and Fabric interfaces. This port and the 10/100 management port on the rear transition module can be used at the same time.
FIGURE 5-6 shows the pinouts for the 10/100BASE-TX management port.
FIGURE 5-6 Base 10/100BASE-TX Management Port Connector Diagram
TABLE 5-9 gives the pinout information for the 10/100BASE-TX management port.
The Extended Fabric gigabit Ethernet serial port and Base serial port on the switch use standard RJ-45 connectors. Note that the front serial port and rear transition module serial port are actually the same port. Only one of the interfaces can be used. Jumpers E7 and E8 can be used to steer the port out the front or out the back, or to allow software to control the direction.
FIGURE 5-7 shows the pinouts for the Fabric gigabit Ethernet serial port and Base serial port.
FIGURE 5-7 Fabric Gigabit Ethernet and Base Serial Ports Connector Diagram
TABLE 5-10 gives the pinout information for the Fabric gigabit Ethernet serial port and Base serial port.
TABLE 5-11 gives the minimum crossover cable pinouts needed to create a special cable or adapter to convert the serial ports’ RJ-45 connectors on the switch to the more standard DB-9 connectors.
This blue LED communicates the hot-swap status of the switch. TABLE 5-12 shows the different states the Hot-Swap LED goes through.
| Note - A board should be hot-swapped only when the LED is solid blue. |
There are separate Reset push buttons for the Base and Fabric GbE interfaces. The buttons are recessed and a paper clip or pin should be used to press the buttons. When pushed, just the ports in the button respective subsystem resets. The IPMI subsystem is not reset. It resets only with a board hot-swap.
The switch has been designed for maximum flexibility. Many features can be configured by the user for specific applications. Most configuration options are selected through the switch software covered in detail in the Netra CT 900 Server Switch Software Reference Manual. Some options cannot be software controlled and are configured with jumpers. Those options are covered in this section.
TABLE 5-13 lists the configuration features that are controlled by jumpers on the switch, and FIGURE 5-8 shows the locations of the jumpers on the switch.
FIGURE 5-8 Jumper Locations on the Netra CP3140 Switch
This jumper is used to control ShMC cross-connect. ShMC cross-connect is the ability to connect to two ShMCs at 10/100 each rather than using a single 10/100/1000 for a single ShMC. When cross-connect is enabled, the front panel Base port (J23) is redirected to the secondary ShMC. Therefore the front panel port no longer works (TABLE 5-14).
This jumper is used in the manufacturer’s test. Leave this jumper off (TABLE 5-15).
This jumper allows the Intelligent Platform Management Interface (IPMI) to send a reset signal that resets the entire board (TABLE 5-16).
|
IPMI subsystem cannot reset the switch (use this to run without a ShMC). |
|
This jumper is used to control whether the switch is forced on when it is powered on or if the IPMI subsystem controls the power on of the switch. Note that forcing power to the board is not enough; the board could still be held in reset. Install E3(1-2) as well or alternatively install E4(3-4) to either bring the board out of reset or hold the IPMI in reset (TABLE 5-17).
The IPMI watchdog should be enabled so that the IPMI subsystem can reset itself if the shelf is not ready or if there is a problem. Use this jumper to disable the watchdog (TABLE 5-18).
If on, this jumper holds the IPMI subsystem in reset (TABLE 5-19).
This jumper is used to tell the Fabric GbE CPU to use the default Reset Configuration Word, or use the one in the flash memory (TABLE 5-20).
This jumper is used to tell the Base CPU to use the default Reset Configuration Word, or use the one in the flash memory (TABLE 5-21).
This jumper is used to adjust the Joint Test Action Group (JTAG) chain of the IPMI subsystem during programming. It has no effect under normal operation (TABLE 5-22).
The front panel serial port and the rear transition module serial port are mutually exclusive. Only one can be used at any one time. The serial port can be forced to the front card or the rear transition module, or it can be controlled by the software (TABLE 5-23).
|
Front serial port active, rear transition module serial port disabled |
|
|
Front serial port disabled, rear transition module serial port active |
|
The front panel serial port and the rear transition module serial port are mutually exclusive. Only one can be used at any one time. The serial port can be forced to the front card or the rear transition module, or it can be controlled by the software (TABLE 5-24).
|
Front serial port active, rear transition module serial port disabled |
|
|
Front serial port disabled, rear transition module serial port active |
|
This jumper is connected to the field-programmable gate array (FPGA). It is reserved for future use (TABLE 5-25).
The switch, and the whole AdvancedTCA shelf for that matter, separate the ground of the chassis itself from the digital ground for EMI protection. This jumper connects those two grounds (TABLE 5-26).
| Netra CT 900 Server Product Overview | 819-1174-14 |
|
Copyright © 2009 Sun Microsystems, Inc. All rights reserved.
