Exit Print View

Sun Datacenter InfiniBand Switch 648 Topic Set

Get PDF Book Print View
 

Document Information

Using This Documentation

Related Documentation

Documentation, Support, and Training

Documentation Feedback

Installation

Preparing the Site

Installing the Switch

Understanding the Installation

Supported Configurations

Fabric Card Distribution

Fabric Card Handling Guidelines

Line Card Distribution

Line Card Handling Guidelines

Fabric Card and Line Card Fillers

Installation Preparation

Suggested Tools for Installation

Antistatic Precautions for Installation

Installation Responsibilities

Installation Sequence

Installing the InfiniBand Software Stack

Install the HPC Software Suite

Unpacking the Chassis

Remove the Shipping Carton Contents

Unsecure the Chassis From the Pallet

Preparing the Chassis

Install the Rail Assemblies in the Rack

Install the Cable Trees

Install the Cable Curves

Install the Chassis in the Rack

Inspecting the Components

Inspecting the Midplane for Installation

Identify the Midplane Fabric Card Connectors

Inspect the Pins on the Rear Side of the Midplane

Identify the Midplane Line Card Connectors

Inspect the Pins on the Front Side of the Midplane

Inspecting the Fabric Cards for Installation

Identify the Fabric Cards for Installation

Inspect the Fabric Card Chassis for Installation

Inspect the Fabric Card Retainer Bolts for Installation

Inspect the Fans for Installation

Inspect the Fabric Card XBOW Connectors for Installation

Inspecting the Line Cards for Installation

Identify the Line Cards for Installation

Inspect the Line Card Chassis for Installation

Inspect the Line Card Retainer Bolts for Installation

Inspect the Line Card XBOW Connectors for Installation

Inspect the Line Card CXP Connectors for Installation

Inspecting the InfiniBand Cables for Installation

Identify the InfiniBand Cable for Installation

Inspect the InfiniBand Cable Hardware for Installation

Inspect the InfiniBand Cable Connectors for Installation

Installing the Components

Install the Midplane Stiffener

Install the Fabric Cards or Fabric Card Fillers

Remove the Midplane Stiffener

Install the Line Cards or Line Card Fillers

Powering On the CMCs

Attach the Management Cables

Attach the Power Cords

Check the Power Supply Status LEDs

Check the CMC Status LEDs

Accessing the CMC

Access the CMC From the Serial Management Port

Set the Initial UBoot Parameters

Access the CMC From the Network Management Port

Verify the CMC Status

Identify the CMC Addresses

Display the CMC FRU ID Information

Verify the Power Supply Status

Identify the Power Supply Addresses

Display the Power Supply FRU ID Information

Powering On the Fabric Cards

Enable the Fabric Cards for Installation

Create the guid.txt File for Installation

Power On the Fabric Cards for Installation

Check the Fabric Card Status LEDs

Verify the Fabric Card Status

Identify the Fabric Card Addresses

Display the Fabric Card FRU ID Information

Powering On the Line Cards

Enable the Line Cards for Installation

Power On the Line Cards for Installation

Check the Line Card Status LEDs

Verify the Line Card Status

Identify the Line Card Addresses

Display the Line Card FRU ID Information

Connecting InfiniBand Cables

Attach Cables to the CXP Connectors

Check the CXP Connector Status LEDs

Check Link Status

Identify the CXP Connector Addresses

Display the InfiniBand Cable FRU ID Information

Verifying the InfiniBand Fabric

Discover the InfiniBand Fabric Topology

Perform Diagnostics on the InfiniBand Fabric

Validate the InfiniBand Fabric and Report Errors

Administration

Administering the Switch

Diagnosing a Problem

Remote Management

Understanding ILOM on the Switch

Installing the ILOM Firmware

Administering ILOM (CLI)

Administering ILOM (Web Interface)

Administering ILOM (SNMP)

Administering Hardware (IPMI)

Understanding ILOM Commands

Service

Servicing the Switch

Reference

Command Reference

Index

Attach Cables to the CXP Connectors


Note - This procedure describes cabling the switch in a center split, left-right symmetrical configuration.


  1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to install the switch.

    See the table in Installation Sequence.

  2. Bring the cables to the switch.

  3. Install a cable shelf into the slot below the lowest line card.

    1. Lift the cable shelf into a horizontal orientation, with the long rolled edge towards you.

    2. Locate where you want to install the cable shelf.

    3. Move the cable shelf between and just above the branches of the cable tree for the slot where you want to install the shelf.

    4. Continue to move the shelf in, then lower the shelf into the tabs of the branches.


      Illustration shows the cable shelf being installed.

      Note - Install cable shelves only as they are needed. If all cable shelves are installed before the switch is cabled, the shelves’ presence makes cabling difficult.


  4. Plug the first cable into the CXP connector at 11B.


    Note - For each connector group, A is the upper connector, B is the lower connector.



    Illustration shows an InfiniBand cable being aligned with the connector.
    1. Remove the protective cap from the connector and visually inspect the cable connector.

      The shell should not be bent and should be parallel to the inner boards. If the connector is bent or damaged, use a different cable.

    2. Ensure that the retraction strap is forward.

    3. Orient the cable connector to the line card CXP receptacle squarely and horizontally.

      Ensure that the upper shell just touches the underside of the top of the receptacle on the line card.


      Illustration shows an InfiniBand cable being properly connected.
    4. Slowly move the connector in.

      As you slide the connector in, the top of the shell should scrape against the underside of the top of the CXP receptacle.

      • If the cable stops or binds after about 5 mm (1/4 in.) travel, back out and repeat from Step c.

      • If the connector stops or binds with about 2 mm (1/8 in.) still to go, back out and repeat Step d.

    5. Continue to push the connector in until the hooks catch onto the top of the receptacle.

  5. Curve the cable to the right through the branches of the cable tree.


    Illustration shows the InfiniBand cable feeding through the cable tree.

    Caution

    Caution - Do not allow any copper core InfiniBand cable to bend tighter than a 5 inch (127 mm) radius. Do not allow any optical InfiniBand cable to bend tighter than a 3.4 inch (85 mm) radius. Tight bends can damage the cable internally.


  6. Place the cable over the cable curve and move the cable in toward the rack.


    Illustration shows InfiniBand cables being tidied.
  7. Repeat Step 4 for the second cable at CXP connector 0B.

  8. Curve the cable to the left through the branches of the cable tree.


    Caution

    Caution - Do not force a bend radius of less than 5 inches (127 mm). Any bend tighter than this damages the inner fibers of the cable, rendering it useless.


  9. Place the cable over the cable curve and move the cable in toward the rack.

  10. Repeat Step 4 for the third cable at CXP connector 10B.

  11. Curve the cable to the right through the branches of the cable tree, next to the cable already there.

  12. Place the cable over the cable curve and move the cable against the cable already there.

  13. Repeat Step 4 for the fourth cable at CXP connector 1B.

  14. Curve the cable to the left through the branches of the cable tree, next to the cable already there.

  15. Place the cable over the cable curve and move the cable against the cable already there.

  16. Repeat Step 4 for cables connecting to CXP connectors 9B, 2B, and so on, to and including 6B and 5B.

    Place the cables next to the cables that are already there.

  17. Repeat Step 4 for cables connecting to CXP connectors 11A, 0A, 10A, 1A, and so on, to and including 6A and 5A.

    Place the cables on top of the cables that are already there.

  18. When you have finished for the lowest line card, repeat from Step 3 through Step 17 for the remaining line cards, working your way up.


    Note - There are two cable curves on each side of the switch. The cables from line cards 0 through 4 route over the lowest cable curve. The cables from line cards 5 through 8 route over the next cable curve up.


  19. Install the EMI screen.

    1. Hook the lower half of the screen at the two square holes at the bottom of the cable trees.

    2. Expand the screen and set the upper half over the top of the switch chassis and cable trees.

    3. Insure that the screen is seated against the cable trees.

  20. Verify the cabled CXP connector status LEDs indicate a link.

    The Link LED should be lit and not flash. See Check the CXP Connector Status LEDs.

Related Information