JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun Datacenter InfiniBand Switch 648 Topic Set
search filter icon
search icon

Document Information

Preface

Related Documentation

Documentation, Support, and Training

Third-Party Web Sites

Sun Welcomes Your Comments

Administration

Administering the Switch

Understanding Administrative Commands

Monitoring the Switch

Monitoring the InfiniBand Fabric

Controlling the Switch

Controlling the InfiniBand Fabric

Diagnosing a Problem

Identifying the Problem

Troubleshooting the Switch

Understanding Signal Routing Through the Switch

Understanding Switch Startup and Component Addition

Switch GUIDs Overview

Remote Management

Understanding ILOM on the Switch

ILOM Documentation

ILOM Overview

Supported Features

Understanding ILOM Targets

Installing the ILOM Firmware

Firmware Layout

rupgrade_tool Application

Download the ILOM Firmware

Install the ILOM Firmware

Administering ILOM (CLI)

CLI Overview

Access ILOM From the CLI

Switching Between the ILOM Shell and the Linux Shell

Monitoring ILOM Targets (CLI)

Controlling ILOM Targets (CLI)

Upgrading the Switch Firmware Through ILOM (CLI)

Administering ILOM (Web Interface)

Web Interface Overview

Access ILOM From the Web Interface

Monitoring ILOM Targets (Web Interface)

Controlling ILOM Targets (Web Interface)

Upgrading the Switch Firmware Through ILOM (Web Interface)

Understanding ILOM Commands

cd Command

create Command

delete Command

dump Command

exit Command

help Command

load Command

set Command

show Command

version Command

Installation

Preparing the Site

Routing Service Cables

Understanding InfiniBand Cabling

Understanding Switch Specifications

Installing the Switch

Understanding the Installation

Supported Configurations

Fabric Card Distribution

Line Card Distribution

Fabric Card and Line Card Fillers

Installation Preparation

Suggested Tools for Installation

Antistatic Precautions for Installation

Installation Responsibilities

Installer Experience

Installation Sequence

Installing the InfiniBand Software Stack

Install the HPC Software Suite

Unpacking the Chassis

Remove the Shipping Carton Contents

Unsecure the Chassis From the Pallet

Preparing the Chassis

Install the Rail Assemblies in the Rack

Install the Cable Trees

Install the Cable Curves

Install the Chassis in the Rack

Inspecting the Components

Inspecting the Midplane Condition

Midplane Fabric Card Connector Nomenclature

Inspect the Pins on the Rear Side of the Midplane

Midplane Line Card Connector Nomenclature

Inspect the Pins on the Front Side of the Midplane

Inspecting the Fabric Cards Condition

Fabric Card Features

Fabric Card Handling Guidelines

Inspect the Fabric Card Chassis Condition

Inspect the Fabric Card Retainer Bolts Condition

Inspect the Fans Condition

Inspect the Fabric Card XBOW Connectors Condition

Inspecting the Line Cards Condition

Line Card Features

Line Card Handling Guidelines

Inspect the Line Card Chassis Condition

Inspect the Line Card Retainer Bolts Condition

Inspect the Line Card XBOW Connectors Condition

Inspect the Line Card CXP Connectors Condition

Inspecting the InfiniBand Cables Condition

InfiniBand Cable Features

Inspect the InfiniBand Cables Hardware Condition

Inspect the InfiniBand Cables Connector Condition

Installing the Components

Install the Midplane Stiffener

Install the Fabric Cards or Fabric Card Fillers

Remove the Midplane Stiffener

Install the Line Cards

Powering On the CMCs

Power Supply Addresses

Power Supply Status LEDs

CMC Addresses

CMC Status LEDs

Attach Management Cables

Attach Power Cords

Accessing the CMC

Access the CMC From the Serial Management Port

Set the Initial UBoot Parameters

Access the CMC From the Network Management Port

Verify the CMC Status

Display the CMC FRU ID Information

Verify the Power Supply Status

Display the Power Supply FRU ID Information

Powering On the Fabric Cards

Fabric Card Addresses

Fabric Card Status LEDs

Enable the Fabric Cards

Create the guid.txt File for Installation

Power On the Fabric Cards

Verify the Fabric Card Status

Display the Fabric Card FRU ID Information

Powering On the Line Cards

Line Card Addresses

Line Card Status LEDs

Enable the Line Cards

Power On the Line Cards

Verify the Line Card Status

Display the Line Card FRU ID Information

Connecting InfiniBand Cables

CXP Connector Addresses

CXP Connector Status LEDs

Attach Cables to the CXP Connectors

Check Link Status

Display the InfiniBand Cable FRU ID Information

Verifying the InfiniBand Fabric

Discover the InfiniBand Fabric Topology

Perform Diagnostics on the InfiniBand Fabric

Validate the InfiniBand Fabric and Report Errors

Service

Servicing the Switch

Understanding Service Procedures

Servicing the Power Supplies

Servicing the CMCs

Servicing the Fans

Servicing the Fabric Cards and Fabric Card Fillers

Servicing the InfiniBand Cables

Servicing the Line Cards

Doing Supportive Software Tasks

Reference

Command Reference

Understanding Switch-Specific Commands

Understanding CLIA Commands

Understanding InfiniBand Software Commands

Index

Attach Cables to the CXP Connectors


Note - This procedure describes cabling the switch in a center split, left-right symmetrical configuration.


  1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to install the switch.

    See the table in Installation Sequence.

  2. Bring the cables to the switch.
  3. Install a cable shelf into the slot below the lowest line card.
    1. Lift the cable shelf into a horizontal orientation, with the long rolled edge towards you.
    2. Locate where you want to install the cable shelf.
    3. Move the cable shelf between and just above the branches of the cable tree for the slot where you want to install the shelf.
    4. Continue to move the shelf in, then lower the shelf into the tabs of the branches.
      Illustration shows the cable shelf being installed.

      Note - Install cable shelves only as they are needed. If all cable shelves are installed before the switch is cabled, the shelves’ presence makes cabling difficult.


  4. Plug the first cable into the CXP connector at 11B.

    Note - For each connector group, A is the upper connector, B is the lower connector.



    Illustration shows an InfiniBand cable being aligned with the connector.
    1. Remove the protective cap from the connector and visually inspect the cable connector.

      The shell should not be bent and should be parallel to the inner boards. If the connector is bent or damaged, use a different cable.

    2. Ensure that the retraction strap is forward.
    3. Orient the cable connector to the line card CXP receptacle squarely and horizontally.

      Ensure that the upper shell just touches the underside of the top of the receptacle on the line card.


      Illustration shows an InfiniBand cable being properly connected.
    4. Slowly move the connector in.

      As you slide the connector in, the top of the shell should scrape against the underside of the top of the CXP receptacle.

      • If the cable stops or binds after about 5 mm (1/4 in.) travel, back out and repeat from Step c.
      • If the connector stops or binds with about 2 mm (1/8 in.) still to go, back out and repeat Step d.
    5. Continue to push the connector in until the hooks catch onto the top of the receptacle.
  5. Curve the cable to the right through the branches of the cable tree.
    Illustration shows the InfiniBand cable feeding through the cable tree.

    Caution

    Caution - Do not allow any copper core InfiniBand cable to bend tighter than a 5 inch (127 mm) radius. Do not allow any optical InfiniBand cable to bend tighter than a 3.4 inch (85 mm) radius. Tight bends can damage the cable internally.


  6. Place the cable over the cable curve and move the cable in toward the rack.
    Illustration shows InfiniBand cables being tidied.
  7. Repeat Step 4 for the second cable at CXP connector 0B.
  8. Curve the cable to the left through the branches of the cable tree.

    Caution

    Caution - Do not force a bend radius of less than 5 inches (127 mm). Any bend tighter than this damages the inner fibers of the cable, rendering it useless.


  9. Place the cable over the cable curve and move the cable in toward the rack.
  10. Repeat Step 4 for the third cable at CXP connector 10B.
  11. Curve the cable to the right through the branches of the cable tree, next to the cable already there.
  12. Place the cable over the cable curve and move the cable against the cable already there.
  13. Repeat Step 4 for the fourth cable at CXP connector 1B.
  14. Curve the cable to the left through the branches of the cable tree, next to the cable already there.
  15. Place the cable over the cable curve and move the cable against the cable already there.
  16. Repeat Step 4 for cables connecting to CXP connectors 9B, 2B, and so on, to and including 6B and 5B.

    Place the cables next to the cables that are already there.

  17. Repeat Step 4 for cables connecting to CXP connectors 11A, 0A, 10A, 1A, and so on, to and including 6A and 5A.

    Place the cables on top of the cables that are already there.

  18. When you have finished for the lowest line card, repeat from Step 3 through Step 17 for the remaining line cards, working your way up.

    Note - There are two cable curves on each side of the switch. The cables from line cards 0 through 4 route over the lowest cable curve. The cables from line cards 5 through 8 route over the next cable curve up.


  19. Install the EMI screen.
    1. Hook the lower half of the screen at the two square holes at the bottom of the cable trees.
    2. Expand the screen and set the upper half over the top of the switch chassis and cable trees.
    3. Insure that the screen is seated against the cable trees.
  20. Check link status.

    See Check Link Status.

Related Information