Sun HPC 3.0 SCI Guide

Chapter 5 Adding or Replacing SCI Adapter Cards

If you add or replace an SCI adapter card on a node that has already been configured by sm_config, perform the steps described in this chapter to initialize the new adapter card.

Note -

Because the node has already been configured by sm_config, it should also already contain SUNWsci and the other SCI-related packages. If not--that is, if you will be installing these SUNW packages as part of the adapter card installation process--perform the steps described in Chapter 2, Network Connection Procedure and Chapter 3, Configuring the SCI Network Interface instead, but only on the node receiving the SCI-related packages and new adapter card.

Install/Replace SCI Adapter Card

Turn the node's power switch off and disconnect it from the power outlet. Note that this only needs to be done on the node receiving the new adapter card.

Check the scrubber jumper setting on the new card and change its setting if needed.

If replacing an existing SCI adapter card without changing any other features of the network, simply give the new adapter card the same scrubber jumper setting as the one it is replacing.

If installing a new adapter card that will be connected directly to another adapter card--not through a switch--set its scrubber jumper to the opposite of the other adapter card. That is, if the other card's scrubber jumper is ON, set this card's jumper to OFF. If the other is OFF, set this one to ON.

If installing a new adapter card that will be connected to an SCI switch, set its scrubber jumper to ON.

If replacing an existing SCI adapter card, simply install the new card in the same SBus slot.

If adding a new SCI adapter card, install it in an available SBus slot. If possible, install the new card in the next higher slot. This will promote an orderly relationship between SBus slot numbers and network ID values, which can simplify network administration.

Connect New Adapter Card to Network

Connect the new card to the network, either directly to an SCI adapter in another node or to a port on an SCI switch. See the sample two-, three-, and four-node topologies illustrated in Figure 2-1, Figure 2-2, and Figure 2-3.

Create a Temporary Network Map

Either update the temporary network map that was made when the network was created or last changed or create a new temporary network map as described in Chapter 3, Configuring the SCI Network Interface. Include the new adapter card in the map, identifying its SBus location and serial number.

Run `sciconf`

Run the command sciconf as root on the node that contains the new SCI adapter card. This will initialize the card so it can be used by the local SCI driver software. In the following example, sciconf is run on a node that has been added to an existing network. This node contains a single SCI adapter whose serial number is 6269. The sciconf output indicates the network ID and SBus slot of this SCI adapter.

# /opt/SUNWsci/bin/sciconf
Dolphin SCI SBus2b card configuration script
== Doing drvconfig, please wait ...
== Doing devlinks, please wait ...
== drvconfig and devlinks completed ...
== 1 SCI card was detected in the system
== Programming SCI card #1 with nodeid, adapter #, and firmware ...
== this takes 20 seconds, please wait ...
== Programming is done ... Please verify the following information:
sciadm $Revision: 2.30 $
DOLPHIN SBus card (SBus2b) found in SBus slot 0 on Board# 0, card slot 0.
Adapter number:          0 
NodeId:                  220 (0xdc)
Slot Number:             0   (0x00)
System Board Number:     0   (0x00)
Card Slot Number:        0   (0x00)
Adapter Serial Number:   6269
Fcode Version:           Revision: 2.18
Fcode checksum:          0x8906bca8
Basic FPGA checksum:     0xaf0c0c7d
Alternative FPGA checksum: 0xaf0c0c7d
== Programming done ... remember to reboot the system when finished

Update `sci_config.hpc`

Update the /opt/SUNWsma/sci_config.hpc file to incorporate the information provided by the sciconf output.

Run `sm_config`

Execute the sm_config command as root on the node that contains the sci_config.hpc file.

If possible, do this from a console terminal so you can see the output it generates. If this is not possible, examine the output in /var/adm/messages. The following example shows the output that would be generated for the four-node, nonstriped sample configuration shown in Figure 3-1.

Note -

Do not press Return when the sm_config output pauses. Go to the next section first.

# /opt/SUNWsma/bin/sm_config -f ../sci_config.hpc
For Host #0 (node3), adapter details :-
        Adp #0 :- serial no = 6148; bus slot = 0;
For Host #1 (node4), adapter details :
        Adp #0 :- serial no = 6520; bus slot = 0;
For Host #2 (node5), adapter details :
-        Adp #0 :- serial no = 6527; bus slot = 0;
For Host #3 (node6), adapter details :-
        Adp #0 :- serial no = 6269; bus slot = 0;
Press Return to continue:

Confirm `sci_config.hpc` Contents

Compare the list of serial numbers in the sm_config output with the serial numbers in the temporary network map.

If the sm_config output conflicts with Section 6 of the sci_config.hpc file, stop execution of sm_config (press Control-C) and correct the configuration file. Then run sm_config again and compare its output with sci_config.hpc again.

When the contents of the sci_config.hpc file are confirmed by the sm_config output, press Return to allow sm_config to complete execution.

Reboot the Node

The sm_config output will also identify which nodes need to be rebooted. Reboot those nodes.

Verify the New Network

Perform the network verification steps described in Chapter 4, Verify That the Network Is Functional to be certain that the network still functions correctly.