x86: PCI Hot-Plugging With cfgadm

The cfgadm command is updated in the Solaris 7 11/99 software release to provide PCI hot-plugging for supported PCI adapter cards on x86 systems. This information supplements information on managing devices found in "Managing Devices" in System Administration Guide, Volume I.

Hot-plugging PCI adapter cards provides the ability to add and remove a hot-pluggable PCI adapter card on a running system without rebooting the system.

As part of Sun's high availability strategy, this feature is expected to be used in conjunction with additional layered products, such as alternate pathing or fail-over software, which provide fault tolerance in the event of a device failure.

Without any high availability software, you can replace a failed device by manually stopping the appropriate applications, unconfiguring the device, and then proceeding with the add or remove operations.

x86: Hardware That Supports Hot-Plugging

Currently, PCI hot-plugging is supported on the Intel platform. Please refer to x86 Hardware Compatibility List for a list of PCI hardware that supports hot-plugging.

x86: PCI Hot-Plugging With the cfgadm Command

The following section describes different hot-plugging operations and then provides step-by-step instructions for hot-plugging PCI adapter cards.

Detaching PCI Adapter Cards

A PCI adapter card hosting non-vital system resources can be removed if the device driver supports hot-plugging.

A PCI adapter card is not detachable if it is a vital system resource. For a PCI adapter card to be detachable:

• The device driver must support hot-plugging.

• Critical resources must be accessible through an alternate pathway.

For example, if a system has only one ethernet card installed in it, the ethernet card cannot be detached without losing network connection. This replacement requires additional layered software support to keep the network connection active.

See "x86: How to Remove a PCI Adapter Card" for step-by-step instructions on removing a PCI adapter card.

Attaching PCI Adapter Cards

A PCI adapter card can be added to the system as long as:

• There are slots available.

• The device driver supports hot-plugging for this adapter card.

See "Attaching PCI Adapter Cards" for step-by-step instructions on adding a PCI adapter card.

In the following examples, only PCI attachment points are listed, for brevity. The attachment points displayed on your system will depend on your system configuration.

x86: How to Display PCI Slot Configuration Information

The cfgadm (1M) command displays the status of PCI hot-pluggable devices and slots on a system.

1. Become superuser.

2. Display PCI slot configuration information.

   # cfgadm pci Ap_Id Receptacle Occupant Condition pci1:hpc0_slot0 empty unconfigured unknown pci1:hpc0_slot1 empty unconfigured ok pci1:hpc0_slot2 empty unconfigured ok pci1:hpc0_slot3 connected configured ok pci1:hpc0_slot4 empty unconfigured ok # cfgadm -s "cols=ap_id:type:info" pci Ap_Id Type Information pci1:hpc0_slot0 unknown Slot 7 pci1:hpc0_slot1 unknown Slot 8 pci1:hpc0_slot2 unknown Slot 9 pci1:hpc0_slot3 Ethernet/HP Slot 10 pci1:hpc0_slot4 unknown Slot 11

The logical ap_id, pci1:hpc0_slot0, is the logical ap_id for that particular hot-pluggable slot, Slot 7, (physical identification of this slot). The component hpc0 indicates the hot-pluggable adapter card for this slot and pci1 indicates the PCI bus instance. The Type field indicates the type of PCI adapter card present in the slot.

x86: How to Remove a PCI Adapter Card

1. Become superuser.

2. Determine which slot the controller is in.

   # cfgadm Ap_Id Receptacle Occupant Condition pci1:hpc0_slot0 empty unconfigured unknown pci1:hpc0_slot1 empty unconfigured unknown pci1:hpc0_slot2 empty unconfigured unknown pci1:hpc0_slot3 connected configured ok pci1:hpc0_slot4 empty unconfigured unknown

3. Stop the application that has the device open.

   For example, if this is an ethernet card, use ifconfig(1M) to bring down the interface and unplumb the interface.

4. Unconfigure the device.

   # cfgadm -c unconfigure pci1:hpc0_slot3

5. Confirm the device has been unconfigured.

   # cfgadm Ap_Id Receptacle Occupant Condition pci1:hpc0_slot0 empty unconfigured unknown pci1:hpc0_slot1 empty unconfigured unknown pci1:hpc0_slot2 empty unconfigured unknown pci1:hpc0_slot3 connected unconfigured unknown pci1:hpc0_slot4 empty unconfigured unknown

6. Disconnect the power to the slot.

   # cfgadm -c disconnect pci1:hpc0_slot3

7. Confirm the device has been disconnected.

   # cfgadm Ap_Id Receptacle Occupant Condition pci1:hpc0_slot0 empty unconfigured unknown pci1:hpc0_slot1 empty unconfigured unknown pci1:hpc0_slot2 empty unconfigured unknown pci1:hpc0_slot3 disconnected unconfigured unknown pci1:hpc0_slot4 empty unconfigured unknown

8. Open the slot latches and remove the board.

x86: How to Add a PCI Adapter Card

1. Become superuser.

2. Identify the hot-pluggable slot and open latches.

3. Insert the adapter card into a hot-pluggable slot.

4. Determine which slot the adapter card is in once it is inserted and the latches are closed.

   # cfgadm Ap_Id Receptacle Occupant Condition pci1:hpc0_slot0 empty unconfigured unknown pci1:hpc0_slot1 empty unconfigured unknown pci1:hpc0_slot2 empty unconfigured unknown pci1:hpc0_slot3 disconnected unconfigured unknown pci1:hpc0_slot4 empty unconfigured unknown

5. Connect the power to the slot.

   # cfgadm -c connect pci1:hpc0_slot3

6. Confirm the slot is connected.

   # cfgadm Ap_Id Receptacle Occupant Condition pci1:hpc0_slot0 empty unconfigured unknown pci1:hpc0_slot1 empty unconfigured unknown pci1:hpc0_slot2 empty unconfigured unknown pci1:hpc0_slot3 connected unconfigured unknown pci1:hpc0_slot4 empty unconfigured unknown

7. Configure the PCI hot-pluggable adapter card.

   # cfgadm -c configure pci1:hpc0_slot3 Ap_Id Receptacle Occupant Condition pci1:hpc0_slot3 connected configured ok

8. Verify the configuration of the adapter card in the slot.

   # cfgadm pci1:hpc0_slot3

9. Configure any supporting software if this is a new device.

   For example, if this is an ethernet card, use the ifconfig(1M) command to set up the interface.

x86: Troubleshooting PCI Configuration Problems

Error Message

cfgadm: Configuration operation invalid: invalid transition

Cause

An invalid transition was attempted.

Solution

Check whether the cfgadm -c command was issued appropriately. Use cfgadm to check the current receptacle and occupant state and make sure the ap_id is correct.

Error Message

cfgadm: Attachment point not found

Cause

Specified attachment point was not found.

Solution

Check whether the attachment point is correct. Use the cfgadm to display a list of available attachment points. Also check the physical path to see if the attachment point is still there.

In addition to cfgadm command, several other commands are helpful during hot-plug operations. The prtconf(1M) command confirms whether or not Solaris recognizes the hardware. After a successful configuration of a hot-pluggable adapter card, use the prtconf command to verify that the specific PCI bus instance for the new hardware displays in the prtconf output. The prtconf output also indicates if the driver for the new hardware is attached.