Known DR Limitations

This section contains known DR software limitations of the Sun Fire 6800, 4810, 4800, and 3800 systems.

General DR Limitations

• If you add a system board to a domain without using DR procedures, such as by running the addboard command-line interface (CLI) command on the system controller (SC), you must run the setkeyswitch off command and then the setkeyswitch on command to bring the board into the system.

• This release of the DR software does not support Sun StorEdge Traffic Manager.

• Before performing any DR operation on an I/O (IBx) board, enter the following command to stop the vold daemon:

  # sh /etc/init.d/volmgt stop

  After the DR operation has successfully completed, enter the following command to re-start the vold daemon:

  # sh /etc/init.d/volmgt start

• On Sun Fire 6800, 4810, 4800, and 3800 systems, DR does not support HIPPI/P, nor SAI/P (bug 4466378), nor the SunHSI/P driver (bug 4496362).

• You must execute the devfsadm(1M) command to see any changes that have been made, especially in regard to changes from PCI to cPCI.

• Do not reboot nor reset the system controller (SC) during DR operations. In addition, do not perform a flashupdate, which requires a reboot upon completion.

Limitations Specific to CompactPCI

• You can unconfigure a CompactPCI (cPCI) I/O assembly only if all the cards in the board are in an unconfigured state. If any cPCI card is busy (such as with a plumbed/up interface or a mounted disk), the board unconfigure operation fails with the status “busy.” All cPCI cards should be unconfigured before attempting to unconfigure the cPCI I/O assembly.

• When a multipath disk is connected to two cPCI cards, it is possible to see disk activity across the cards when none is expected. For this reason, make sure that there is no activity on the local side of the resource. This is more likely to occur when attempting to perform DR operations on a cPCI card that shows a busy status, even when there is no activity on the local side of the resource. A subsequent DR attempt may be required.

• When a user lists attachment points using the cfgadm(1M) command with the -a option, cPCI slots and PCI buses are all listed as attachment points. The cfgadm -a command displays an attachment point for a PCI bus as N0.IB8::pci0. There are four such attachment points for each cPCI board. The user should not perform DR operations on these, nor on the sghsc attachment point (which the cfgadm -a command displays as N0.IB8::sghsc4), because DR is not actually performed, and some internal resources are removed. However, there is no harm in doing so.

• In order for DR to function properly with cPCI cards, the levers on all cPCI cards that are inserted at Solaris boot time must be fully engaged.

Procedures for Bringing a cPCI Network Interface (IPMP) Online or Offline

To Take a cPCI Network Interface (IPMP) Offline and Remove It

1. Retrieve the group name, test address, and interface index by typing the following command.

   # ifconfig interface

   For example, ifconfig hme0

2. Use the if_mpadm(1M) command as follows:

   # if_mpadm -d interface

   This takes the interface offline and causes the failover addresses to be failed over to another active interface in the group. If the interface is already in a failed state, then this step simply marks and ensures that the interface is offline.

3. (Optional) Unplumb the interface.

   This step is required only if you want to use DR to reconfigure the interface automatically at a later time.

4. Remove the physical interface.

   Refer to the cfgadm(1M) man page and the Sun Fire 6800, 4810, 4800 and 3800 Systems Dynamic Reconfiguration User Guide for more information.

To Attach and Bring Online a cPCI Network Interface (IPMP)

1. Attach the physical interface.

   Refer to the cfgadm(1M) man page and the Sun Fire 6800, 4810, 4800 and 3800 Systems Dynamic Reconfiguration User Guide for more information.

After you attach the physical interface, it is automatically configured using settings in the hostname configuration file (/etc/hostname.interface, where interface is a value such as hme1 or qfe2).

This triggers the in.mpathd daemon to resume probing and detect repairs. Consequently, in.mpathd causes original IP addresses to failback to this interface. The interface should now be online and ready for use under IPMP.

If the interface had not been unplumbed and set to the OFFLINE status prior to a previous detach, then the attach operation described here would not automatically configure it. To set the interface back to the ONLINE status and failback its IP address after the physical attach is complete, enter the following command: if_mpadm -r interface

Operating System Quiescence

This section discusses permanent memory, and the requirement to quiesce the operating system when unconfiguring a system board that has permanent memory.

A quick way to determine whether a board has permanent memory is to run the following command as superuser:

The system responds with output such as the following, which describes system board 0 (zero):

Permanent memory is where the Solaris kernel and its data reside. The kernel cannot be released from memory in the same way that user processes residing in other boards can release memory by paging out to the swap device. Instead, cfgadm uses the copy-rename technique to release the memory.

The first step in a copy-rename operation is to stop all memory activity on the system by pausing all I/O operations and thread activity; this is known as quiescence. During quiescence, the system is frozen and does not respond to external events such as network packets. The duration of the quiescence depends on two factors: how many I/O devices and threads need to be stopped; and how much memory needs to be copied. Typically the number of I/O devices determines the required quiescent time, because I/O devices must be paused and unpaused. Typically, a quiescent state lasts longer than two minutes.

Because quiescence has a noticeable impact, cfgadm requests confirmation before effecting quiescence. If you enter:

The system responds with a prompt for confirmation:

If you are using SunMC to perform the DR operation, a pop-up window displays this prompt.

Enter yes to confirm that the impact of the quiesce is acceptable, and to proceed.