Chapter 3 Using Metadisks and Disk Pathgroups

You can create metadisks and disk pathgroups only for disks that are accessible via two paths. You should generally use two separate controllers on different system boards.

AP does not modify a disk's data when that disk is placed under AP control or when a pathgroup is deleted (except for the data on the slices that contain AP database copies). AP does not repartition a disk. If a pathgroup is deleted, you can continue to access the data by using its physical device name.

Device Nodes for Metadisks

Here are two examples of physical device nodes for disk devices under Solaris 2.x:

• /dev/dsk/c0t0d0s0

• /dev/rdsk/c0t0d0s0

where:

c references the I/O port on the host (not the disk array) t is the bus within the disk array d is the target ID of the disk on that bus s is the slice number on the disk

These physical device nodes represent a particular physical path to a partition on a disk.

Each controller port has both a port number (such as c0) and a port name (such as pln2 or sf3). The port name consists of the port type and instance number. See /etc/path_to_inst for more information.

When a disk array is connected to two ports, it can potentially be accessed from either path via the physical device node, for example, /dev/dsk/c0t0d0s0 or /dev/dsk/c1t0d0s0.

The device node for a metadisk is derived from the physical device node of the primary path for a pathgroup. Here are two examples of metadisk device nodes:

• /dev/ap/dsk/mc0t0d0s0

• /dev/ap/rdsk/mc0t0d0s0

As you can see, an ap directory has been added, and an m (for "meta") is prepended to the device specification. The device node for a metadisk has the ability to access the underlying physical disk drive from multiple paths.

Automatic Switching of Metadisks

Metadisks can be automatically switched from the active path to the alternate path in two situations:

• The active path fails.

• The board containing the controller for the active path is detached by using a DR Detach operation. (Automatic switching during a DR detach is available only on the Sun Enterprise 10000 server.)

When the active path fails, an automatic switch is attempted only if an alternate path is available. The failed path is then marked unavailable, or tried. You can identify the tried paths with apconfig -S:

# apconfig -S

c1      pln0  P A
c2      pln1  T
        metadiskname(s):
                mc1t5d0  
                mc1t4d0   
                mc1t3d0   
                mc1t2d0   
                mc1t1d0   
                mc1t0d0   

In this example, the currently inactive path, pln1, is marked with a "T", which indicates that path was tried but failed.

The tried flag is only significant for automatic switch operations (not for manual switch operations). AP never attempts to automatically switch to a tried path. (This prevents thrashing in the case that both paths may have failed.)

You can reset the tried flag with any of the following actions:

• Reboot the corresponding domain.

• Perform a DR detach followed by a DR attach of a board that contains the controller marked as tried.

• Manual reset the tried flag for a particular controller.

You can manually reset the tried flag as shown in this example:

# apdisk -w pln1

In this example, pln1 is a controller with the tried flag set to true. The apdisk -w feature should be used judiciously. This command merely clears the tried flag; it does not address any potential problems with the controller or device. This command should only be used in situations where the failed path has been restored without an intervening DR operation or reboot.

Note that you can attempt to perform a manual switch to a path that is marked as tried.

Working With Disk Pathgroups and Metadisks

The example commands in this section use pln ports (for SSA disk arrays). If you have Sun^TM StorEdge^TM A5000 disk arrays, you would simply specify sf ports wherever pln ports are shown.

To Create a Disk Pathgroup and Metadisk

1. Decide which two ports will make up the alternate paths for the pathgroup.

   1. You can use the apinst(1M) command to display all ports (for example, pln0 and pln1) and their disk device nodes (for example, /dev/dsk/c1t0d0):

      # apinst pln0 /dev/dsk/c1t0d0 /dev/dsk/c1t1d0 /dev/dsk/c1t2d0 /dev/dsk/c1t3d0 /dev/dsk/c1t4d0 /dev/dsk/c1t5d0 pln1 /dev/dsk/c2t0d0 /dev/dsk/c2t1d0 /dev/dsk/c2t2d0 /dev/dsk/c2t3d0 /dev/dsk/c2t4d0 /dev/dsk/c2t5d0

   2. You must know your system hardware configuration to recognize when two ports are connected to the same disk array.

      In this example, it is assumed that the SSA contains six disks and two SSA ports. One SSA port is connected to pln port c1, and the other SSA port is connected to pln port c2.

2. Use apdisk(1M) with the -c , -p, and -a options to create an uncommitted disk pathgroup:

   # apdisk -c -p pln0 -a pln1

   where:

   -p specifies the primary path -a specifies the alternate path -c specifies that this information is to be created.

   This apdisk(1M) command creates a metadisk name, as well as all of the necessary information in the AP database for maintaining the two alternate paths for all six disks.

3. Verify the results:

   # apconfig -S -u c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 U mc1t4d0 U mc1t3d0 U mc1t2d0 U mc1t1d0 U mc1t0d0 U

   The apconfig -S -u command lists the uncommitted metadisks. -S lists SCSI devices only (that is, disks rather than networks). -u lists uncommitted devices only. The U next to each metadisk name indicates that the metadisk entry is uncommitted.

   The P next to pln0 indicates that pln0 is the primary path, and the A indicates that pln0 is the active alternate. Although you can change the active alternate, the primary path always remains constant. The significance of the primary path is that it is initially the active alternate, it is used when the metadisk is named, and it is used to identify the metadisk. In this case, c1t0d0 from the primary path name becomes part of mc1t0d0 in the metadisk name.

4. If you are satisfied with the results shown in the previous step, use apdb(1M) with the -C option to commit the uncommitted database entries:

   # apdb -C

   apconfig -S lists the committed SCSI entries in the database. As shown in the example, this listing is exactly the same as the previous listing, except the U no longer appears after the metadisk names, indicating that the metadisks are no longer uncommitted.

5. Before continuing, you may want to verify the results by using apconfig -S to view the committed disk entries in the database:

   # apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0 mc1t0d0

6. Use drvconfig(1M):

   # drvconfig -i ap_dmd

   drvconfig rebuilds the devices directory, which represents the device tree in the kernel. The AP disk metadriver is a pseudo-device.

7. Use the following command to verify the results:

   # ls /devices/pseudo/ap_dmd* /devices/pseudo/ap_dmd@0:128,blk /devices/pseudo/ap_dmd@0:128,raw /devices/pseudo/ap_dmd@0:129,blk /devices/pseudo/ap_dmd@0:129,raw /devices/pseudo/ap_dmd@0:130,blk /devices/pseudo/ap_dmd@0:130,raw ...

   As you can see from the listing, drvconfig created minor nodes for the alternately pathed device.

8. Use apconfig(1M) with the -R option to create symbolic links from the devices directories, /dev/ap/dsk and /dev/ap/rdsk, to the metadisk special files under /devices/pseudo:

   # apconfig -R

9. Use the following command to view the symbolic links and verify the results:

   # ls -l /dev/ap/dsk total 8 lrwxrwxrwx 1 root 40 Jul 27 16:47 mc1t0d0s0 -> ../../../devices/pseudo/ap_dmd@0:128,blk lrwxrwxrwx 1 root 40 Jul 27 16:47 mc1t0d0s1 -> ../../../devices/pseudo/ap_dmd@0:129,blk lrwxrwxrwx 1 root 40 Jul 27 16:47 mc1t0d0s2 -> ../../../devices/pseudo/ap_dmd@0:130,blk

   The device nodes that you need--under /dev/ap/dsk as well as /dev/ap/rdsk--are now ready to be used.

10. Modify every reference that uses a physical device node (that is, a path that begins with /dev/dsk or /dev/rdsk) to use the corresponding metadisk device node (that is, a path that begins with /dev/ap/dsk or /dev/ap/rdsk).

    If a partition is currently mounted under a physical path name, it should be unmounted and remounted under the metadisk path name.

    If you do not want to unmount a partition, perhaps because it is heavily used, you could delay placing the partition under AP control until you are ready to bring the system down for maintenance and then reboot. In this scenario, you would modify the /etc/vfstab file so that when the system is rebooted, the partition comes up under an AP device. (If you are placing the boot disk under AP control, you would also need to modify /etc/vfstab using apboot(1M) as described in Chapter 4, Using AP Boot Devices.)

To Switch From the Primary Path to the Alternate Path

You can perform a switch at any time, even while I/O is occurring on the device. You might want to experiment with the switching process to verify that you understand it and that your system is setup properly, rather than wait until a critical situation occurs.

When you switch paths, AP does not check to verify that data can be transferred over the path to which you are switching (although it does determine whether or not that path is detached or offline). You may want to verify the status of the path before switching to it by performing an I/O operation such as prtvtoc(1M). AP does not produce any error or warning messages if you switch to a path that is not functioning properly. If you switch to a non-functioning path for your boot disk, your system may crash if the path is not switched back immediately.

1. Use the apconfig -S command to view the current configuration:

   # apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0

   In this example, pln0 is the active alternate since it is followed by an A. (It is also the primary path, since it is followed by a P.)

2. To perform the switch, use apconfig(1M) with the -P and -a options:

   # apconfig -P pln0 -a pln1

   -P specifies the primary path and thereby identifies the pathgroup for which you want to change the active alternate. Thus, -P pln0 in the example above identifies the pathgroup for which pln0 is the primary path. -a specifies the alternate that you want to make active.

3. You can verify the results using apconfig(1M) with the -S option to view the committed metadisks in the database:

   # apconfig -S c1 pln0 P c2 pln1 A metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0

   The active alternate has been switched to pln1.

   Note that you do not have to commit a switch operation.

To Switch Back to the Primary Path

1. You can switch back to the primary path using the following commands:

   # apconfig -P pln0 -a pln0 # apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0

The first apconfig command, above, switches the active alternate for the pathgroup that has the primary controller pln0. The active alternate becomes pln0.

To Delete a Disk Pathgroup

All usage of the metadisks in a pathgroup must be discontinued (for example, the file systems must be unmounted) before you delete the pathgroup.

1. Use apdisk(1M) with the -d option to specify the primary path:

   # apdisk -d pln0

2. To verify the results, use apconfig(1M) with the -S option to view the committed SCSI disk entries in the database:

   # apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 D mc1t4d0 D mc1t3d0 D mc1t2d0 D mc1t1d0 D mc1t0d0 D

   If the pathgroup was not previously committed, the apdisk -d command deletes it from the database. However, if the pathgroup was previously committed, the apdisk -d command simply marks it as deleted, but the deletion is not completed until the next time you commit the entries in the database. In the example above, the pln0 pathgroup was previously committed, so the letter D indicates that it is marked for deletion.

3. Use apdb(1M) as follows to commit the database entries, thereby completing the deletion:

   # apdb -C

4. You can verify the deletion with apconfig -S:

   # apconfig -S

   ---

   Note -

   You can undo a deletion if the deletion is uncommitted. To undo a deletion, use apdisk -z, specifying the same port that you previously specified.

   ---