| Skip Navigation Links | |
| Exit Print View | |
|
Oracle Solaris Cluster Software Installation Guide Oracle Solaris Cluster 3.3 3/13 |
| Skip Navigation Links | |
| Exit Print View | |
|
|
Oracle Solaris Cluster Software Installation Guide Oracle Solaris Cluster 3.3 3/13 |
1. Planning the Oracle Solaris Cluster Configuration
2. Installing Software on Global-Cluster Nodes
3. Establishing the Global Cluster
4. Configuring Solaris Volume Manager Software
Configuring Solaris Volume Manager Software
How to Create State Database Replicas
How to Mirror the Root (/) File System
How to Mirror the Global Devices Namespace on a Dedicated Partition
How to Mirror File Systems Other Than Root (/) That Cannot Be Unmounted
Creating Disk Sets in a Cluster
How to Add Drives to a Disk Set
How to Repartition Drives in a Disk Set
Configuring Dual-String Mediators
Requirements for Dual-String Mediators
How to Check the Status of Mediator Data
5. Creating a Cluster File System
6. Creating Non-Global Zones and Zone Clusters
The following table lists the tasks that you perform to configure Solaris Volume Manager software for Oracle Solaris Cluster configurations. Complete the procedures in the order that is indicated.
Table 4-1 Task Map: Configuring Solaris Volume Manager Software
|
Perform this procedure on each node in the global cluster.
Use the physical name (cNtXdY sZ), not the device-ID name (dN), to specify the slices to use.
phys-schost# metadb -af slice-1 slice-2 slice-3
Tip - To provide protection of state data, which is necessary to run Solaris Volume Manager software, create at least three replicas for each node. Also, you can place replicas on more than one device to provide protection if one of the devices fails.
See the metadb(1M) man page and Creating State Database Replicas in Solaris Volume Manager Administration Guide for details.
phys-schost# metadb
The metadb command displays the list of replicas.
Example 4-1 Creating State Database Replicas
The following example shows three state database replicas. Each replica is created on a different device.
phys-schost# metadb -af c0t0d0s7 c0t1d0s7 c1t0d0s7 phys-schost# metadb flags first blk block count a u 16 8192 /dev/dsk/c0t0d0s7 a u 16 8192 /dev/dsk/c0t1d0s7 a u 16 8192 /dev/dsk/c1t0d0s7
Next Steps
To mirror file systems on the root disk, go to Mirroring the Root Disk.
Otherwise, go to Creating Disk Sets in a Cluster to create Solaris Volume Manager disk sets.
Mirroring the root disk prevents the cluster node itself from shutting down because of a system disk failure. Four types of file systems can reside on the root disk. Each file-system type is mirrored by using a different method.
Use the following procedures to mirror each type of file system.
How to Mirror the Global Devices Namespace on a Dedicated Partition
How to Mirror File Systems Other Than Root (/) That Cannot Be Unmounted
 | Caution - For local disk mirroring, do not use /dev/global as the path when you specify the disk name. If you specify this path for anything other than cluster file systems, the system cannot boot. |
Use this procedure to mirror the root (/) file system.
Note - If the global-devices namespace is on a lofi-created file, this procedure includes the mirroring of the global-devices namespace.
Specify the physical disk name of the root-disk slice (cNtXdYsZ).
phys-schost# metainit -f submirror1 1 1 root-disk-slice
phys-schost# metainit submirror2 1 1 submirror-disk-slice
phys-schost# metainit mirror -m submirror1
Note - If the device is a local device to be used to mount a global-devices file system, /global/.devices/node@nodeid, the volume name for the mirror must be unique throughout the cluster.
phys-schost# metaroot mirror
This command edits the /etc/vfstab and /etc/system files so the system can be booted with the root (/) file system on a metadevice or volume. For more information, see the metaroot(1M) man page.
phys-schost# lockfs -fa
This command flushes all transactions out of the log and writes the transactions to the master file system on all mounted UFS file systems. For more information, see the lockfs(1M) man page.
phys-schost# clnode evacuate from-node
Specifies the name of the node from which to evacuate resource or device groups.
This command remounts the newly mirrored root (/) file system.
phys-schost# shutdown -g0 -y -i6
phys-schost# metattach mirror submirror2
See the metattach(1M) man page for more information.
Disabling fencing for the device prevents unintentional fencing of a node from its boot device if the boot device is connected to multiple nodes.
phys-schost# cldevice set -p default_fencing=nofencing submirror-disk
Specifies a device property.
Disables fencing for the specified device.
For more information about the default_fencing property, see the cldevice(1CL) man page.
If the primary boot device fails, you can then boot from this alternate boot device. See Creating a RAID-1 Volume in Solaris Volume Manager Administration Guide for more information about alternate boot devices.
phys-schost# ls -l /dev/rdsk/root-disk-slice
Ensure that each volume name for a mirror on which a global-devices file system, /global/.devices/node@nodeid, is to be mounted is unique throughout the cluster.
Example 4-2 Mirroring the Root (/) File System
The following example shows the creation of mirror d0 on the node phys-schost-1, which consists of submirror d10 on partition c0t0d0s0 and submirror d20 on partition c2t2d0s0. Device c2t2d0 is a multihost disk, so fencing is disabled. The example also displays the alternate boot path for recording.
phys-schost# metainit -f d10 1 1 c0t0d0s0 d11: Concat/Stripe is setup phys-schost# metainit d20 1 1 c2t2d0s0 d12: Concat/Stripe is setup phys-schost# metainit d0 -m d10 d10: Mirror is setup phys-schost# metaroot d0 phys-schost# lockfs -fa phys-schost# clnode evacuate phys-schost-1 phys-schost# shutdown -g0 -y -i6 phys-schost# metattach d0 d20 d0: Submirror d20 is attachedphys-schost# cldevice set -p default_fencing=nofencing c2t2d0 phys-schost# ls -l /dev/rdsk/c2t2d0s0 lrwxrwxrwx 1 root root 57 Apr 25 20:11 /dev/rdsk/c2t2d0s0 –> ../../devices/node@1/pci@1f,0/pci@1/scsi@3,1/disk@2,0:a,raw
Next Steps
To mirror a global devices namespace that is configured on a dedicated partition, /global/.devices/node@nodeid, go to How to Mirror the Global Devices Namespace on a Dedicated Partition.
To mirror file systems than cannot be unmounted, go to How to Mirror File Systems Other Than Root (/) That Cannot Be Unmounted.
To mirror user-defined file systems, go to How to Mirror File Systems That Can Be Unmounted.
Otherwise, go to Creating Disk Sets in a Cluster to create a disk set.
Troubleshooting
Some of the steps in this mirroring procedure might cause an error message similar to metainit: dg-schost-1: d1s0: not a metadevice. Such an error message is harmless and can be ignored.
If you configured a dedicated partition for the global devices namespace, use this procedure to mirror the namespace, /global/.devices/node@nodeid/.
Note - Do not use this procedure if the global-devices namespace is on a lofi-based file. Instead, go to How to Mirror the Root (/) File System.
Use the physical disk name of the disk slice (cNtXdY sZ).
phys-schost# metainit -f submirror1 1 1 diskslice
phys-schost# metainit submirror2 1 1 submirror-diskslice
phys-schost# metainit mirror -m submirror1
Note - The volume name for a mirror on which a global-devices file system, /global/.devices/node@nodeid, is to be mounted must be unique throughout the cluster.
This attachment starts a synchronization of the submirrors.
phys-schost# metattach mirror submirror2
Replace the names in the device to mount and device to fsck columns with the mirror name.
phys-schost# vi /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # /dev/md/dsk/mirror /dev/md/rdsk/mirror /global/.devices/node@nodeid ufs 2 no global
Use the metastat(1M) command to view mirror status and to verify that mirror synchronization is complete.
phys-schost# metastat mirror
Disabling fencing for the device prevents unintentional fencing of a node from its boot device if the boot device is connected to multiple nodes.
phys-schost# cldevice set -p default_fencing=nofencing submirror-disk
Specifies a device property.
Disables fencing for the specified device.
For more information about the default_fencing property, see the cldevice(1CL) man page.
Example 4-3 Mirroring a Global Devices Namespace That Is Configured on a Dedicated Partition
The following example shows creation of mirror d101, which consists of submirror d111 on partition c0t0d0s3 and submirror d121 on partition c2t2d0s3. The /etc/vfstab file entry for /global/.devices/node@1 is updated to use the mirror name d101. Device c2t2d0 is a multihost disk, so fencing is disabled.
phys-schost# metainit -f d111 1 1 c0t0d0s3 d111: Concat/Stripe is setup phys-schost# metainit d121 1 1 c2t2d0s3 d121: Concat/Stripe is setup phys-schost# metainit d101 -m d111 d101: Mirror is setup phys-schost# metattach d101 d121 d101: Submirror d121 is attached phys-schost# vi /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # /dev/md/dsk/d101 /dev/md/rdsk/d101 /global/.devices/node@1 ufs 2 no global phys-schost# metastat d101 d101: Mirror Submirror 0: d111 State: Okay Submirror 1: d121 State: Resyncing Resync in progress: 15 % done … phys-schost# cldevice show phys-schost-3:/dev/rdsk/c2t2d0 === DID Device Instances === DID Device Name: /dev/did/rdsk/d2 Full Device Path: phys-schost-1:/dev/rdsk/c2t2d0 Full Device Path: phys-schost-3:/dev/rdsk/c2t2d0 … phys-schost# cldevicegroup show | grep dsk/d2 Device Group Name: dsk/d2 … Node List: phys-schost-1, phys-schost-3 … localonly: false phys-schost# cldevicegroup remove-node -n phys-schost-3 dsk/d2 phys-schost# cldevice set -p default_fencing=nofencing c2t2d0
Next Steps
To mirror file systems other than root (/) that cannot be unmounted, go to How to Mirror File Systems Other Than Root (/) That Cannot Be Unmounted.
To mirror user-defined file systems, go to How to Mirror File Systems That Can Be Unmounted
Otherwise, go to Creating Disk Sets in a Cluster to create a disk set.
Troubleshooting
Some of the steps in this mirroring procedure might cause an error message similar to metainit: dg-schost-1: d1s0: not a metadevice. Such an error message is harmless and can be ignored.
Use this procedure to mirror file systems other than root (/) that cannot be unmounted during normal system usage, such as /usr, /opt, or swap.
Specify the physical disk name of the disk slice (cNtX dYsZ).
phys-schost# metainit -f submirror1 1 1 diskslice
phys-schost# metainit submirror2 1 1 submirror-diskslice
phys-schost# metainit mirror -m submirror1
Note - The volume name for this mirror does not need to be unique throughout the cluster.
Replace the names in the device to mount and device to fsck columns with the mirror name.
phys-schost# vi /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # /dev/md/dsk/mirror /dev/md/rdsk/mirror /filesystem ufs 2 no global
phys-schost# clnode evacuate from-node
Specifies the name of the node from which to move resource or device groups.
phys-schost# shutdown -g0 -y -i6
This attachment starts a synchronization of the submirrors.
phys-schost# metattach mirror submirror2
Use the metastat(1M) command to view mirror status and to verify that mirror synchronization is complete.
phys-schost# metastat mirror
Disabling fencing for the device prevents unintentional fencing of a node from its boot device if the boot device is connected to multiple nodes.
phys-schost# cldevice set -p default_fencing=nofencing submirror-disk
Specifies a device property.
Disables fencing for the specified device.
For more information about the default_fencing property, see the cldevice(1CL) man page.
Example 4-4 Mirroring File Systems That Cannot Be Unmounted
The following example shows the creation of mirror d1 on the node phys-schost-1 to mirror /usr, which resides on c0t0d0s1. Mirror d1 consists of submirror d11 on partition c0t0d0s1 and submirror d21 on partition c2t2d0s1. The /etc/vfstab file entry for /usr is updated to use the mirror name d1. Device c2t2d0 is a multihost disk, so fencing is disabled.
phys-schost# metainit -f d11 1 1 c0t0d0s1 d11: Concat/Stripe is setup phys-schost# metainit d21 1 1 c2t2d0s1 d21: Concat/Stripe is setup phys-schost# metainit d1 -m d11 d1: Mirror is setup phys-schost# vi /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # /dev/md/dsk/d1 /dev/md/rdsk/d1 /usr ufs 2 no global … phys-schost# clnode evacuate phys-schost-1 phys-schost# shutdown -g0 -y -i6 phys-schost# metattach d1 d21 d1: Submirror d21 is attached phys-schost# metastat d1 d1: Mirror Submirror 0: d11 State: Okay Submirror 1: d21 State: Resyncing Resync in progress: 15 % done … phys-schost# cldevice show phys-schost-3:/dev/rdsk/c2t2d0 … DID Device Name: /dev/did/rdsk/d2 phys-schost# cldevicegroup show dsk/d2 Device Group Name: dsk/d2 … Node List: phys-schost-1, phys-schost-3 … localonly: false phys-schost# cldevicegroup remove-node -n phys-schost-3 dsk/d2 phys-schost# cldevice set -p default_fencing=nofencing c2t2d0
Next Steps
To mirror user-defined file systems, go to How to Mirror File Systems That Can Be Unmounted.
Otherwise, go to Creating Disk Sets in a Cluster to create a disk set.
Troubleshooting
Some of the steps in this mirroring procedure might cause an error message similar to metainit: dg-schost-1: d1s0: not a metadevice. Such an error message is harmless and can be ignored.
Use this procedure to mirror user-defined file systems that can be unmounted. In this procedure, the nodes do not need to be rebooted.
Ensure that no processes are running on the file system.
phys-schost# umount /mount-point
See the umount(1M) man page and Mounting and Unmounting Oracle Solaris File Systems in System Administration Guide: Devices and File Systems for more information.
Specify the physical disk name of the disk slice (cNtX dYsZ).
phys-schost# metainit -f submirror1 1 1 diskslice
phys-schost# metainit submirror2 1 1 submirror-diskslice
phys-schost# metainit mirror -m submirror1
Note - The volume name for this mirror does not need to be unique throughout the cluster.
Replace the names in the device to mount and device to fsck columns with the mirror name.
phys-schost# vi /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # /dev/md/dsk/mirror /dev/md/rdsk/mirror /filesystem ufs 2 no global
This attachment starts a synchronization of the submirrors.
phys-schost# metattach mirror submirror2
Use the metastat(1M) command to view mirror status.
phys-schost# metastat mirror
Disabling fencing for the device prevents unintentional fencing of a node from its boot device if the boot device is connected to multiple nodes.
phys-schost# cldevice set -p default_fencing=nofencing submirror-disk
Specifies a device property.
Disables fencing for the specified device.
For more information about the default_fencing property, see the cldevice(1CL) man page.
phys-schost# mount /mount-point
See the mount(1M) man page and Mounting and Unmounting Oracle Solaris File Systems in System Administration Guide: Devices and File Systems for more information.
Example 4-5 Mirroring File Systems That Can Be Unmounted
The following example shows creation of mirror d4 to mirror /export, which resides on c0t0d0s4. Mirror d4 consists of submirror d14 on partition c0t0d0s4 and submirror d24 on partition c2t2d0s4. The /etc/vfstab file entry for /export is updated to use the mirror name d4. Device c2t2d0 is a multihost disk, so fencing is disabled.
phys-schost# umount /export phys-schost# metainit -f d14 1 1 c0t0d0s4 d14: Concat/Stripe is setup phys-schost# metainit d24 1 1 c2t2d0s4 d24: Concat/Stripe is setup phys-schost# metainit d4 -m d14 d4: Mirror is setup phys-schost# vi /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # # /dev/md/dsk/d4 /dev/md/rdsk/d4 /export ufs 2 no global phys-schost# metattach d4 d24 d4: Submirror d24 is attached phys-schost# metastat d4 d4: Mirror Submirror 0: d14 State: Okay Submirror 1: d24 State: Resyncing Resync in progress: 15 % done … phys-schost# cldevice show phys-schost-3:/dev/rdsk/c2t2d0 … DID Device Name: /dev/did/rdsk/d2 phys-schost# cldevicegroup show dsk/d2 Device Group Name: dsk/d2 … Node List: phys-schost-1, phys-schost-2 … localonly: false phys-schost# cldevicegroup remove-node -n phys-schost-3 dsk/d2 phys-schost# cldevice set -p default_fencing=nofencing c2t2d0 phys-schost# mount /export
Next Steps
To create a disk set, go to Creating Disk Sets in a Cluster. Alternatively, if you will create a multi-owner disk set for use by Oracle Real Application Clusters, go to How to Create a Multi-Owner Disk Set in Solaris Volume Manager for Sun Cluster for the Oracle RAC Database in Oracle Solaris Cluster Data Service for Oracle Real Application Clusters Guide.
If you have sufficient disk sets for your needs, go to one of the following:
If your cluster contains disk sets that are configured with exactly two disk enclosures and two nodes, you must add dual-string mediators. Go to Configuring Dual-String Mediators.
If your cluster configuration does not require dual-string mediators, go to How to Create Cluster File Systems.
Troubleshooting
Some of the steps in this mirroring procedure might cause an error message that is similar to metainit: dg-schost-1: d1s0: not a metadevice. Such an error message is harmless and can be ignored.
|