This chapter explains how to configure storage for the Oracle Real Application Clusters database.
Overview of Storage Configuration Tasks for the Oracle Real Application Clusters Database
Registering and Configuring the RAC Framework Resource Group
SPARC: Creating a VxVM Shared-Disk Group for the Oracle Real Application Clusters Database
Table 2–1 summarizes the tasks for configuring storage for the Oracle Real Application Clusters database.
Perform these tasks in the order in which they are listed in the table.
Table 2–1 Tasks for Configuring Storage for the Oracle Real Application Clusters Database
Task |
Instructions |
---|---|
Register and configure the Real Application Clusters (RAC) framework resource group |
If you are using the scsetup utility for this task, seeRegistering and Configuring the RAC Framework Resource Group. If you are using the scrgadm utility for this task, see the following documentation: |
Create a multi-owner disk set in Solaris Volume Manager for Sun Cluster for the Oracle Real Application Clusters database | |
Create a VxVM shared-disk group for the Oracle Real Application Clusters database |
SPARC: Creating a VxVM Shared-Disk Group for the Oracle Real Application Clusters Database |
Registering and configuring the RAC framework resource group enables Sun Cluster Support for Oracle Real Application Clusters to be managed by using Sun Cluster commands.
You can register and configure the RAC framework resource group for the following entities:
A cluster. For more information, see How to Register and Configure the RAC Framework Resource Group for a Cluster.
Selected nodes in a cluster. For more information, see How to Register and Configure the RAC Framework Resource Group for Selected Nodes.
The RAC framework resource group enables Oracle Real Application Clusters to be managed by using Sun Cluster commands. This resource group contains an instance of the following single-instance resource types:
SUNW.rac_framework, which represents the framework that enables Oracle Real Application Clusters to be managed by using Sun Cluster commands
SUNW.rac_udlm, which represents the UNIX Distributed Lock Manager (Oracle UDLM) component of Sun Cluster Support for Oracle Real Application Clusters
In addition, the RAC framework resource group contains an instance of a single-instance resource type that represents the storage management scheme that you are using for the Oracle Real Application Clusters database.
Solaris Volume Manager for Sun Cluster is represented by the SUNW.rac_svm resource type.
VxVM with the cluster feature is represented by the SUNW.rac_cvm resource type.
Any combination of storage management schemes without a volume manager is represented by the SUNW.rac_hwraid resource type.
If you are using the Sun StorEdge QFS shared file system, an additional resource is required to represent the metadata server for the Sun StorEdge QFS shared file system. This resource is not contained in the RAC framework resource group. For information about this resource, see the following documentation:
SPARC: Resources for the Sun StorEdge QFS Shared File System
Sun StorEdge QFS and Sun StorEdge SAM-FS File System Administration Guide
The resource types that are defined for the RAC framework resource group do not enable the Resource Group Manager (RGM) to manage instances of Oracle Real Application Clusters.
The procedures that follow explain how to configure resources for the RAC framework resource group by using the scsetup(1M) utility. The scsetup utility enables you to configure resources for the RAC framework resource group interactively. Configuring these resources interactively reduces the possibility for configuration errors that might result from command syntax errors or omissions. The scsetup utility ensures that all required resources are created on the basis of the software that is installed on your cluster nodes.
For instructions about how to configure resources for the RAC framework resource group by using the scrgadm(1M) utility, see Appendix B, Creating, Modifying, and Removing the RAC Framework Resource Group by Using the scrgadm Utility.
When you register and configure the RAC framework resource group for a cluster, the RAC framework resource group is created.
Perform this procedure if you require Sun Cluster Support for Oracle Real Application Clusters to run on all nodes in a cluster.
Before you perform this procedure, ensure that the required Sun Cluster Support for Oracle Real Application Clusters software packages are installed on each node. The packages that are required depend on the storage management scheme that you are using for the Oracle Real Application Clusters database. For more information, see Installing Sun Cluster Support for Oracle Real Application Clusters Packages.
Become superuser.
From one node only, create the RAC framework resource group.
The following instructions explain how to perform this operation by using the scsetup utility. For information about how to perform this operation by using the scrgadm utility, see How to Create the RAC Framework Resource Group by Using the scrgadm Utility.
Start the scsetup utility.
# scsetup |
The scsetup main menu is displayed.
Type the number that corresponds to the option for data services.
The Data Services menu is displayed.
Type the number that corresponds to the option for configuring Sun Cluster Support for Oracle Real Application Clusters.
The Sun Cluster Support for Oracle RAC menu is displayed.
Type the number that corresponds to the option for creating the RAC framework resource group.
The scsetup utility displays a list of the cluster nodes on which the Sun Cluster Support for Oracle Real Application Clusters packages are installed. The scsetup utility also asks you to confirm that the RAC framework resource group should be created on these nodes.
To confirm that the scsetup utility should create the RAC framework resource group on the listed nodes, type y.
The scsetup utility creates the RAC framework resource group and the resources that this resource group contains on the listed nodes.
If the value of a resource property is changed from its default value, the scsetup utility sets the property to the changed value for the resource that the scsetup utility creates.
Determine if the RAC framework resource group and its resources are online.
Use the scstat(1M) utility for this purpose. The resource group to verify is named rac-framework-rg. The scsetup utility automatically assigns this name when you create the resource group.
# scstat -g |
If the RAC framework resource group and its resources are not online, bring them online.
# scswitch -Z -g rac-framework-rg |
Enables the resource and monitor, moves the resource group to the MANAGED state, and brings the resource group online
Specifies that the resource group to be moved to the MANAGED state and brought online is rac-framework-rg
The next step depends on the volume manager that you are using, as shown in the following table.
Volume Manager |
Next Step |
---|---|
Solaris Volume Manager for Sun Cluster | |
VxVM with the cluster feature |
SPARC: Creating a VxVM Shared-Disk Group for the Oracle Real Application Clusters Database |
None |
When you register and configure the RAC framework resource group for selected nodes, the nodes are added to the RAC framework resource group.
Perform this procedure in the following situations:
You are adding nodes to a cluster and you require Sun Cluster Support for Oracle Real Application Clusters to run on the nodes.
You require Sun Cluster Support for Oracle Real Application Clusters to run on only a subset of nodes in a cluster.
Ensure that the required Sun Cluster Support for Oracle Real Application Clusters software packages are installed on each node for which you are registering and configuring the RAC framework resource group. The packages that are required depend on the storage management scheme that you are using for the Oracle Real Application Clusters database. For more information, see Installing Sun Cluster Support for Oracle Real Application Clusters Packages.
Become superuser.
Add the nodes to the RAC framework resource group.
The following instructions explain how to perform this operation by using the scsetup utility. For information about how to perform this operation by using the scrgadm utility, see How to Add Nodes to the RAC Framework Resource Group by Using the scrgadm Utility.
Start the scsetup utility.
# scsetup |
Type the number that corresponds to the option for data services.
The Data Services menu is displayed.
Type the number that corresponds to the option for configuring Sun Cluster Support for Oracle Real Application Clusters.
The Sun Cluster Support for Oracle RAC menu is displayed.
Type the number that corresponds to the option for adding a node to the RAC framework resource group.
The scsetup utility responds as follows:
The list of nodes that you can add to the RAC framework resource group is displayed. This list contains the cluster nodes on which the Sun Cluster Support for Oracle Real Application Clusters packages are installed, but on which the RAC framework resource group does not yet exist.
The scsetup utility prompts you for the list of nodes that you are adding.
Type a comma-separated list of the nodes that you are adding to the RAC framework resource group.
Bring the RAC framework resource group and its resources online.
# scswitch -Z -g rac-framework-rg |
Enables the resources, moves the resource group to the MANAGED state, and brings the resource group online
Specifies that the resource group to be moved to the MANAGED state and brought online is rac-framework-rg
The next step depends on the volume manager that you are using, as shown in the following table.
Volume Manager |
Next Step |
---|---|
Solaris Volume Manager for Sun Cluster | |
VxVM with the cluster feature |
SPARC: Creating a VxVM Shared-Disk Group for the Oracle Real Application Clusters Database |
None |
Perform this task only if you are using Solaris Volume Manager for Sun Cluster.
If you are using Solaris Volume Manager for Sun Cluster, Solaris Volume Manager requires a multi-owner disk set for the Oracle Real Application Clusters database to use. For information about Solaris Volume Manager for Sun Cluster multi–owner disk sets, see Multi-Owner Disk Set Concepts in Solaris Volume Manager Administration Guide.
Ensure that the required Sun Cluster Support for Oracle Real Application Clusters software packages are installed on each node. For more information, see Installing Sun Cluster Support for Oracle Real Application Clusters Packages.
Do not create any file systems in the multi-owner disk set because only the raw data file uses this disk set.
Disk devices that you add to the multi-owner disk set must be directly attached to all of the cluster nodes.
Create a multi-owner disk set.
Use the metaset(1M) command for this purpose.
# metaset -s setname -M -a -h nodelist |
Specifies the name of the disk set that you are creating.
Specifies that the disk set that you are creating is a multi-owner disk set.
Specifies that the nodes that the -h option specifies are to be added to the disk set.
Specifies a space-separated list of nodes that are to be added to the disk set. The Sun Cluster Support for Oracle Real Application Clusters software packages must be installed on each node in the list.
Add global devices to the disk set that you created in Step 1.
# metaset -s setname -a devicelist |
Specifies that you are modifying the disk set that you created in Step 1.
Specifies that the devices that devicelist specifies are to be added to the disk set.
Specifies a space-separated list of full device ID path names for the global devices that are to be added to the disk set. To enable consistent access to each device from any node in the cluster, ensure that each device ID path name is of the form /dev/did/dsk/dN, where N is the device number.
For the disk set that you created in Step 1, create the volumes that the Oracle Real Application Clusters database will use.
If you are creating many volumes for Oracle data files, you can simplify this step by using soft partitions. For more information, see Chapter 12, Soft Partitions (Overview), in Solaris Volume Manager Administration Guide and Chapter 13, Soft Partitions (Tasks), in Solaris Volume Manager Administration Guide.
Create each volume by concatenating slices on global devices that you added in Step 2. Use the metainit(1M) command for this purpose.
# metainit -s setname volume-abbrev numstripes width slicelist |
Specifies that you are creating a volume for the disk set that you created in Step 1.
Specifies the abbreviated name of the volume that you are creating. An abbreviated volume name has the format dV, where V is the volume number.
Specifies the number of slices in each stripe. If you set width to greater than 1, the slices are striped.
Specifies a space-separated list of slices that the volume contains. Each slice must reside on a global device that you added in Step 2.
Verify that each node is correctly added to the multi-owner disk set.
Use the metastat(1M) command for this purpose.
# metastat -s setname |
Specifies that you are verifying the disk set that you created in Step 1
This command displays a table that contains the following information for each node that is correctly added to the disk set:
The Host column contains the node name.
The Owner column contains the text multi-owner.
The Member column contains the text Yes.
Verify that the multi-owner disk set is correctly configured.
# scconf -pvv | grep setname |
Specifies that configuration information only for the disk set that you created in Step 1 is displayed
This command displays the device group information for the disk set. For a multi-owner disk set, the device group type is Multi-owner_SVM.
Verify the online status of the multi-owner disk set.
# scstat -D |
This command displays the node names of nodes in the multi-owner disk set that are online.
On each node that can own the disk set, change the ownership of each volume that you created in Step 3 as follows:
Owner: oracle
Group: dba
Ensure that you change ownership only of volumes that the Oracle Real Application Clusters database will use.
# chown oracle:dba volume-list |
Specifies a space-separated list of the logical names of the volumes that you created for the disk set. The format of these names depends on the type of device where the volume resides, as follows:
For block devices: /dev/md/setname/dsk/dV
For raw devices: /dev/md/setname/rdsk/dV
The replaceable items in these names are as follows:
Specifies the name of the multi-owner disk set that you created in Step 1
Specifies the volume number of a volume that you created in Step 3
Ensure that this list specifies each volume that you created in Step 3.
On each node that can own the disk set, grant the oracle user read access and write access to each volume for which you changed the ownership in Step 7.
Ensure that you change access permissions only of volumes that the Oracle Real Application Clusters database will use.
# chmod u+rw volume-list |
Specifies a space-separated list of the logical names of the volumes to which you are granting the oracle user read access and write access. Ensure that this list contains the volumes that you specified in Step 7.
This example shows the sequence of operations that is required to create a multi-owner disk set in Solaris Volume Manager for Sun Cluster. This example assumes that the volumes reside on raw devices.
To create the multi-owner disk set, the following command is run:
# metaset -s racdbset -M -a -h rachost1 rachost2 rachost3 rachost4 |
The multi-owner disk set is named racdbset. The nodes rachost1, rachost2, rachost3, and rachost4 are added to this disk set.
To add the global device /dev/did/dsk/d0 to the disk set, the following command is run:
# metaset -s racdbset -a /dev/did/dsk/d0 |
To create a volume for the disk set, the following command is run:
# metainit -s racdbset d0 1 1 /dev/did/dsk/d0s0 |
The volume is named d0. This volume is created by a one-on-one concatenation of the slice /dev/did/dsk/d0s0. The slice is not striped.
To verify that each node is correctly added to the multi-owner disk set, the following command is run:
# metastat -s racdbset Multi-owner Set name = racdbset, Set number = 1, Master = rachost2 Host Owner Member rachost1 multi-owner Yes rachost2 multi-owner Yes rachost3 multi-owner Yes rachost4 multi-owner Yes Drive Dbase d6 Yes d10 Yes |
To verify that the multi-owner disk set is correctly configured, the following command is run:
# scconf -pvv | grep racdbset Device group name: racdbset (racdbset) Device group type: Multi-owner_SVM (racdbset) Device group failback enabled: no (racdbset) Device group node list: rachost1, rachost2, rachost3, rachost4 (racdbset) Device group ordered node list: no (racdbset) Device group desired number of secondaries: 0 (racdbset) Device group diskset name: racdbset |
To verify the online status of the multi-owner disk set, the following command is run:
# scstat -D -- Device Group Servers -- Device Group Primary Secondary ------------ ------- --------- -- Device Group Status -- Device Group Status ------------ ------ -- Multi-owner Device Groups -- Device Group Online Status ------------ ------------- Multi-owner device group: racdbset rachost1,rachost2,rachost3,rachost4 |
To change the ownership of the volume in the disk set to owner oracle in group dba, the following command is run:
# chown oracle:dba /dev/md/racdbset/rdsk/d0 |
This command is run on each node that can own the disk set.
To grant the oracle user read access to the volume in the disk set, the following command is run:
# chmod u+rw /dev/md/racdbset/rdsk/d0 |
This command is run on each node that can own the disk set.
Go to Installing Oracle Real Application Clusters Software.
Perform this task only if you are using VxVM with the cluster feature.
If you are using VxVM with the cluster feature, VxVM requires a shared-disk group for the Oracle Real Application Clusters database to use.
Ensure that the required Sun Cluster Support for Oracle Real Application Clusters software packages are installed on each node. For more information, see Installing Sun Cluster Support for Oracle Real Application Clusters Packages.
Do not register the shared-disk group as a cluster device group with the cluster.
Do not create any file systems in the shared-disk group because only the raw data file uses this disk group.
Disks that you add to the shared-disk group must be directly attached to all of the cluster nodes.
Ensure that your VxVM license is current. If your license expires, the node panics.
Use VERITAS commands that are provided for creating a VxVM shared-disk group.
For information about VxVM shared-disk groups, see your VxVM documentation.