Extend an existing configuration of Sun Cluster Support for Oracle RAC in any of the following situations:
You are adding nodes to a cluster and you require Sun Cluster Support for Oracle RAC to run on the nodes. See How to Add Sun Cluster Support for Oracle RAC to Selected Nodes .
You are adding a volume manager. See How to Add a Volume Manager Resource to the Framework Resource Group.
Perform this procedure if you are adding nodes to a cluster and you require Sun Cluster Support for Oracle RAC to run on the nodes. Perform this procedure from only one node.
This procedure provides the long forms of the Sun Cluster maintenance commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands, in Sun Cluster Data Services Planning and Administration Guide for Solaris OS.
This task involves adding the selected nodes from the following resource groups in the following order:
Resource groups for scalable file-system mount-point resources
The multiple-owner volume-manager framework resource group, if used.
The RAC framework resource group
Resource groups for scalable device group resources
Resource groups that contain resources for the Sun QFS metadata server
Resource groups for logical hostname resources
The resource group for the RAC database
Ensure that the required Sun Cluster Support for Oracle RAC software packages are installed on each node to which you are adding Sun Cluster Support for Oracle RAC. For more information, see Installing the Sun Cluster Support for Oracle RAC Packages.
Ensure that the node that you add is connected to the shared storage that is used by the Oracle RAC configuration.
Become superuser on any cluster node.
Add the nodes to any resource groups that contain scalable file-system mount-point resources.
If no resource groups that contain scalable file-system mount-point resources are configured, omit this step.
For each resource group to which you are adding nodes, run the following command:
# clresourcegroup add-node -S -n nodelist scal-mp-rg |
Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.
Specifies the name of the resource group to which you are adding nodes.
Add the nodes to the multiple-owner volume-manager framework resource group, if used.
# clresourcegroup add-node -S -n nodelist vucmm-fmwk-rg |
Specifies a comma-separated list of cluster nodes to which you are adding the resource group.
Specifies the name of the resource group to which you are adding nodes.
Add the nodes to the RAC framework resource group.
# clresourcegroup add-node -S -n nodelist rac-fmwk-rg |
Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.
Specifies the name of the resource group to which you are adding nodes.
Add the nodes to any scalable device groups that you are using for Oracle files.
If you are not using any scalable device groups for Oracle files, omit this step.
How to perform this step depends on the type of the scalable device group.
For each Solaris Volume Manager for Sun Cluster multi-owner disk set, type the following command:
# metaset -s set-name -M -a -h nodelist |
Specifies the Solaris Volume Manager for Sun Cluster multi-owner disk set to which you are adding nodes.
Specifies a space-separated list of cluster nodes that you are adding to the multi-owner disk set.
For each VxVM shared-disk group, use Veritas commands to add the nodes to the VxVM shared-disk group.
For more information, see your VxVM documentation.
Add the nodes to any resource groups that contain scalable device group resources.
If no resource groups that contain scalable device group resources are configured, omit this step.
For each resource group to which you are adding nodes, run the following command:
# clresourcegroup add-node -S -n nodelist scal-dg-rg |
Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.
Specifies the name of the resource group to which you are adding nodes.
Mount each shared file system that is to be accessed from the nodes that you are adding.
If no shared file systems are to be accessed from the nodes that you are adding, omit this step.
For each file system that you are mounting, type the following command:
# mount mount-point |
Specifies the mount point of the file system that you are mounting.
Add the nodes to any resource groups that contain resources for the Sun QFS metadata server.
If no resource groups that contain resources for the Sun QFS metadata server are configured, omit this step.
For each resource group to which you are adding nodes, run the following command:
# clresourcegroup add-node -n nodelist qfs-mds-rg |
Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.
Specifies the name of the resource group to which you are adding nodes.
Bring online all resource groups to which you added nodes in Step 6.
These resource groups contain scalable device group resources.
If no resource groups that contain scalable device group resources are configured, omit this step.
For each resource group that you are bringing online, type the following command:
# clresourcegroup online scal-dg-rg |
Specifies the name of the resource group that you are bringing online.
(Oracle 10g Release 2 or 11g only) Start the Oracle CRS.
If you are using Oracle 9i or Oracle 10g Release 1, omit this step.
# /etc/init.d/init.crs start Startup will be queued to init within 30 seconds. |
(Oracle 9i only) Add the nodes to all resource groups that contain logical hostname resources for each RAC database that is to run on the nodes.
If you are using Oracle 10g Release 1 or Oracle 10g Release 2 or 11g, omit this step. For Oracle 10g Release 1 and Oracle 10g Release 2 or 11g, no resource groups for logical hostname resources are configured.
For each resource group to which you are adding nodes, run the following command:
# clresourcegroup add-node -n nodelist lh-rg |
Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.
Specifies the name of the resource group to which you are adding nodes.
(Oracle 9i and Oracle 10g Release 2 or 11g only) Add the nodes to the resource group for each RAC database that is to run on the nodes.
If you are using Oracle 10g Release 1, omit this step. For Oracle 10g Release 1, no resource groups for RAC databases are configured.
For each resource group to which you are adding nodes, run the following command:
# clresourcegroup add-node -S -n nodelist rac-db-rg |
Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.
Specifies the name of the resource group to which you are adding nodes.
(Oracle 10g Release 2 or 11g only) For each node that you are adding, create the Oracle CRS resources that are required to represent Sun Cluster resources.
Create an Oracle CRS resource for each Sun Cluster resource for scalable device groups and scalable file-system mount points on which Oracle components depend. For more information, see How to Create an Oracle CRS Resource for Interoperation With Sun Cluster.
(Oracle 9i and Oracle 10g Release 2 or 11g only) Modify each resource for RAC databases to set a value of each per-node property for each node that you are adding.
If you are using Oracle 10g Release 1, omit this step. For Oracle 10g Release 1, no resource groups for RAC databases are configured.
For each resource that you are modifying, perform these steps:
Disable the resource.
# clresource disable rac-db-rs |
Specifies the name of the RAC database resource that you are disabling.
Set a value of each per-node property for each node that you are adding.
The per-node properties of each resource type for RAC databases are shown in the following table.
Resource Type |
Properties |
---|---|
SUNW.scalable_rac_server_proxy |
oracle_sid |
SUNW.scalable_rac_listener |
listener_name |
SUNW.scalable_rac_server |
alert_log_file oracle_sid |
For information about extension properties of resource types for RAC databases, see the following sections:
# clresource set \ -p property{node}=value[…] \ [-p property{node}=value[…]][…] \ rac-db-rs |
Specifies the name of a per-node property that you are setting.
Specifies the node for which you are setting a value for property.
Specifies the value to which you are setting property for node.
Specifies the name of the RAC database resource whose per-node properties you are setting.
Enable the resource.
# clresource enable rac-db-rs |
Specifies the name of the RAC database resource that you are enabling.
(Oracle 9i and Oracle 10g Release 2 or 11g only) Bring online each resource group for RAC databases.
If you are using Oracle 10g Release 1, omit this step. For Oracle 10g Release 1, no resource groups for RAC databases are configured.
For each resource group that you are bringing online, run the following command:
# clresourcegroup online rac-db-rg |
Specifies the name of the resource group that you are bringing online.
This example shows the sequence of operations that is required to add Sun Cluster Support for Oracle RAC to nodes pclus3 and pclus4 of a four-node cluster.
The configuration of Sun Cluster Support for Oracle RAC in this example is as follows:
The version of Oracle RAC is 10g Release 2.
Sun QFS shared file system on Solaris Volume Manager for Sun Cluster is used to store Oracle files.
The mount points of file systems that are used for Oracle files are as follows:
Oracle database files: /db_qfs/OraData
Oracle binary files and related files: /db_qfs/OraHome
The oradg disk set is used only by the RAC database.
The name of the Oracle RAC database is swb.
The Sun QFS shared file systems use a Solaris Volume Manager for Sun Cluster multi-owner disk set that is named oradg. The creation of this disk set is shown in Example 2–1.
The configuration uses a multiple-owner volume-manager framework resource group.
The configuration of resource groups in this example is shown in the following table.
Resource Group |
Purpose |
---|---|
vucmm-framework-rg |
Multiple-owner volume-manager resource group. |
rac-framework-rg |
RAC framework resource group. |
scaldg-rg |
Resource group for scalable device-group resources. |
qfsmds-rg |
Resource group for Sun QFS metadata server resources. |
scalmnt-rg |
Resource group for scalable file-system mount-point resources. |
rac_server_proxy-rg |
RAC database resource group. |
The resource groups that are required for this configuration are shown in Figure A–2.
To add the nodes to the resource group that contains scalable file-system mount-point resources, the following command is run:
# clresourcegroup add-node -S -n pclus3,pclus4 scalmnt-rg |
To add the nodes to the multiple-owner volume-manager framework resource group, the following command is run:
# clresourcegroup add-node -S -n pclus3,plcus4 vucmm-framework-rg |
To add the nodes to the RAC framework resource group, the following command is run:
# clresourcegroup add-node -S -n pclus3,plcus4 rac-framework-rg |
To add the nodes to the Solaris Volume Manager for Sun Cluster multi-owner disk set oradg, the following command is run:
# metaset -s oradg -M -a -h pclus3 pclus4 |
To add the nodes to the resource group that contains scalable device group resources, the following command is run:
# clresourcegroup add-node -S -n pclus3,pclus4 scaldg-rg |
To mount the shared file systems that are to be accessed from the nodes that are being added, the following commands are run:
# mount /db_qfs/OraData # mount /db_qfs/OraHome |
To add the nodes to the resource group that contains resources for the Sun QFS metadata server, the following command is run:
# clresourcegroup add-node -n pclus3,pclus4 qfsmds-rg |
To bring online the resource group that contains scalable device group resources, the following command is run:
# clresourcegroup online scaldg-rg |
To start the Oracle CRS and to verify the correct startup of the Oracle CRS, the following commands are run:
# /etc/init.d/init.crs start Startup will be queued to init within 30 seconds. # /db_qfs/OraHome/crs/bin/crsctl check crs CSS appears healthy CRS appears healthy EVM appears healthy |
To add the nodes to the resource group for the RAC database, the following command is run:
# clresourcegroup add-node -S -n pclus3,pclus4 rac_server_proxy-rg |
After the addition of the nodes to the resource group for the RAC database, the required Oracle CRS resources are created. The creation of these Oracle CRS resources is beyond the scope of this example.
To set required per-node properties for the RAC database resource, the following commands are run:
# clresource disable rac_server_proxy-rs # clresource set -p oracle_sid\{3\}=swb3 -p \ oracle_sid\{4\}=swb4 rac_server_proxy-rs # clresource enable rac_server_proxy-rs |
The per-node property oracle_sid is set to swb3 on node pclus3 and to swb4 on node pclus4.
To bring online the resource group for the RAC database, the following command is run:
# clresourcegroup online rac_server_proxy-rg |
Perform this task if you are adding a volume manager to an existing configuration of Sun Cluster Support for Oracle RAC. The framework resource group must contain a resource that represents the volume manager that you are adding. You can add a volume manager resource only if the framework resource is disabled and if the framework daemon is stopped on all cluster nodes.
If the cluster contains a SUNW.vucmm_framework based resource group, you add an instance of the SUNW.vucmm_svm or SUNW.vucmm_cvm resource type to that resource group.
Do not add an instance of the SUNW.rac_svm or SUNW.rac_cvm resource type to the SUNW.rac_framework based resource group when a SUNW.vucmm_framework based resource group exists in the cluster.
If the cluster does not contain a SUNW.vucmm_framework based resource group, you add an instance of the SUNW.rac_svm or SUNW.rac_cvm resource type to the SUNW.rac_framework based resource group.
This task requires downtime because you must disable the framework resource and reboot the nodes where Oracle RAC is running.
Ensure that the volume manager for which you are adding a resource is installed and configured on all nodes where Oracle RAC is to run.
Become superuser on any cluster node.
Disable the framework resource in the framework resource group and any other resources that depend on this resource.
# clresource disable -r fmwk-rs |
Specifies the name of the resource of type SUNW.vucmm_framework or SUNW.rac_framework that you are disabling.
Reboot all the nodes that are in the node list of the framework resource group.
Register and add an instance of the resource type that represents the volume manager that you are adding.
If you are adding Solaris Volume Manager for Sun Cluster, register and add the instance as follows:
Register the Solaris Volume Manager for Sun Cluster resource type.
Add an instance of the Solaris Volume Manager for Sun Cluster resource type to the framework resource group.
Ensure that this instance depends on the resource that you disabled in Step 2.
# clresource create -g fmwk-rg \ -t svm-rt \ -p resource_dependencies=fmwk-rs svm-rs |
Specifies the name of the framework resource group. This resource group contains the resource of type SUNW.vucmm_framework or SUNW.rac_framework that you disabled in Step 2.
Specifies the name of the Solaris Volume Manager for Sun Cluster resource type.
Specifies that this instance depends on the resource that you disabled in Step 2.
Specifies the name that you are assigning to the resource of type SUNW.vucmm_svm or SUNW.rac_svm.
SPARC: If you are adding VxVM with the cluster feature, register and add the instance as follows.
Register the VxVM volume manager resource type.
Add an instance of the VxVM volume manager resource type to the resource group that you disabled in Step 2.
Ensure that this instance depends on the resource that you disabled in Step 2.
# clresource create -g fmwk-rg \ -t cvm-rt \ -p resource_dependencies=fmwk-rs cvm-rs |
Specifies the name of the framework resource group. This resource group contains the resource that you disabled in Step 2.
Specifies the name of the Solaris Volume Manager for Sun Cluster resource type.
Specifies that this instance depends on the resource that you disabled in Step 2.
Specifies the name that you are assigning to the resource of type SUNW.vucmm_cvm or SUNW.rac_cvm.
Bring online and in a managed state the framework resource group and its resources.
# clresourcegroup online -emM fmwk-rg |
Specifies that the framework resource group is to be moved to the MANAGED state and brought online. This resource group contains the resource that you disabled in Step 2.
The next step depends on the volume manager that you are adding, as shown in the following table.
Volume Manager |
Next Step |
---|---|
Solaris Volume Manager for Sun Cluster | |
SPARC: VxVM with the cluster feature |
How to Create a VxVM Shared-Disk Group for the Oracle RAC Database |