9
Adding Nodes and Instances and Deleting Instances in Real Application Clusters

This chapter provides detailed procedures on how to add nodes and instances and delete instances in Real Application Clusters. The topics in this chapter are:

• Adding Nodes to a Cluster

• Overview of Procedures for Adding Nodes

• Adding a Node at the Clusterware Layer

• Adding a Node at the Oracle Layer

• Deleting Instances

  See Also: Oracle9i Real Application Clusters Installation and Configuration for procedures on using the Oracle Database Configuration Assistant (DBCA) to create and delete databases

Adding Nodes to a Cluster

This section explains how to dynamically add nodes to Real Application Clusters environments. The topics in this section are:

• Overview of Procedures for Adding Nodes

• Adding a Node at the Clusterware Layer

• Adding a Node at the Oracle Layer

  See Also: "Deleting Instances" for procedures on deleting an instance

Overview of Procedures for Adding Nodes

The procedure for adding nodes to Real Application Clusters comprises two main steps:

• Adding a Node at the Clusterware Layer

• Adding a Node at the Oracle Layer

The method for adding nodes at the clusterware layer is operating-system dependent as described in this chapter. For the second step, adding a node at the Oracle layer, the procedures in this chapter require using the Oracle Universal Installer (OUI) and the Oracle Database Configuration Assistant (DBCA). The procedures for the second step are generic and therefore apply to all platforms.

Adding a Node at the Clusterware Layer

On UNIX, unless you preconfigure your clusterware to accommodate additional nodes, you cannot dynamically add nodes. Therefore, you must stop the clusterware and reconfigure your cluster to accommodate additional nodes. Then follow the procedures under the heading "Adding a Node at the Clusterware Layer on UNIX".

On Windows NT and Windows 2000, Oracle supplies the cluster software or operating system-dependent (OSD) layer. This enables you to dynamically add nodes as described under the heading "Adding a Node at the Clusterware Layer on Windows NT and Windows 2000".

The topics in this section are:

• Adding a Node at the Clusterware Layer on UNIX

• Adding a Node at the Clusterware Layer on Windows NT and Windows 2000

Adding a Node at the Clusterware Layer on UNIX

You can take advantage of the scalability and high availability features of Real Application Clusters by adding new nodes to existing UNIX clusters that uses a Real Application Clusters database. As mentioned, you can add nodes without stopping your database if you have preconfigured your UNIX cluster to accommodate additional nodes.

Oracle Corporation recommends using the following procedures to add new nodes to existing Real Application Clusters in UNIX environments. These procedures assume that there is already a UNIX cluster running your Real Application Clusters database.

To prepare the new node, ensure that it is an exact clone of the existing nodes in terms of vendor operating system, clusterware software, and so on. Then perform the following tasks in the order shown to add a node at the clusterware layer on UNIX:

1. Connect Node to Cluster

2. Install Vendor Cluster Software

3. Create Raw Devices

Connect Node to Cluster

Connect the new node's hardware to the existing cluster. This includes electrical connections, network interconnects, shared disk subsystem connections, and so on. Refer to your hardware vendor documentation for details on this step.

Install Vendor Cluster Software

Install the cluster software by referring to your vendor's installation procedures. Also use your vendor documentation for procedures to make the new node an active member of your cluster.

Create Raw Devices

If your platform supports a cluster file system and you not are using raw devices for your cluster database, then you do not need to create raw devices as described in this section. Proceed to "Adding a Node at the Oracle Layer".

As part of the preinstallation steps before installing the Real Application Clusters software on the new node, create at least two new disk volumes to accommodate the redo logs for the new instance. Refer to Oracle9i Real Application Clusters Installation and Configuration for the requirements for these redo logs and the preinstallation steps.

Make the disk volumes for the redo logs the same size as the redo log volumes that you configured for your existing nodes' instances. Refer to your vendor documentation for procedures on creating disk volumes and raw devices.

If the cluster database to which you want to add a new node and instance uses automatic undo management, then create an additional raw volume for the new undo tablespace for the new instance. After completing this step, proceed to "Adding a Node at the Oracle Layer" to add the node at the Oracle layer.

See Also: Oracle9i Real Application Clusters Installation and Configuration for more information about creating raw devices

Adding a Node at the Clusterware Layer on Windows NT and Windows 2000

You can take advantage of the scalability and high availability features of Real Application Clusters by dynamically adding new nodes to an existing Windows cluster that uses a Real Application Clusters database.

Oracle Corporation recommends using the following procedures to add a node in Windows environments. These procedures assume that there is already a Windows cluster running your Real Application Clusters database. These procedures also assume that the new node uses the same operating system that is running on the existing nodes of your cluster. In other words, you have installed Microsoft Windows NT 4.0 or Windows 2000 on all the nodes in your cluster.

Perform the following tasks in the order shown to add a new node at the clusterware layer on Windows NT and Windows 2000:

1. Connect Node to Cluster

2. Install Cluster Software

3. Create Disk Partitions

4. Create Links to Disk Partitions

Connect Node to Cluster

Connect the new node's hardware to the existing cluster. This includes electrical connections, network interconnects, shared disk subsystem connections, and so on. Refer to your hardware vendor documentation for details on this step.

Install Cluster Software

If you are using your operating system vendor supplied clusterware on the existing nodes, also install this vendor software on the new node using your vendor's installation procedures.

However, if you are using Oracle Corporation's reference clusterware/operating system-dependent software on the existing nodes of your cluster, you must also install these Oracle OSD clusterware on the new node using the Cluster Setup Wizard. To do this, perform the following steps from one of the existing nodes before running the wizard:

1. If you use the Virtual Interface Architecture (VIA) interconnect on the existing cluster, make sure that VIA interconnect is also installed on the new node.

2. If you have a private interconnect network on the existing cluster, make sure that the private interconnect network is also installed on the new node.

3. Determine the disk drive on which the cluster software was installed on the existing nodes. Make sure that you have at least 2MB of free space on the same drive on the new node to install the OSD clusterware and the Object Link Manager.

4. Make sure that you can execute following from each of the existing nodes of your cluster:

   NET USE \\host_name\C$
   
   

   Where the host_name is the public network name of the new node. You have administrative privileges on each node if the operating system responds with:

   Command completed successfully. 
   
   

5. To install the Oracle OSD clusterware when the new node already has a version of your vendor OSD clusterware installed, make sure the vendor OSD clusterware is stopped. Otherwise, continue to the next step.

6. Insert CD number one from the Oracle9i release 1 (9.0.1) CD set into the new node's CD-ROM drive.

7. Navigate to the PREINSTALL\CLUSTERSETUP directory, for example:

   cd \PREINSTALL\CLUSTERSETUP
   
   

8. Run the Cluster Setup Wizard by entering the following from the \PREINSTALL\CLUSTERSETUP directory:

   clustersetup
   
   

9. The cluster setup wizard should display its Welcome Page. Click Next to proceed.

10. Check the Node Addition option and click Next.

11. Provide a public name for the new node you want to add. If the existing cluster also uses a high speed private network, provide a name for the private network interface for the new node and click Next.

12. Click Finish.

    See Also: Cluster Setup Wizard online help Oracle9i Database installation guide for Windows for more information

Create Disk Partitions

As part of the preinstall requirements before installing the Oracle software on a node for Oracle Real Application Clusters, create raw disk partitions. Create at least two new disk partitions to accommodate the redo logs for the new instance. Make these disk partitions the same size as the redo log partitions that you configured for the existing nodes' instances. If the database to which you want to add a node uses automatic undo management, then create an additional logical partition for the undo tablespace.

See Also: Oracle9i Real Application Clusters Installation and Configuration for more information on configuring raw partitions on Windows NT and Windows 2000 platforms

For Windows 2000 only, use a basic disk as an extended partition for creating raw partitions.

From one of the existing nodes of the cluster:

1. Start Disk Administrator using the path: Start >Program >Administrative Tools >Disk Administrator

2. Click inside an unallocated part of the extended partition.

3. Choose Create from the Partition menu. A dialog box appears in which you should enter the size of the partition. Ensure you use the same sizes as those you used on your existing nodes.

4. Click on the newly created partition and select Assign Drive Letter from the Tool menu.

5. Select Don't Assign Drive Letter, and click OK.

6. Repeat the last four steps, Steps 2 through 5, for the second and any additional partitions.

7. Select Commit Changes Now from the Partition menu to save the new partition information.

8. For Windows NT only, you may need to reboot all the nodes in your cluster if you are using a pre-4.0 Service Pack. This makes all of the nodes recognize the new partitions.

Create Links to Disk Partitions

Create the links to the partitions that you created in the "Create Disk Partitions" section so the existing nodes recognize these links. In addition, the new node should also recognize the pre-existing symbolic links to logical drives as well as the new links. To create these links, perform the following steps from one of the existing nodes:

1. Start the Object Link Manager by typing the following from the
   %ORACLE_HOME%\bin directory:

   GUIOracleOBJManager
   
   

The Object Link Manager automatically detects the symbolic links to the logical drives and displays them in OLM's graphical user interface (GUI).

2. Recall the disk and partition numbers for the partitions you created. Look for the disk and partition numbers in the OLM GUI and perform the following tasks:
   1. Right-click next to the box under the New Link column and enter the link name for the first partition.

   2. Repeat step a. for the second and any additional partitions.

   For example, if your Real Application Clusters database name is db and it consists of two instances running on two nodes and you are adding a third instance on the third node, your link names for your redo logs are db_redo3_1, db_redo3_2, and so on.

   If your existing database uses automatic undo management, then enter the link name the logical partition for the undo tablespace. For example, if your Real Application Clusters database name is db and it consists of two instances running on two nodes and you are adding a third instance on the third node, your link names for the undo tablespace will be db_undotbs3.

3. Select Commit from the Options menu. This creates the new links on the current node.

4. Select Sync Nodes from the Options menu. This makes the new links visible to the existing nodes of the cluster.

5. Select Exit from the Options menu to exit the Object Link Manager.

Adding a Node at the Oracle Layer

At this point, you have added the new node to the cluster at the clusterware layer. To add a node to an existing cluster at the Oracle layer, on all platforms perform the following steps:

• Install the Oracle Software on the New Node

• Post-Installation Steps

• Add a Database Instance on the New Node

The following procedures explain these steps in detail.

Install the Oracle Software on the New Node

Follow these steps to install the Oracle software on the new node:

1. If your platform supports a cluster file system then proceed to the next section titled "Post-Installation Steps".

2. On the new node, insert the Oracle9i release 1 (9.0.1) CD-ROM into the new node's CD-ROM drive.

3. Run the installer using one of the following procedures for your platform.
   • On UNIX, run the Installer on by executing the runInstaller command from the root directory of the first Oracle CD. The Installer displays the Welcome page.

   • On Windows NT and Windows 2000, run the Installer by executing the setup.exe command. The installer displays the Welcome page.

4. On the File Locations page in the Path field under the Destination heading, enter the ORACLE_HOME into which you are going to install the Oracle Enterprise Edition and Real Application Clusters software. The ORACLE_HOME you enter should be the same as the ORACLE_HOME that the existing nodes use.

5. On the Available Products page, select Oracle9i Database.

6. On the Install Types page, select Enterprise Edition. The Installer displays the Database Configuration page.

7. Follow the OUI instructions to install the same products and options that you installed on the existing nodes. Or select the Software Only configuration option on the Database Configuration page.

8. On the Node Selection page, select only the current or new node name. Do not select other pre-existing node names; the Oracle software is already on them.

9. When the OUI completes the installation of the Oracle software, run the post installation script root.sh. Then exit the Oracle Universal Installer and proceed to the next section, "Post-Installation Steps".

Post-Installation Steps

From the new node, perform the following post-installation steps:

1. Run the Oracle Net Configuration Assistant from the bin directory in ORACLE_HOME to configure a new listener for the node.

2. Complete any post-installation steps for your platform by referring to your platform-specific Oracle documentation.

Add a Database Instance on the New Node

To add an instance on the new node, go to one of the existing nodes that has a running instance and follow these steps:

1. Make sure the Global Services Daemon (GSD) is running on all the existing nodes as well as on the new node. If the GSD is not running:

   --On UNIX, enter the following command to start it:

   $ORACLE_HOME/bin/gsd 
   
   

   --On Windows NT and Windows 2000, enter the following to start the OracleGSDService:

   %ORACLE_HOME%\bin\gsdservice -start 
   
   

2. Launch the Oracle Database Configuration Assistant by typing dbca in the bin directory in $ORACLE_HOME on UNIX, or on Windows NT and Windows 2000, choose Start >Programs >Oracle for Windows NT/2000 - [HOME_NAME] >Database Administration >Database Configuration Assistant.

   The DBCA wizard starts and displays the Welcome Page for Real Application Clusters shown in Figure 9-1.

   Figure 9-1 DBCA Welcome Page for Real Application Clusters

   
   
   Text description of the illustration welcome.gif

   Note: If the Welcome Page for Real Application Clusters does not display, it means the Oracle Database Configuration Assistant was unable to: Communicate with the Cluster Manager (CM) software on Windows NT or Windows 2000 Detect the Global Cache Service (GCS) software or the lists of nodes in the cluster on UNIX operating systems To resolve this, refer to your vendor's operating system-dependent documentation and then restart the DBCA.

3. Select the Oracle Real Application Clusters Database option and click Next. After you click Next, the DBCA displays the Operations Page as shown in Figure 9-2:

   Figure 9-2 Operations Page

   
   
   Text description of the illustration page1.gif
4. Select the Instance Management option and click Next.

   After you click Next, the DBCA displays the Instance Management shown in Figure 9-3.

   Figure 9-3 Instance Management Page

   
   
   Text description of the illustration instance.gif
5. Select the Add Instance option and click Next.

   After you click Next, the DBCA displays the List of Databases and their current status, such as ACTIVE, or INACTIVE, as shown in Figure 9-4.

   Figure 9-4 List of Databases

   
   
   Text description of the illustration instanca.gif
6. Select an active Real Application Clusters database name to add an instance to from the List of Databases page. If your user ID is not operating-system authenticated, the DBCA also prompts you for a user ID and password for a database user that has SYSDBA privileges. If prompted, enter a valid user ID and password. Click Next.

   After you click Next, the DBCA displays the List of Instances Page showing the names of the instances that exist for the selected Real Application Clusters database as shown in Figure 9-5.

   Figure 9-5 List of Instances

   
   
   Text description of the illustration add5.gif
7. Click Next.

   After you click Next, the DBCA displays the Adding an Instance page as shown in Figure 9-6.

   Figure 9-6 Adding an Instance

   
   
   Text description of the illustration add6.gif
8. Enter the instance name in the field at the top of this page if the instance name that the DBCA provides does not match your existing instance name sequence. Then select the new node name from the list and click Next.

   After you click Next, the DBCA displays the Storage Page as shown in Figure 9-7.

   Figure 9-7 Instance Storage Page

   
   Text description of the illustration storage.gif

   9. If your database uses automatic undo management, and your platform supports a cluster file system, then you can skip this step unless you want to change the default undo tablespace data file name that the DBCA provides. Otherwise, select the Tablespaces folder and expand it. Select the undo tablespace storage object. Another dialog will appear on the right-hand side. Change the default data file name to the raw device name (or change it to the cluster file system file name, if you do not want to use the default name) for the tablespace. Oracle Corporation recommends against changing the storage properties for the undo tablespace.

   10. If your database uses rollback undo management, then select the Rollback Segment folder to change the rollback segment names if you do not want to use the default rollback segment names that the DBCA provides. Oracle Corporation recommends against changing the storage properties for the rollback segments.

   11. If your platform supports a cluster file system, then skip this step unless you want to change the default redo log group file names. Otherwise, select the Redo Log Groups folder and expand it. For each redo log group number you select, the DBCA displays another dialog box. For UNIX, enter the raw device name that you created in the section "Create Raw Devices" in the File Name field. On Windows NT or Windows 2000 enter the symbolic link name that you created in the section, "Create Links to Disk Partitions".

   12. Repeat the previous step for all other redo log groups.

   13. Click Finish.

   After you click Finish, the DBCA displays a summary page.

   Figure 9-8 DBCA Summary Dialog

   
   
   Text description of the illustration dbca_inb.jpg
14. Click OK.

    The DBCA displays a progress dialog that shows the DBCA performing the instance addition operation during which the DBCA adds the instance and creates the instance's Oracle Net configuration. When the DBCA completes the instance addition operation, the DBCA displays a dialog asking whether you want to perform another operation. Click No and exit the DBCA.

    At this point, you have accomplished the following:

    • Created a new database instance on the new node

    • Created the new instance's required services for Windows NT or Windows 2000

    • Configured Oracle Net

    • Started the required services for Windows NT or Windows 2000

    • Started the new instance and its listeners

Deleting Instances

The following procedures explain how to delete an instance with the Oracle Database Configuration Assistant. Where noted, refer to previous figures in this chapter. To delete an instance:

1. Go to the DBCA Operations Page shown in Figure 9-2, select Instance Management, and click Next.

After you click Next, the DBCA displays the Instance Management Page shown in Figure 9-9.

2. Select the Delete Instance option and click Next.

   Figure 9-9 DBCA Instance Management Operation Selection Page

   
   
   Text description of the illustration dbca_ins.jpg

   After you click Next, the DBCA displays the List of Databases page shown in Figure 9-10.

   Figure 9-10 List of Databases

   
   
   Text description of the illustration delete4.gif
3. Select a Real Application Clusters database from which to delete an instance. If your user ID is not operating-system authenticated, then the DBCA also prompts you for a user ID and password for a database user that has SYSDBA privileges. If the DBCA prompts you for this, then enter a valid user ID and password. Click Next.

   After you click Next, the DBCA displays the List of Instances Page shown in Figure 9-11. The List of Instance Page shows the instances associated with the Real Application Clusters database that you selected as well as the status of each instance.

   Figure 9-11 List of Instances

   
   
   Text description of the illustration delete5.gif
4. Select the instance you want to delete and click Finish.

5. If you select the local instance, the DBCA displays a warning as shown in Figure 9-12.

   Figure 9-12 DBCA Warning Dialog for Selecting the Local Instance

   
   
   Text description of the illustration dbca_inc.jpg
6. To proceed with the operation, click OK on the warning dialog and select an instance other than the local instance and click Finish.

7. The DBCA displays a summary dialog as shown in Figure 9-13. Click OK on the summary dialog.

   Figure 9-13 DBCA Summary Dialog

   
   
   Text description of the illustration dbca_ina.jpg

   After you click OK, the DBCA displays a progress dialog that shows the DBCA performing the instance deletion operation during which the DBCA removes the instance and the instance's Oracle Net configuration. When the DBCA completes the deletion operation, the DBCA displays a dialog asking whether you want to perform another operation. Click No and exit the DBCA.

   At this point, you have accomplished the following:

   • Stopped the listeners associated with the selected instance

   • Deleted the selected database instance from the instance's configured node

   • Deleted the selected instance's services for Windows NT or Windows 2000

   • Removed the Oracle Net configuration

   • Deleted the Oracle Flexible Architecture directory structure from the instance's configured node