8
Adding Instances and Nodes

This chapter describes how to convert from a single instance Oracle8 database to a multi-instance Oracle8 database using the parallel server option, and how to add a third or fourth node to an existing two or three node configuration.

Specific topics discussed are:

• Moving from a Single Instance to Parallel Server

• Adding Additional Nodes to a Cluster

Moving from a Single Instance to Parallel Server

This section explains how to migrate from a single instance Oracle8 database to multi-instance. In addition, it can help you extend an existing OPS configuration to additional nodes.

This section should be used as a supplement to Chapter 23, "Migrating from Single Instance to Parallel Server", of the Oracle8 Parallel Server Concepts & Administrator's Guide.

Step 1: Configure Hardware

See your vendor documentation for information about setting up Oracle Parallel Server hardware.

Step 2: Evaluate Tablespaces and Log Files of Single Instance

Because tablespaces must be identical on the Oracle Parallel Server as a single instance database, it is important that you do any consolidation, addition of, or renaming of tablespaces on the single instance at this time.

Each additional node in the cluster requires two log files. Typically, a single instance database only has two log files. You must add additional log files for each node using the ALTER DATABASE ADD LOGFILE statement.

Step 3: Create Raw Partitions

When building a database that is to be shared and managed by Oracle Parallel Server, it is necessary to access all disks as raw disks. In the case of a cluster, it is not possible to share a file system between two separate nodes. The file system is in effect a private cache resident on just one node.

The process of creating a raw disk involves creating an extended partition on a disk and assigning logical drives to it.

To create raw partitions:

1. Create raw partitions and assign logical drives, as described in Chapter 3, "Performing Pre-Installation Tasks". Ensure enough raw partitions have been created for:

   File Type	Number of Partitions	Space Per Partition
   control files	2	2 MB
   log files1	2 per node	small number of transactions: 20-50K each medium number of transaction: 200-500 K each large number of transactions: 2 MB - 5 MB each

   1 Each node requires at least two log files.

1. Assign meaningful symbolic link names to the logical drives, as described in "Step 2: Assign Symbolic Links to Each Logical Drive" of Chapter 5, "Configuring Oracle Parallel Server". You may need to modify the ORALINK*.TBL files to suit your database.

Step 4: Install Oracle8 Enterprise Edition and Oracle Parallel Server Option

Except on the server already running, install Oracle8 Enterprise Edition along with the Oracle Parallel Server option, as described in Chapter 4, "Installing Oracle Parallel Server". If your current single instance database does not have the supporting hardware, you must perform a clean install on new cluster hardware.

On the server already running Oracle8 Enterprise Edition, install the Oracle Parallel Server Option, as described in Chapter 4, "Installing Oracle Parallel Server".

Step 5: Install OSD Files

For all nodes, see your Operating System Dependent (OSD) vendor documentation for instructions about installing OSD files.

Step 6: Move Initialization Files

The initialization files, INIT_COM.ORA and INITSID.ORA are installed in the ORACLE_HOME\OPS directory. In order to avoid having to specify the PFILE parameter when starting the database (STARTUP PFILE=C: ORANT\OPS\INITSID.ORA), move these file to the ORACLE_HOME\DATABASE file where initialization files are normally located.

To change the location of the initialization files:

1. Move INIT_COM.ORA and INITSID.ORA form the ORACLE_HOME\OPS directory to ORACLE_HOME\DATABASE directory.

1. Modify the INITSID.ORA file so the IFILE points to the new ORACLE_HOME\DATABASE directory.

Step 7: Export Data from Old Database

Export the entire database from the single instance database. Use one of the following tools:

• Oracle Data Manager

• Export Utility

Oracle Data Manager

To start Oracle Data Manager from the Enterprise Manager Console:

• Choose View > Launch Palettes > Applications.

  The Applications palette appears. Click the Data Manager icon.

• Choose Tools > Applications > Data Manager

  Additional Information: See the Oracle Enterprise Manager Administrator's Guide.

Export Utility

Enter the following at the MS-DOS command prompt followed by your user name and password.

Note: To export an entire database, you must use the user name SYSTEM. Do not use INTERNAL or SYS.

To use the Export utility to export all data from an existing database to the new database:

1. Set ORACLE_SID to the database service of the database whose contents you want to export. For example, export the starter database ORCL. Note there are no spaces around the equal sign (=) character.

   C:\MYDIR> SET ORACLE_SID=ORCL 
   

1. Run the Export utility:

   C:\MYDIR> EXP80 SYSTEM/PASSWORD FILE=MYEXP.DMP FULL=Y LOG=MYEXP.LOG
   

   You now have a full database export of the starter database ORCL in the file MYEXP.DMP, with all messages from the Export utility logged in the MYEXP.LOG file.

   When running the Export utility, the default values for the following parameters under Windows NT are:

   BUFFER 4 KB

   RECORDLENGTH 2 KB

Step 8: Prepare CREATE DATABASE Script

A new database must be created on the raw partitions.

To prepare the CREATE DATABASE script:

1. Use the BUILD_DB.SQL script located in ORACLE_HOME\RDBMS80\ADMIN to build a script:

   Make the following changes to the BUILD_DB.SQL script:

   1. Set PFILE so it points to the ORACLE_HOME\DATABASE\INITOPS1.ORA initialization file.

   2. Modify the CREATE DATABASE ORACLE to CREATE DATABASE OPS.

   3. Modify the log file and data file names with the symbolic link names you created in "Step 2: Assign Symbolic Links to Each Logical Drive" of Chapter 5, "Configuring Oracle Parallel Server".

   4. Modify the log file and data files with a naming convention of \\.\SYMBOLIC_LINK_NAME. The symbolic link name should match the name you used in ORALINK*.TBL file, as described in "Step 3: Create Raw Partitions".

   5. Create new entries for additional log files for each additional node. (Each node requires two log files.

   6. Modify the log file and data file sizes for the Oracle Parallel Server.

      Enter the following command to find out the current size of data files:

      SELECT * FROM DBA_DATA_FILES
      

   7. Create enough private (acquired explicitly by an instance when an instance opens a database) rollback segments (16 is the default) for the number of concurrent users per transaction. With the exception of the SYSTEM rollback segment, public rollback segments cannot be shared among nodes.

      A sample script follows:

      create database ops
          controlfile reuse
          logfile GROUP 1 '\\.\OPS_log1t1'size 200K reuse,
                  GROUP 1 '\\.\OPS_log2t1'size 200K reuse,
                  GROUP 2 '\\.\OPS_log1t2'size 200K reuse,
                  GROUP 2 '\\.\OPS_log2t2'size 200K reuse,
          datafile '\\.\OPS_sys01' size 50M,
          character set WE8ISO8859P1;
      
      create rollback segment rb_temp;
      
      create tablespace user_data
          datafile `\\.\OPS_usr01' size 15M;
      create tablespace rollback_data
          datafile '\\.\OPS_rbs01' size 50M;
      create tablespace temporary_data
          datafile '\\.\OPS_tmp01' size 10M;
      alter rollback segment rb_temp online;
      
      -- Change the SYSTEM users' password, default tablespace and
      -- temporary tablespace.
      
      alter user system temporary tablespace temporary_data;
      alter user system default tablespace user_data;
      
      -- Create 16 rollback segments.  Allows 16 concurrent users with open
      -- transactions updating the database. This should be enough.
      
      create private rollback segment rb1 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb2 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb3 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb4 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb5 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb6 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb7 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb8 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb9 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb10 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb11 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb12 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb13 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb14 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb15 storage(initial 50K next 50K)
        tablespace rollback_data;
      create private rollback segment rb16 storage(initial 50K next 50K)
        tablespace rollback_data;
      

Step 9: Configure OSD Layer, Create Services, and Configure Network

Follow "Step 3: Configure OSD Layer" to "Step 6: Configure the Network" in Chapter 5, "Configuring Oracle Parallel Server".

Step 10: Create an Oracle Parallel Server Database

To create a database:

1. On the primary node, start the database, and run the CREATE DATABASE script you just created along with the following scripts:

   CATALOG.SQL	Generates the data dictionary.
   CATPROC.SQL	Installs the objects used by the Oracle7 database's PL/SQL functionality.
   CATPARR.SQL	Creates objects for Oracle Parallel Server.

   C:\> SVRMGR30
   SVRMGR> CONNECT INTERNAL/ORACLE
   SVRMGR> STARTUP 
   SVRMGR> @%ORACLE_HOME%\RDBMS80\ADMIN\CREATE_DATABASE.SQL
   SVRMGR> @%ORACLE_HOME%\RDBMS80\ADMIN\CATALOG.SQL
   SVRMGR> @%ORACLE_HOME%\RDBMS80\ADMIN\CATPROC.SQL
   SVRMGR> @%ORACLE_HOME%\RDBMS80\ADMIN\CATPARR.SQL
   

1. If you modified the number of rollback segments, change the ORACLE_HOME\DATABASE\INITSID.ORA files to reflect the change.

2. Shut down the database.

   SVRMGR> SHUTDOWN
   

Step 11: Transfer Data From Old Data to Empty Database

Import the entire database into the empty database. Use one of the following tools:

• Oracle Data Manager

• Import Utility

Oracle Data Manager

To start Oracle Data Manager from the Enterprise Manager Console:

• Choose View > Launch Palettes > Applications. The Applications palette appears. Click the Data Manager icon.

• Choose Tools > Applications > Data Manager

  Additional Information: See the Oracle Enterprise Manager Administrator's Guide

Import Utility

To import the full export created in the section "Step 7: Export Data from Old Database" into the new database:

To run the Import utility:

C:\MYDIR> IMP80 SYSTEM/PASSWORD FILE=MYEXP.DMP FULL=Y LOG=MYIMP.LOG

ATTENTION: If the original database from which the export file was generated contains tablespaces that are not in the new database, then the Import utility attempts to create those tablespaces with associated data files. The easy solution is to ensure both databases contain the same tablespaces. The data files do not have to be identical. Only the tablespace names are important.

When running the Import utility, the default values for the following parameters under Windows NT are:

BUFFER 4 KB
RECORDLENGTH 2 KB

Step 12: Start the Database

See "Step 8: Start the Database in Parallel Mode" in Chapter 5, "Configuring Oracle Parallel Server" to start the Oracle Parallel Server.

Adding Additional Nodes to a Cluster

This release of Oracle Parallel Server supports up to four nodes in a cluster. If you have already configured a two- or three-node cluster and want to add nodes three and/or four, you must add the new configuration information for these nodes through scripts. The database does need to be recreated.

Step:	For additional information, see:
Create additional logical drives. Each additional node requires two log files. Because of this, you must add two additional logical drives for each additional node.	"Step 2: Create Logical Drives In an Extended Partition" in Chapter 3, "Performing Pre-Installation Tasks"
Unassign drive letters.	"Step 3: Unassign Drive Letters" in Chapter 3, "Performing Pre-Installation Tasks"
Assign symbolic links to each additional logical drives.	"Assign Symbolic Links to Each Logical Drive" in this chapter
Shut down services.	"Shut Down Services" in this chapter
Configure OSD layer.	"Step 3: Configure OSD Layer" in Chapter 5, "Configuring Oracle Parallel Server"
Create OracleServiceSID services.	"Step 4: Create Services" in Chapter 5, "Configuring Oracle Parallel Server"
Add the ORACLE_SID entry to the registry.	"Step 5: Add the ORACLE_SID Entry to the Registry" in Chapter 5, "Configuring Oracle Parallel Server"
Configure the network. With the addition of nodes, additional service names must be created in order for Net8 clients to communicate with Oracle8 servers.	"Step 6: Configure the Network" in Chapter 5, "Configuring Oracle Parallel Server"
Create additional rollback segments for new nodes.	"Create Rollback Segments For Additional Nodes" in this chapter.
Start the database.	"Step 8: Start the Database in Parallel Mode" in Chapter 5, "Configuring Oracle Parallel Server"
Verify instances are running.	"Step 9: Verify Instances Are Running" in Chapter 5, "Configuring Oracle Parallel Server"
Reconfigure Performance Manager.	"Reconfigure Oracle Performance Manager" in this chapter

Assign Symbolic Links to Each Logical Drive

Use the SETLINKS utility to create symbolic links to the new logical partitions. The SETLINKS utility uses a file named ORALINK2.TBL for the third node in a cluster and ORALINK3.TBL for the fourth node in a cluster. These files will create the symbolic links to the log files' logical partitions:

Figure 8-1 ORALINK2.TBL

Figure 8-2 ORALINK3.TBL

To create symbolic links to a raw partition:

1. On the primary node, edit the partition and hard disk numbers appropriately in the ORALINK2.TBL file (for the third node) and ORALINK3.TBL (for the fourth node) located in ORACLE_HOME\OPS to match the disk numbers and partitions number you wrote down in section Step 2: Unassign Drive Letters.

   You can use a worksheet similar the one below to assist with the edits:

   Symbolic Link	Node 3
   OPS_log1t3	Harddiskx Partitionx
   OPS_log2t3	Harddiskx Partitionx

   Symbolic Link	Node 4
   OPS_log1t4	Harddiskx Partitionx
   OPS_log2t4	Harddiskx Partitionx

   WARNING: Do not change the symbolic link names in OPSLINK1.TBL. If you do, symbolic link names must be modified for the following files located in: ORACLE_HOME\OPS\OPS.SQL ORACLE_HOME\DATABASE\INITSID.ORA ORACLE_HOME\DATABASE\INIT_COM.ORA

1. Run the ORALINK2.TBL and/or ORALINK3.TBL through the SETLINKS application as shown below:

   C:\> CD ORACLE_HOME\OPS
   C:\ORACLE_HOME\OPS> SETLINKS /F:ORALINK2.TBL
   C:\ORACLE_HOME\OPS> SETLINKS /F:ORALINK3.TBL
   

   The symbolic links are created:

   Oracle Corporation. Copyright (c) 1997. All rights reserved.
   Created Link:OPS_log1t3 = Device:\Device\Harddisk3\Partition15
   Created Link:OPS_log2t3 = Device:\Device\Harddisk3\Partition16
   Dos devices updated successfuly.
   

2. Make sure the drives have been mapped with the correct names with the SETLINKS application (as shown below):

   C:\ORACLE_HOME\OPS> SETLINKS /D
   

3. Repeat Steps 1-3 on the other nodes.

Shut Down Services

Some vendors may require you stop the services and stop the Cluster Manager prior to configuring the OSD layer. See your vendor documentation for further information.

To stop the services:

Stop OracleServiceSID, OracleHOME_NAMETNSListener80, OraclePGMSService, and OracleAgent services on each node:

• From the MS-DOS command line, enter:

  C:\> NET STOP SERVICE
  

• From the Control Panel's Services window, select the service, and click Stop.

Create Rollback Segments For Additional Nodes

To create rollback segment for the additional nodes:

1. On each node, delete the ORACLE_HOME\RDBMS80\TRACE\SIDLMON.TRC file.

1. Start OracleServiceSID on each node.

   Note: Prior to starting a OracleServiceSID service, the OraclePGMSService service must be running. If you used the CRTSRV script in "Step 4: Create Services" in Chapter 5, "Configuring Oracle Parallel Server", OraclePGMSService automatically starts when the OracleServiceSID service is started. If you chose to create your services with another method, you can still have OraclePGMSService start up automatically with a OracleServiceSID service by entering the following at the command for each node: C:\> OPSREG80 ADD SID

   • From the MS-DOS command line, enter:

     C:\> NET START OracleServiceSID
     

   • From the Control Panel's Services window, select OracleServiceSID, then choose Start.

2. On the primary node, start the database, and run the C_OPS3.SQL (for the third node) and/or C_OPS4.SQL (for the fourth node) scripts:

   C:\> SVRMGR30
   SVRMGR> CONNECT INTERNAL/PASSWORD
   SVRMGR> STARTUP
   SVRMGR> @C_OPSX.SQL
   

   The C_OPSX.SQL script creates the necessary rollback segments for the third and fourth nodes.

3. On each node, ensure there is a "Reconfiguration complete" message in the ORACLE_HOME\RDBMS80\TRACE\SIDLMON.TRC file for the correct number of node, signifying the cluster was started without errors:

4. Stop OracleServiceSID:
   • From the MS-DOS command line, enter:

     C:\> NET STOP OracleServiceSID
     

   • From the Control Panel's Services window, select the service, and click Stop.

Reconfigure Oracle Performance Manager

Reconfiguring the Performance Manager involves re-running OPS_MON.SQL.

To reconfigure Performance Manager:

1. Regenerate the OPS_DBL.SQL script:
   • Use OPSCONF, as described in "Step 6b: Configure the Service Names" of Chapter 5, "Configuring Oracle Parallel Server".
   • Modify the OPS_DBL.SQL script by adding the following lines for the third node:

     drop public database link ops@HOSTNAME_ops3
     /
     create public database link ops@HOSTNAME_ops3 using 'ops3'
     /
     

     Adding the following lines for the fourth node:

     drop public database link ops@HOSTNAME_ops4
     /
     create public database link ops@HOSTNAME_ops4 using 'ops4'
     /
     

1. Run the OPS_MON.SQL script in "Step 3: Configure Oracle Performance Manager" of Chapter 6, "Installing and Configuring Oracle Parallel Server Manager".

---