5
Configuring Oracle Parallel Server

This chapter describes the configuration tasks to create an operating Oracle Parallel Server. Specific topics discussed are:

• Step 1: Copy Initialization Files

• Step 2: Assign Symbolic Links to Each Logical Partition

• Step 3: Configure OSD Layer

• Step 4: Create Services

• Step 5: Add the ORACLE_SID Entry to the Registry

• Step 6: Configure the Network

• Step 7: Create an Oracle Parallel Server Database

• Step 8: Start the Database in Parallel Mode

• Step 9: Verify InstancesAre Running

Step 1: Copy Initialization Files

The initialization files, INIT_COM.ORA and INITSID.ORA are installed in the ORACLE_HOME\OPS directory. To use Oracle Parallel Server Manager and to avoid specifying the PFILE parameter when starting the database (STARTUP PFILE=C: ORANT\OPS\INITSID.ORA), copy these file to the ORACLE_HOME\DATABASE file where initialization files are normally located.

To change the location of the initialization files:

1. Copy INIT_COM.ORA and INITSID.ORA form the ORACLE_HOME\OPS directory to ORACLE_HOME\DATABASE directory.

2. Modify the INITSID.ORA file so the IFILE points to the new ORACLE_HOME\DATABASE directory.

Step 2: Assign Symbolic Links to Each Logical Partition

The Oracle SETLINKS utility creates symbolic links to raw logical partitions.

For example, with the SETLINKS utility, you can create a symbolic link called OPS_log1t1 to the second logical partition on the second physical disk:

OPS_log1t1 \Device\Harddisk2\Partition2

The SETLINKS utility comes with the following ASCII files, with symbolic link names already associated with raw partitions. Each node requires two log files. If you have more than a three to four cluster, additional symbolic link names must be linked to the files:

.

This File...	Contains...
ORALINK1.TBL	symbolic link names already associated with raw partitions for a two-node cluster
ORALINK2.TBL	symbolic link names to accommodate a third node's additional log files
ORALINK3.TBL	symbolic link names to accommodate a fourth node's additional log files

Figure 5-1 ORALINK1.TBL

Figure 5-2 ORALINK2.TBL

Figure 5-3 ORALINK3.TBL

The following is a description of the symbolic link names:

Symbolic link name...	File Description	Symbolic link name used in the file ORACLE_HOME...
OPS_log1t1	recovery log file	\OPS\OPS.SQL
OPS_log2t1	recovery log file	\OPS\OPS.SQL
OPS_sys01	system data file	\OPS\OPS.SQL
OPS_usr01	user data file	\OPS\OPS.SQL
OPS_rbs01	rollback data file	\OPS\OPS.SQL
OPS_tmp01	temporary data file	\OPS\OPS.SQL
OPS_log1t2	first log file for thread 1	\OPS\OPS.SQL
OPS_log2t2	second log file for thread 2	\OPS\OPS.SQL
OPS_cntr01	control file	\DATABASE\ INITSID.ORA, INIT_COM.ORA

To create symbolic links to the logical partitions:

1. On the primary node, edit the partition and hard disk numbers appropriately in the ORALINK1.TBL file located in ORACLE_HOME\OPS to match the disk numbers and partitions number you wrote down in "Step 3: Unassign Drive Letters". If the cluster has a third and/or fourth node, edit ORALINK2.TBL (for the third node) and ORALLINK3.TBL for the fourth node.

   You can use a worksheet similar the one below to assist with the process:

   Symbolic Link	Node 1	Node 2
   OPS_log1t1	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_log2t1	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_sys01	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_usr01	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_rbs01	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_tmp01	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_log1t2	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_log2t2	Harddiskx Partitionx	Harddiskx Partitionx
   OPS_cntr01	Harddiskx Partitionx	Harddiskx Partitionx

   For example, the worksheet for the two nodes shown in section "Step 2: Create Logical Partitions In an Extended Partition" looks likethe following:

   Symbolic Link	Node 1	Node 2
   OPS_log1t1	Harddisk1 Partition1	Harddisk2 Partition1
   OPS_log2t1	Harddisk1 Partition2	Harddisk2 Partition2
   OPS_sys01	Harddisk1 Partition3	Harddisk2 Partition3
   OPS_usr01	Harddisk1 Partition4	Harddisk2 Partition4
   OPS_rbs01	Harddisk1 Partition5	Harddisk2 Partition5
   OPS_tmp01	Harddisk1 Partition6	Harddisk2 Partition6
   OPS_log1t2	Harddisk2 Partition1	Harddisk3 Partition1
   OPS_log2t2	Harddisk2 Partition2	Harddisk3 Partition2
   OPS_cntr01	Harddisk2 Partition3	Harddisk3 Partition3

   Additional Information: See Chapter 3, "Performing Pre-Installation Tasks", for further information about the Disk Administrator.

   WARNING: Do not change the symbolic link names in OPSLINK1.TBL. If you do, symbolic link names must be modified for the following files: ORACLE_HOME\OPS\OPS.SQL ORACLE_HOME\DATABASE\INIT_COM.ORA

2. Run the ASCII input file, ORALINK1.TBL, through the SETLINKS utility as shown below:

   C:\> CD ORACLE_HOME\OPS
   C:\ORACLE_HOME\OPS> SETLINKS /F:ORALINK1.TBL
   
   

   SETLINKS maps the drives to the symbolic label names:

   Oracle Corporation. Copyright (c) 1997. All rights reserved.
   Created Link:
   Created Link:OPS_log1t1 = Device:\Device\Harddisk1\Partition1
   Created Link:OPS_log2t1 = Device:\Device\Harddisk1\Partition2
   Created Link:OPS_sys01 = Device:\Device\Harddisk1\Partition3
   Created Link:OPS_usr01 = Device:\Device\Harddisk1\Partition4
   Created Link:OPS_rbs01 = Device:\Device\Harddisk1\Partition5
   Created Link:OPS_tmp01 = Device:\Device\Harddisk1\Partition6
   Created Link:OPS_log1t2 = Device:\Device\Harddisk2\Partition1
   Created Link:OPS_log2t2 = Device:\Device\Harddisk2\Partition2
   Created Link:OPS_cntr01 = Device:\Device\Harddisk3\Partition3
   Dos devices updated successfully.
   
   

   If the cluster has third or fourth node, SETLINKS must be run again with OPSLINK2.TBL (for the third node) or OPSLINK3.TBL for the fourth node.

   C:\> CD ORACLE_HOME\OPS
   C:\ORACLE_HOME\OPS> SETLINKS /F:ORALINK2.TBL
   C:\ORACLE_HOME\OPS> SETLINKS /F:ORALINK3.TBL
   
   

3. Make sure the drives have been mapped with the correct names as shown below:

   C:\ORACLE_HOME\OPS> SETLINKS /D
   
   

   SETLINKS shows how the symbolic link names have been mapped:

   Oracle Corporation. Copyright (c) 1997. All rights reserved.
   OPS_log1t1  = \Device\Harddisk1\Partition1
   OPS_log2t1  = \Device\Harddisk1\Partition2
   OPS_sys01   = \Device\Harddisk1\Partition2
   OPS_usr01   = \Device\Harddisk1\Partition2
   OPS_rbs01   = \Device\Harddisk1\Partition5
   OPS_tmp01   = \Device\Harddisk1\Partition6
   OPS_log1t2  = \Device\Harddisk2\Partition1
   OPS_log2t2  = \Device\Harddisk2\Partition2
   OPS_cntr01  = \Device\Harddisk2\Partition3
   ...
   
   

4. Repeat Steps 1-3 on the other nodes.

Removing or Ignoring Links

Links can be removed or ignored by changing the contents of the ORALINKx.TBL files and running it through SETLINKS again.

Ignoring Links

Insert a `#" at the beginning of a line to denote a comment, which is ignored by SETLINKS.

Removing an Existing Symbolic Llink

List a symbolic link without a device path in the ORALINKx.TBL file.

For example:

OPS_log1t1 
OPS_log2t1 \Device\Harddisk1\Partition2
OPS_sys01  \Device\Harddisk1\Partition3
OPS_usr01  \Device\Harddisk1\Partition4
OPS_rbs01  \Device\Harddisk1\Partition5
OPS_tmp01  \Device\Harddisk1\Partition6
OPS_log1t2 \Device\Harddisk2\Partition1
OPS_log2t2 \Device\Harddisk2\Partition2
OPS_cntr01 \Device\Harddisk2\Partition3

This removes the OPS_log1t1 link and creates the other links specified.

Step 3: Configure OSD Layer

Note: When following the steps below, note that some tasks are performed on the primary node only, while other tasks are performed on each node.

To configure the Operating System Dependent (OSD) components:

See your OSD vendor documentation for instructions about configuring the OSD components, making sure the:

• domain name you specify is identical to the database name defined by the DB_NAME parameter in the INIT_COM.ORA file located in ORACLE_HOME\DATABASE. The database name is OPS by default.

• instance number you specify must be one less than the digit defined by the INSTANCE_NUMBER parameter in the INITSID.ORA file located in ORACLE_HOME\DATABASE.

• machine names you specify are alpha-numeric

  CAUTION: When configuring domains, two instances cannot reside on the same machine and be in the same domain.

Step 4: Create Services

After the OSD components have been configured, an OracleServiceSID for each instance must be created.

To create OracleServiceSID:

1. On each node, go to the ORACLE_HOME\BIN directory:

   C:\> CD ORACLE_HOME\BIN
   
   

2. On each node, use the CRTSRV batch file to create a unique service corresponding to the instance of the node. This batch file also sets up the service so that each time it started, the OraclePGMSService is started automatically.

   C:\ORACLE_HOME\BIN> CRTSRV SID
   
   

   For example, to create OracleServiceOPS1 on the primary node:

   C:\ORACLE_HOME\BIN> CRTSRV OPS1
   
   

3. Verify OracleServiceSID exists by choosing Services from the Control Panel. The Services window appears:

   
   

4. Some vendors may require changing the password for OracleServiceSID and OracleHOME_NAMETNSListener80 to the Administrator's password for the machine:

   Additional Information: See Chapter 6, "Using Multiple Oracle Homes", in the Oracle8 Enterprise Edition Getting Started for Windows NT for a description of HOME_NAME and multiple Oracle homes.

   1. Double-click the service. The Service dialog box appears:

      
      

   2. Select the This Account option button.

   3. Select the "..." button. The Add User dialog box appears:

      
      

   4. Select Administrator from the Names list box, then click Add.

   5. Click OK. The Service dialog box reappears.

   6. Enter the NT administrator's password in the Password and Confirm Password fields.

   7. Click OK.

      The Services window reappears.

   8. Click Close to exit the Services window.

5. Stop OracleServiceSID:
   • From the MS-DOS command line, enter:

     C:\> NET STOP OracleServiceSID
     
     

   • From the Control Panel's Services window, select OracleServiceSID, then click Stop.

Step 5: Add the ORACLE_SID Entry to the Registry

The SID for an instance must be set in the registry to allow instance recognition when you access a database from a node. When you connect to a database, Oracle Parallel Server will be able to resolve the SID name.

Note: If the SID is not set in the registry, Oracle Parallel Server tries to see if the SID is set as an environment variable. If it cannot find it, the SID name defaults to ORCL. ORCL is not a valid SID for an Oracle Parallel Server. Therefore, you cannot connect with it.

Oracle Parallel Server requires the ORACLE_SID registry value to be manually added to the following location:

If you have...	Parameter is located in...
One Oracle home directory	HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE
Two or more Oracle home directories	HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\ HOMEID where ID is incremented for each additional Oracle home directory on your machine.

To add the ORACLE_SID registry value:

1. From each node, start the registry in either one of two ways:
   • From the command line, enter:

     C:\> REGEDT32
     
     

   • Choose Start > Run, enter REGEDT32 in the Open field, then click OK.

   The Registry Editor window appears:

   
   
2. Double-click the SOFTWARE key.

3. Navigate to the proper Oracle home key.

4. Choose Add Value from Edit menu. The Add Value dialog box appears:

   
   

5. In the Value Name field, enter ORACLE_SID.

6. In the Data Type box, select REG_SZ.

7. Click OK. The String Editor dialog box appears:

   
   

8. Enter the SID of the instance for the node you are on, then click OK.

   Registry Editor adds ORACLE_SID.

9. Choose Exit from the Registry menu.

   The registry exits.

Step 6: Configure the Network

Configure the nodes and clients for connections. Net8 clients communicate with Oracle8 servers through service names, which are easy to remember aliases for database addresses.

Each node must have the following configuration files in ORACLE_HOME\NET80\ADMIN:

Configuration File	Description	Configuration Requirements
LISTENER.ORA	Includes addresses of all network listeners on a server, the system identifiers (SIDs) of the databases for which they listen, and various control parameters used by the network listener.	On each node, LISTENER.ORA must contain an entry for the database name and SID of the node in the SID_LIST section. Do not make an entry for GLOBAL_DBNAME. For example: SID_LIST_LISTENER = (SID_LIST = (SID_DESC = (SID_NAME = OPS) ) (SID_DESC = (SID_NAME = ops1) )
TNSNAMES.ORA	Includes a list of service names of databases.	A service name for the database and the name of the node's instance.

The OPSCONF utility must be used to generates a TNSNAMES.ORA file with the service names already configured for all the nodes in a cluster and a LISTENER.ORA file with SID_NAME parameter already set.

The steps to perform to configure the network include:

Step 6a: Stop the Network Listener

Step 6b: Configure the Service Names

Step 6c: Start the Network Listener

Step 6d: Test the Configuration

Step 6a: Stop the Network Listener

The network listener must be stopped on each node.

To stop the network listener:

• From the MS-DOS command line, enter:

  C:\> NET STOP OracleHOME_NAMETNSListener80
  C:\> NET STOP OracleHOME_NAMETNSListener80LSNR 
  

  (where LSNR is the non-default network listener name)

• From the Control Panel's Services window, select OracleHOME_NAMETNSListener80 (the service name if you are using the default network listener name LISTENER), or OracleHOME_NAMETNSListener80LSNR (where LSNR is the non-default network listener name), then click Stop.

  A message appears, confirming you want to stop the network listener. Click Yes.

Step 6b: Configure the Service Names

Configuration files can be created with the OPSCONF utility. The OPSCONF utility creates properly configured files for all nodes, including:

• TNSNAMES.ORA

• LISTENER_NODE_HOSTNAME.ORA (created for each node in the cluster)

• TOPOLOGY.OPS defines the parallel servers and instances in the network. This file is not needed if Oracle Enterprise Manager's auto-discovery feature is working.

  These files are created on the node in ORACLE_HOME\OPS. After creation, you need to move the files to their proper configuration file names and to the proper machines.

To create the configuration files:

1. On one of the nodes, go to the ORACLE_HOME\BIN directory:

   C:\> CD ORACLE_HOME\BIN
   
   

2. Run OPSCONF:

   C:\ORACLE_HOME\BIN> OPSCONF
   
   

OPSCONF runs in interactive mode, prompting you with configuration questions. Bold indicates user input:

1. Enter the name of the Oracle Parallel Server database.

   database_name
   
   

   The database name is defined in the INIT_COM.ORA file in ORACLE_HOME\DATABASE. The database name is defaulted to OPS.

2. Enter the port number of the listener (Default=1526).

   port_number
   
   

   Press [Enter] to accept the default value.

3. Enter the port number of the Oracleagent (Default=1748).

   Press [Enter] to accept the default value unless you have changed the port number.

All the configuration files are created in ORACLE_HOME\OPS.

Move each LISTENER_node_hostname.ORA file to the ORACLE_HOME\NET80\ADMIN directory to the appropriate node, renaming it to LISTENER.ORA.

Copy TNSNAMES.ORA file to the ORACLE_HOME\NET80\ADMIN directory on each of the nodes, all client machines, and the Oracle Enterprise Manager console.

Move TOPOLOGY.OPS to the ORACLE_HOME\NET80\ADMIN directory on the Oracle Enterprise Manager console workstation, renaming it to TOPOLOGY.ORA. Or, merge it with your existing TOPOLOGY.ORA.

Step 6c: Start the Network Listener

The network listener must be started on each node.

To start the network listener:

• From the MS-DOS command line, enter:

  C:\> NET START OracleHOME_NAMETNSListener80
  C:\> NET START OracleHOME_NAMETNSListener80LSNR 
  

  (where LSNR is the non-default network listener name)

• From the Control Panel's Services window, select OracleHOME_NAMETNSListener80 (the service name if you are using the default network listener name LISTENER), or OracleHOME_NAMETNSListener80LSNR (where LSNR is the non-default network listener name), then click Start.

Step 6d: Test the Configuration

To ensure the files are configured correctly:

1. On any node or client machine, start Server Manager. Enter:

   C:\> SVRMGR30
   
   

2. Connect to an instance:

   SVRMGR> CONNECT INTERNAL/PASSWORD@SERVICE_NAME
   
   

3. Server Manager returns a "Connected" message to the screen.

   If there is an error in connecting, you must troubleshoot your installation. Typically, there is a problem with the IP address, host name, or SID.

Step 7: Create an Oracle Parallel Server Database

Creating an Oracle Parallel Server database involves generating:

• database files and rollback segments

• tablespaces

• SYSTEM tables

  CAUTION: The instructions below assume the domain you specified during the OSD configuration is identical to the database name defined in the INIT_COM.ORA file.

To create the Oracle Parallel Server database:

1. Start the OracleServiceOPS1 service

   Note: Prior to starting a OracleServiceSID service, the OraclePGMSService service must be running. If you used the CRTSRV script in "Step 4: Create Services", OraclePGMSService automatically starts when the OracleServiceSID service is started. If you chose to create your services with another method, you can still have OraclePGMSService start up automatically with a OracleServiceSID service by entering the following at the command for each node: C:\> OPSREG80 ADD SID

   • From the MS-DOS command line, enter:

     C:\> NET START OracleServiceOPS1
     
     

   • From the Control Panel's Services window, select OracleServiceOPS1, then click Start.

2. Connect to the instance and run the OPSALL.SQL script to create the database:

   C:\> CD ORACLE_HOME\OPS
   C:\ORACLE_HOME\OPS> SVRMGR30
   SVRMGR> @OPSALL.SQL
   
   

3. Enter the password for INTERNAL when prompted.

   The OPSALL.SQL script:

   • Calls the OPS.SQL script to create the Oracle Parallel Server database files, rollback segments, and tablespaces

   • Builds a system data dictionary

   • Creates users

   • Creates views

   • Creates redo log files for each instance

   • Shuts down the database

     Note: You may modify the OPS.SQL script in ORACLE_HOME\OPS to better suit your database needs.

4. Stop the OracleServiceOPS1 service:
   • From the MS-DOS command line, enter:

     C:\> NET STOP OracleServiceOPS1
     
     

   • From the Control Panel's Services window, select OracleServiceSID, then click Stop.

Step 8: Start the Database in Parallel Mode

To start the Oracle Parallel Server database in parallel mode:

Note: When following the steps below, note that some tasks are performed on the primary node only, while other tasks are performed on each node.

1. On each node, delete the ORACLE_HOME\RDBMS80\TRACE\SIDLMON.TRC file.

   This file is re-created with new Oracle Parallel Server messages.

2. Start OracleServiceSID instance on each node.

   Note: Prior to starting a OracleServiceSID service, the OraclePGMSService service must be running. If you used the CRTSRV script in "Step 4: Create Services", OraclePGMSService automatically starts up when the OracleServiceSID service is started. If you chose to create your services with another method, you can still have OraclePGMSService start up automatically with a OracleServiceSID service by entering the following at the command for each node: C:\> OPSREG80 ADD SID

   • From the MS-DOS command line, enter:

     C:\> NET START OracleServiceOPS1
     
     

   • From the Control Panel's Services window, select OracleServiceSID, then click Start.

3. On the primary node, start the database:

   C:\> SVRMGR30
   SVRMGR> CONNECT INTERNAL/PASSWORD
   SVRMGR> STARTUP 
   
   

   The first instance to start up in shared mode determines the values of the global constant parameters for the other instances. The control file records the values of the GC_* parameters when the first instance starts up.

   When another instance attempts to start up in shared mode, the Oracle Server compares the values of the global constant parameters in its parameter file with those already in use and issues a message if any values are incompatible. The instance cannot mount the database unless it has the correct values for its global constant parameters.

4. On the remaining nodes, start the database:

   C:\> SVRMGR30
   SVRMGR> CONNECT INTERNAL/PASSWORD
   SVRMGR> STARTUP
   

   Note: If PARALLEL_SERVER (the default) is set in the INIT_COM.ORA file, you do not have specify STARTUP PARALLEL. You also do not have specify the PFILE parameter when starting the database (STARTUP PFILE=C: ORANT\OPS\INITSID.ORA) if you followed the procedure in "Step 1: Copy Initialization Files".

5. On each node, ensure there is a "Reconfiguration complete" message in the ORACLE_HOME\RDBMS80\TRACE\SIDLMON.TRC file for the correct number of node, signifying the cluster was started without errors:

   Below is a sample of the trace file, indicating successful startup for two nodes:

   Thu Oct 30 15:22:59 1997
   Reconfiguration started
   List of nodes: 1,2,
    Lock DB frozen
    Non-local Process blocks cleaned out
    Non-local Group blocks cleaned out
    Resources and locks cleaned out
    Traffic controller Initialized
    Communication channels reestablished
    Submitted all rdomain info
    Submitted all remote-lock requests
    Update rdomain variables
    Dwn-cvts replayed, VALBLKs dubious
    All grantable locks granted
   Thu Oct 30 15:23:02 1997
   Reconfiguration complete
   

Step 9: Verify InstancesAre Running

To verify instances are running:

1. On any node, enter the following:

   C:\> SVRMGR30
   SVRMGR> CONNECT INTERNAL/PASSWORD
   SVRMGR> SELECT * FROM V$ACTIVE_INSTANCES;
   
   

   The following output is returned:

   INST_NUMBRE  INST_NAME
   -----------  ---------
              1 node1:ops1
              2 node2:ops2
              3 node3:ops3
   . . 
   

   .

   CAUTION: If another node is reconfigured after this table has been accessed and this statement is issued again, only the last node whose database was started displays.

2. On all the nodes, connect as SCOTT/TIGER, and view the EMP table.

   C:\> SVRMGR30
   SVRMGR> CONNECT SCOTT/TIGER
   SVRMGR> SELECT * FROM EMP;
   
   

   The employee table displays:

       EMPNO ENAME      JOB             MGR HIREDATE        SAL      COMM    DEPTNO
   --------- ---------- --------- --------- --------- --------- --------- ---------
        7369 SMITH      CLERK          7902 17-DEC-80       800                  20
        7499 ALLEN      SALESMAN       7698 20-FEB-81      1600       300        30
        7521 WARD       SALESMAN       7698 22-FEB-81      1250       500        30
        7566 JONES      MANAGER        7839 02-APR-81      2975                  20
        7654 MARTIN     SALESMAN       7698 28-SEP-81      1250      1400        30
        7698 BLAKE      MANAGER        7839 01-MAY-81      2850                  30
        7782 CLARK      MANAGER        7839 09-JUN-81      2450                  10
        7788 SCOTT      ANALYST        7566 19-APR-87      3000                  20
        7839 KING       PRESIDENT           17-NOV-81      5000                  10
        7844 TURNER     SALESMAN       7698 08-SEP-81      1500         0        30
        7876 ADAMS      CLERK          7788 23-MAY-87      1100                  20
        7900 JAMES      CLERK          7698 03-DEC-81       950                  30
        7902 FORD       ANALYST        7566 03-DEC-81      3000                  20
        7934 MILLER     CLERK          7782 23-JAN-82      1300                  10
   14 rows selected.
   
   

3. On the one node, increase MILLER's salary by $1000 and commit the change:

   SVRMGR> UPDATE EMP
   SET SAL = SAL + 1000
   WHERE ENAME = `MILLER';
   SVRMGR> COMMIT;
   
   

4. On the other nodes, select the EMP table again:

   SVRMGR> SELECT * FROM EMP;
   
   

   MILLER's salary should now be $2,300, indicating all the instances can see the database.

5. Continue to Chapter 6, "Installing and Configuring Oracle Parallel Server Manager", if you want to configure Oracle Parallel Server Manager with Oracle Enterprise Manager.