2
Configuring the Shared Disks

This chapter describes how to configure shared disk subsystems to deploy Real Application Clusters. This chapter explains how to configure your subsystem so that the nodes within a cluster can read from and write to the shared disks. The topics in this chapter include:

• Configuring Shared Disk Subsystems for Real Application Clusters

• Configuring Raw Volumes for Real Application Clusters on UNIX

• Configuring Logical Drives on Windows NT and Windows 2000

  Note: Perform the procedures in this chapter only after ensuring that your system complies with the hardware and software requirements described in Chapter 1, and before continuing with the procedures for installing the Oracle9i Enterprise Edition software in Chapter 3.

Configuring Shared Disk Subsystems for Real Application Clusters

Real Application Clusters requires that all each instance be able to access a set of unformatted devices on a shared disk subsystem. These shared disks are also referred to as raw devices. If your platform supports an Oracle-certified cluster file system, however, you can store the files that Real Application Clusters requires directly on the cluster file system.

The Oracle instances in Real Application Clusters write data onto the raw devices to update the control file, server parameter file, each datafile, and each redo log file. All instances in the cluster share these files. The number and type of raw devices required depends on several factors as described in the following sections.

You must configure at least one device for the configuration information as described under the following heading, "The Configuration Raw Device". If you select one of the preconfigured database options on the Installer's Database Configuration screen, or if you use the DBCA interactively, you must also configure additional raw devices as described under the heading "Additional Raw Devices Required by the DBCA".

See Also: The file size requirements as described in this chapter may change. Therefore, consult the README for the latest file size recommendations.

The Configuration Raw Device

You must create at least one shared raw device as an information repository for the database server configuration. You must do this:

• If this installation is the first Oracle9i Server installation on your cluster

• To migrate from a pre-release 1 (9.0.1) Oracle9i Real Application Clusters database to release 1 (9.0.1)

• To manually create your database

Oracle Enterprise Manager, the Oracle Database Configuration Assistant (DBCA), the SRVCTL Utility, and other management tools use this raw device to store configuration information about:

• The instances in your Real Application Clusters database

• The node that each instance runs on

• The Oracle home within which the database runs

  Note: In previous releases, Oracle stored this information in either the db_name.conf file on UNIX platforms, or in the Registry on Windows NT platforms.

The configuration process for this raw device depends on your operating system:

• On UNIX, configure a raw device for the Server Management (SRVM) configuration raw device; the Oracle Universal Installer (OUI) prompts you for the name of this raw device.

• On Windows NT and Windows 2000, configure a Cluster Manager (CM) voting disk/SRVM Configuration Disk. Label this raw device srvcfg by creating an Object Link Manager (OLM) symbolic link with a fixed name.

  Note: On Windows NT and Windows 2000, SRVM uses the same raw device as the Cluster Manager voting disk.

To use the DBCA to create your database, the procedures for which appear in Chapter 4, configure additional raw devices as described in the next section.

Additional Raw Devices Required by the DBCA

To use the Oracle Database Configuration Assistant, you must configure raw devices as described in this section. These devices are in addition to the configuration device mentioned in the previous section. Create these devices before running the Oracle Universal Installer to install the Oracle Enterprise Edition software. The DBCA cannot create a Real Application Clusters database unless you have properly configured these raw devices:

• Seven for datafiles

• Two for control files

• One device for each instance for its own tablespace for automatic undo management (or one rollback segment tablespace for the database if you do not use automatic undo management)

• At least two for redo log files for each instance

• One for context

• Recommended: One for the server parameter file

• One for Oracle9i interMedia

• One for OLAP

  See Also: The Oracle9i Database installation guide for Windows for additional information about raw device requirements for Windows NT and Windows 2000

  Note: Each instance of a Real Application Clusters database has its own log files, but all instances in a cluster share the control files and datafiles. Each instance's log files must be readable by all other instances for several types of recovery, such as instance recovery.

Planning Your Raw Device Creation Strategy

Before installing the Oracle software, create enough partitions of specific sizes to support your database and leave a few spare partitions of the same size for future expansion. For example, if you have space on your shared disk array, select a limited set of standard partition sizes for your entire database. Partition sizes of 50MB, 100MB, 500MB, and 1GB are suitable for most databases. Also create a few very small and very large spare partitions that are, for example, 1MB and perhaps 5GB or greater in size.

Based your plans for using each partition, determine the placement of these spare partitions using a couple of methods. For example, you can mix different sizes on one disk, or segment each disk into same-sized partitions.

Note: Ensuring that there are one or two spare partitions enables you to perform emergency file relocations or additions if a tablespace unexpectedly fills. For example, you can drop and re-create unused partitions into two or more partitions if needed.

DBCA Database Configuration Options

The DBCA supports several database templates for Real Application Clusters, General Purpose, Transaction Processing, Data Warehouse, and New Database. Chapter 3 describes these configuration types in more detail. The first three options are preconfigured database options and they include datafiles. As already mentioned, the New Database option does not include datafiles and is fully customizable.

If you use the General Purpose, Transaction Processing, or Data Warehouse configuration, you must create specific tablespaces using specific minimum sizes as described under the heading "Recommended Tablespace and File Capacities".

You can customize these tablespaces if you select the Customized database configuration type on the Universal Installer's Database Configuration screen. You can also specify the file and block sizes. To use the Customized configuration option, ensure that the raw volumes on UNIX, or logical drives on Windows NT and Windows 2000, have enough space to accommodate the customized sizes.

Special Considerations for UNIX: Using Veritas Volume Manager

Use Veritas Volume Manager to create your volumes and then change the permissions and ownership on the volumes to the correct user, for example, oracle user. Also make sure that all volumes on the disks have the same names on each node. More information about Veritas Volume Manager is available at:

http://www.veritas.com

Recommended Tablespace and File Capacities

Use the tablespace size requirements in Table 2-1 for the General Purpose, Transaction Processing, and Data Warehouse configuration types. These requirements apply to both UNIX and Windows NT and Windows 2000 platforms. The preconfigured database options automatically create tablespaces and files with the capacities shown in Table 2-1. If you use the Customized database configuration, use these recommended sizes as guidelines.

Table 2-1 Minimum Tablespace Sizes

Create a Raw Device For	With File Size
SYSTEM tablespace	400MB
server parameter file	5MB
USERS tablespace	120MB
TEMP tablespace	100MB
UNDOTBS tablespace	625MB (total)
The DBCA uses automatic undo management by default. You should have one undo tablespace for each instance.
Note: Undo tablespace raw device data files for the preconfigured Transaction Processing and General Purpose database templates should be at least 200MB in size. If you use the preconfigured Data Warehousing database, then create a raw device data file for an undo tablespace for the first instance that is at least 625MB in size. For subsequent instances, create undo tablespace raw device data files that are at least 200MB in size. If you use the New Database template with automatic undo management, then divide the 625MB among the instances. If you have more than two instances, make sure that the undo tablespace raw device data files are at least 50MB in size. If you use manual undo management, make the RBS tablespace raw device data file at least 625MB in size.
EXAMPLE tablespace	160MB
CWMLITE tablespace	100MB
INDX tablespace	70MB
TOOLS tablespace	12MB
DRSYS tablespace	90MB
First control file	110MB
Second control file	110MB
Two redo log files for each instance	120MB (for each file)
srvcfg configuration raw device	100MB

Note: To expand a datafile after your Real Application Clusters database is in use, refer to your platform-specific documentation. Then to use the additional space, execute the ALTER DATABASE AUTOEXTEND ON MAXSIZE nM statement, where n is the maximum size for the tablespace.

Some operating systems require additional overhead for the partition sizes in Table 2-1. Refer to your operating system-specific documentation for the exact raw partition size requirements.

If you do not use the DBCA and instead create your database manually, the number of raw devices you create depends on the number of instances and database options that you install.

See Also: Chapter 5, "Manually Creating Real Application Clusters Databases" for more information about manual database creation

Configuring Logical Devices

The configuration of raw devices is operating system-specific as described in the following sections:

• Configuring Raw Volumes for Real Application Clusters on UNIX

• Configuring Logical Drives on Windows NT and Windows 2000

Refer to the section that corresponds to your operating system.

Configuring Raw Volumes for Real Application Clusters on UNIX

Use the following procedures to configure raw volumes on UNIX:

1. Obtain root privileges.

   See Also: Oracle9i Administrator's Reference for your UNIX operating system

2. Oracle Corporation recommends that you create the datafiles, control files, and redo log files using the file name format in Table 2-2. Although you can use any file naming scheme, the format in Table 2-2 simplifies administration by using names that identify the database with the db_name entry, and that also relate to the raw volume type:

   Table 2-2 Recommended Names for Oracle Database Files

   Example File Name	Raw Volume
   db_name_raw_system_400m	SYSTEM tablespace raw volume
   db_name_raw_spfile_5m	Server parameter file tablespace raw volume
   db_name_raw_users_120m	USERS tablespace raw volume
   db_name_raw_temp_100m	TEMP tablespace raw volume
   db_name_raw_undotbs1_312m	UNDOTBS 1 tablespace raw volume
   db_name_raw_undotbs2_312m	UNDOTBS 2 tablespace raw volume
   db_name_raw_example_160m	EXAMPLE tablespace raw volume
   db_name_raw_cwmlite_100m	CWMLITE (OLAP) tablespace raw volume
   db_name_raw_indx_70m	INDX tablespace raw volume
   db_name_raw_tools_12m	TOOLS tablespace raw volume
   db_name_raw_drsys_90m	DRSYS tablespace raw volume
   db_name_raw_controlfile1_110m	controlfile1 tablespace raw volume
   db_name_raw_controlfile2_110m	controlfile2 tablespace raw volume
   db_name_raw_thread_lognumber_120m	Two redo log files for each instance
   Where thread is the thread ID of the instance and number is the log number, 1 or 2, of the instance.

   Note: If you do not use automatic undo management, then to accommodate the RBS tablespace raw volume, substitute the entries for undotbs1 and undotbs2 with the following: db_name_raw_rbs_625m

If you select the General Purpose, Transaction Processing, or Data Warehouse database configuration type on the Universal Installer's Database Configuration screen, or if you select the Customized option on this screen and then use one of the DBCA templates that uses datafiles, then you must perform the following step:

3. On the node from which you run the Oracle Universal Installer, create an ASCII file identifying the raw volume objects as shown in Table 2-3. The DBCA requires that these objects exist during installation and database creation. When creating the ASCII file content for the objects, name them using the format:

   database_object=raw_device_file_path
   

Where database_object represents the raw volume object and raw_device_file_path is the path of the datafile, control file, or redo log file.

Table 2-3 UNIX ASCII File Contents for the Oracle Universal Installer

Database Object	Used For
system1	SYSTEM tablespace datafile
spfile1	Server parameter file
users1	USERS tablespace datafile
temp1	TEMP tablespace datafile
undotbs1	Undo tablespace 1 datafile
undotbs2	Undo tablespace 2 datafile
example1	EXAMPLE tablespace datafile
cwmlite1	CWMLITE (OLAP) tablespace datafile
indx1	INDX tablespace datafile
tools1	TOOLS tablespace datafile
drsys1	DRSYS tablespace datafile
control1	Control file 1 tablespace datafile
control2	Control file 2 tablespace datafile
redo1_1	First redo log file for the first instance
redo1_2	Second redo log file for the first instance
redo2_1	First redo log file for the second instance
redo2_2	Second redo log file for the second instance

Use the following format for your undo tablespace files:

undotbs_number

Where number is the number of the instance.

Use the following format for your redo log files:

redothread_number

Where thread is the thread ID of the instance and number is the log number (1 or 2) of the instance.

Note: You must create at least two redo log file entries for each instance. If you do not use automatic undo management, you must make substitute entries for undotbs1 and undotbs2 with an entry for rbs1 to use manual undo management.

When you create the ASCII file, separate the database objects from the paths with equals (=) signs as shown in Example 2-1:

Example 2-1 Example UNIX ASCII File for a Two-Instance General Purpose Real Application Clusters Database Configuration

system1=/dev/vx/rdsk/oracle_dg/clustdb_raw_system_400m
spfile1=/dev/vx/rdsk/oracle_dg/clustdb_raw_spfile_5m
users1=/dev/vx/rdsk/oracle_dg/clustdb_raw_users_120m
temp1=/dev/vx/rdsk/oracle_dg/clustdb_raw_temp_100m
undotbs1=/dev/vx/rdsk/oracle_dg/clustdb_raw_undotbs1_312m
undotbs2=/dev/vx/rdsk/oracle_dg/clustdb_raw_undotbs2_312m
example1=/dev/vx/rdsk/oracle_dg/clustdb_raw_example_160m 
cwmlite1=/dev/vx/rdsk/oracle_dg/clustdb_raw_cwmlite_100m 
indx1=/dev/vx/rdsk/oracle_dg/clustdb_raw_indx_70m
tools1=/dev/vx/rdsk/oracle_dg/clustdb_raw_tools_12m
drsys1=/dev/vx/rdsk/oracle_dg/clustdb_raw_dr_90m
control1=/dev/vx/rdsk/oracle_dg/clustdb_raw_controlfile1_110m
control2=/dev/vx/rdsk/oracle_dg/clustdb_raw_controlfile2_110m
redo1_1=/dev/vx/rdsk/oracle_dg/clustdb_raw_log11_120m
redo1_2=/dev/vx/rdsk/oracle_dg/clustdb_raw_log12_120m
redo2_1=/dev/vx/rdsk/oracle_dg/clustdb_raw_log21_120m
redo2_2=/dev/vx/rdsk/oracle_dg/clustdb_raw_log22_120m

Note: If you are not using automatic undo management, then substitute the entries for undotbs1 and undotbs2 with: rbs1=/dev/vx/rdsk/oracle_dg/clustdb_raw_rbs1_625m

Also make sure to use the following format for your redo log files:

redothread_number

Where thread is the thread ID of the instance and number is the log number (1 or 2) of the instance.

Note: You must create at least two redo log file entries and one undo tablespace for each instance.

You must specify that Oracle should use this file to determine the raw device volume names by setting the following environment variable where filename is the name of the ASCII file that contains the entries shown in Example 2-1:

setenv DBCA_RAW_CONFIG filename 

UNIX Preinstallation Steps

After configuring the raw volumes, perform the following steps prior to installation as root user:

Note: You cannot complete these preinstallation procedures until you configure your raw volumes.

1. Make sure you have an osdba group defined in the /etc/group file on all nodes of your cluster. To designate an osdba group name and group number and osoper group during installation, these group names must be identical on all nodes of your UNIX cluster that will be part of the Real Application Clusters database. The default UNIX group name for the osdba and osoper groups is dba.

2. Create an oracle account on each node so that the account:
   1. Is a member of the osdba group

   2. Is used only to install and update Oracle software

   3. Has write permissions on remote directories

3. Create a mount point directory on each node to serve as the top of your Oracle software directory structure so that:
   1. The name of the mount point on each node is identical to that on the initial node

   2. The oracle account has read, write, and execute privileges

4. On the node from which you will run the Oracle Universal Installer, set up user equivalence by adding entries for all nodes in the cluster, including the local node, to the .rhosts file of the oracle account, or the /etc/hosts.equiv file.

5. Exit the root account when you are done.

6. As oracle account user, check for user equivalence for the oracle account by performing a remote login (rlogin) to each node in the cluster.

7. As oracle account user, if you are prompted for a password, you have not given the oracle account the same attributes on all nodes. You must correct this because the Oracle Universal Installer cannot use the rcp command to copy Oracle products to the remote node's directories without user equivalence.

   Note: UNIX clusters also require an environment setup similar to single-instance environments. For these instructions and other operating system-specific Real Application Clusters preinstallation instructions, see the Oracle9i Installation Guide for your UNIX operating system.

After configuring your raw volumes, proceed to Chapter 4 to install the Oracle9i Enterprise Edition software and to configure your Oracle9i Real Application Clusters database.

Configuring Logical Drives on Windows NT and Windows 2000

Datafiles, control files, and redo log files must reside on unformatted raw devices on Windows NT and Windows 2000 platforms. On Windows, these are more commonly referred to as logical drives that reside within extended partitions. The extended partitions point to raw space on the disk. To configure the logical drives, create multiple logical partitions using Windows NT Disk Administrator or Windows 2000 Computer Manager.

Before creating the logical partitions, first create extended partitions that point to the raw space on the disk. Then create multiple logical partitions within the extended partitions and assign symbolic link names to them using the Object Link Manager (OLM).

See Also: Oracle9i Database installation guide for Windows for further information about configuring logical drives for Real Application Clusters on Windows NT and Windows 2000

The DBCA General Purpose, Transaction Processing, and Data Warehouse database configuration types require the symbolic link names shown in the left-hand column of Table 2-4 for a two-instance Real Application Clusters database:

Table 2-4 Symbolic Link Names for Preconfigured Database Types

Symbolic Link Name...	Used for...
db_name_system1	SYSTEM tablespace
db_name_spfile1	Server parameter file
db_name_users1	USERS tablespace
db_name_temp1	TEMP tablespace
db_name_undotbs1	First UNDO tablespace
db_name_undotbs2	Second UNDO tablespace
db_name_example1	EXAMPLE tablespace
db_name_cwmlite1	CWMLITE tablespace
db_name_indx1	INDX tablespace
db_name_tools1	TOOLS tablespace
db_name_drsys1	DRSYS tablespace
db_name_control1	First control file
db_name_control2	Second control file
srvcfg	SRVM Configuration Repository
db_name_redo_thread_number Where thread is the thread ID of the instance and number is the log number (1 or 2) for the instance.	redo log files Each instance must have two redo log files. If the database name is db, the link names for the first instance should be: db_redo1_1 db_redo1_2 The link names for the second instance should be: db_redo2_1 db_redo2_2

Note: If you do not use automatic undo management, then to accommodate a rollback segment tablespace, replace the db_name_undotbs1 and db_name_undotbs2 symbolic link names with: db_name_rbs1

To install the Oracle operating system-dependent clusterware, run the Oracle Cluster Setup Wizard. The Cluster Setup Wizard installs the Object Link Manager tool with which you create persistent symbolic links to the logical drives.

If you do not install the Oracle OSD clusterware, then copy the Object Link Manager software from the preinstall directory on the Oracle CD. Refer to the online document, the Oracle9i Database installation guide for Windows for further procedures for completing the configuration using OLM.

After configuring your logical drives, proceed to Chapter 3 to install the Oracle9i Enterprise Edition software and to configure your Real Application Clusters database.