|Oracle9i Real Application Clusters Setup and Configuration
Release 2 (9.2)
Part Number A96600-02
This chapter discusses considerations and procedures for manually creating Oracle Real Application Clusters databases. The topics in this chapter are:
This section describes the following
DATABASE options specific to Real Application Clusters.
Use this information when writing database creation scripts. A sample database creation script for Real Application Clusters databases appears in the script
clustdb.sql that resides in the
$ORACLE_HOME/srvm/admin directory on UNIX or in the
%ORACLE_HOME%\srvm\admin directory on Windows NT and Windows 2000 platforms.
MAXINSTANCES option of
DATABASE limits the number of instances that can access a database concurrently. For Real Application Clusters, set
MAXINSTANCES to a value greater than the maximum number of instances you expect to run concurrently.
MAXLOGFILES option of
DATABASE specifies the maximum number of redo log groups that can be created for the database. The
MAXLOGMEMBERS option specifies the maximum number of members or copies for each group. Set
MAXLOGFILES to the maximum number of threads possible, multiplied by the maximum anticipated number of groups for each thread multiplied by
MAXLOGMEMBERS for each group.
MAXLOGHISTORY clause of the
CREATE DATABASE statement specifies the maximum number of archived redo log files that can be recorded in the log history of the control file. The log history is used for automatic media recovery of Real Application Clusters databases.
For Real Application Clusters, set
MAXLOGHISTORY to a large value, such as 1000. The control files can then only store information for this number of redo log files, which in this case is 1000 files. When the log history exceeds this limit, Oracle overwrites the oldest entries. The default for
0 (zero), which disables log history.
This parameter is useful only if you are using Oracle in ARCHIVELOG mode with Real Application Clusters. Specify the maximum number of archived redo log files for automatic media recovery. Oracle uses this value to determine how much space in the control file to allocate for the names of archived redo log files. The minimum value is 0. The default value is a multiple of the
MAXINSTANCES value and depends on your operating system. The maximum value is limited only by the maximum size of the control file.
MAXDATAFILES option is generic, but Real Application Clusters databases tend to have more datafiles and log files than single-instance Oracle databases.
Create your database using the default of
NOARCHIVE log mode. This reduces system overhead during database creation. You can later implement archive logging using the
DATABASE statement with the
ARCHIVELOG option. Refer to "Setting the Log Mode" for information on setting the log mode.
Oracle9i Database Administrator's Guide for more information about archive logging
You can use the
CONTROLFILE statement to change the value of the following database parameters for a database:
Oracle9i SQL Reference for a description of the
To manually prepare a new database for Real Application Clusters, create and configure additional database objects as described under the following headings:
Oracle strongly recommends that you use automatic undo management. This feature automatically manages undo space. To use automatic undo management, use the
DATABASE statement with the
TABLESPACE clause to create an undo tablespace. You can also use the
TABLESPACE statement to create additional undo tablespaces for additional instances.
When you use the
DATABASE statement and you have enabled automatic undo management, if you do not specify the
TABLESPACE clause, then Oracle creates an undo tablespace by default. The name and size of the default file varies depending on your operating system. However, if you are not using raw devices, this automatically created file will be on your file system where it cannot be shared unless you are using a cluster file system. In this case, you must specify a shared raw device datafile name for the undo tablespace.
If you create your database in manual undo management mode, you must first create and bring online one additional rollback segment in the
SYSTEM tablespace before creating rollback segments in other tablespaces. The instance that creates the database can create this additional rollback segment and new tablespaces, but it cannot create database objects in non-system tablespaces until you bring the additional rollback segment online.
Then you must create at least two rollback segments for each Real Application Clusters instance. To avoid performance issues, create these rollback segments in a tablespace other than the
SYSTEM tablespace, for example, use the
Oracle Corporation strongly recommends against storing these rollback segments in the
Oracle9i Real Application Clusters Administration for more information about automatic undo management
If you cannot use automatic undo management, then you can manually create rollback segments. Real Application Clusters databases need at least as many rollback segments as the maximum number of concurrent instances plus one; the extra rollback segment is for the
SYSTEM rollback segment. An instance cannot start up in shared mode without exclusive access to at least one rollback segment, whether it is public or private.
You can create new rollback segments in any tablespace except for the temporary tablespace. To reduce performance issues between rollback data and table data, partition your rollback segments in a separate tablespace. This facilitates taking tablespaces offline; you cannot take a tablespace offline if it contains active rollback segments.
In general, make each rollback segment extent the same size by specifying identical values for the
NEXT storage parameters. To ensure you have correctly created the rollback segments, examine the data dictionary view
DBA_ROLLBACK_SEGS. This view shows each rollback segment's name, segment ID number, and owner (
PUBLIC or other).
The Oracle9i Database Administrator's Guide for information about rollback segment performance and for information on the implications of adding rollback segments
If you use manual undo management and manually manage rollback segments, Oracle Corporation recommends that you make the rollback segments private. This enables you to closely control which instances use which rollback segments. To do this follow these steps:
ROLLBACK_SEGMENTSparameter to specify the rollback segment in the server parameter file by naming it as a value for the parameter. For example,
RBS2). This reserves the two rollback segments, RBS1 and RBS 2, for the instance identified by the
ALTER ROLLBACK SEGMENTto bring the rollback segment online. You can also restart the instance to use the reserved rollback segment.
You should specify a particular private rollback segment in either the server parameter file with the appropriate instance identifier, or in only one instance-specific initialization parameter file so that the segment is associated with only one instance. If an instance attempts to acquire a public rollback segment that another instance has already acquired, then Oracle generates an error message and prevents the instance from starting up. Private rollback segments stay offline until brought online or until the owning instance restarts and acquires it.
Any instance can create public rollback segments and once created, public rollback segments are available for any instance. When an instance uses a rollback segment, the instance uses it exclusively until the instance shuts down. When the instance shuts down, the instance releases the rollback segment for use by other instances.
Use the SQL statement
CREATE PUBLIC ROLLBACK SEGMENT to create public rollback segments. Public rollback segments appear as
PUBLIC in the data dictionary view
DBA_ROLLBACK_SEGS. If you do not assign a rollback segment to an instance by setting a value for the
ROLLBACK_SEGMENTS parameter, then the instance uses public rollback segments. The procedures you use to create and manage rollback segments are the same regardless of whether you have enabled or disabled Real Application Clusters.
Typically, the parameter file does not specify public rollback segments because they are by default available to any instance needing them. However, if another instance is not already using a particular public rollback segment, then you can assign the rollback segment to the instance by specifying it in the
ROLLBACK_SEGMENTS parameter for that instance.
An instance brings a public rollback segment online when the instance acquires the rollback segment at startup. However, starting an instance that uses public rollback segments does not ensure that the instance uses a particular public rollback segment. The exception to this is when the instance acquires all available public rollback segments.
If you need to keep a public rollback segment offline and do not want to drop it and re-create the segment, then you must prevent other instances that require public rollback segments from starting up.
Oracle9i Database Administrator's Guide for more information about rollback segments
This section explains how to configure online redo threads for cluster database. Each instance has its own thread of online redo, consisting of its own online redo log groups. Oracle Corporation recommends that you create at least two members for each redo log group to prevent data loss. Create each thread with at least two redo log groups and enable each thread so the instance can use it.
For improved performance and to minimize the overhead of software mirroring, or multiplexing, put the members of each redo log group on separate physical disks or on separate disk arrays. The
DATABASE statement creates thread number 1 as a public thread and enables it automatically. Use the
ALTER DATABASE statement to create and enable subsequent threads.
Threads can be either public or private. The
THREAD initialization parameter assigns a unique thread number to an instance. If you set
THREAD to zero, which is the default, the instance acquires a public thread.
DATABASE statement creates thread number 1 as a public thread and enables it automatically. Subsequent threads must be created and enabled with the
DATABASE statement. For example, the following statements create and enable thread 2 with two groups of three members each.
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 3 (disk1_file4, disk2_file4, disk3_file4) SIZE 100M REUSE, GROUP 4 (disk1_file5, disk2_file5, disk3_file5) SIZE 100M REUSE; ALTER DATABASE ENABLE PUBLIC THREAD 2;
If you do not specify the
THREAD parameter in your initialization file, you must specify the
THREAD clause when creating new redo log groups. If you specify the
THREAD parameter, then you can omit the
THREAD clause when creating new redo log groups and the newly created redo log groups will be assigned to the thread of the instance that you used to create them.
Oracle9i Real Application Clusters Administration for more information about threads of redo
Disable a public or private thread with the
THREAD statement. You cannot disable a thread if an instance using the thread has the database mounted. To change a thread from public to private, or from private to public, disable the thread and then enable it. However, an instance cannot disable its own thread. The database must be open when you disable or enable a thread.
When you disable a thread with the database in
ARCHIVELOG mode, Oracle marks its current redo log file as needing to be archived. If you want to drop that file, you might need to first archive it manually.
An error or failure while a thread is being enabled can result in a thread that has a current set of log files but is not enabled. You cannot drop or archive these log files. In this case, disable the thread, even though it is already disabled, then re-enable it.
You typically set the redo log mode,
NOARCHIVELOG, immediately after you create your database. You can later change the archive mode using the
DATABASE statement. When archiving is enabled, online redo log files cannot be reused until they are archived.
The redo log mode is associated with the database rather than with individual instances. If the redo log is being used in ARCHIVELOG mode, for most purposes all instances should use the same archiving method, either automatic or manual.
To switch archiving modes:
falsein the parameter file.
To specify the archive log destinations on a per instance basis for a two-instance cluster database, for example, set the parameter as follows:
Ensure that these archive log destinations are not on a file system which is on a shared disk. If the archive log destinations are on a shared disk, then they should not be cross mounted across the nodes because this would corrupt the file system meta data and Oracle cannot use the archive logs for recovery.
If the archive destinations are on private disks, then these can be cross mounted. Ensure that each instance's archive log destination is mounted as read-write on its node and read only on other nodes. When one of the instances fails, mount its archive log destination as read-only onto the surviving instance's node if it has not been already mounted.
ALTER DATABASEstatement with either the
CLUSTER_DATABASEinitialization parameter set to true.
You can change the configuration of the redo log by adding, dropping, or renaming a log file or log file member while the database is mounted with Real Application Clusters either enabled or disabled. The only restriction is that you cannot drop or rename a log file or log file member currently in use by any thread. Moreover, you cannot drop a log file if that would reduce the number of log groups to less than two for the thread to which the log file is assigned.
Any instance can add or rename redo log files, or members, of any group for any other instance. As long as there are more than two groups for an instance, a redo log group can be dropped from that instance by any other instance. Changes to redo log files and log members take effect on the next log switch.
Oracle9i Real Application Clusters Administration for more information about archiving redo log files
Create your database manually if you already have scripts, or if you have database requirements that differ greatly from the types of databases that the DBCA creates as described in Chapter 4. The three primary steps for manually creating a database are:
Perform the following tasks before manually creating a Real Application Clusters database:
Perform the following tasks to manually create your database:
Oracle Corporation recommends that you review all the steps in this chapter before performing them.
Oracle strongly recommends that you make complete backups of all databases before creating a new database in case database creation accidentally affects existing files. Your backups should include parameter files, database files, redo log files, control files, and network configuration files.
In Real Application Clusters, each node typically has one instance. Being aware of database- and instance-level information enables you to more easily complete Tasks 2 through 12.
To determine database- and instance-level information:
The name of your database.
The domain name of your database.
Global Database Name
A name that comprises the database name and database domain.
A prefix for the Oracle system identifier (SID). The instance's thread number, or number of the redo thread assigned to the instance, is appended to the SID prefix to create the SID for the node's instance.
|Database Name||Database Domain||Global Database Name||SID Prefix|
The node name defined by the Cluster Manager (CM) software
On UNIX, Windows NT, and Windows 2000, use the command
Each instance requires a unique thread number. The thread ID is appended to the SID prefix to create the SID for the instance on the node. Threads are usually numbered sequentially beginning with
|Node Name||Host Name||Thread ID||SID|
If this is the first Oracle9i database created on this cluster database, then you must initialize the clusterwide SRVM configuration. Do this by executing the following command:
The first time you use the SRVCTL Utility to create the configuration, start the Global Services Daemon (GSD) on each node with the
start command so that SRVCTL can access your cluster configuration information. Then execute the
add command so that Server Management (SRVM) knows what instances belong to your cluster database. For example, if you are using the server parameter file, then execute the following command:
If your database does not have a domain name, then do not specify the
-m option. If you do not use the server parameter file, then do not specify the -s option. Then for each instance enter the command:
To use Oracle Enterprise Manager, manually create an entry in the oratab file on each node. This entry identifies the database. Oracle Enterprise Manager uses the information in this file during service discovery to determine the database name and the Oracle home from which the database runs.
oratab file is stored in
/var/opt/oracle/oratab, depending on your operating system. The syntax for this entry is as follows where db_name is your database's database name,
$ORACLE_HOME is the directory path to the database, and
N indicates that the database should not be started at restart time:
Use the database name and Oracle home you specified in "Task 2: Determine the Database and Instance Parameter Settings".
The following is an example entry for a database named
The SID must be defined for each node's instance in the cluster database, and the value you set for
ORACLE_SID must be unique for each instance. To simplify administration, Oracle Corporation recommends that you use SID
s that consist of the database name as the common base and the number of the thread assigned to the instance that you specified in "Task 2: Determine the Database and Instance Parameter Settings". For example, if
db is the database name, then the first instance in the cluster has a SID of
db1 and the second instance has a SID of
db2. The SID specification is operating system-specific as described under the following headings:
On UNIX, set the
ORACLE_SID environment variable.
Oracle9i Administrator's Reference for your UNIX operating system for further information about setting this environment variable
On Windows NT and Windows 2000, create an
ORACLE_SID Registry key under the following Registry key. Then set the value of your instance
SID in the
ORACLE_SID registry key:
Oracle9i Database Getting Started for Windows for further information about this Registry value
After creating the SIDs, create an OracleServicesid service. You can use this service to start or stop an instance from the Control Panel. To create OracleServicesid:
CRTSRVbatch file to create a unique service corresponding to the instance on the node.
For example, to create a service for a SID of
OracleServicedb1, enter the following:
When an Oracle instance starts, it refers to the parameter file for configuration information. Oracle Corporation recommends that you use a single server parameter file to designate both global and instance-specific settings. Using this type of parameter file greatly simplifies parameter administration.
Name the server parameter file initdbname.ora. To designate instance-specific settings in this file, use the SID prefix and place these entries after the generic, global entries. Specify instance-specific settings using the instance_name.parameter_name=value syntax.
You can also make parameter files for the database you are about to create by copying the initialization parameter file located in the
$ORACLE_HOME/srvm/admin directory on UNIX or by using the file in the
%ORACLE_HOME%\srvm\admin directory on Windows NT and Windows 2000. Rename this file and customize it for your database.
Examples of instance-specific settings in the server parameter file are:
REMOTE_LOGIN_PASSWORDFILEis set to
"Initialization Parameter Files" for further information about initialization parameter files and the parameters to set
Use the Password Utility
ORAPWD to create password files.
ORAPWD is automatically installed with the Oracle9i utilities. Password files are located in the
$ORACLE_HOME/dbs directory on UNIX and in the
%ORACLE_HOME%\database directory on Windows NT and Windows 2000. They are named
orapwsid on UNIX and
.ora on Windows NT and Windows 2000, where sid identifies the database instance you specified in "Task 2: Determine the Database and Instance Parameter Settings".
To create a password file on each node:
ORAPWDto create the password file.
$ORACLE_HOME/binwith the following syntax:
%ORACLE_HOME%\binwith the following syntax:
FILE specifies the password file name and
PASSWORD sets the password for the
CREATE DATABASE script on one of the nodes by using the
clustdb.sql sample script, located in the
$ORACLE_HOME/srvm/admin directory on UNIX or in the
%ORACLE_HOME%\srvm\admin directory on Windows NT and Windows 2000. The sample script is for a two-node cluster. If you use the sample script, edit the following:
CONNECT SYS/oracle AS SYSDBAline to use the password you created in "Task 7: Create the Password Files". You must also be a member of the
$ORACLE_HOME/rdbms/adminon UNIX and
%ORACLE_HOME%\rdms\adminon Windows NT and Windows 2000, to reflect the Oracle home you specified in "Task 2: Determine the Database and Instance Parameter Settings".
SYSTEMrollback segment, instances cannot share public rollback segments. An instance explicitly acquires private rollback segments when it opens a database.
To create the new database, run the
CREATE DATABASE SQL script (
clustdb.sql) from the SQL*Plus prompt:
The location of the
clustdb.sql script is
$ORACLE_HOME/srvm/admin on UNIX and
%ORACLE_HOME%\srvm\admin on Windows NT and Windows 2000 platforms. When you execute this script, Oracle creates the following:
SYSTEMtablespace and the
Then Oracle mounts and opens the local database instance for use.
Make a full backup of the database to ensure you have a complete set of files from which to recover in case of media failure.
|Configuration File||Description||Configuration Requirements|
Includes addresses of each network listener on a server, the SIDs of the databases for which they listen, and various control parameters used by the listener.
Includes the names resolution method.
See Also: "Profile (sqlnet.ora File)" for a sample configuration
Oracle9i Net Services Administrator's Guide for information about creating these files
Because you manually created your Real Application Clusters database, you must run the
CATCLUST.SQL script to create Real Application Clusters-related views and tables. You must have
SYSDBA privileges to run this script.
Oracle9i Database Reference for more information about dynamic performance views
If you did not enable operating system authentication for the database server, then Oracle uses a password file for each instance of a Real Application Clusters database to authenticate user access to the database. Because these password files are private to each instance, you must administer passwords and roles in Real Application Clusters as follows:
USERSQL statement on all instances of your cluster database.
SYSOPERRoles--If you grant or revoke the
SYSOPERroles to a user, then you must execute the same
REVOKE ROLESQL statement on all instances of the cluster database.