Skip Headers
Oracle® Real Application Clusters Installation and Configuration Guide
10g Release 1 (10.1.0.3) for Linux x86-64
Part No. B14406-01
  Go To Table Of Contents
Contents
Go To Index
Index

Previous
Previous
Next
Next
 

2 Pre-Installation Tasks for Installing RAC on Linux-Based Systems

This chapter describes the tasks that you must complete before you start the Oracle Universal Installer (OUI). It includes information about the following tasks:

Log In to the System as root

Before you install the Oracle software, you must complete several tasks as the root user. To log in as the root user, complete one of the following procedures:


Note:

Unless you intend to complete a silent-mode installation, you must install the software from an X Window System workstation, an X terminal, or a PC or other system with X server software installed.

For more information about silent-mode installations, see Appendix B.


Check the Hardware Requirements

Each system must meet the following minimum hardware requirements:

To ensure that each system meets these requirements, follow these steps:

  1. To determine the physical RAM size, enter the following command:

    # grep MemTotal /proc/meminfo
    
    

    If the size of the physical RAM installed in the system is less than the required size, you must install more memory before continuing.

  2. To determine the size of the configured swap space, enter the following command:

    # grep SwapTotal /proc/meminfo
    
    

    If necessary, see your operating system documentation for information about how to configure additional swap space.

  3. To determine the amount of disk space available in the /tmp directory, enter the following command:

    # df -h /tmp
    
    

    If there is less than 400 MB of disk space available in the /tmp directory, complete one of the following steps:

    • Delete unnecessary files from the /tmp directory to achieve the required disk space.

    • Set the TEMP and TMPDIR environment variables when setting the oracle user's environment (described later).

    • Extend the file system that contains the /tmp directory. If necessary, contact your system administrator for information about extending file systems.

  4. To determine the amount of free disk space on the system, enter the following command:

    # df -h
    
    
  5. To determine whether the system architecture can run the software, enter the following command:

    # grep "model name" /proc/cpuinfo
    
    

    This command displays the processor type. Verify that the processor architecture matches the Oracle software release that you want to install.


    Note:

    If you do not see the expected output, then you cannot install the software on this system.

Check the Network Requirements

Check that you have the networking hardware and internet protocol (IP) addresses required for an Oracle Real Application Clusters installation.


Note:

For the most up-to-date information about supported network protocols and hardware for RAC installations, see the Certify pages on the OracleMetaLink Web site:

http://metalink.oracle.com


Network Hardware Requirements

Each node in the cluster must meet the following requirements:

IP Address Requirements

Before starting the installation, you must identify or obtain the following IP addresses for each node:

For example, if each node has two public and two private interfaces, you might have the following host names and IP addresses on one of the nodes (rac1) and similar host names and IP addresses on the other nodes:

Host Name Type IP Address Registered In
rac1.mydomain.com Public 143.47.43.100 DNS
rac1-2.mydomain.com Public 143.46.51.101 DNS
rac1-vip.mydomain.com Virtual 143.47.43.104 DNS
rac1-priv1 Private 10.0.0.1 /etc/hosts
rac1-priv2 Private 10.0.0.2 /etc/hosts

Checking the Network Requirements

To verify that each node meets the requirements, follow these steps:

  1. If necessary, install the network adapters for the public and private networks and configure them with either public or private IP addresses.

  2. Register the host names and IP addresses for the public network interfaces in DNS.

  3. For each node, register one virtual host name and IP address in DNS.

  4. For each private interface on every node, add a line similar to the following to the /etc/hosts file on all nodes, specifying the private IP address and associated private host name:

    10.0.0.1     rac1-priv1
    
    
  5. To identify the interface name and associated IP address for every network adapter, enter the following command:

    # /sbin/ifconfig
    
    

    From the output, identify the interface name and IP address for all network adapters that you want to specify as public or private network interfaces.


    Note:

    When you install Oracle CRS and RAC, you will require this information.

Check the Software Requirements

Refer to the following section for information about checking the software requirements.


Note:

The Oracle Universal Installer performs checks on your system to verify that it meets the requirements listed for your platform. To ensure that these checks pass, verify the requirements before you start the Installer.

Checking the Software Requirements

Check that the required software and patches are installed on the system.

Check for Required Software

Depending on the products that you intend to install, verify that the following software is installed on the system. The procedure following the table describes how to check these requirements.

Installation Type or Product Requirement
All installations One of the following operating system versions:
  • Red Hat Enterprise Linux 3 (update 2)

  • SuSE Linux Enterprise Server (SLES) 8 with service pack 3 or later

  • SuSE Linux Enterprise Server 9

All installations The system must be running the following kernel version (or a later version):

Red Hat Enterprise Linux 3 (update 2):

2.4.21-15.EL

SuSE Linux Enterprise Server 8 (x86-64):

2.4.21-185

SuSE Linux Enterprise Server 9:

2.6.5-7.97

All installations The following packages (or later versions) must be installed:

Red Hat Enterprise Linux 3 (update 2):

make-3.79.1
gcc-3.2.3-34
glibc-2.3.2-95.20
glibc-devel-2.3.2-95.20
glibc-devel-2.3.2-95.20 (32 bit)
compat-db-4.0.14-5
compat-gcc-7.3-2.96.128
compat-gcc-c++-7.3-2.96.128
compat-libstdc++-7.3-2.96.128
compat-libstdc++-devel-7.3-2.96.128
openmotif21-2.1.30-8
setarch-1.3-1
gnome-libs-1.4.1.2.90-34.1 (32 bit)
libaio-0.3.96-3
libaio-devel-0.3.96-3

SuSE Linux Enterprise Server 8 (x86-64):

make-3.79.1
gcc-3.3-43
gcc-c++-3.3-43
glibc-2.2.5-213
glibc-32bit-8.1-9
glibc-devel-32bit-8.1-9
openmotif-2.2.2-124
libaio-0.3.96-3
libaio-devel-0.3.96-3

SuSE Linux Enterprise Server 9:

gcc-3.3.3-43
gcc-c++-3.3.3-43
glibc-2.3.3-98
libaio-0.3.98-18
libaio-devel-0.3.98-18
make-3.80
openmotif-libs-2.2.2-519.1
Oracle Spatial X Window System development package:
  • Red Hat:

    XFree86
    XFree86-devel
    
  • SuSE:

    xf86
    xdevel
    

Note: This software is required only to build the sample programs.

Pro*C/C++, Oracle Call Interface, Oracle C++ Call Interface, Oracle XML Developer's Kit (XDK) The version of GNU C and C++ compilers listed previously for your distribution is supported for use with these products.

Oracle JDBC/OCI Drivers On x86-64 systems only, you can use the following optional JDK version with the Oracle JDBC/OCI drivers, however it is not required for the installation:
  • Sun JDK 1.4.2_03 with the JNDI extension

Real Application Clusters Oracle Cluster File System (OCFS) version 1.0.11-1 or later:

Note: OCFS is required only if you want to use a cluster file system for database file storage. If you want to use Automatic Storage Management or raw devices for database file storage, you do not need to install OCFS.

ocfs-supportocfs-toolsocfs-kernel_version


To ensure that the system meets these requirements, follow these steps:

  1. To determine which distribution and version of Linux is installed, enter the following command:

    # cat /etc/issue
    
    

    Note:

    Only the distributions and versions listed in the previous table are supported. Do not install the software on other versions of Linux.

  2. To determine whether the required kernel version is installed, enter the following command:

    # uname -r
    
    

    If the kernel version is lower than the required version, download and install the required version or a later version from your Linux vendor's Web site.

  3. To determine whether the required packages are installed, enter commands similar to the following:

    # rpm -q package_name
    
    

    If a package is not installed, install it from your Linux distribution media or download the required package version from your Linux vendor's Web site.

  4. To determine whether OCFS is installed, enter the following command:

    # rpm -qa | grep ocfs
    
    

    If you want to install the Oracle database files on an OCFS file system and the packages are not installed, download them from the following Web site. Follow the instructions listed with the kit to install the packages and configure the file system:

    http://oss.oracle.com/projects/ocfs/

  5. If you require a CSD for WebSphere MQ, refer to the following Web site for download and installation information:

    http://www.ibm.com/software/integration/mqfamily/support/summary/lin.html

Create Required UNIX Groups and User

Depending on whether this is the first time Oracle software is being installed on this system and on the products that you are installing, you may need to create several UNIX groups and a UNIX user.

The following UNIX groups and user are required if you are installing Oracle Database:

The following UNIX group and user are required for all installations:

A single Oracle Inventory group is required for all installations of Oracle software on the system. After the first installation of Oracle software, you must use the same Oracle Inventory group for all subsequent Oracle software installations on that system. However, you can choose to create different Oracle software owner users, OSDBA groups, and OSOPER groups (other than oracle, dba, and oper) for separate installations. By using different groups for different installations, members of these different groups have DBA privileges only on the associated databases rather than on all databases on the system.


See Also:

The Oracle Database Administrator's Reference for UNIX Systems and the Oracle Database Administrator's Guide contain more information about the OSDBA and OSOPER groups and the SYSDBA and SYSOPER privileges.

The following sections describe how to create the required UNIX user and groups.


Note:

The following sections describe how to create local users and groups. As an alternative to creating local users and groups, you could create the appropriate users and groups in a directory service, for example, Network Information Services (NIS). For information about using directory services, contact your system administrator or see your operating system documentation.

Creating the Oracle Inventory Group

You must create the Oracle Inventory group if it does not already exist. The following subsections describe how to determine the Oracle Inventory group name, if it exists, and how to create it if necessary.

Determine Whether the Oracle Inventory Group Exists

When you install Oracle software on the system for the first time, the Installer creates the oraInst.loc file. This file identifies the name of the Oracle Inventory group and the path of the Oracle Inventory directory. To determine whether the Oracle Inventory group exists, enter the following command:

# more /etc/oraInst.loc

If the oraInst.loc file exists, the output from this command is similar to the following:

inventory_loc=/u01/app/oracle/oraInventory
inst_group=oinstall

The inst_group parameter shows the name of the Oracle Inventory group (oinstall).

Create the Oracle Inventory Group

If the oraInst.loc file does not exist, enter the following command to create the oinstall group:

# /usr/sbin/groupadd oinstall

Creating the OSDBA Group

You must create an OSDBA group in the following circumstances:

  • An OSDBA group does not exist, for example, if this is the first installation of Oracle Database software on the system

  • An OSDBA group exists, but you want to give a different group of UNIX users database administrative privileges in a new Oracle installation

If the OSDBA group does not exist or if you require a new OSDBA group, enter the following command to create the dba group. Use the group name dba unless a group with that name already exists.

# /usr/sbin/groupadd dba

Creating an OSOPER Group (Optional)

Create an OSOPER group only if you want to identify a group of UNIX users with a limited set of database administrative privileges (SYSOPER operator privileges). For most installations, it is sufficient to create only the OSDBA group. If you want to use an OSOPER group, you must create it in the following circumstances:

  • If an OSOPER group does not exist, for example, if this is the first installation of Oracle Database software on the system

  • If an OSOPER group exists, but you want to give a different group of UNIX users database operator privileges in a new Oracle installation

If you require an new OSOPER group, enter the following command to create the oper group. Use the group name oper unless a group with that name already exists.

# /usr/sbin/groupadd oper

Creating the Oracle Software Owner User

You must create an Oracle software owner user in the following circumstances:

  • If an Oracle software owner user does not exist, for example, if this is the first installation of Oracle software on the system

  • If an Oracle software owner user exists, but you want to use a different UNIX user, with different group membership, to give database administrative privileges to those groups in a new Oracle Database installation

Determine Whether an Existing Oracle Software Owner User Exists

To determine whether an Oracle software owner user named oracle exists, enter the following command:

# id oracle

If the oracle user exists, the output from this command is similar to the following:

uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)

If the user exists, determine whether you want to use the existing user or create a new user. If you want to use the existing user, ensure that the user's primary group is the Oracle Inventory group and that it is a member of the appropriate OSDBA and OSOPER groups. Refer to one of the following sections for more information:


Note:

If necessary, contact your system administrator before using or modifying an existing user.

Create a New Oracle Software Owner User

If the Oracle software owner user does not exist or if you require a new Oracle software owner user, create it as follows. Use the user name oracle unless a user with that name already exists.

  1. To create the oracle user, enter a command similar to the following:

    # /usr/sbin/useradd -g oinstall -G dba[,oper] oracle
    
    

    In this command:

    • The -g option specifies the primary group, which must be the Oracle Inventory group, for example oinstall

    • The -G option specifies the secondary groups, which must include the OSDBA group and if required, the OSOPER group, for example dba or dba,oper

  2. Set the password of the oracle user:

    # passwd oracle
    
    

Refer to the "Verifying that the UNIX User nobody Exists" section to continue.

Modify an Existing Oracle Software Owner User

If the oracle user exists, but its primary group is not oinstall or it is not a member of the appropriate OSDBA or OSOPER groups, enter a command similar to the following to modify it. Specify the primary group using the -g option and any required secondary groups using the -G option:

# /usr/sbin/usermod -g oinstall -G dba[,oper] oracle

Verifying that the UNIX User nobody Exists

Before installing the software, verify that the UNIX user nobody exists on the system:

  1. To determine whether the user exists, enter the following command:

    # id nobody
    
    

    If this command displays information about the nobody user, you do not have to create that user.

  2. If the nobody user does not exist, enter the following command to create it:

    # /usr/sbin/useradd nobody
    
    
  3. Repeat this procedure on all of the other cluster nodes.

Create Identical Users and Groups on Other Cluster Nodes


Note:

You must complete the following procedures only if you are using local users and groups. If you are using users and groups defined in a directory service such as NIS, they are already identical on each cluster node.

The Oracle software owner user and the Oracle Inventory, OSDBA, and OSOPER groups must exist and be identical on all cluster nodes. To create these identical users and groups, you must identify the user ID and group IDs assigned them on the node where you created them, then create the user and groups with the same name and ID on the other cluster nodes.

Identifying the User and Group IDs

To determine the user ID (UID) of the Oracle software owner user and the group IDs (GID) of the Oracle Inventory, OSDBA, and OSOPER groups, follow these steps:

  1. Enter following command:

    # id oracle
    
    

    The output from this command is similar to the following:

    uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)
    
    
  2. From the output, identify the UID for the oracle user and the GIDs for the groups to which it belongs.

Create the User and Groups on the Other Cluster Nodes

To create the user and groups on the other cluster nodes, repeat the following procedure on each node:

  1. Log in to the next cluster node as root.

  2. Enter commands similar to the following to create the oinstall and dba groups, and if required, the oper group. Use the -g option to specify the correct GID for each group.

    # /usr/sbin/groupadd -g 200 oinstall
    # /usr/sbin/groupadd -g 201 dba
    # /usr/sbin/groupadd -g 202 oper
    
    

    Note:

    If the group already exists, use the groupmod command to modify it if necessary. If you cannot use the same group ID for a particular group on this node, view the /etc/group file on all nodes to identify a group ID that is available on every node. You must then specify that ID for the group on all of the nodes.

  3. To create the oracle user, enter a command similar to the following:

    # /usr/sbin/useradd -u 200 -g oinstall -G dba[,oper] oracle
    
    

    In this command:

    • The -u option specifies the user ID, which must be the user ID that you identified in the previous subsection

    • The -g option specifies the primary group, which must be the Oracle Inventory group, for example oinstall

    • The -G option specifies the secondary groups, which must include the OSDBA group and if required, the OSOPER group, for example dba or dba,oper


      Note:

      If the user already exists, use the usermod command to modify it if necessary. If you cannot use the same user ID for the oracle user on this node, view the /etc/passwd file on all nodes to identify a user ID that is available on every node. You must then specify that ID for the user on all of the nodes.

  4. Set the password of the oracle user:

    # passwd oracle
    
    

Configure SSH on All Cluster Nodes

Before you install and use Oracle Real Application clusters, you must configure secure shell (SSH) for the oracle user on all cluster nodes. The Installer uses the ssh and scp commands during installation to run remote commands on and copy files to the other cluster nodes. You must configure SSH so that these commands do not prompt for a password.


Note:

This section describes how to configure OpenSSH version 3. If SSH is not available, the Installer attempts to use rsh and rcp instead. However, these services are disabled by default on most Linux systems.

To configure SSH, complete the following steps on each cluster node:

  1. Log in as the oracle user.

  2. If necessary, create the .ssh directory in the oracle user's home directory and set the correct permissions on it:

    $ mkdir ~/.ssh
    $ chmod 755 ~/.ssh
    
    
  3. Enter the following commands to generate an RSA key for version 2 of the SSH protocol:

    $ /usr/bin/ssh-keygen -t rsa
    
    

    At the prompts:

    • Accept the default location for the key file

    • Enter and confirm a pass phrase that is different from the oracle user's password

    This command writes the public key to the ~/.ssh/id_rsa.pub file and the private key to the ~/.ssh/id_rsa file. Never distribute the private key to anyone.

  4. Enter the following commands to generate a DSA key for version 2 of the SSH protocol:

    $ /usr/bin/ssh-keygen -t dsa
    
    

    At the prompts:

    • Accept the default location for the key file

    • Enter and confirm a pass phrase that is different from the oracle user's password

    This command writes the public key to the ~/.ssh/id_dsa.pub file and the private key to the ~/.ssh/id_dsa file. Never distribute the private key to anyone.

  5. Copy the contents of the ~/.ssh/id_rsa.pub and ~/.ssh/id_dsa.pub files to the ~/.ssh/authorized_keys file on this node and to the same file on all other cluster nodes.


    Note:

    The ~/.ssh/authorized_keys file on every node must contain the contents from all of the ~/.ssh/id_rsa.pub and ~/.ssh/id_dsa.pub files that you generated on all cluster nodes.

  6. Change the permissions on the ~/.ssh/authorized_keys file on all cluster nodes:

    $ chmod 644 ~/.ssh/authorized_keys
    
    

At this point, if you use ssh to log in to or run a command on another node, you are prompted for the pass phrase that you specified when you created the DSA key.

To enable the Installer to use the ssh and scp commands without being prompted for a pass phrase, follow these steps:

  1. On the system where you want to install the software, log in as the oracle user.

  2. Enter the following commands:

    $ exec /usr/bin/ssh-agent $SHELL
    $ /usr/bin/ssh-add
    
    
  3. At the prompts, enter the pass phrase for each key that you generated.

    If you have configured SSH correctly, you can now use the ssh or scp commands without being prompted for a password or a pass phrase.

  4. To test the SSH configuration, enter the following commands from the same terminal session, testing the configuration of each cluster node:

    $ ssh nodename1 date
    $ ssh nodename2 date
        .
        .
    
    

    These commands should display the date set on each node. If any node prompts for a password or pass phrase, verify that the ~/.ssh/authorized_keys file on that node contains the correct public keys.


    Note:

    The first time you use SSH to connect to a node from a particular system, you might see a message stating that the authenticity of the host could not be established. Enter yes at the prompt to continue. You should not see this message again when you connect from this system to that node.

    If you see any other messages or text, apart from the date, the installation might fail. Make any changes required to ensure that only the date is displayed when you enter these commands.


  5. To ensure that X11 forwarding will not cause the installation to fail, create a user-level SSH client configuration file for the Oracle software owner user, as follows:

    1. Using any text editor, edit or create the ~oracle/.ssh/config file.

    2. Make sure that the ForwardX11 attribute is set to no, for example:

      Host *
            ForwardX11 no
      
      
  6. You must run the Installer from this session or remember to repeat steps 2 and 3 before you start the Installer from a different terminal session.

Configure Kernel Parameters and Shell Limits

Refer to the following section for information about configuring the system kernel parameters and shell limits:

Configuring Kernel Parameters


Note:

The kernel parameter and shell limit values shown in the following sections are recommended values only. For production database systems, Oracle recommends that you tune these values to optimize the performance of the system. See your operating system documentation for more information about tuning kernel parameters.

On all cluster nodes, verify that the kernel parameters shown in the following table are set to values greater than or equal to the recommended value shown. The procedure following the table describes how to verify and set the values.

Parameter Value File
semmsl semmns semopm semmni 250 32000 100 128 /proc/sys/kernel/sem
shmall 2097152 /proc/sys/kernel/shmall
shmmax Half the size of physical memory (in bytes) /proc/sys/kernel/shmmax
shmmni 4096 /proc/sys/kernel/shmmni
file-max 65536 /proc/sys/fs/file-max
ip_local_port_range 1024 65000 /proc/sys/net/ipv4/ip_local_port_range
rmem_default 262144 /proc/sys/net/core/rmem_default
rmem_max 262144 /proc/sys/net/core/rmem_max
wmem_default 262144 /proc/sys/net/core/wmem_default
wmem_max 262144 /proc/sys/net/core/wmem_max


Note:

If the current value for any parameter is higher than the value listed in this table, do not change the value of that parameter.

To view the current value specified for these kernel parameters, and to change them if necessary, follow these steps:

  1. Enter the commands shown in the following table to view the current values of the kernel parameters:


    Note:

    Make a note of the current values and identify any values that you must change.

    Parameter Command
    semmsl, semmns, semopm, and semmni # /sbin/sysctl -a | grep sem

    This command displays the value of the semaphore parameters in the order listed.

    shmall, shmmax, and shmmni # /sbin/sysctl -a | grep shm
    file-max # /sbin/sysctl -a | grep file-max
    ip_local_port_range # /sbin/sysctl -a | grep ip_local_port_range

    This command displays a range of port numbers.

    rmem_default, rmem_max, wmem_default, and wmem_max # /sbin/sysctl -a | grep net.core

  2. If the value of any kernel parameter is different to the recommended value, complete the following steps:

    1. Using any text editor, create or edit the /etc/sysctl.conf file and add or edit lines similar to the following:


      Note:

      Include lines only for the kernel parameter values that you want to change. For the semaphore parameters (kernel.sem), you must specify all four values. However, if any of the current values are larger than the recommended value, specify the larger value.

      kernel.shmall = 2097152
      kernel.shmmax = 2147483648
      kernel.shmmni = 4096
      kernel.sem = 250 32000 100 128
      fs.file-max = 65536
      net.ipv4.ip_local_port_range = 1024 65000
      net.core.rmem_default = 262144
      net.core.rmem_max = 262144
      net.core.wmem_default = 262144
      net.core.wmem_max = 262144
      
      

      By specifying the values in the /etc/sysctl.conf file, they persist when you restart the system.

    2. Enter the following command to change the current values of the kernel parameters:

      # /sbin/sysctl -p
      
      

      Review the output from this command to verify that the values are correct. If the values are incorrect, edit the /etc/sysctl.conf file, then enter this command again.

    3. On SuSE systems only, enter the following command to cause the system to read the /etc/sysctl.conf file when it reboots:

      # /sbin/chkconfig boot.sysctl on
      
      
  3. Repeat this procedure on all other cluster nodes.

Set Shell Limits for the oracle User

To improve the performance of the software on Linux systems, you must increase the following shell limits for the oracle user:

Shell Limit Item in limits.conf Hard Limit
Maximum number of open file descriptors nofile 65536
Maximum number of processes available to a single user nproc 16384

To increase the shell limits:

  1. Add the following lines to /etc/security/limits.conf file:

    oracle              soft    nproc   2047
    oracle              hard    nproc   16384
    oracle              soft    nofile  1024
    oracle              hard    nofile  65536
    
    
  2. Add or edit the following line in the /etc/pam.d/login file, if it does not already exist:

    session    required     /lib/security/pam_limits.so
    
    
  3. Depending on the oracle user's default shell, make the following changes to the default shell start-up file:

    • For the Bourne, Bash, or Korn shell, add the following lines to the /etc/profile file (or the /etc/profile.local file on SuSE systems):

      if [ $USER = "oracle" ]; then
              if [ $SHELL = "/bin/ksh" ]; then
                    ulimit -p 16384
                    ulimit -n 65536
              else
                    ulimit -u 16384 -n 65536
              fi
      fi
      
      
    • For the C shell (csh or tcsh), add the following lines to the /etc/csh.login file (or the /etc/csh.login.local file on SuSE systems):

      if ( $USER == "oracle" ) then
              limit maxproc 16384
              limit descriptors 65536
      endif
      
      
  4. Repeat this procedure on all other cluster nodes.

Identify Required Software Directories

You must identify or create four directories for the Oracle software, as follows:

The following subsections describe the requirements for these directories.

Oracle Base Directory

The Oracle base directory acts as a top-level directory for Oracle software installations. It is analogous to the C:\Oracle directory used for Oracle software installations on Windows systems. On Linux systems, the Optimal Flexible Architecture (OFA) guidelines recommend that you use a path similar to the following for the Oracle base directory:

/mount_point/app/oracle_sw_owner

In this example:

You can use the same Oracle base directory for more than one installation or you can create separate Oracle base directories for different installations. If different UNIX users install Oracle software on the same system, each user must create a separate Oracle base directory. The following example Oracle base directories could all exist on the same system:

/u01/app/oracle
/u01/app/orauser
/opt/oracle/app/oracle

The following sections describe how to identify existing Oracle base directories that might be suitable for your installation and how to create a new Oracle base directory if necessary.

Regardless of whether you create a new Oracle base directory or decide to use an existing one, you must set the ORACLE_BASE environment variable to specify the full path to this directory.


Note:

The Oracle base directory can be on a local file system or on an NFS file system on a certified NAS device. Do not create the Oracle base directory on an OCFS version 1 file system.

Oracle Inventory Directory

The Oracle Inventory directory (oraInventory) stores an inventory of all software installed on the system. It is required by, and shared by, all Oracle software installations on a single system. The first time you install Oracle software on a system, the Installer prompts you to specify the path to this directory. If you are installing the software on a local file system, Oracle recommends that you choose the following path:

oracle_base/oraInventory

The Installer creates the directory that you specify and sets the correct owner, group, and permissions on it. You do not need to create it.


Note:

All Oracle software installations rely on this directory. Make sure that you back it up regularly.

Do not delete this directory unless you have completely removed all Oracle software from the system.


CRS Home Directory

The CRS home directory is the directory where you choose to install the software for Oracle Cluster Ready Services. You must install CRS in a separate home directory. When you run the Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. The directory that you specify must be a subdirectory of the Oracle base directory. Oracle recommends that you specify a path similar to the following for the CRS home directory:

/u01/crs/oracle/product/10.1.0/crs

If the Oracle base directory is on a cluster file system or an NFS file system on a NAS device, you must specify a path for the Oracle Inventory directory on a local file system. The Oracle base directory must be on a local file system to enable all of the nodes to have separate inventories.

The Installer creates the directory path that you specify under the Oracle base directory. It also sets the correct owner, group, and permissions on it. You do not need to create this directory.


Note:

Because you must change the permissions of all of the parent directories of the CRS home directory after installing the software to grant write access only to the root user, the CRS home directory must not be a subdirectory of the Oracle base directory.

Oracle Home Directory

The Oracle home directory is the directory where you choose to install the software for a particular Oracle product. You must install different Oracle products, or different releases of the same Oracle product, in separate Oracle home directories. When you run the Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. Oracle recommends that you specify a path similar to the following for the Oracle home directory:

oracle_base/product/10.1.0/db_1

The Installer creates the directory path that you specify under the Oracle base directory. It also sets the correct owner, group, and permissions on it. You do not need to create this directory.

Identify or Create an Oracle Base Directory

Before starting the installation, you must either identify an existing Oracle base directory or if required, create a new one. This section contains information about the following:


Note:

You can choose to create a new Oracle base directory, even if other Oracle base directories exist on the system.

Identifying an Existing Oracle Base Directory

Existing Oracle base directories might not have paths that comply with OFA guidelines. However, if you identify an existing Oracle Inventory directory or existing Oracle home directories, you can usually identify the Oracle base directories, as follows:

Before deciding to use an existing Oracle base directory for this installation, make sure that it satisfies the following conditions:

To continue:

Creating a New Oracle Base Directory

Before you create a new Oracle base directory, you must identify an appropriate file system with sufficient free disk space, as follows:

Requirement Free Disk Space
The Oracle base directory will contain only software files. Up to 3 GB
The Oracle base directory will contain both software and database files (not recommended for production databases). Up to 4 GB

To identify an appropriate file system, follow these steps:

  1. Use the df -h command to determine the free disk space on each mounted file system.

  2. From the display, identify a file system that has appropriate free space.


    Note:

    The file system can be a local file system or an NFS file system on a certified NAS device. Do not create the Oracle base directory on an OCFS file system.

    The path to the Oracle base directory must be the same on all nodes.

  3. Note the name of the mount point directory for the file system that you identified.

To create the Oracle base directory and specify the correct owner, group, and permissions for it, follow these steps:

  1. Enter commands similar to the following to create the recommended subdirectories in the mount point directory that you identified and set the appropriate owner, group, and permissions on them:

    # mkdir -p /mount_point/app/oracle_sw_owner
    # chown -R oracle:oinstall /mount_point/app/oracle_sw_owner
    # chmod -R 775 /mount_point/app/oracle_sw_owner
    
    

    If the mount point you identified is /u01 and oracle is the user name of the Oracle software owner, the recommended Oracle base directory path is as follows:

    /u01/app/oracle
    
    
  2. If necessary, repeat the commands listed in the previous step to create the same directory on the other cluster nodes.

  3. When you are configuring the oracle user's environment later in this chapter, set the ORACLE_BASE environment variable to specify this directory.

Create the CRS Home Directory

You must create a CRS home directory before installing Oracle CRS. You can create the CRS home directory on the same file system as the Oracle base directory, or you can choose a different file system for this directory. If you choose the same file system as the Oracle base directory, you must not use a subdirectory of the Oracle base directory as the CRS home directory.

To identify an appropriate file system, follow these steps:

  1. Use the df -h command to determine the free disk space on each mounted file system.

  2. From the display, identify a file system that has at least 1 MB of free disk space.

    If you are using the same file system for the Oracle base directory, this 1 MB of disk space is additional to the free disk space requirement that you identified previously.


    Note:

    The file system can be a local file system or an NFS file system on a certified NAS device. Do not create the CRS home directory on an OCFS file system.

    The path to the CRS home directory must be the same on all nodes.

  3. Note the name of the mount point directory for the file system that you identified.

To create the CRS home directory and specify the correct owner, group, and permissions for it, follow these steps:

  1. Enter commands similar to the following to create the recommended subdirectories in the mount point directory that you identified and set the appropriate owner, group, and permissions on them:

    # mkdir -p /mount_point/crs/oracle_sw_owner/product/10.1.0/crs
    # chown -R oracle:oinstall /mount_point/crs/oracle_sw_owner
    # chmod -R 775 /mount_point/crs/oracle_sw_owner
    
    

    If the mount point you identified is /u01 and oracle is the user name of the Oracle software owner, the recommended CRS home directory path is as follows:

    /u01/crs/oracle/product/10.1.0/crs
    
    
  2. If necessary, repeat the commands listed in the previous step to create the same directory on the other cluster nodes.

  3. When you are installing Oracle CRS, set the ORACLE_HOME environment variable to specify this directory.


    Note:

    During the installation, before you run the root.sh script, you must change the permissions on the parent directories of the CRS home directory to permit only the root user to write to those directories.

Choose a Storage Option for Oracle CRS, Database, and Recovery Files

The following table shows the storage options supported for storing Oracle Cluster Ready Services (CRS) files, Oracle database files, and Oracle database recovery files. Oracle database files include datafiles, control files, redo log files, the server parameter file, and the password file. Oracle CRS files include the Oracle Cluster Registry (OCR) and the CRS voting disk.

For all installations, you must choose the storage option that you want to use for Oracle CRS files and Oracle database files. If you want to enable automated backups during the installation, you must also choose the storage option that you want to use for recovery files (the flash recovery area). You do not have to use the same storage option for each file type.


Note:

For the most up-to-date information about supported storage options for RAC installations, see the Certify pages on the OracleMetaLink Web site:

http://metalink.oracle.com


Storage Option File Types Supported
CRS Database Recovery
Automatic Storage Management No Yes Yes
Cluster file system (OCFS) Yes Yes Yes
NFS file system

Note: Requires a certified NAS device

Yes Yes Yes
Shared raw partitions Yes Yes No

Use the following guidelines when choosing the storage options that you want to use for each file type:

For information about how to configure disk storage before you start the installation, see one of the following sections depending on your choice:

Create Directories for Oracle CRS, Database, or Recovery Files

If you decide to place the Oracle CRS, database, or recovery files on a file system, use the following guidelines when deciding where to place them.

Guidelines for Placing Oracle CRS Files on a File System

The Installer does not suggest a default location for the Oracle Cluster Registry (OCR) or the Oracle CRS voting disk. If you choose to create these files on a file system, use the following guidelines when deciding where to place them:


Note:

If you are upgrading from Oracle9i Release 2, you can continue to use the raw device or shared file that you used for the SRVM configuration repository instead of creating a new file for the OCR.

Guidelines for Placing Oracle Database Files on a File System

If you choose to place the Oracle database files on a file system, use the following guidelines when deciding where to place them:

Guidelines for Placing Oracle Recovery Files on a File System


Note:

You must choose a location for recovery files only if you intend to enable automated backups during the installation.

If you choose to place the Oracle recovery files on a file system, use the following guidelines when deciding where to place them:

Creating Required Directories


Note:

You must complete this procedure only if you want to place the Oracle CRS, database, or recovery files on a separate file system to the Oracle base directory.

To create directories for the Oracle CRS, database, or recovery files on separate file systems to the Oracle base directory, follow these steps:

  1. If necessary, configure the shared file systems that you want to use and mount them on each node.


    Note:

    The mount point that you use for the file system must be identical on each node. Make sure that the file systems are configured to mount automatically when a node reboots.

  2. Use the df -h command to determine the free disk space on each mounted file system.

  3. From the display, identify the file systems that you want to use:

    File Type File System Requirements
    CRS files Choose a file system with at least 120 MB of free disk space
    Database files Choose either:
    • A single file system with at least 1.2 GB of free disk space

    • Two or more file systems with at least 1.2 GB of free disk space in total

    Recovery files Choose a file system with at least 2 GB of free disk space.

    If you are using the same file system for more than one type of file, add the disk space requirements for each type to determine the total disk space requirement.

  4. Note the names of the mount point directories for the file systems that you identified.

  5. Enter commands similar to the following to create the recommended subdirectories in each of the mount point directories and set the appropriate owner, group, and permissions on them:

    • CRS file directory:

      # mkdir /mount_point/oracrs
      # chown oracle:oinstall /mount_point/oracrs
      # chmod 775 /mount_point/oracrs
      
      
    • Database file directory:

      # mkdir /mount_point/oradata
      # chown oracle:oinstall /mount_point/oradata
      # chmod 775 /mount_point/oradata
      
      
    • Recovery file directory (flash recovery area):

      # mkdir /mount_point/flash_recovery_area
      # chown oracle:oinstall /mount_point/flash_recovery_area
      # chmod 775 /mount_point/flash_recovery_area
      
      
  6. If you also want to use ASM or raw devices for storage, see one of the following sections:

    Otherwise, refer to the "Verify that the Required Software is Running" section.

Configure Disks for Automatic Storage Management

This section describes how to configure disks for use with ASM. Before you configure the disks, you must determine the number of disks and the amount of free disk space that you require. The following sections describe how to identify the requirements and configure the disks:


Note:

Although this section refers to disks, you can also use zero-padded files on a certified NAS storage device in an ASM disk group. Refer to the Oracle Database Installation Guide for Linux x86-64 for information about creating and configuring NAS-based files for use in an ASM disk group.

Identifying Storage Requirements for ASM

To identify the storage requirements for using ASM, you must determine how many devices and the amount of free disk space that you require. To complete this task, follow these steps:

  1. Determine whether you want to use ASM for Oracle database files, recovery files, or both.


    Note:

    You do not have to use the same storage mechanism for database files and recovery files. You can use the file system for one file type and ASM for the other.

    For RAC installations, if you choose to enable automated backups and you do not have a shared file system available, you must choose ASM for recovery file storage.


    If you enable automated backups during the installation, you can choose ASM as the storage mechanism for recovery files by specifying an ASM disk group for the flash recovery area. Depending on how you choose to create a database during the installation, you have the following options:

    • If you select an installation method that runs DBCA in interactive mode, by choosing the Advanced database configuration option for example, you can decide whether you want to use the same ASM disk group for database files and recovery files, or you can choose to use different disk groups for each file type.

      The same choice is available to you if you use DBCA after the installation to create a database.

    • If you select an installation method that runs DBCA in non-interactive mode, you must use the same ASM disk group for database files and recovery files.

  2. Choose the ASM redundancy level that you want to use for the ASM disk group.

    The redundancy level that you choose for the ASM disk group determines how ASM mirrors files in the disk group and determines the number of disks and amount of disk space that you require, as follows:

    • External redundancy

      An external redundancy disk group requires a minimum of one disk device. The effective disk space in an external redundancy disk group is the sum of the disk space in all of its devices.

      Because ASM does not mirror data in an external redundancy disk group, Oracle recommends that you use only RAID or similar devices that provide their own data protection mechanisms as disk devices in this type of disk group.

    • Normal redundancy

      In a normal redundancy disk group, ASM uses two-way mirroring by default, to increase performance and reliability. A normal redundancy disk group requires a minimum of two disk devices (or two failure groups). The effective disk space in a normal redundancy disk group is half the sum of the disk space in all of its devices.

      For most installations, Oracle recommends that you use normal redundancy disk groups.

    • High redundancy

      In a high redundancy disk group, ASM uses three-way mirroring to increase performance and provide the highest level of reliability. A high redundancy disk group requires a minimum of three disk devices (or three failure groups). The effective disk space in a high redundancy disk group is one-third the sum of the disk space in all of its devices.

      While high redundancy disk groups do provide a high level of data protection, you must consider the higher cost of additional storage devices before deciding to use this redundancy level.

  3. Determine the total amount of disk space that you require for the database files and recovery files.

    Use the following table to determine the minimum number of disks and the minimum disk space requirements for the installation:

    Redundancy Level Minimum Number of Disks Database Files Recovery Files Both File Types
    External 1 1.15 GB 2.3 GB 3.45 GB
    Normal 2 2.3 GB 4.6 GB 6.9 GB
    High 3 3.45 GB 6.9 GB 10.35 GB

    For RAC installations, you must also add additional disk space for the ASM metadata. You can use the following formula to calculate the additional disk space requirements (in MB):

    15 + (2 * number_of_disks) + (126 * number_of_ASM_instances)

    For example, for a four-node RAC installation, using three disks in a high redundancy disk group, you require an additional 525 MB of disk space:

    15 + (2 * 3) + (126 * 4) = 525

    If an ASM instance is already running on the system, you can use an existing disk group to meet these storage requirements. If necessary, you can add disks to an existing disk group during the installation.

    The following section describes how to identify existing disk groups and determine the free disk space that they contain.

  4. Optionally identify failure groups for the ASM disk group devices.


    Note:

    You need to complete this step only if you intend to use an installation method that runs DBCA in interactive mode, for example, if you intend to choose the Custom installation type or the Advanced database configuration option. Other installation types do not enable you to specify failure groups.

    If you intend to use a normal or high redundancy disk group, you can further protect your database against hardware failure by associating a set of disk devices in a custom failure group. By default, each device comprises its own failure group. However, if two disk devices in a normal redundancy disk group are attached to the same SCSI controller, the disk group becomes unavailable if the controller fails. The controller in this example is a single point of failure.

    To protect against failures of this type, you could use two SCSI controllers, each with two disks, and define a failure group for the disks attached to each controller. This configuration would enable the disk group to tolerate the failure of one SCSI controller.


    Note:

    If you define custom failure groups, you must specify a minimum of two failure groups for normal redundancy disk groups and three failure groups for high redundancy disk groups.

  5. If you are sure that a suitable disk group does not exist on the system, install or identify appropriate disk devices to add to a new disk group. Use the following guidelines when identifying appropriate disk devices:

    • All of the devices in an ASM disk group should be the same size and have the same performance characteristics.

    • Do not specify more than one partition on a single physical disk as a disk group device. ASM expects each disk group device to be on a separate physical disk.

    • Although you can specify a logical volume as a device in an ASM disk group, Oracle does not recommend their use. Logical volume managers can hide the physical disk architecture, preventing ASM from optimizing I/O across the physical devices.

    For information about completing this task, refer to the "Configuring Disks for ASM" section.

Using an Existing ASM Disk Group

If you want to store either database or recovery files in an existing ASM disk group, you have the following choices, depending on the installation method that you select:

  • If you select an installation method that runs DBCA in interactive mode, by choosing the Advanced database configuration option for example, you can decide whether you want to create a new disk group or use an existing one.

    The same choice is available to you if you use DBCA after the installation to create a database.

  • If you select an installation method that runs DBCA in non-interactive mode, you must choose an existing disk group for the new database; you cannot create a new disk group. However, you can add disk devices to an existing disk group if it has insufficient free space for your requirements.


Note:

The ASM instance that manages the existing disk group can be running in a different Oracle home directory.

To determine whether an existing ASM disk group exists, or to determine whether there is sufficient disk space in a disk group, you can use Oracle Enterprise Manager Grid Control or Database Control. Alternatively, you can use the following procedure:

  1. View the contents of the oratab file to determine whether an ASM instance is configured on the system:

    # more /etc/oratab
    
    

    If an ASM instance is configured on the system, the oratab file should contain a line similar to the following:

    +ASM:oracle_home_path:N
    
    

    In this example, +ASM is the system identifier (SID) of the ASM instance and oracle_home_path is the Oracle home directory where it is installed. By convention, the SID for an ASM instance begins with a plus sign.

  2. Set the ORACLE_SID and ORACLE_HOME environment variables to specify the appropriate values for the ASM instance that you want to use.

  3. Connect to the ASM instance as the SYS user with SYSDBA privilege and start the instance if necessary:

    # $ORACLE_HOME/bin/sqlplus "SYS/SYS_password as SYSDBA"
    SQL> STARTUP
    
    
  4. Enter the following command to view the existing disk groups, their redundancy level, and the amount of free disk space in each one:

    SQL> SELECT NAME,TYPE,TOTAL_MB,FREE_MB FROM V$ASM_DISKGROUP;
    
    
  5. From the output, identify a disk group with the appropriate redundancy level and note the free space that it contains.

  6. If necessary, install or identify the additional disk devices required to meet the storage requirements listed in the previous section.


    Note:

    If you are adding devices to an existing disk group, Oracle recommends that you use devices that have the same size and performance characteristics as the existing devices in that disk group.

Configuring Disks for ASM

Oracle provides an ASM library driver that you can use to improve the I/O performance of databases that use ASM for storage management. If you intend to use ASM for database storage, Oracle recommends that you install the ASM library driver and associated utilities and use them to configure the devices that you want to include in an ASM disk group. If you do not use the ASM library driver, you must bind each disk device that you want to use to a raw device. The following sections describe how to configure disks using each method:


Note:

If you choose to configure disks using the ASM library driver, you must use Database Configuration Assistant (DBCA) in interactive mode to create the database. You can run DBCA in interactive mode by choosing the Custom installation type or the Advanced database configuration option. You must also change the default disk discovery string to ORCL:*.

Configuring Disks for ASM Using the ASM Library Driver

To use the ASM library driver to configure ASM devices, complete the following tasks.

Install and Configure the ASM Library Driver Software

To install and configure the ASM library driver software, follow these steps:

  1. Enter the following command to determine the kernel version and architecture of the system:

    # uname -rm
    
    
  2. If necessary, download the required ASM library driver packages from the OTN Web site:

    http://www.oracle.com/technology/software/tech/linux/asmlib/index.html

    You must install the following three packages, where version is the version of the ASM library driver, arch is the system architecture, and kernel is the version of the kernel that you are using:

    oracleasm-support-version.arch.rpm
    oracleasm-kernel-version.arch.rpm
    oracleasmlib-version.arch.rpm
    
    
  3. Switch user to the root user:

    $ su -
    
    
  4. Enter a command similar to the following to install the packages:

    # rpm -Uvh oracleasm-support-version.arch.rpm \
               oracleasm-kernel-version.arch.rpm \
               oracleasmlib-version.arch.rpm
    
    

    For example, if you are using the Red Hat Enterprise Linux AS 2.1 enterprise kernel on an x86 system, enter a command similar to the following:

    # rpm -Uvh oracleasm-support-1.0.0-1.i386.rpm \
               oracleasm-2.4.9-e-enterprise-1.0.0-1.i686.rpm \
               oracleasmlib-1.0.0-1.i386.rpm
    
    
  5. Enter a command similar to the following to determine the UID of the Oracle software owner user that you are using for this installation (typically oracle) and the GID of the OSDBA group (typically dba):

    # id oracle
    
    
  6. Enter the following command to run the oracleasm initialization script with the configure option:

    # /etc/init.d/oracleasm configure
    
    
  7. Enter the following information in response to the prompts that the script displays:

    Prompt Suggested Response
    Default UID to own the driver interface: Specify the UID of the Oracle software owner user (oracle).
    Default GID to own the driver interface: Specify the GID of the OSDBA group (dba).
    Start Oracle ASM Library driver on boot (y/n): Enter y to start the Oracle ASM library driver when the system starts.

    The script completes the following tasks:

    • Creates the /etc/sysconfig/oracleasm configuration file

    • Creates the /dev/oracleasm mount point

    • Loads the oracleasm kernel module

    • Mounts the ASM library driver file system


      Note:

      The ASM library driver file system is not a regular file system. It is used only by the ASM library to communicate with the ASM driver.

  8. Repeat this procedure on all cluster nodes where you want to install RAC.

Configure the Disk Devices to Use the ASM Library Driver

To configure the disk devices that you want to use in an ASM disk group, follow these steps:

  1. If necessary, install the shared disks that you intend to use for the disk group and restart the system.

  2. To identify the device name for the disks that you want to use, enter the following command:

    # /sbin/fdisk -l
    
    

    Depending on the type of disk, the device name can vary:

    Disk Type Device Name Format Description
    IDE disk
    /dev/hdxn
    
    In this example, x is a letter that identifies the IDE disk and n is the partition number. For example, /dev/hda is the first disk on the first IDE bus.
    SCSI disk
    /dev/sdxn
    
    In this example, x is a letter that identifies the SCSI disk and n is the partition number. For example, /dev/sda is the first disk on the first SCSI bus.
    RAID disk
    /dev/rd/cxdypz
    /dev/ida/cxdypz
    
    Depending on the RAID controller, RAID devices can have different device names. In the examples shown, x is a number that identifies the controller, y is a number that identifies the disk, and z is a number that identifies the partition. For example, /dev/ida/c0d1 is the second logical drive on the first controller.

    To include devices in a disk group, you can specify either whole-drive device names or partition device names.


    Note:

    On Linux systems, Oracle recommends that you create a single whole-disk partition on each disk that you want to use.

  3. Using either fdisk or parted, create a single whole-disk partition on the disk device that you want to use.

  4. Enter a command similar to the following to mark a disk as an ASM disk:

    # /etc/init.d/oracleasm createdisk disk1 /dev/sdb1
    
    

    In this example, disk1 is a name that you want to assign to the disk.


    Note:

    If you are using a multi-pathing disk driver with ASM, make sure that you specify the correct logical device name for the disk.

  5. To make the disk available on the other cluster nodes, enter the following command as root on each node:

    # /etc/init.d/oracleasm scandisks
    
    

    This command identifies shared disks attached to the node that are marked as ASM disks.


Note:

To create a database during the installation using the ASM library driver, you must choose an installation method that runs DBCA in interactive mode, for example by choosing the Custom installation type or the Advanced database configuration option. You must also change the default disk discovery string to ORCL:*.

Administer the ASM Library Driver and Disks

To administer the ASM library driver and disks, use the oracleasm initialization script with different options, as follows:

Option Description
configure
Use the configure option to reconfigure the ASM library driver, if necessary:
# /etc/init.d/oracleasm configure
enable
disable
Use the disable and enable options to change the behavior of the ASM library driver when the system boots. The enable option causes the ASM library driver to load when the system boots:
# /etc/init.d/oracleasm enable
start
stop
restart
Use the start, stop, and restart options to load or unload the ASM library driver without rebooting the system:
# /etc/init.d/oracleasm restart
createdisk
Use the createdisk option to mark a disk device for use with the ASM library driver and give it a name:
# /etc/init.d/oracleasm createdisk diskname devicename
deletedisk
Use the deletedisk option to unmark a named disk device:
# /etc/init.d/oracleasm deletedisk diskname

Caution: Do not use this command to unmark disks that are being used by an ASM disk group. You must drop the disk from the ASM disk group before you unmark it.

querydisk
Use the querydisk option to determine whether a disk device or disk name is being used by the ASM library driver:
# /etc/init.d/oracleasm querydisk {diskname | devicename}
listdisks
Use the listdisks option to list the disk names of marked ASM library driver disks:
# /etc/init.d/oracleasm listsdisks
scandisks
Use the scandisks option to enable cluster nodes to identify which shared disks have been marked as ASM library driver disks on another node:
# /etc/init.d/oracleasm scandisks

Configuring Disks for ASM Using Raw Devices


Note:

For improved performance and easier administration, Oracle recommends that you use the ASM library driver rather than raw devices to configure ASM disks.

To configure disks for ASM using raw devices, follow these steps:

  1. If necessary, install the shared disks that you intend to use for the disk group and reboot the system.

  2. To identify the device names for the disks that you want to use, enter the following command:

    # /sbin/fdisk -l
    
    

    Depending on the type of disk, the device name can vary:

    Disk Type Device Name Format Description
    IDE disk
    /dev/hdxn
    
    In this example, x is a letter that identifies the IDE disk and n is the partition number. For example, /dev/hda is the first disk on the first IDE bus.
    SCSI disk
    /dev/sdxn
    
    In this example, x is a letter that identifies the SCSI disk and n is the partition number. For example, /dev/sda is the first disk on the first SCSI bus.
    RAID disk
    /dev/rd/cxdypz
    /dev/ida/cxdypz
    
    Depending on the RAID controller, RAID devices can have different device names. In the examples shown, x is a number that identifies the controller, y is a number that identifies the disk, and z is a number that identifies the partition. For example, /dev/ida/c0d1 is the second logical drive on the first controller.

    To include devices in a disk group, you can specify either whole-drive device names or partition device names.


    Note:

    On Linux systems, Oracle recommends that you create a single whole-disk partition on each disk that you want to use.

  3. Using either fdisk or parted, create a single whole-disk partition on the disk devices that you want to use.

  4. On Red Hat systems, complete the following steps on each node to bind the disk devices to raw devices:


    Note:

    If the nodes are configured differently, the disk device names might be different on some nodes. In the following procedure, make sure that you specify the correct disk device names on each node.

    1. To determine what raw devices are already bound to other devices, enter the following command on every node:

      # /usr/bin/raw -qa
      
      

      Raw devices have device names in the form /dev/raw/rawn, where n is a number that identifies the raw device.

      For each device that you want to include in the disk group, identify a raw device name that is unused on all nodes.

    2. Open the /etc/sysconfig/rawdevices file in any text editor and add a line similar to the following for each device that you want to include in a disk group:

      /dev/raw/raw1 /dev/sdb1
      
      

      Note:

      If you are using a multi-pathing disk driver with ASM, make sure that you specify the correct logical device name for the disk.

      Specify an unused raw device for each disk device.

    3. For each raw device that you specified in the rawdevices file, enter commands similar to the following to set the owner, group, and permissions on the device file:

      # chown oracle:dba /dev/raw/rawn
      # chmod 660 /dev/raw/rawn
      
      
    4. To bind the disk devices to the raw devices, enter the following command:

      # /sbin/service rawdevices restart
      
      

      The system automatically binds the devices listed in the rawdevices file when it restarts.

  5. On SuSE systems, complete the following steps on each node to bind the disk devices to raw devices:


    Note:

    If the nodes are configured differently, the disk device names might be different on some nodes. In the following procedure, make sure that you specify the correct disk device names on each node.

    1. To determine what raw devices are already bound to other devices, enter the following command on every node:

      # /usr/sbin/raw -qa
      
      

      Raw devices have device names in the form /dev/raw/rawn, where n is a number that identifies the raw device.

      For each device that you want to include in the disk group, identify a raw device name that is unused on all nodes.

    2. Open the /etc/raw file in any text editor and add a line similar to the following for each device that you want to include in a disk group:

      raw1:sdb1
      
      

      Note:

      If you are using a multi-pathing disk driver with ASM, make sure that you specify the correct logical device name for the disk.

      Specify an unused raw device for each disk device.

    3. For each raw device that you specified in the /etc/raw file, enter commands similar to the following to set the owner, group, and permissions on the device file:

      # chown oracle:dba /dev/raw/rawn
      # chmod 660 /dev/raw/rawn
      
      
    4. To bind the disk devices to the raw devices, enter the following command:

      # /etc/init.d/raw start
      
      
    5. To ensure that the raw devices are bound when the system restarts, enter the following command:

      # /sbin/chkconfig raw on
      
      
  6. If you also want to use raw devices for storage, see the "Configure Raw Partitions" section.

    Otherwise, see the "Verify that the Required Software is Running" section.

Configure Raw Partitions

Refer to the following section for information about configuring raw devices.


Note:

If you are using ASM for database file storage, you need only create raw devices for the Oracle CRS files. However, if a cluster file system is available on your platform, Oracle recommends that you use that file system to store the Oracle CRS files instead of using raw devices for them.

Configuring Raw Partitions

The following subsections describe how to configure raw partitions on Linux.

Review Important Information

The procedures contained in this section describe how to create raw partitions for Oracle CRS and database file storage. Although Red Hat Enterprise Linux 3 and SuSE Linux Enterprise Server 8 provide a Logical Volume Manager (LVM), this LVM is not cluster aware. For this reason, Oracle does not support the use of logical volumes with RAC for either CRS or database files.

Create Raw Partitions

To create the required raw partitions, follow these steps:

  1. If necessary, install the shared disks that you intend to use and restart the system.


    Note:

    Because the number of partitions that you can create on a single device is limited, you might need to create the required raw partitions on more than one device.

  2. To identify the device name for the disks that you want to use, enter the following command:

    # /sbin/fdisk -l
    
    

    Depending on the type of disk, the device name can vary:

    Disk Type Device Name Format Description
    IDE disk
    /dev/hdxn
    
    In this example, x is a letter that identifies the IDE disk and n is the partition number. For example, /dev/hda is the first disk on the first IDE bus.
    SCSI disk
    /dev/sdxn
    
    In this example, x is a letter that identifies the SCSI disk and n is the partition number. For example, /dev/sda is the first disk on the first SCSI bus.
    RAID disk
    /dev/rd/cxdypz
    /dev/ida/cxdypz
    
    Depending on the RAID controller, RAID devices can have different device names. In the examples shown, x is a number that identifies the controller, y is a number that identifies the disk, and z is a number that identifies the partition. For example, /dev/ida/c0d1 is the second logical drive on the first controller.

    You can create the required raw partitions either on new devices that you added or on previously partitioned devices that have unpartitioned free space. To identify devices that have unpartitioned free space, examine the start and end cylinder numbers of the existing partitions and determine whether the device contains unused cylinders.

  3. To create raw partitions on a device, enter a command similar to the following:

    # /sbin/fdisk devicename
    
    

    Use the following guidelines when creating partitions:

    • Use the p command to list the partition table of the device.

    • Use the n command to create a new partition.

    • After you have created the required partitions on this device, use the w command to write the modified partition table to the device.

    • Refer to the fdisk man page for more information about creating partitions.

    Table 2-1 lists the number and size of the raw disk devices that you must configure for database files. Table 2-2 lists the number and size of the raw disk devices that you must configure for CRS files.

    Table 2-1 Raw Partitions Required for Database Files on Linux

    Number Partition Size (MB) Purpose
    1 500 SYSTEM tablespace
    1 300 + (Number of instances * 250) SYSAUX tablespace
    Number of instances 500 UNDOTBSn tablespace (One tablespace for each instance)
    1 250 TEMP tablespace
    1 160 EXAMPLE tablespace
    1 120 USERS tablespace
    2 * number of instances 120 Two online redo log files for each instance
    2 110 First and second control files
    1 5 Server parameter file (SPFILE)
    1 5 Password file


    Note:

    If you prefer to use manual, instead of automatic, undo management, create a single RBS tablespace raw device at least 500 MB in size instead of the UNDOTBSn raw devices.

    Table 2-2 Raw Partitions Required for CRS Files on Linux

    Number Partition Size (MB) Purpose
    1 100 Oracle Cluster Registry

    Note: You need to create this raw partition only once on the cluster. If you create more than one database on the cluster, they all share the same Oracle Cluster Registry (OCR).

    If you are upgrading from Oracle9i Release 2, you can continue to use the raw device that you used for the SRVM configuration repository instead of creating this new raw device.

    1 20 Oracle CRS voting disk

    Note: You need to create this raw partition only once on the cluster. If you create more than one database on the cluster, they all share the same Oracle CRS voting disk.


Bind the Partitions to Raw Devices

After you have created the required partitions, you must bind the partitions to raw devices on every node. However, you must first determine what raw devices are already bound to other devices. The procedure that you must follow to complete this task varies, depending on the Linux distribution that you are using:


Note:

If the nodes are configured differently, the disk device names might be different on some nodes. In the following procedure, make sure that you specify the correct disk device names on each node.

  • Red Hat:

    1. To determine what raw devices are already bound to other devices, enter the following command on every node:

      # /usr/bin/raw -qa
      
      

      Raw devices have device names in the form /dev/raw/rawn, where n is a number that identifies the raw device.

      For each device that you want to use, identify a raw device name that is unused on all nodes.

    2. Open the /etc/sysconfig/rawdevices file in any text editor and add a line similar to the following for each partition that you created:

      /dev/raw/raw1 /dev/sdb1
      
      

      Specify an unused raw device for each partition.

    3. For the raw device that you created for the Oracle Cluster Registry, enter commands similar to the following to set the owner, group, and permissions on the device file:

      # chown root:oinstall /dev/raw/rawn
      # chmod 640 /dev/raw/rawn
      
      
    4. For each other raw device that you specified in the rawdevices file, enter commands similar to the following to set the owner, group, and permissions on the device file:

      # chown oracle:dba /dev/raw/rawn
      # chmod 660 /dev/raw/rawn
      
      
    5. To bind the partitions to the raw devices, enter the following command:

      # /sbin/service rawdevices restart
      
      

      The system automatically binds the devices listed in the rawdevices file when it restarts.

    6. Repeat steps 2 to 5 on the other cluster nodes.

  • SuSE:

    1. To determine what raw devices are already bound to other devices, enter the following command on every node:

      # /usr/sbin/raw -qa
      
      

      Raw devices have device names in the form /dev/raw/rawn, where n is a number that identifies the raw device.

      For each device that you want to use, identify a raw device name that is unused on all nodes.

    2. Open the /etc/raw file in any text editor and add a line similar to the following to associate each partition with an unused raw device:

      raw1:sdb1
      
      
    3. For the raw device that you created for the Oracle Cluster Registry, enter commands similar to the following to set the owner, group, and permissions on the device file:

      # chown root:oinstall /dev/raw/rawn
      # chmod 640 /dev/raw/rawn
      
      
    4. For each other raw device that you specified in the /etc/raw file, enter commands similar to the following to set the owner, group, and permissions on the device file:

      # chown oracle:dba /dev/raw/rawn
      # chmod 660 /dev/raw/rawn
      
      
    5. To bind the partitions to the raw devices, enter the following command:

      # /etc/init.d/raw start
      
      
    6. To ensure that the raw devices are bound when the system restarts, enter the following command:

      # /sbin/chkconfig raw on
      
      
    7. Repeat steps 2 to 6 on the other cluster nodes.

Create the DBCA Raw Device Mapping File


Note:

You must complete this procedure only if you are using raw devices for database files. You do not specify the raw devices for the Oracle CRS files in the DBCA raw device mapping file.

To allow Database Configuration Assistant (DBCA) to identify the appropriate raw device for each database file, you must create a raw device mapping file, as follows:

  1. Set the ORACLE_BASE environment variable to specify the Oracle base directory that you identified or created previously:

    • Bourne, Bash, or Korn shell:

      $ ORACLE_BASE=/u01/app/oracle ; export ORACLE_BASE
      
      
    • C shell:

      % setenv ORACLE_BASE /u01/app/oracle
      
      
  2. Create a database file subdirectory under the Oracle base directory and set the appropriate owner, group, and permissions on it:

    # mkdir -p $ORACLE_BASE/oradata/dbname
    # chown -R oracle:oinstall $ORACLE_BASE/oradata
    # chmod -R 775 $ORACLE_BASE/oradata
    
    

    In this example, dbname is the name of the database that you chose previously.

  3. Change directory to the $ORACLE_BASE/oradata/dbname directory.

  4. Edit the dbname_raw.conf file in any text editor to create a file similar to the following:


    Note:

    The following example shows a sample mapping file for a two-instance RAC cluster.

    system=/dev/raw/raw1
    sysaux=/dev/raw/raw2
    example=/dev/raw/raw3
    users=/dev/raw/raw4
    temp=/dev/raw/raw5
    undotbs1=/dev/raw/raw6
    undotbs2=/dev/raw/raw7
    redo1_1=/dev/raw/raw8
    redo1_2=/dev/raw/raw9
    redo2_1=/dev/raw/raw10
    redo2_2=/dev/raw/raw11
    control1=/dev/raw/raw12
    control2=/dev/raw/raw13
    spfile=/dev/raw/raw14
    pwdfile=/dev/raw/raw15
    
    

    Use the following guidelines when creating or editing this file:

    • Each line in the file must have the following format:

      database_object_identifier=raw_device_path
      
      
    • For a RAC database, the file must specify one automatic undo tablespace datafile (undotbsn) and two redo log files (redon_1, redon_2) for each instance.

    • Specify at least two control files (control1, control2).

    • To use manual instead of automatic undo management, specify a single RBS tablespace datafile (rbs) instead of the automatic undo management tablespace datafiles.

  5. Save the file and note the file name that you specified.

  6. When you are configuring the oracle user's environment later in this chapter, set the DBCA_RAW_CONFIG environment variable to specify the full path to this file.

Verify that the Required Software is Running

Before installing Oracle Real Application Clusters on Linux systems, verify that the hangcheck-timer module is loaded and configured correctly. If you are using Oracle Cluster File System for Linux, you must also verify that it is configured correctly. The following sections describe how to complete these tasks.

Check the Configuration of the hangcheck-timer Module

The hangcheck-timer module monitors the Linux kernel for extended operating system hangs that could affect the reliability of a RAC node and cause a database corruption. If a hang occurs, the module restarts the node in seconds.

You can use the hangcheck_tick and hangcheck_margin parameters to control the behavior of the module, as follows:

If the kernel fails to respond within the sum of the hangcheck_tick and hangcheck_margin parameter values, the hangcheck-timer module restarts the system. Using the default values, the node would be restarted if the kernel fails to respond within 240 seconds.

To verify that the hangcheck-timer module is running on every node:

  1. Enter the following command on each node to determine which kernel modules are loaded:

    # /sbin/lsmod
    
    
  2. If the hangcheck-timer module is not listed for any node, enter a command similar to the following to start the module on that node:

    # /sbin/insmod hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
    
    
  3. To ensure that the module is loaded every time the system restarts, verify that the local system startup file contains the command shown in the previous step, or add it if necessary:

    • Red Hat:

      On Red Hat Enterprise Linux systems, add the command to the /etc/rc.d/rc.local file.

    • SuSE:

      On SuSE systems, add the command to the /etc/init.d/boot.local file.

Check the Configuration of OCFS

If you intend to use Oracle Cluster File System for database files, verify that it is configured correctly, as follows:


Note:

For more information about configuring OCFS, refer to the documentation provided with the kit or available from the following Web site:

http://oss.oracle.com/projects/ocfs


  1. Enter the following command to verify that OCFS is configured to start at runlevels 2, 3, and 5:

    # /usr/sbin/chkconfig --list ocfs
    
    
  2. Verify that the OCFS file systems you want to use are mounted and are specified in the /etc/fstab file.

Stop Existing Oracle Processes


Caution:

If you are installing additional Oracle Database 10g products in an existing Oracle home, stop all processes running in the Oracle home. You must complete this task to enable the Installer to relink certain executables and libraries.

If you choose to create a database during the installation, most installation types configure and start a default Oracle Net listener using TCP/IP port 1521 and the IPC key value EXTPROC. However, if an existing Oracle Net listener process is using the same port or key value, the Installer can only configure the new listener; it cannot start it. To ensure that the new listener process starts during the installation, you must shut down any existing listeners before starting the Installer.

To determine whether an existing listener process is running and to shut it down if necessary, follow these steps:

  1. Switch user to oracle:

    # su - oracle
    
    
  2. Enter the following command to determine whether a listener process is running and to identify its name and the Oracle home directory in which it is installed:

    $ ps -ef | grep tnslsnr
    
    

    This command displays information about the Oracle Net listeners running on the system:

    ... oracle_home1/bin/tnslsnr LISTENER -inherit
    
    

    In this example, oracle_home1 is the Oracle home directory where the listener is installed and LISTENER is the listener name.


    Note:

    If no Oracle Net listeners are running, refer to the "Configure the Oracle User's Environment" section to continue.

  3. Set the ORACLE_HOME environment variable to specify the appropriate Oracle home directory for the listener:

    • Bourne, Bash, or Korn shell:

      $ ORACLE_HOME=oracle_home1
      $ export ORACLE_HOME
      
      
    • C or tcsh shell:

      % setenv ORACLE_HOME oracle_home1
      
      
  4. Enter the following command to identify the TCP/IP port number and IPC key value that the listener is using:

    $ $ORACLE_HOME/bin/lsnrctl status listenername
    

    Note:

    If the listener uses the default name LISTENER, you do not have to specify the listener name in this command.

  5. Enter a command similar to the following to stop the listener process:

    $ $ORACLE_HOME/bin/lsnrctl stop listenername
    
    
  6. Repeat this procedure to stop all listeners running on this system and on all other cluster nodes.

Configure the Oracle User's Environment

You run the Installer from the oracle account. However, before you start the Installer you must configure the environment of the oracle user. To configure the environment, you must:

To set the oracle user's environment, follow these steps:

  1. Start a new terminal session, for example, an X terminal (xterm).

  2. Enter the following command to ensure that X Window applications can display on this system:

    $ xhost +
    
    
  3. If you are not already logged in to the system where you want to install the software, log in to that system as the oracle user.

  4. If you are not logged in as the oracle user, switch user to oracle:

    $ su - oracle
    
    
  5. To determine the default shell for the oracle user, enter the following command:

    $ echo $SHELL
    
    
  6. Open the oracle user's shell startup file in any text editor:

    • Bash shell on Red Hat Enterprise Linux:

      $ vi .bash_profile
      
      
    • Bourne shell (sh), Bash shell (bash), or Korn shell (ksh):

      $ vi .profile
      
      
    • C shell (csh or tcsh):

      % vi .login
      
      
  7. Enter or edit the following line, specifying a value of 022 for the default file mode creation mask:

    umask 022
    
    
  8. If the ORACLE_SID, ORACLE_HOME, or ORACLE_BASE environment variables are set in the file, remove the appropriate lines from the file.

  9. Save the file and exit from the editor.

  10. To run the shell startup script, enter one of the following commands:

    • Bash shell on Red Hat Enterprise Linux:

      $ . ./.bash_profile
      
      
    • Bourne, Bash, or Korn shell:

      $ . ./.profile
      
      
    • C shell:

      % source ./.login
      
      
  11. If you are not installing the software on the local system, enter a command similar to the following to direct X applications to display on the local system:

    • Bourne, Bash, or Korn shell:

      $ DISPLAY=local_host:0.0 ; export DISPLAY
      
      
    • C shell:

      % setenv DISPLAY local_host:0.0
      
      

    In this example, local_host is the host name or IP address of the system that you want to use to display the Installer (your workstation or PC).

  12. If you determined that the /tmp directory has less than 400 MB of free disk space, identify a file system with at least 400 MB of free space and set the TEMP and TMPDIR environment variables to specify a temporary directory on this file system:

    1. Use the df -h command to identify a suitable file system with sufficient free space.

    2. If necessary, enter commands similar to the following to create a temporary directory on the file system that you identified, and set the appropriate permissions on the directory:

      $ su - root
      # mkdir /mount_point/tmp
      # chmod a+wr /mount_point/tmp
      # exit
      
      
    3. Enter commands similar to the following to set the TEMP and TMPDIR environment variables:

      • Bourne, Bash, or Korn shell:

        $ TEMP=/mount_point/tmp
        $ TMPDIR=/mount_point/tmp
        $ export TEMP TMPDIR
        
        
      • C shell:

        % setenv TEMP /mount_point/tmp
        % setenv TMPDIR /mount_point/tmp
        
        
  13. Enter commands similar to the following to set the ORACLE_BASE environment variable:

    • Bourne, Bash, or Korn shell:

      $ ORACLE_BASE=/u01/app/oracle
      $ export ORACLE_BASE
      
      
    • C shell:

      % setenv ORACLE_BASE /u01/app/oracle
      
      

    In these examples, /u01/app/oracle is the Oracle base directory that you created or identified earlier.

  14. If you are using raw devices for database file storage, set the DBCA_RAW_CONFIG environment variable to specify the full path to the raw device mapping file:

    • Bourne, Bash, or Korn shell:

      $ DBCA_RAW_CONFIG=$ORACLE_BASE/oradata/dbname/dbname_raw.conf
      $ export DBCA_RAW_CONFIG
      
      
    • C shell:

      % setenv DBCA_RAW_CONFIG=$ORACLE_BASE/oradata/dbname/dbname_raw.conf
      
      
  15. Enter the following command to ensure that the ORACLE_HOME and TNS_ADMIN environment variables are not set:

    • Bourne, Bash, or Korn shell:

      $ unset ORACLE_HOME
      $ unset TNS_ADMIN
      
      
    • C shell:

      % unsetenv ORACLE_HOME
      % unsetenv TNS_ADMIN
      

    Note:

    If the ORACLE_HOME environment variable is set, the Installer uses the value that it specifies as the default path for the Oracle home directory. However, if you set the ORACLE_BASE environment variable, Oracle recommends that you unset the ORACLE_HOME environment variable and choose the default path suggested by the Installer.

  16. To verify that the environment has been set correctly, enter the following commands:

    $ umask
    $ env | more
    
    

    Verify that the umask command displays a value of 22, 022, or 0022 and that the environment variables you set in this section have the correct values.