This chapter describes the Oracle Clusterware storage configuration tasks that you must complete before you start Oracle Universal Installer (OUI). This chapter includes information about the following tasks:
Complete the following steps to prepare shared disks for storage:
You must disable write caching on all disks that will be used to share data between the nodes in your cluster. Perform these steps to disable write caching:
Click Start, then click Settings, then Control Panel, then Administrative Tools, then Computer Management, then Device Manager, and then Disk drives
Expand the Disk drives and double-click the first drive listed
Under the Disk Properties tab for the selected drive, uncheck the option that enables the write cache.
For Windows 2000, select the option Optimize for quick removal on the Policies tab.
Double-click each of the other drives that will be used by Oracle Clusterware and Oracle RAC and disable write caching as described in the previous step.
Caution:Any disks that you use to store files, including database files, that will be shared between nodes, must have write caching disabled.
If you are using Windows 2003, then you must enable disk automounting, depending on the Oracle products that you are installing and on other conditions. You must enable automounting when using:
Raw partitions for Oracle Real Application Clusters (Oracle RAC)
Cluster file system for Oracle RAC
Raw partitions for single-node database installations
Logical drives for Automatic Storage Management (ASM)
To enable automounting:
Enter the following commands at a command prompt:
c:\> diskpart DISKPART> automount enable Automatic mounting of new volumes enabled.
exit to end the
Repeat steps 1 and 2 for each node in the cluster.
When you have prepared all of the cluster nodes in your Windows 2003 system as described in the previous steps, restart all of the nodes.
Note:All nodes in the cluster must have automatic mounting enabled in order to correctly install Oracle RAC and Oracle Clusterware. Oracle recommends that you enable automatic mounting before creating any logical partitions for use by the database, ASM, or the Oracle Cluster File System.
You must restart each node after enabling disk automounting. After it is enabled and the node is restarted, automatic mounting remains active until it is disabled.
Use the following sections to help you select your storage option and take the first steps in configuring storage:
There are two ways to store Oracle Clusterware files:
Oracle Cluster File System (OCFS): The cluster file system Oracle provides a storage solution for Windows users.
Raw storage: Raw storage is a logical partition created inside an extended partition that does not yet have a file system configured on it. These unformatted partitions are created and managed by Microsoft Windows disk management tools or by tools provided by third party vendors.
Note:You cannot use Direct NFS to store Oracle Clusterware files. You can only use Direct NFS to store Oracle Database files. See Chapter 4 for more information on using Direct NFS.
Oracle Clusterware files include voting disks, used to monitor cluster node status, and Oracle Cluster Registry (OCR) which contains configuration information about the cluster. The voting disks and OCR are shared files on a cluster file system. If you do not use a cluster file system, then you must place these files on shared raw devices.
The OCR is a file that contains the configuration information and status of the cluster. Oracle Universal Installer (OUI) automatically initializes the OCR during the Oracle Clusterware installation. Database Configuration Assistant uses the OCR for storing the configurations for the cluster databases that it creates.
For either storage option to meet high availability requirements, the files stored on the disk need to be protected by data redundancy, so that if one or more disks fail, the data stored on the failed disks can be recovered. This redundancy can be provided externally using Redundant Array of Independent Disks (RAID) devices, or logical volumes on more than one physical device and implement the stripe-and-mirror- everything methodology, also known as SAME. If you do not have a RAID devices or logical volumes, you can create additional copies, or mirrors, of the files on different file systems. If you choose to mirror the files, you must provide disk space for an additional Oracle Cluster Registry (OCR) file and at least two additional voting disk files.
The OCR mirror should be placed on a different disk than the primary OCR file. For voting disk file placement, ensure that each file is configured so that it does not share any hardware device or disk, or other single point of failure with the other voting disks. Any node that does not have available to it an absolute majority of voting disks configured (more than half) will be restarted.
Note:For the most up-to-date information about supported storage options for Oracle Clusterware installations, refer to the Certify pages on the OracleMetaLink Web site:
The following table shows the storage options supported for storing Oracle Clusterware files. Oracle Clusterware files:
|Storage Option||File Types Supported|
|OCR and Voting Disks||Oracle CRS Software|
|Automatic Storage Management||No||No|
|Shared disk partitions||Yes||No|
Use the following guidelines when choosing the storage options that you want to use for Oracle Clusterware:
You can choose any combination of the supported storage options for each file type provided that you satisfy all requirements listed for the chosen storage options.
You cannot use ASM to store Oracle Clusterware files, because these files must be accessible before any Oracle instance starts, which includes the ASM instances.
If you do not have a storage option that provides external file redundancy, then you must configure at least three voting disk areas to provide voting disk redundancy.
Note:Do not create an NTFS partition on a disk that you are using for OCFS.
When you have determined your disk storage options, you must perform the following tasks in the following order:
To use OCFS for Oracle Clusterware files, refer to "Configuring Storage for Oracle Clusterware Files on a Shared File System".
To use raw devices (partitions) for Oracle Clusterware files, refer to "Configuring Storage for Oracle Clusterware Files on Raw Devices".
To check for all shared file systems available across all nodes on the cluster, log in as the installation owner user (
crs), and use the following syntax:
mountpoint\runcluvfy.bat comp ssa -n node_list
If you want to check the shared accessibility of a specific shared storage type to specific nodes in your cluster, then use the following command syntax:
mountpoint\runcluvfy.bat comp ssa -n node_list -s storageID_list
In the preceding syntax examples, the variable
mountpoint is the mountpoint path of the installation media, the variable
node_list is the list of nodes you want to check, separated by commas, and the variable
storageID_list is the paths for the storage devices that you want to check.
For example, if you want to check the shared accessibility from
node2 of storage devices
\\host2\voting_disks, and your mountpoint is
F:, then enter the following command:
F:\runcluvfy.bat comp ssa -n node1,node2 -s \\host1\ocr_dir,\\host2\voting_disks
If you do not specify storage device IDs in the command, then CVU searches for all available storage devices connected to the nodes on the list.
Note:Prior to installing Oracle Clusterware, use the
runcluvfyutility on the installation media. If Oracle Clusterware is installed, use the
Oracle Universal Installer (OUI) does not suggest a default location for the Oracle Cluster Registry (OCR) or the Oracle Clusterware voting disk. If you choose to create these files on a file system, then review the following sections to complete storage requirements for Oracle Clusterware files:
Note:The OCR is a shared file in a cluster file system environment. If you do not use a cluster file system, then you must place this file on a shared storage device.
To use OCFS for Oracle Clusterware files, you must comply with the following requirements:
If you choose to place your Oracle Cluster Registry (OCR) files on a shared file system, then Oracle recommends that one of the following is true:
The disks used for the file system are on a highly available storage device, (for example, a RAID device that implements file redundancy)
At least three file systems are mounted, and use the features of Oracle Clusterware 11g release 1 (11.1) to provide redundancy for the OCR and voting disks
If you use a RAID device to store the Oracle Clusterware files, you must have a partition that has at least 560 MB of available space for the OCR and voting disk.
If you use the redundancy features of Oracle Clusterware to provide high availability for the OCR and voting disk files, then:
A second file system must have 560 MB of available space for the OCR mirror and a voting disk.
A third file system must have at least 500 MB for a voting disk (500 MB is the smallest disk size that can be used by OCFS).
If you intend to store Oracle Clusterware binaries on OCFS, then you must have at least 650 MB of available space on one of the file systems.
Note:Oracle does not recommend the use of a shared Oracle Clusterware home directory because it creates a single point-of-failure, and removes the ability to perform rolling upgrades of Oracle Clusterware.
The total required volume size listed in the previous paragraph is cumulative. For example, to store all OCR and voting disk files on a shared file system that does not provide redundancy at the hardware level (external redundancy), you should have at least 1.4 GB of storage available over a minimum of three volumes (two separate volume locations for the OCR and OCR mirror, and one voting disk on each volume). If you use a file system that provides data redundancy, then you need only one physical disk with 560 MB of available space to store the OCR and voting disk files.
The OCR and voting disk files are stored in the OCFS data file directory (
datafile_disk is the OCFS partition and
clustername is the name of your cluster.
Oracle Universal Installer (OUI) does not suggest a default location for the Oracle Cluster Registry (OCR) or the Oracle Clusterware voting disk. If you choose to create these files on a file system, then perform the steps described in this section to set up the unformatted partitions on the shared disks for OCFS. Windows refers to unformatted partitions as logical drives. If you need more information about creating partitions, then refer to the Windows online help from within the disk administration tools.
Run Windows Disk Management from one node to create an extended partition. Use a basic disk: dynamic disks are not supported.
If you plan to store the Oracle Clusterware binaries on OCFS, create at least two logical partitions: one for the Oracle Clusterware home and one for the Oracle Clusterware files.
You do not need to create separate partitions for the OCR and voting disk if you plan to use OCFS. OCFS creates individual files for the OCR and voting disk.
The number of partitions used for OCFS affects performance. Therefore, you should create the minimum number of partitions needed for the OCFS option you choose.
If your file system is not based on a RAID system, then you must provide storage space for creating redundant Oracle Clusterware files. Create an extended partition and logical partition for each additional file system or disk drive that will be used by Oracle Clusterware. The additional storage used for Oracle Clusterware files should not share any hardware device or disk to avoid single points of failure.
To create the required partitions, perform the following steps:
From one of the existing nodes of the cluster, run the Windows disk administration tool as follows:
Click Start, then select Settings, Control Panel, Administrative Tools, and then Computer Management
Expand the Storage folder to Disk Management. Use a basic disk with a Master Boot Record (MBR) partition style as an extended partition for creating partitions.
Right click inside an unallocated part of an extended partition and choose Create Logical Drive. A wizard presents pages for configuring the logical drive. Select the select logical drive radio button and click Next.
Enter the size that you want for the partition and click Next.
Repeat steps 2 through 4 for the second partition and any additional partitions. An optimal configuration is one partition for the Oracle home and one partition for Oracle Clusterware files, or a single partition for both the OCR and voting disk.
If you are preparing drives on a Windows 2003 system, then you should restart all nodes in the cluster after you have created the logical drives.
Check all nodes in the cluster to ensure that the partitions are visible on all the nodes and to ensure that none of the Oracle partitions have drive letters assigned. If any partitions have drive letters assigned, then remove them by performing these steps:
Right-click the partition in the Windows disk administration tool
Select "Change Drive Letters and Paths..." from the menu
Click Remove in the "Change Drive Letter and Paths" window
To use raw devices, you must create a separate partition for each file. If you are using three raw devices to store the Oracle Clusterware files, then:
Create two partitions on the second raw device: one 280 MB partition for the voting disk and one 280 MB partition for the OCR mirror.
Create a single partition on the third raw device for the voting file.
If you are not using OCFS or ASM for your data files, then you must also create raw partitions for your database files as described in "Configuring Logical Volumes or Unformatted Partitions".
To create and configure raw volumes or partitions, use the disk administration tools provided by the operating system or third party vendors. The following administration tools are provided by the operating system:
Disk Management snap-in to manage disks.
To access this tool, type
diskmgmt.msc at the command prompt. Alternatively, from the Start menu, select Programs, then Administrative Tools, then Computer Management. Then select the Disk Management node in the Storage tree.
A command line tool to manage disks.
To access this tool, type
diskpart.exe at the command prompt.
Note:If you need to download the
diskmgmt.msctool, consult Microsoft documentation on the Microsoft Web site
See Also:The online help or documentation for the administration tool you are using
You can use the
diskpart tool command
create partition to create primary or extended partitions, or create logical drives. The following example uses the
diskpart tool to create a 280 MB extended partition on disk 100. In this syntax,
diskpart.exe is the command line tool for managing disks.
c:\> diskpart.exe DISKPART> select disk 100 DISKPART> create partition extended size=280
Optionally, mount the partition on an NTFS folder instead of assigning a drive letter:
DISKPART> assign mount=C:\mnt\raw_1
Note:Be aware of the following restrictions for partitions:
You cannot use primary partitions for storing Oracle Clusterware files while running the OUI to install Oracle Clusterware as described in Chapter 5, "Installing Oracle Clusterware". You must create logical drives inside extended partitions for the disks to be used by Oracle Clusterware files and Oracle ASM.
With 32-bit Windows, you cannot create more than four primary disk partitions for each disk. One of the primary partitions can be an extend partition, which can then be subdivided into multiple logical partitions.
You can assign mount points only to primary partitions and logical drives.
You must create logical drives inside extended partitions for the disks to be used by Oracle Clusterware files and Oracle ASM.
Oracle recommends that you limit the number of partitions you create on a single disk to prevent disk contention. Therefore, you may prefer to use extended partitions rather than primary partitions.
For these reasons, you might prefer to use extended partitions for storing Oracle software files and not primary partitions.