Oracle® Clusterware

Installation Guide

11g Release 1 (11.1) for Microsoft Windows

B28250-07

July 2010

Oracle Clusterware Installation Guide, 11g Release 1 (11.1) for Microsoft Windows

B28250-07

Copyright © 2007, 2010 Oracle and/or its affiliates. All rights reserved.

Primary Authors: Mark Bauer, Janet Stern, Lyju Vadassery, Douglas Williams

Contributing Authors: Jonathan Creighton, Pat Huey, Raj Kumar

Contributors: Chris Allison, Karin Brandauer, Robert Chang, Sudip Datta, Luann Ho, Rajiv Jayaraman, Roland Knapp, Diana Lorentz, Barb Lundhild, Vijay Lunawat, John Patrick McHugh, Anil Nair, Randy Neville, Philip Newlan, Michael Polaski, Dipak Saggi, Sudheendra Sampath, Janelle Simmons, Clive Simpkins, Khethavath P. Singh, Nitin Vengurlekar, Gary Young

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this software or related documentation is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle USA, Inc., 500 Oracle Parkway, Redwood City, CA 94065.

This software is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications which may create a risk of personal injury. If you use this software in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure the safe use of this software. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software in dangerous applications.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

This software and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

Contents

List of Tables

Preface

- Audience

- Documentation Accessibility

- Related Documents

- Conventions

What's New in Oracle Clusterware Installation and Configuration?

- Changes in Installation Documentation

- Enhancements and New Features for Installation

1 Summary List: Installing Oracle Clusterware

- 1.1 Verify System Requirements

- 1.2 Check Network Requirements

- 1.3 Configure Users

- 1.4 Create Directories

- 1.5 Create Storage

- 1.5.1 Create Disk Partitions for OCR and Voting Disk Files

- 1.5.2 Create and Configure Disk Partitions for ASM

- 1.6 Verify Oracle Clusterware Requirements with CVU

- 1.7 Install Oracle Clusterware Software

2 Server and Network Preinstallation Tasks

- 2.1 Installation Differences Between Windows and Linux or UNIX

- 2.2 Reviewing Upgrade Best Practices

- 2.3 Checking Hardware and Software Certification

- 2.3.1 Web Browser Support

- 2.3.2 Telnet and Terminal Services Support

- 2.3.2.1 Windows Telnet Services Support

- 2.3.2.2 Windows Terminal Services and Remote Desktop Support

- 2.4 Checking Hardware Requirements

- 2.4.1 Hard Disk Space Requirements

- 2.5 Verifying Hardware Requirements

- 2.6 Checking Software Requirements

- 2.7 Configuring User Accounts

- 2.7.1 Managing User Accounts with User Account Control

- 2.8 Checking Network Requirements

- 2.8.1 Network Hardware Requirements

- 2.8.2 IP Address Requirements

- 2.8.3 Checking Network Requirements

- 2.8.3.1 Check Hardware and IP Addresses

- 2.8.3.2 (Optional) Rename Network Interfaces

- 2.8.3.3 Configure Network Bind Order

- 2.8.3.4 Disable Windows Media Sensing for TCP/IP

- 2.9 Checking Individual Component Requirements

- 2.9.1 Oracle Advanced Security Requirements

- 2.9.2 Oracle Enterprise Manager Requirements

- 2.10 Verifying Cluster Privileges

- 2.11 Understanding and Using Cluster Verification Utility

- 2.11.1 Entering Cluster Verification Utility Commands

- 2.11.2 Using CVU to Determine if Installation Prerequisites are Complete

- 2.11.3 Using the Cluster Verification Utility Help

- 2.11.4 Using Cluster Verification Utility with Oracle Database 10g Release 1 or 2

- 2.11.5 Verbose Mode and "Unknown" Output

3 Configuring Oracle Clusterware Storage

- 3.1 Preliminary Shared Disk Preparation

- 3.1.1 Disabling Write Caching

- 3.1.2 Enabling Automounting for Windows 2003

- 3.2 Reviewing Storage Options for Oracle Clusterware Files

- 3.2.1 Overview of Oracle Clusterware Storage Options

- 3.2.2 General Storage Considerations

- 3.2.3 After You Have Selected Disk Storage Options

- 3.3 Checking for Available Shared Storage with CVU

- 3.4 Configuring Storage for Oracle Clusterware Files on a Shared File System

- 3.4.1 Requirements for Using a File System for Oracle Clusterware Files

- 3.4.2 Creating Required Partitions for Oracle Clusterware on Shared File Systems

- 3.5 Configuring Storage for Oracle Clusterware Files on Raw Devices

4 Configuring Oracle Real Application Clusters Storage

- 4.1 Preliminary Shared Disk Preparation

- 4.1.1 Disabling Write Caching

- 4.1.2 Enabling Automounting for Windows 2003

- 4.2 Reviewing Storage Options for Oracle Database and Recovery Files

- 4.2.1 Overview of Oracle Database and Recovery File Storage Options

- 4.2.2 General Storage Considerations

- 4.2.3 Guidelines for Placing Oracle Data Files on a File System

- 4.2.4 Guidelines for Placing Oracle Recovery Files on a File System

- 4.2.5 After You Have Selected Disk Storage Options

- 4.3 Configuring Storage for Oracle Database Files on a Shared File System

- 4.4 Configuring Disks for Automatic Storage Management

- 4.4.1 General Steps for Configuring Automatic Storage Management

- 4.4.2 Step 1: Identifying Storage Requirements for Automatic Storage Management

- 4.4.3 Step 2 (Optional): Using an Existing Automatic Storage Management Disk Group

- 4.4.4 Step 3: Creating DAS or SAN Disk Partitions for Automatic Storage Management

- 4.4.5 Step 4: Manually Configuring Disks for Automatic Storage Management

- 4.4.5.1 Overview of asmtoolg and asmtool

- 4.4.5.2 Using asmtoolg (Graphical User Interface)

- 4.4.5.3 Using asmtool (Command Line)

- 4.5 Configuring Logical Volumes or Unformatted Partitions

- 4.5.1 Creating Partitions for Logical Volumes

- 4.5.2 Assigning Logical Names

- 4.5.3 Creating the DBCA Raw Device Mapping File

- 4.6 Configuring Direct NFS Storage for Data Files

- 4.6.1 About Direct NFS Storage

- 4.6.2 Using the Oranfstab File with Direct NFS

- 4.6.3 Mounting NFS Storage Devices with Direct NFS

- 4.6.4 Specifying Network Paths with the Oranfstab File

- 4.6.5 Enabling Direct NFS Client

- 4.6.6 Disabling Direct NFS Client

- 4.6.7 Checking NFS Buffer Size

- 4.7 Requirements for Files Managed by Oracle

5 Installing Oracle Clusterware

- 5.1 Verifying Oracle Clusterware Requirements with CVU

- 5.1.1 Troubleshooting Clusterware Setup for Windows

- 5.2 Preparing to Install Oracle Clusterware with OUI

- 5.3 Installing Oracle Clusterware with OUI

- 5.3.1 Running OUI to Install Oracle Clusterware

- 5.3.2 Installing Oracle Clusterware Using a Cluster Configuration File

- 5.3.3 Silent Installation of Oracle Clusterware

- 5.4 Formatting Drives to Use Oracle Cluster File System after Installation

6 Oracle Clusterware Post-Installation Procedures

- 6.1 Backing Up the Voting Disk after Installation

- 6.2 Downloading and Installing Patches

- 6.3 Required Tasks to Install Oracle RAC Database Release 9.2

A Troubleshooting the Oracle Clusterware Installation Process

- A.1 General Installation Issues

- A.2 Oracle Clusterware Install Actions Log Errors and Causes

- A.2.1 The OCFS format is not recognized on one or more of the remote cluster nodes

- A.2.2 Timing issue with start of the OracleCSService:

- A.2.3 You are on a Windows 2003 system, and Automount of new drives is not enabled:

- A.2.4 You have entered a period in one of the node names during Oracle Clusterware install

- A.2.5 Node1 failed to startup service OracleEVMService, err(1053)

- A.2.6 Step 1: checking status of Oracle Clusterware clusterStep 2: configuring OCR repositoryignoring upgrade failure of ocr(-1073740972)

B Deinstalling Oracle Clusterware

- B.1 Deinstalling Oracle Clusterware from Windows Environments

- B.1.1 Deinstalling Oracle Clusterware with No Previous Cluster Software Versions

C Configuring Raw Devices for Oracle Database

- C.1 Support for Raw Devices

- C.2 Raw Devices Required by Database Configuration Assistant

- C.2.1 Planning Your Raw Device Creation Strategy

D How to Perform Oracle Clusterware Rolling Upgrades

- D.1 Back Up the Oracle Software Before Upgrades

- D.2 Restrictions for Clusterware Upgrades to Oracle Clusterware 11g

- D.2.1 Overview of Upgrading from Oracle Clusterware 10g Release 2 to Oracle Clusterware 11g

- D.2.2 Overview of Upgrading from Oracle Cluster Ready Services 10g Release 1 to Oracle Clusterware 11g

- D.3 Verify System Readiness for Patches and Upgrades

- D.4 How to Perform Oracle Clusterware Rolling Upgrades for Patches

- D.4.1 Copy Patch Software to the Primary Upgrade Node

- D.4.2 Stop All Oracle Processes on Upgrade Nodes

- D.4.3 Start OUI and Complete the Upgrade Processes on Upgrade Nodes

- D.5 How to Perform Rolling Upgrades From an Earlier Release to 11g

Index

List of Tables

- 2-1 Hard Disk Space Requirements for Oracle RAC

- 2-2 Oracle RAC Software Requirements for Windows Systems

Preface

Welcome to Oracle Clusterware Installation Guide. This document provides configuration information for network and system administrators, and database installation information for database administrators (DBAs) who install and configure Oracle Clusterware for Microsoft Windows.

This preface contains the following topics:

- Audience

- Documentation Accessibility

- Related Documents

- Conventions

Audience

This document is intended for network and system administrators and database administrators (DBAs) who install and configure Oracle Clusterware.

Documentation Accessibility

Our goal is to make Oracle products, services, and supporting documentation accessible to all users, including users that are disabled. To that end, our documentation includes features that make information available to users of assistive technology. This documentation is available in HTML format, and contains markup to facilitate access by the disabled community. Accessibility standards will continue to evolve over time, and Oracle is actively engaged with other market-leading technology vendors to address technical obstacles so that our documentation can be accessible to all of our customers. For more information, visit the Oracle Accessibility Program Web site at

http://www.oracle.com/accessibility/

Accessibility of Code Examples in Documentation

Screen readers may not always correctly read the code examples in this document. The conventions for writing code require that closing braces should appear on an otherwise empty line; however, some screen readers may not always read a line of text that consists solely of a bracket or brace.

Accessibility of Links to External Web Sites in Documentation

This documentation may contain links to Web sites of other companies or organizations that Oracle does not own or control. Oracle neither evaluates nor makes any representations regarding the accessibility of these Web sites.

Access to Oracle Support for Hearing-Impaired Customers

Oracle customers have access to electronic support through My Oracle Support or by calling Oracle Support at 1.800.223.1711. Hearing-impaired customers in the U.S. who wish to speak to an Oracle Support representative may use a telecommunications relay service (TRS). Information about the TRS is available at http://www.fcc.gov/cgb/consumerfacts/trs.html, and a list of telephone numbers is available at http://www.fcc.gov/cgb/dro/trsphonebk.html. International hearing-impaired customers should use the TRS at +1.605.224.1837. An Oracle Support engineer will respond to technical issues according to the standard service request process.

Related Documents

For more information, refer to the following Oracle resources:

Oracle Clusterware and Oracle Real Application Clusters Documentation

Most Oracle error message documentation is in Oracle Database Error Messages. If you only have access to the Oracle Documentation media, then browse the error messages by range. Once you find a range, use your browser's find feature to locate a specific message. When connected to the Internet, you can search for a specific error message using the error message search feature of the Oracle online documentation. Additional error messages for Oracle Clusterware and Oracle RAC tools are included in the Oracle Real Application Clusters Administration and Deployment Guide.

Installation Guides

- Oracle Diagnostics Pack Installation Guide

- Oracle Real Application Clusters Installation Guide for Microsoft Windows

Operating System-Specific Administrative Guides

- Oracle Database Platform Guide for Microsoft Windows

Oracle 11g Clusterware Management

- Oracle Clusterware Administration and Deployment Guide

Oracle Database 11g Real Application Clusters Management

- Oracle Real Application Clusters Administration and Deployment Guide

- Oracle Database 2 Day DBA

- Oracle Database 2 Day + Real Application Clusters Guide

- Getting Started with the Oracle Diagnostics Pack

Generic Documentation

- Oracle Database New Features Guide

- Oracle Database Net Services Administrator's Guide

- Oracle Database Concepts

- Oracle Database Reference

To download free release notes, installation documentation, white papers, or other collateral, please visit the Oracle Technology Network (OTN). You must register online before using OTN; registration is free and can be done at the following Web site:

http://otn.oracle.com/membership/

If you already have a username and password for OTN, then you can go directly to the documentation section of the OTN Web site at the following Web site:

http://otn.oracle.com/documentation/

Oracle error message documentation is available only in HTML. You can browse the error messages by range in the Documentation directory of the installation media. When you find a range, use your browser's "find in page" feature to locate a specific message. When connected to the Internet, you can search for a specific error message using the error message search feature of the Oracle online documentation.

If you already have a username and password for OTN, then you can go directly to the documentation section of the OTN Web Site:

http://otn.oracle.com/documentation/

Conventions

This section describes the conventions used in the text and code examples of this documentation set. It describes:

- Conventions in Text

- Conventions in Code Examples

Conventions in Text

We use various conventions in text to help you more quickly identify special terms. The following table describes those conventions and provides examples of their use.

Convention

Meaning

Example

Bold

Bold typeface indicates terms that are defined in the text or terms that appear in a glossary, or both.

When you specify this clause, you create an index-organized table.

Italics

Italic typeface indicates book titles or emphasis.

Oracle Database Concepts
Ensure that the recovery catalog and target database do not reside on the same disk.

UPPERCASE monospace (fixed-width) font

Uppercase monospace typeface indicates elements supplied by the system. Such elements include parameters, privileges, data types, RMAN keywords, SQL keywords, SQL*Plus or utility commands, packages and methods, as well as system-supplied column names, database objects and structures, usernames, and roles.

You can specify this clause only for a NUMBER column.
You can back up the database by using the BACKUP command.

Query the TABLE_NAME column in the USER_TABLES data dictionary view.

Use the DBMS_STATS.GENERATE_STATS procedure.

lowercase monospace (fixed-width) font

Lowercase monospace typeface indicates executables, filenames, directory names, and sample user-supplied elements. Such elements include computer and database names, net service names, and connect identifiers, as well as user-supplied database objects and structures, column names, packages and classes, usernames and roles, program units, and parameter values.
Note: Some programmatic elements use a mixture of UPPERCASE and lowercase. Enter these elements as shown.

Enter sqlplus to start SQL*Plus.
The password is specified in the orapwd file.

Back up the data files and control files in the /disk1/oracle/dbs directory.

The department_id, department_name, and location_id columns are in the hr.departments table.

Set the QUERY_REWRITE_ENABLED initialization parameter to true.

Connect as oe user.

The JRepUtil class implements these methods.

lowercase italic monospace (fixed-width) font

Lowercase italic monospace font represents placeholders or variables.

You can specify the parallel_clause.
Run old_release.SQL where old_release refers to the release you installed prior to upgrading.

Conventions in Code Examples

Code examples illustrate SQL, PL/SQL, SQL*Plus, or other command-line statements. They are displayed in a monospace (fixed-width) font and separated from normal text as shown in this example:

SELECT username FROM dba_users WHERE username = 'MIGRATE';

The following table describes typographic conventions used in code examples and provides examples of their use.

Convention

Meaning

Example

[ ]

Brackets enclose one or more optional items. Do not enter the brackets.

DECIMAL (digits [ , precision ])

{ }

Braces enclose two or more items, one of which is required. Do not enter the braces, and do not enter more than one option.

{ENABLE | DISABLE}

|

A vertical bar represents a choice of two or more options within brackets or braces. Enter one of the options. Do not enter the vertical bar.

{ENABLE | DISABLE}
[COMPRESS | NOCOMPRESS]

...

Horizontal ellipsis points indicate either:

- That we have omitted parts of the code that are not directly related to the example

- That you can repeat a portion of the code

CREATE TABLE ... AS subquery;

SELECT col1, col2, ... , coln FROM employees;

.
.
.

Vertical ellipsis points indicate that we have omitted several lines of code not directly related to the example.

SQL> SELECT NAME FROM V$DATAFILE;
NAME
------------------------------------
/fsl/dbs/tbs_01.dbf
/fs1/dbs/tbs_02.dbf
.
.
.
/fsl/dbs/tbs_09.dbf
9 rows selected.

Other notation

You must enter symbols other than brackets, braces, vertical bars, and ellipsis points as shown.

acctbal NUMBER(11,2);
acct    CONSTANT NUMBER(4) := 3;

Italics

Italicized text indicates placeholders or variables for which you must supply particular values.

CONNECT SYSTEM/system_password
DB_NAME = database_name

UPPERCASE

Uppercase typeface indicates elements supplied by the system. We show these terms in uppercase in order to distinguish them from terms you define. Unless terms appear in brackets, enter them in the order and with the spelling shown. However, because these terms are not case sensitive, you can enter them in lowercase.

SELECT last_name, employee_id FROM employees;
SELECT * FROM USER_TABLES;
DROP TABLE hr.employees;

lowercase

Lowercase typeface indicates programmatic elements that you supply. For example, lowercase indicates names of tables, columns, or files.
Note: Some programmatic elements use a mixture of UPPERCASE and lowercase. Enter these elements as shown.

SELECT last_name, employee_id FROM employees;
sqlplus hr/hr
CREATE USER mjones IDENTIFIED BY ty3MU9;

What's New in Oracle Clusterware Installation and Configuration?

This section describes Oracle Database 11g release 1 (11.1) features as they pertain to the installation and configuration of Oracle Clusterware and Oracle Real Application Clusters (Oracle RAC). The topics in this section are:

- Changes in Installation Documentation

- Enhancements and New Features for Installation

Changes in Installation Documentation

With Oracle Database 11g release 1, Oracle Clusterware can be installed or configured as an independent product. Additional documentation is provided about storage administration. For installation planning, note the following documentation:

Oracle Database 2 Day + Real Application Clusters Guide

This book provides an overview and examples of the procedures to install and configure a two-node Oracle Clusterware and Oracle RAC environment.

Oracle Clusterware Installation Guide

This book (the guide that you are reading) provides procedures either to install Oracle Clusterware as a standalone product, or to install Oracle Clusterware with either Oracle Database, or Oracle RAC. It contains system configuration instructions that require system administrator privileges.

Oracle Real Application Clusters Installation Guide

This book provides procedures to install Oracle RAC after Oracle Clusterware installation has been completed successfully. It contains database configuration instructions for database administrators.

Oracle Database Storage Administrator's Guide

This book provides information for database and storage administrators who administer and manage storage, or who configure and administer Automatic Storage Management (ASM).

Oracle Clusterware Administration and Deployment Guide

This is the platform-specific administrator's reference for Oracle Clusterware. It contains information about administrative tasks, including those that involve changes to operating system configuration.

Oracle Real Application Clusters Administration and Deployment Guide

This is the platform-specific administrator's reference for Oracle RAC. It contains information about administrative tasks, including those that involve changes to operating system configuration.

Enhancements and New Features for Installation

The following is a list of enhancements and new features for Oracle Database 11g release 1 (11.1):

Oracle HTTP Server Update

To install Oracle HTTP Server, use the "Oracle Fusion Middleware Web Tier Utilities 11g (11.1.1.2.0)" media or download.

Windows Server 2008 (32-Bit) and Windows Server 2008 x64

Oracle Clusterware and Oracle RAC run on Microsoft Windows Server 2008 (32-bit) and Windows Server 2008 x64. Oracle Database 11g Release 1 (11.1.0.7 or later) is certified on Microsoft Windows Server 2008 (32-Bit) and Microsoft Windows Server 2008 x64 (all editions except HPC). The specific operating system components that are not supported in Windows Server 2008 (32-Bit) and Windows Server 2008 x64 are Hyper-V and Server Core.

New SYSASM Privilege for ASM Administration

This feature introduces a new SYSASM privilege that is specifically intended for performing ASM administration tasks. Using the SYSASM privilege instead of the SYSDBA privilege provides a clearer division of responsibility between ASM administration and database administration.

1 Summary List: Installing Oracle Clusterware

The following is a summary of the installation configuration requirements and commands. This summary is intended to provide an overview of the installation process.

In addition to providing a summary of the Oracle Clusterware installation process, this list also contains configuration information for preparing a system for Automatic Storage Management (ASM) and Oracle Real Application Clusters (Oracle RAC) installation.

1.1 Verify System Requirements

For more information, review "Checking Hardware Requirements".

Memory Requirements

In the Windows Task Manager window, select the Performance tab to view the available memory for your system.

To view the Virtual memory settings, from the Control panel, select System. In the System Properties window, select the Advanced tab, then, under Performance, click Performance Options. In the Performance Options window, the virtual memory, or page file, settings are displayed at the bottom of the window.

The minimum required RAM is 1 GB, and the minimum required swap space is 1 GB. Oracle recommends that you set the paging file size to twice the amount of RAM for systems with 2 GB of RAM or less. For systems with 2 GB to 8 GB RAM, use a paging file size equal to RAM. For systems with over 8 GB RAM, use .75 times the size of RAM for the paging file size.

Disk Requirements

From the Control Panel, select Administrative Tools, then select Computer Management. In the menu on the left-hand side of the Computer Management window, under Storage, select Disk Management.

The Disk Management window displays the available space on file systems. If you use standard redundancy for Oracle Clusterware files, which is 2 Oracle Cluster Registry (OCR) partitions and 3 voting disk partitions, then you should have at least 1 GB of disk space available on separate physical disks reserved for Oracle Clusterware files. Each Oracle Clusterware file should be at least 256 MB in size.

Note:
When you create partitions by specifying a size for the partition, such as 256 MB, the actual size of the created partition may be smaller than the size requested, based on the cylinder geometry of the disk. Oracle configuration software checks that the devices being used for Oracle Clusterware files contain a minimum of 256 MB of available disk space, not total disk space. Therefore, Oracle recommends using at least 280MB for the device size, to ensure there is 256 MB of available disk space.

The Oracle Clusterware home requires 650 MB of disk space.

TEMP Space Requirements

Ensure that you have at least 400 MB of disk space in the Windows TEMP directory. If this space is not available, then increase the partition size, or delete unnecessary files in the TEMP directory. Make sure the environment variables TEMP and TMP both point to the location of the TEMP directory, for example:

TEMP=C:\WINDOWS\TEMP
TMP=C:\WINDOWS\TEMP

1.2 Check Network Requirements

For more information, review "Checking Network Requirements".

Network Address Requirements

The following is a list of address requirements that you must configure on a domain name server (DNS), or configure in the %SystemRoot%\system32\drivers\etc\hosts file for each cluster node:

- You must have three network addresses for each node:

- A public internet protocol (IP) address

- A virtual IP (VIP) address, which is used by applications for failover in the event of node failure

- A private IP address, which is used by Oracle Clusterware and Oracle RAC for internode communication

- The virtual IP address has the following requirements:

- The IP address and host name are currently unused (it can be registered in a DNS, but should not be accessible by a ping command)

- The virtual IP address is on the same subnet as your public interface

- The private IP address has the following requirements:

- It should be on a subnet reserved for private networks, such as 10.0.0.0 or 192.168.0.0

- It should use dedicated switches or a physically separate, private network, reachable only by the cluster member nodes, preferably using high-speed NICs

- It must use the same private interfaces for both Oracle Clusterware and Oracle RAC private IP addresses

- It cannot be registered on the same subnet that is registered to a public IP address

After you obtain the IP addresses from a network administrator, to assign the public and private IP addresses to NICs, right-click on My Network Places and select Properties. In the Network and Dial-up Connections window, right-click the device for which you want to change the IP address, and select Properties. Select Internet Protocol (TCP/IP), then click Properties. Do not assign the VIP address.

Host Name Resolution Requirements

Ensure that the public (external) host names are defined in your DNS and that the correct IP addresses resolve for all nodes in the cluster. Ensure that all public (external) and private (internal) host names are defined in the hosts file on all nodes of the cluster. Test your cluster configuration by pinging all host names from each node and check for proper names resolution. The public and private IP addresses should respond to ping commands. The VIP addresses should not respond.

Network Adaptor Naming and Binding Order

Check the network adaptor binding order on each node. Ensure that your public network adaptor is first in the bind order, and the private network adaptor is second. Follow these steps to configure the network adaptor bind order:

1. Right-click My Network Places and choose Properties.

2. In the Advanced menu, click Advanced Settings.

3. If the public interface name is not the first name listed under the Adapters and Bindings tab, then select it and click the arrow to move it to the top of list.

4. Increase the priority of IPv4 over IPv6.

5. Click OK to save the setting and then exit the network setup dialog.

The names used for each class of network adaptor (such as public) must be consistent across all nodes. You can use nondefault names for the network adaptors, for example, PublicLAN, as long as the same names are used for the same class of network adaptors on each node in the network.

Disable DHCP Media Sense

Media Sense allows Windows to uncouple an IP address from a network adaptor when the link to the local switch is lost. To disable Windows Media Sensing for TCP/IP, you must set the value of the DisableDHCPMediaSense parameter to 1 on each node. Because you need to modify the Windows registry to disable Media Sensing, you should first backup the registry and confirm that you can restore it, using the methods described in your Windows documentation. Disable Media Sensing by completing the following steps on each node of your cluster:

1. Backup the Windows registry.

2. Use Registry Editor (Regedt32.exe) to view the following key in the registry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters

3. Add the following registry value:

Value Name: DisableDHCPMediaSense
Data Type: REG_DWORD -Boolean
Value: 1

4. Restart the computer.

Verify Privileges for Copying Files in the Cluster

During installation, OUI copies the software from the local node to the remote nodes in the cluster. Verify that you have administrative privileges on the other nodes in the cluster by running the following command on each node, where nodename is the node name:

net use \\nodename\C$

1.3 Configure Users

For this example, assume that you have one Oracle installation owner, and that this Oracle installation software owner name is oracle. The oracle user must be either a local Administrator user or a domain user, and the same user must exist (same user name, password, and domain) on each node in the cluster. If you use a domain user, then the user on each node must be in the same domain. For example, you cannot have a dba1 user on the first node in the DBADMIN domain and a dba1 user on the second node in the RACDBA domain.

If you intend to install Oracle Database, then the oracle user must be part of the ORA_DBA group.

Tip:
For more information, review the section "Configuring User Accounts".

1.4 Create Directories

To install properly across all nodes, the Oracle Universal Installer will need to use the temporary folders defined within Microsoft Windows. The TEMP and TMP folders should be the same across all nodes in the cluster. By default, these settings are defined as %USERPROFILE%\Local Settings\Temp and %USERPROFILE%\Local Settings\Tmp in the Environment Settings of My Computer. It is recommended to explicitly redefine these as WIN_DRIVE:\temp and WIN_DRIVE:\tmp, for example, C:\temp or C:\tmp for all nodes, if Windows is installed on the C drive.

For installations with Oracle Clusterware only, Oracle recommends that you let Oracle Universal Installer (OUI) create the Oracle Clusterware and Oracle Central Inventory (oraInventory) directories for you. However, as Administrator, you must create a path compliant with Oracle Optimal Flexible Architecture (OFA) guidelines, so that OUI can select that directory during installation. For OUI to recognize the path as an Oracle software path, it must be in the form C:\... .

1.5 Create Storage

Oracle RAC requires that each node be able to access a set of files on a shared disk subsystem. All instances in the cluster share these files.

The following sections outline the procedure for creating OCR and voting disk partitions, and preparing ASM disks.

For additional information, review the following sections:

- "Configuring Storage for Oracle Clusterware Files on a Shared File System"

- "Configuring Storage for Oracle Clusterware Files on Raw Devices"

- "Configuring Disks for Automatic Storage Management"

1.5.1 Create Disk Partitions for OCR and Voting Disk Files

The following steps outline the procedure for creating an OCR and voting disk partition:

1. Run Windows Disk Management from one node to create an extended partition. Use a basic disk: dynamic disks are not supported.

2. If you plan to store the Oracle Clusterware binaries on OCFS, create at least two logical partitions: one for the Oracle Clusterware home and one for the Oracle Clusterware files. You do not need to create separate partitions for the OCR and voting disk if you plan to use OCFS. OCFS creates individual files for the OCR and voting disk.

3. If your file system does not use RAID, then create an extended partition and logical partition for each additional file system or disk drive that will be used by Oracle Clusterware.

To create the required partitions, use the Disk Management utilities available with Microsoft Windows. Use a basic disk with a Master Boot Record (MBR) partition style as an extended partition for creating partitions.

1. From one of the existing nodes of the cluster, run the Windows disk administration tool as follows:

- Click Start, then select Settings, Control Panel, Administrative Tools, and then Computer Management

- Expand the Storage folder to Disk Management. Use a basic disk with a Master Boot Record (MBR) partition style as an extended partition for creating partitions. Right click inside an unallocated part of an extended partition and choose Create Logical Drive.
If using OCFS, specify a size for the partition that is at least 520 MB to store both the OCR and voting disk, or a size of 500 MB (the minimum size) to store just the voting disk or OCR. If using raw devices, then specify a size that is at least 280 MB to store a single OCR or voting disk file.

When specifying options for the logical drive, choose the option "Do not assign a drive letter or path" and "Do not format this partition". Repeat these steps to create enough partitions to store all the required files.

2. Repeat Step 1 to create all the required partitions.

3. If you are preparing drives on a Windows 2003 system, then you should restart all nodes in the cluster after you have created the logical drives.

4. Check all nodes in the cluster to ensure that the partitions are visible on all the nodes and to ensure that none of the Oracle partitions have drive letters assigned. If any partitions have drive letters assigned, then remove them by performing these steps:

- Right-click the partition in the Windows disk administration tool

- Select "Change Drive Letters and Paths..." from the menu

- Click Remove in the "Change Drive Letter and Paths" window

1.5.2 Create and Configure Disk Partitions for ASM

Use Microsoft Computer Management utility or the command line tool diskpart to create the partitions. Ensure that you create the partitions without drive letters. After you have created the partitions, the disks must be stamped with a header.

If you install Oracle Database in interactive mode, then Oracle Universal Installer configures the disks' headers during the installation process. If you plan to install Oracle Database in noninteractive mode, then you need to configure the disks manually before installation either by using asmtoolg (GUI version) or using asmtool (command line version).

For more information on configuring your disks with asmtoolg, refer to the section "Using asmtoolg (Graphical User Interface)". To configure the disks with asmtool, refer to the section "Using asmtool (Command Line)".

1.6 Verify Oracle Clusterware Requirements with CVU

For information, review the following section in "Verifying Oracle Clusterware Requirements with CVU".

Using the following command syntax, log in as the administrative user, and start Cluster Verification Utility (CVU) to check system requirements prior to installing Oracle Clusterware. In the following syntax example, replace the variable mountpoint with the path to the installation media, and replace the variable node_list with the names of the nodes in your cluster, separated by commas:

mountpoint\runcluvfy.bat stage -pre crsinst -n node_list

For example, if you want to verify the system requirements have been met for node1 and node2, and your installation media is available on drive F:, then enter the following command:

F:\runcluvfy.bat stage -pre crsinst -n node1,node2

1.7 Install Oracle Clusterware Software

For information, review the following sections in"Verifying Oracle Clusterware Requirements with CVU" AND "Installing Oracle Clusterware with OUI".

1. Navigate to the installation media, and start OUI. For example, if the installation media is available on drive D:, you would use the following commands:

cd D:\Disk1
setup.exe

2. Select Install Oracle Clusterware, and enter the configuration information as prompted.

3. After you have installed Oracle Clusterware, apply any available patch sets.

2 Server and Network Preinstallation Tasks

This chapter describes the tasks that you must complete before you start Oracle Universal Installer (OUI) to install Oracle Clusterware and Oracle Real Application Clusters (Oracle RAC) on Microsoft Windows x86 (32-bit) systems.

This chapter includes the following topics:

- Installation Differences Between Windows and Linux or UNIX

- Checking Hardware and Software Certification

- Checking Hardware Requirements

- Verifying Hardware Requirements

- Checking Software Requirements

- Configuring User Accounts

- Checking Network Requirements

- Checking Individual Component Requirements

- Verifying Cluster Privileges

- Understanding and Using Cluster Verification Utility

2.1 Installation Differences Between Windows and Linux or UNIX

If you are experienced with installing Oracle components in Linux or UNIX environments, note that many manual setup tasks required on Linux or UNIX are not required on Windows. The key differences between Windows and Linux or UNIX and installations are:

- Startup and shutdown services
With Windows, Oracle Universal Installer creates and sets startup and shutdown services at installation time. On Linux and UNIX systems, administrators are responsible for creating these services.

- Environment variables
With Windows, Oracle Universal Installer sets environment variables such as PATH, ORACLE_BASE, ORACLE_HOME, and ORACLE_SID in the registry. On Linux and UNIX systems, you must manually set these environment variables.

- DBA account for database administrators
With Windows, Oracle Universal Installer creates the ORA_DBA group. On Linux and UNIX systems, you must create the DBA account manually.

- Account for running Oracle Universal Installer
With Windows, you log in with Administrator privileges. You do not need a separate account. On Linux and UNIX systems, you must create this account manually.

See Also:
"Oracle Database Windows/UNIX Differences," in Oracle Database Platform Guide for Microsoft Windows

2.2 Reviewing Upgrade Best Practices

If you have an existing Oracle installation, then document version numbers, patches, and other configuration information, and review upgrade procedures for your existing installation. Review Oracle upgrade documentation before proceeding with installation, to decide how you want to proceed.

The following information applies when upgrading clusters:

- If servers run the same operating system binary, then Oracle Clusterware and Oracle Real Application Clusters support servers with different hardware in the same cluster. For example, you can build a cluster using different speeds and size computers, if all cluster members are running the 2.6.24 kernel. In addition, during rolling upgrades of the operating system, Oracle supports using different operating system binaries when both versions of the operating system are certified with the version of Oracle Database you are running.

Note:
Using mixed operating system versions is only supported for the duration of an upgrade, over the period of a few hours. Oracle does not support operating a cluster with mixed operating systems for an extended period. Oracle does not support running Oracle Clusterware and Oracle Real Application Clusters on heterogeneous platforms (servers with different chip architectures) in the same cluster.

- Running 32-bit and 64-bit Oracle software versions in the same cluster stack is not supported.

To find the most recent updates, and to find best practices recommendations about preupgrade, post-upgrade, compatibility, and interoperability, refer to "Oracle Upgrade Companion." "Oracle Upgrade Companion" is available through Note 466181.1 on My Oracle Support:

http://metalink.oracle.com

2.3 Checking Hardware and Software Certification

The hardware and software requirements included in this installation guide were current at the time this guide was published. However, because new platforms and operating system software versions might be certified after this guide is published, review the certification matrix on the My Oracle Support Web site for the most up-to-date list of certified hardware platforms and operating system versions. This Web site also provides compatible client and database versions, patches, and workaround information for bugs. The OracleMetaLink Web site is available at the following URL:

https://metalink.oracle.com/

You must register online before using OracleMetaLink. After logging in, select Certify & Availability from the left-hand column. From the Product Lifecycle page, select the Certifications button. Other Product Lifecycle options include Product Availability, Desupport Notices, and Alerts.

The following sections list the following certification information:

- Web Browser Support

- Telnet and Terminal Services Support

2.3.1 Web Browser Support

On 32-bit Windows systems, the following Web browsers are supported for Oracle Enterprise Manager Database Control:

- Netscape Navigator 7.2 and higher

- Microsoft Internet Explorer 6.0 with service pack 2, and higher

- Mozilla version 1.7 and higher

- Firefox versions 1.0.4, 1.5, 2. 0 and higher

To view or develop Oracle Application Express applications, Web browsers must support Java Script and the HTML 4.0, and CSS 1.0 standards. The following browsers meet this requirement:

- Microsoft Internet Explorer 6.0 or later version

- Firefox 1.0 or a later version

2.3.2 Telnet and Terminal Services Support

This section contains these topics:

- Windows Telnet Services Support

- Windows Terminal Services and Remote Desktop Support

2.3.2.1 Windows Telnet Services Support

Windows 2000 and Windows Server 2003 can use a Telnet Service to enable remote users to log on to the operating system and run console programs using the command line. Oracle supports the use of database command line utilities such as sqlplus, export, import and sqlldr using this feature, but does not support the database GUI tools such as Oracle Universal Installer, Database Configuration Assistant, and Oracle Net Configuration Assistant.

Note:
Ensure that the Telnet service is installed and started.

2.3.2.2 Windows Terminal Services and Remote Desktop Support

Oracle supports installing, configuring, and running Oracle Database through Terminal Services (console mode), on Windows 2000 and Windows 2003 Server. If you do not use Terminal Services in console mode, then you will encounter problems with configuration assistants at the end of the installation.

To start Terminal Services in console mode, enter the following command:

mstsc -v:servername /F /console

Platform-specific support information is as follows:

- Windows 2000: Oracle supports installing, configuring, and running Oracle Database from a remote Terminal Services Client.

- Windows Server 2003: You can configure Windows Server 2003 to use Terminal Services in Remote Desktop for Administration Mode or Terminal Server Mode.

The following products and features are not supported with Windows Terminal Services:

- Oracle Connection Manager

- Oracle Object Link Manager

- Oracle Services for Microsoft Transaction Server

- Server Management (SRVM) (You need to use a Windows Terminal Services console in order to use SRVM.)

See Also:

- The Microsoft Web site for more information about terminal servers
http://www.microsoft.com/

- The OracleMetaLink Web site for the latest Terminal Server certification information
https://metalink.oracle.com

2.4 Checking Hardware Requirements

You must have at least the following hardware component values for installing Oracle Database:

- RAM: At least 1 GB

- Virtual memory: double the amount of RAM

- Hard disk space: See Table 2-1

- Temp disk space: 400 MB

- Video adapter: 256 color

- Processor: 550 MHz minimum for 32-bit

Note:
Oracle provides 32-bit (x86) and 64-bit (x64) versions of Oracle Real Application Clusters for Microsoft Windows. The 32-bit database version runs on the 32-bit version of Windows on either x86 or x64 hardware.

See Also:

- "Configuring Oracle Clusterware Storage"

- "Reviewing Storage Options for Oracle Clusterware Files"

2.4.1 Hard Disk Space Requirements

Oracle recommends that you store Oracle components on NTFS. Because it is difficult for OUI to estimate NTFS and FAT disk sizes on Windows, the system requirements in this section are likely more accurate than the values reported on the Oracle Universal Installer Summary screen. In other words, it is difficult for OUI to provide accurate hard disk values for disk space. This includes the space required to create databases that are over 700 MB in size and the sizes of compressed files that are expanded on the hard drive.

Note:
Oracle Clusterware software cannot be installed on NFS.

Data files are not placed on NTFS partitions because they cannot be shared. Data files can be placed on unformatted (raw) disks used by ASM, Oracle Cluster File System (OCFS), or shared unformatted disks. Oracle Clusterware shared files can be placed only on OCFS or shared unformatted disks.

ASM is recommended for managing Oracle database datafiles in single instance or Oracle Real Application Cluster (Oracle RAC) environments. OCFS is best suited for Oracle binaries (if using a shared Oracle home) and flat files such as alert log and trace files.

To install Oracle Clusterware, it requires 650 MB of available disk space for the software binaries. For the shared files used by Oracle Clusterware, you must have either 1.1 GB of available disk space if using external redundancy for the disks or 1.4 GB of available disk space if using the redundancy features available with Oracle Clusterware.

For both the Enterprise and Standard Editions of Oracle RAC, the hard disk requirements for Oracle Database components include 32 MB required to install Java Runtime Environment (JRE) and OUI on the partition where the operating system is installed. If sufficient space is not detected, then the installation fails and an error message appears. Table 2-1 lists the hard disk space requirements, including the requirement for the starter database.

Table 2-1 Hard Disk Space Requirements for Oracle RAC

Installation Type

TEMP space

SYSTEM_DRIVE:\Program Files\Oracle

Oracle Database Home

Data FilesFoot 1

Total

Standard Edition

400 MB

100 MB

2.95 GB

3 GBFoot 2

6.45 GB

Enterprise Edition

400 MB

100 MB

2.96 GB

3 GB

6.46 GB

Footnote 1 Refers to the contents of the admin, flash_recovery_area, and oradata directories in the ORACLE_BASE directory

Footnote 2 This size can be greater, depending on the installation options selected, such as languages or additional components. If you later plan to install Oracle Database and Oracle Real Application Clusters with automated backups enabled, then include at least an additional 2 GB for data file disk space.

2.5 Verifying Hardware Requirements

To ensure that the system meets these requirements, follow these steps:

1. Determine the physical RAM size. For a computer using Windows 2003, for example, open System in the control panel and select the General tab. If the size of the physical RAM installed in the system is less than the required size, then you must install more memory before continuing.

2. Determine the size of the configured swap space (also known as paging file size). For a computer using Windows 2003, for example, open System in the control panel, select the Advanced tab, and click Settings in the Performance section.
If necessary, refer to your operating system documentation for information about how to configure additional swap space.

3. Determine the amount of free disk space on the system. For a computer using Windows 2003, for example, open My Computer, right-click the drive where the Oracle software is to be installed, and choose Properties.

4. Determine the amount of disk space available in the temp directory. This is equivalent to the total amount of free disk space, minus what will be needed for the Oracle software to be installed.
You require 400 MB of disk space available in the temp directory. If you do not have sufficient space, then first delete all unnecessary files. If the temp disk space is still less than the required amount, then set the TEMP or TMP environment variable to point to a different hard drive. For a computer using Windows 2003, for example, open the System control panel, select the Advanced tab, and click Environment Variables.

Note:
The temporary directory must reside in the same directory path on each node in the cluster.

2.6 Checking Software Requirements

Table 2-2 lists the software requirements for Oracle Clusterware and Oracle RAC 11g release 1.

Table 2-2 Oracle RAC Software Requirements for Windows Systems

Requirement

Value

System Architecture

Processor: Intel (x86), AMD64, or Intel Extended memory (EM64T)

Note: Oracle provides 32-bit (x86), 64-bit (Itanium), versions of Oracle Database with Oracle Real Application Clusters (Oracle RAC) for Windows.

The 32-bit RAC version runs on the 32-bit version of Windows. For additional information, visit My Oracle Support at the following URL:

https://metalink.oracle.com

Operating System for 32-bit Windows

Oracle Real Application Clusters for 32-bit Windows:

- Windows 2000 (32-bit) with service pack 1 or higher. All editions, including Terminal Services and Windows 2000 MultiLanguage Edition (MLE), are supported.

- Windows Server 2003 (32-bit), all editions.

- Windows Server 2003 R2 (32-bit), all editions.
Windows Multilingual User Interface Pack is supported on Windows Server 2003 and Windows Server 2003 R2.

- Windows Server 2008 (32-bit), all editions. Oracle does not support the following two operating system features: Windows Server 2008 Hyper-V and Server Core.

Compiler for 32-bit Windows

Pro*Cobol is supported with Micro Focus Net Express 5.0. Object Oriented COBOL (OOCOBOL) specifications are not supported.

The following components are supported with the Microsoft Visual C++ .NET 2002 7.0 and Microsoft Visual C++ .NET 2003 7.1 compilers:

- Oracle C++ Call Interface

- Oracle Call Interface

- Pro*C/C++

- External callouts

- Oracle XML Developer's Kit (XDK)

Network Protocol

Oracle Net foundation layer uses Oracle protocol support to communicate with the following industry-standard network protocols:

- TCP/IP

- TCP/IP with SSL

- Named Pipes

Note:
The Clusterware API Demo program is not supported on Microsoft Windows.

If you are currently running an operating system version that is not supported by Oracle Database 11g, release 11.1, such as Windows NT Server 4.0, then you must first upgrade your operating system before upgrading to Oracle Database 11g Real Application Clusters.

If you are currently running a cluster with Oracle9i Clusterware and wish to continue to use it, then you must upgrade to Oracle9i, version 9.2.0.4 to ensure compatibility between Cluster Manager Services in Oracle9i and Oracle Database 11g.

2.7 Configuring User Accounts

To install and perform administrative tasks, Oracle recommends using the same local administrative user name and password on every node in a cluster, or the same domain user name with local administrative privileges on all nodes. All nodes must be in the same domain.

Note:
If you change the user name or domain name that was used to install Oracle Clusterware after you install the product, then Oracle Clusterware services will not start properly.

If you use a domain account for the installation, then the domain user must be explicitly granted local administrative privileges on each node in the cluster. It is not sufficient if the domain user has inherited privileges from membership in a group.

Note:
Do not install Oracle RAC on a primary domain controller or backup domain controller. Although this type of installation might succeed, your system may experience many resource contention issues.

2.7.1 Managing User Accounts with User Account Control

To ensure that only trusted applications run on your computer, Windows Server 2008 provides User Account Control. If you have enabled this security feature, then depending on how you have configured it, Oracle Universal Installer prompts you for either your consent or your credentials when installing Oracle Database. Provide either the consent or your Windows Administrator credentials as appropriate.

You must have Administrator privileges to run some Oracle tools, such as Database Configuration Assistant, Net Configuration Assistant, and OPatch, or to run any tool or application that writes to any directory within the Oracle home. If User Account Control is enabled and you are logged in as the local Administrator, then you can successfully run each of these commands. However, if you are logged in as "a member of the Administrator group," then you must explicitly invoke these tasks with Windows Administrator privileges.

All of the Oracle shortcuts that require Administrator privileges are invoked as "Administrator" automatically when you click the shortcuts. However, if you run the previously mentioned tools from a Windows command prompt, then you need to run them from an Administrative command prompt. OPatch does not have a shortcut and has to be run from an Administrative command prompt.

2.8 Checking Network Requirements

Check that you have the networking hardware and internet protocol (IP) addresses required for an Oracle RAC installation.

Note:
For the most up-to-date information about supported network protocols and hardware for Oracle RAC installations, refer to the Certify pages on the OracleMetaLink Web site:

https://metalink.oracle.com

2.8.1 Network Hardware Requirements

Each node in the cluster must meet the following requirements:

- Each node must have at least two network adapters; one for the public network interface and one for the private network interface (the interconnect).

- The private and public private network connection names must be different from each other and cannot contain any multibyte language characters. The names are case-sensitive.

- The private network connection name must be the same on all nodes.

- The public network connection name must be the same on all nodes.

- The public interface on each node must be listed first in the bind order (the order in which network services access the node).

- Oracle supports the TCP/IP protocol for the public and private networks

- Windows Media Sensing must be disabled for the private network connection by setting the value of the DisableDHCPMediaSense parameter to 1.

2.8.2 IP Address Requirements

Before starting the installation, you must have the following IP addresses available for each node:

- An IP address with an associated network name registered in the domain name service (DNS) for the public interface. If you do not have an available DNS, then record the network name and IP address in the system hosts file, %SystemRoot%\system32\drivers\etc\hosts.

- One virtual IP (VIP) address with an associated network name registered in DNS. If you do not have an available DNS, then record the network name and VIP address in the system hosts file, %SystemRoot%\system32\drivers\etc\hosts. Select an address for your VIP that meets the following requirements:

- The IP address and network name are currently unused

- The VIP is on the same subnet as your public interface

Before installation, ensure that the default gateway can be accessed by a ping command. To find the default gateway, use the route print command, as described in your operating system's Help utility. After installation, configure clients to use either the VIP address or the network name associated with the VIP. If a node fails, then the node's virtual IP address fails over to another node.

- A private IP address with a host name for each private interface.
Oracle recommends that you use private network IP addresses for these interfaces (for example: 10.*.*.* or 192.168.*.*). Use the %SystemRoot%\system32\drivers\etc\hosts file on each node to associate private network names with private IP addresses.

Note:
Avoid changing host names after you complete the Oracle Clusterware installation, including adding or deleting domain qualifications. You must remove the node from the cluster configuration and add it back later with the new name.

For example, with a two-node cluster where each node has one public and one private interface, you might have the configuration shown in the following table for your network interfaces, where the hosts file is %SystemRoot%\system32\drivers\etc\hosts:

Node

Interface Name

Type

IP Address

Registered In

rac1

rac1

Public

143.46.43.100

DNS (if available, else the hosts file)

rac1

rac1-vip

Virtual

143.46.43.104

DNS (if available, else the hosts file)

rac1

rac1-priv

Private

10.0.0.1

Hosts file

rac2

rac2

Public

143.46.43.101

DNS (if available, else the hosts file)

rac2

rac2-vip

Virtual

143.46.43.105

DNS (if available, else the hosts file)

rac2

rac2-priv

Private

10.0.0.2

Hosts file

To enable VIP failover, the configuration shown in the preceding table defines the public and VIP addresses of both nodes on the same subnet, 143.46.43.0. When a node or interconnect fails, then the associated VIP is relocated to the surviving instance, enabling fast notification of the failure to the clients connecting through that VIP. If the application and client are configured with transparent application failover options, then the client is reconnected to the surviving instance.

2.8.3 Checking Network Requirements

To verify that the network hardware is configured correctly for each node, you must perform the following tasks:

- Check Hardware and IP Addresses

- (Optional) Rename Network Interfaces

- Configure Network Bind Order

- Disable Windows Media Sensing for TCP/IP

2.8.3.1 Check Hardware and IP Addresses

To verify that each node meets the requirements, follow these steps:

1. If necessary, install the network adapters for the public and private networks and configure them with either public or private IP addresses.

2. Register the host names and IP addresses for the public network interfaces in DNS.

3. For each node, register one virtual host name and IP address in DNS.

4. For each private interface on every node, add a line similar to the following to the %SystemRoot%\system32\drivers\etc\hosts file on all nodes, specifying the private IP address and associated private host name:

10.0.0.1     rac1-priv

2.8.3.2 (Optional) Rename Network Interfaces

If you need to change a network interface name, follow these steps:

1. Click Start, then Settings, then Control Panel, and then Network and Dial-up Connections

2. Right click the icon of the network interface for which you need to change the name

3. Select Rename

4. Enter and save the new name

2.8.3.3 Configure Network Bind Order

To ensure that your public interface is first in the bind order, follow these steps:

1. Right-click My Network Places and choose Properties.

2. In the Advanced menu, click Advanced Settings...

3. If the public interface name is not the first name listed under the Adapters and Bindings tab, then select it and click the arrow to move it to the top of list

4. Click OK to save the setting and then exit network setup dialog

2.8.3.4 Disable Windows Media Sensing for TCP/IP

To disable Windows Media Sensing for TCP/IP, you must set the value of the DisableDHCPMediaSense parameter to 1 on each node. Because you need to modify the Windows registry to disable Media Sensing, you should first backup the registry and confirm that you can restore it, using the methods described in your Windows documentation. Disable Media Sensing by completing the following steps on each node of your cluster:

1. Backup the Windows registry.

2. Use Registry Editor (Regedt32.exe) to view the following key in the registry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters

3. Add the following registry value:

Value Name: DisableDHCPMediaSense
Data Type: REG_DWORD -Boolean
Value: 1

4. Restart the computer.

2.9 Checking Individual Component Requirements

This section contains these topics:

- Oracle Advanced Security Requirements

- Oracle Enterprise Manager Requirements

2.9.1 Oracle Advanced Security Requirements

Satisfy hardware and software requirements to use authentication support with Oracle components. Some Oracle Advanced Security components can use a Lightweight Directory Access Protocol (LDAP) such as Oracle Internet Directory.

See Also:
Oracle Database Advanced Security Administrator's Guide

2.9.2 Oracle Enterprise Manager Requirements

All Oracle Enterprise Manager products that you use on your system must be of the same release. Older versions of Enterprise Manager are not supported with the current release.

Note:
All Oracle Enterprise Manager products, except Oracle Enterprise Manager Database Control, are released on the Enterprise Manager Grid Control installation media. Enterprise Manager Database Control is available on the Oracle Database installation media.

See Also:
Oracle Enterprise Manager Grid Control Installation and Basic Configuration available on the Enterprise Manager Grid Control installation media

2.10 Verifying Cluster Privileges

Before running Oracle Universal Installer, from the node where you intend to run the Installer, verify that you have administrative privileges on the other nodes. To do this, enter the following command for each node that is a part of the cluster where nodename is the node name:

net use \\nodename\C$

If your installation accesses drives in addition to the C: drive, then repeat this command for every node in the cluster, substituting the drive letter for each drive you plan to use.

Note:
For the installation to be successful, you must use the same local administrative user name and password on every node in a cluster or the same domain user name with local administrative privileges on all nodes.
If you use a domain account for the installation, then the domain user must be explicitly granted local administrative privileges on each node in the cluster. It is not sufficient if the domain user has inherited privileges from membership in a group.

2.11 Understanding and Using Cluster Verification Utility

Cluster Verification Utility (CVU) is a tool that performs system checks. This guide provides CVU commands to assist you with confirming that your system is properly configured for Oracle Clusterware and Oracle RAC installation.

This section describes the following topics:

- Entering Cluster Verification Utility Commands

- Using CVU to Determine if Installation Prerequisites are Complete

- Using the Cluster Verification Utility Help

- Using Cluster Verification Utility with Oracle Database 10g Release 1 or 2

- Verbose Mode and "Unknown" Output

2.11.1 Entering Cluster Verification Utility Commands

CVU is provided with two scripts: runcluvfy.bat, which is designed to be used before installation, and cluvfy, which is in the path CRS_home\bin. The script runcluvfy.bat contains temporary variable definitions which enable it to be run before installing Oracle Clusterware or Oracle Database. After you install Oracle Clusterware, use the command cluvfy to check prerequisites and perform other system readiness checks.

Before Oracle software is installed, to enter a CVU command, change directories and start runcluvfy.bat using the following syntax:

cd mountpoint
runcluvfy.bat options

In the preceding example, the variable mountpoint represents the path for the installation media and the variable options represents the CVU command options that you select. For example:

C:\> cd d:\media\db\Disk1\
C:\> runcluvfy.bat comp nodereach -n node1,node2 -verbose

By default, when you enter a CVU command, CVU provides a summary of the test. During preinstallation, Oracle recommends that you obtain detailed output by using the -verbose argument with the CVU command. The -verbose argument produces detailed output of individual checks. Where applicable, it shows results for each node in a tabular layout.

2.11.2 Using CVU to Determine if Installation Prerequisites are Complete

You can use CVU to determine which system prerequisites for installation are already completed. Use this option if you are installing Oracle 11g release 1 (11.1) on a system with a pre-existing Oracle software installation. In using this option, note the following:

- You must complete the prerequisites for using CVU, notably configuring SSH between all nodes in the cluster, before you can complete a cluster-wide status check.

- CVU can assist you by finding preinstallation steps that need to be completed, but it cannot perform preinstallation tasks

Use the following syntax to determine what preinstallation steps are completed, and what preinstallation steps must be performed

runcluvfy.bat stage -pre crsinst -n node_list

In the preceding syntax example, replace the variable node_list with the names of the nodes in your cluster, separated by commas.

For example, for a cluster with the installation media located at D:\media\db and consisting of nodes node1, node2, and node3, you would enter the following command:

C:\> cd d:\media\db\Disk1\
D:\> runcluvfy.bat stage -pre crsinst -n node1,node2,node3

Review the CVU report, and proceed to the sections of the preinstallation chapter to complete additional steps as needed.

2.11.3 Using the Cluster Verification Utility Help

The cluvfy commands have context-sensitive help that shows correct syntax usage based on the command line arguments that you enter.

If you enter an invalid CVU command, then CVU shows the correct usage for that command. For example, if you type runcluvfy.bat stage -pre dbinst, then CVU shows the correct syntax for the database preinstallation checks that CVU performs with the dbinst stage option. The following is a list of context help commands.

- cluvfy -help —CVU displays detailed CVU command information.

- cluvfy comp -list—CVU displays a list of components that can be checked, and brief descriptions of how each component is checked.

- cluvfy comp -help—CVU displays detailed syntax for each of the valid component checks.

- cluvfy stage -list—CVU displays a list of valid stages.

- cluvfy stage -help—CVU displays detailed syntax for each of the valid stage checks.

2.11.4 Using Cluster Verification Utility with Oracle Database 10g Release 1 or 2

You can use CVU on the Oracle Database 11g release 1 (11.1) media to check system requirements for Oracle Database 10g release 1 (10.1) and later installations. To use CVU to check 10. 2 installations, append the command flag -r 10gR2 to the standard CVU system check commands.

For example, to perform a verification check for a Cluster Ready Services 10. 2 installation, on a system where the installation media is located on drive D: and the cluster nodes are node1, node2, and node3, enter the following command:

C:\> cd d:\Disk1\
D:\> runcluvfy.bat stage -pre crsinst -n node1,node2,node3 -r 10gR2

Note:
If you do not specify a release version to check, then CVU checks for 11g release 1 (11.1) requirements.

2.11.5 Verbose Mode and "Unknown" Output

If you run CVU using the -verbose argument, and a CVU command responds with UNKNOWN for a particular node, then this is because the CVU cannot determine whether a check passed or failed. The following is a list of possible causes for an "Unknown" response:

Note:
For Windows Server 2008, you must have administrator privileges and you must run commands from an Administrative command prompt to run executables that reside in the Oracle Clusterware home.

- The node is down

- Executables required by CVU are missing in the Oracle_home\BIN directory, where Oracle_home represents the Oracle Clusterware home or the Oracle Database home directory

- The user account starting CVU does not have privileges to run common operating system executables on the node

- The node is missing an operating system patch, or other required software

3 Configuring Oracle Clusterware Storage

This chapter describes the Oracle Clusterware storage configuration tasks that you must complete before you start Oracle Universal Installer (OUI). This chapter includes information about the following tasks:

- Preliminary Shared Disk Preparation

- Reviewing Storage Options for Oracle Clusterware Files

- Checking for Available Shared Storage with CVU

- Configuring Storage for Oracle Clusterware Files on a Shared File System

3.1 Preliminary Shared Disk Preparation

Complete the following steps to prepare shared disks for storage:

- Disabling Write Caching

- Enabling Automounting for Windows 2003

3.1.1 Disabling Write Caching

You must disable write caching on all disks that will be used to share data between the nodes in your cluster. Perform these steps to disable write caching:

1. Click Start, then click Settings, then Control Panel, then Administrative Tools, then Computer Management, then Device Manager, and then Disk drives

2. Expand the Disk drives and double-click the first drive listed

3. Under the Disk Properties tab for the selected drive, uncheck the option that enables the write cache.
For Windows 2000, select the option Optimize for quick removal on the Policies tab.

4. Double-click each of the other drives that will be used by Oracle Clusterware and Oracle RAC and disable write caching as described in the previous step.

Caution:
Any disks that you use to store files, including database files, that will be shared between nodes, must have write caching disabled.

3.1.2 Enabling Automounting for Windows 2003

If you are using Windows 2003, then you must enable disk automounting, depending on the Oracle products that you are installing and on other conditions. You must enable automounting when using:

- Raw partitions for Oracle Real Application Clusters (Oracle RAC)

- Cluster file system for Oracle RAC

- Oracle Clusterware

- Raw partitions for single-node database installations

- Logical drives for Automatic Storage Management (ASM)

To enable automounting:

1. Enter the following commands at a command prompt:

c:\> diskpart
DISKPART> automount enable
Automatic mounting of new volumes enabled.

2. Type exit to end the diskpart session

3. Repeat steps 1 and 2 for each node in the cluster.

4. When you have prepared all of the cluster nodes in your Windows 2003 system as described in the previous steps, restart all of the nodes.

Note:
All nodes in the cluster must have automatic mounting enabled in order to correctly install Oracle RAC and Oracle Clusterware. Oracle recommends that you enable automatic mounting before creating any logical partitions for use by the database, ASM, or the Oracle Cluster File System.
You must restart each node after enabling disk automounting. After it is enabled and the node is restarted, automatic mounting remains active until it is disabled.

3.2 Reviewing Storage Options for Oracle Clusterware Files

Use the following sections to help you select your storage option and take the first steps in configuring storage:

- Overview of Oracle Clusterware Storage Options

- General Storage Considerations

- After You Have Selected Disk Storage Options

3.2.1 Overview of Oracle Clusterware Storage Options

There are two ways to store Oracle Clusterware files:

- Oracle Cluster File System (OCFS): The cluster file system Oracle provides a storage solution for Windows users.

- Raw storage: Raw storage is a logical partition created inside an extended partition that does not yet have a file system configured on it. These unformatted partitions are created and managed by Microsoft Windows disk management tools or by tools provided by third party vendors.

Note:
You cannot use Direct NFS to store Oracle Clusterware files. You can only use Direct NFS to store Oracle Database files. See Chapter 4 for more information on using Direct NFS.

3.2.2 General Storage Considerations

Oracle Clusterware files include voting disks, used to monitor cluster node status, and Oracle Cluster Registry (OCR) which contains configuration information about the cluster. The voting disks and OCR are shared files on a cluster file system. If you do not use a cluster file system, then you must place these files on shared raw devices.

The OCR is a file that contains the configuration information and status of the cluster. Oracle Universal Installer (OUI) automatically initializes the OCR during the Oracle Clusterware installation. Database Configuration Assistant uses the OCR for storing the configurations for the cluster databases that it creates.

For either storage option to meet high availability requirements, the files stored on the disk need to be protected by data redundancy, so that if one or more disks fail, the data stored on the failed disks can be recovered. This redundancy can be provided externally using Redundant Array of Independent Disks (RAID) devices, or logical volumes on more than one physical device and implement the stripe-and-mirror- everything methodology, also known as SAME. If you do not have a RAID devices or logical volumes, you can create additional copies, or mirrors, of the files on different file systems. If you choose to mirror the files, you must provide disk space for an additional Oracle Cluster Registry (OCR) file and at least two additional voting disk files.

The OCR mirror should be placed on a different disk than the primary OCR file. For voting disk file placement, ensure that each file is configured so that it does not share any hardware device or disk, or other single point of failure with the other voting disks. Any node that does not have available to it an absolute majority of voting disks configured (more than half) will be restarted.

Note:
For the most up-to-date information about supported storage options for Oracle Clusterware installations, refer to the Certify pages on the OracleMetaLink Web site:

https://metalink.oracle.com

The following table shows the storage options supported for storing Oracle Clusterware files. Oracle Clusterware files:

Storage Option

File Types Supported

OCR and Voting Disks

Oracle CRS Software

Automatic Storage Management

No

No

OCFS

Yes

Yes

Shared disk partitions

Yes

No

Direct NFS

No

No

Local storage

No

Yes

Use the following guidelines when choosing the storage options that you want to use for Oracle Clusterware:

- You can choose any combination of the supported storage options for each file type provided that you satisfy all requirements listed for the chosen storage options.

- You cannot use ASM to store Oracle Clusterware files, because these files must be accessible before any Oracle instance starts, which includes the ASM instances.

- If you do not have a storage option that provides external file redundancy, then you must configure at least three voting disk areas to provide voting disk redundancy.

Note:
Do not create an NTFS partition on a disk that you are using for OCFS.

3.2.3 After You Have Selected Disk Storage Options

When you have determined your disk storage options, you must perform the following tasks in the following order:

1: Check for available storage with CVU

Refer to "Checking for Available Shared Storage with CVU"

2: Configure shared storage for Oracle Clusterware files

- To use OCFS for Oracle Clusterware files, refer to "Configuring Storage for Oracle Clusterware Files on a Shared File System".

- To use raw devices (partitions) for Oracle Clusterware files, refer to "Configuring Storage for Oracle Clusterware Files on Raw Devices".

3.3 Checking for Available Shared Storage with CVU

To check for all shared file systems available across all nodes on the cluster, log in as the installation owner user (oracle or crs), and use the following syntax:

mountpoint\runcluvfy.bat comp ssa -n node_list

If you want to check the shared accessibility of a specific shared storage type to specific nodes in your cluster, then use the following command syntax:

mountpoint\runcluvfy.bat comp ssa -n node_list -s storageID_list

In the preceding syntax examples, the variable mountpoint is the mountpoint path of the installation media, the variable node_list is the list of nodes you want to check, separated by commas, and the variable storageID_list is the paths for the storage devices that you want to check.

For example, if you want to check the shared accessibility from node1 and node2 of storage devices \\host1\ocr_dir and \\host2\voting_disks, and your mountpoint is F:, then enter the following command:

F:\runcluvfy.bat comp ssa -n node1,node2 -s \\host1\ocr_dir,\\host2\voting_disks

If you do not specify storage device IDs in the command, then CVU searches for all available storage devices connected to the nodes on the list.

Note:
Prior to installing Oracle Clusterware, use the runcluvfy utility on the installation media. If Oracle Clusterware is installed, use the cluvfy utility instead.

3.4 Configuring Storage for Oracle Clusterware Files on a Shared File System

Oracle Universal Installer (OUI) does not suggest a default location for the Oracle Cluster Registry (OCR) or the Oracle Clusterware voting disk. If you choose to create these files on a file system, then review the following sections to complete storage requirements for Oracle Clusterware files:

- Requirements for Using a File System for Oracle Clusterware Files

- Creating Required Partitions for Oracle Clusterware on Shared File Systems

Note:
The OCR is a shared file in a cluster file system environment. If you do not use a cluster file system, then you must place this file on a shared storage device.

3.4.1 Requirements for Using a File System for Oracle Clusterware Files

To use OCFS for Oracle Clusterware files, you must comply with the following requirements:

- If you choose to place your Oracle Cluster Registry (OCR) files on a shared file system, then Oracle recommends that one of the following is true:

- The disks used for the file system are on a highly available storage device, (for example, a RAID device that implements file redundancy)

- At least three file systems are mounted, and use the features of Oracle Clusterware 11g release 1 (11.1) to provide redundancy for the OCR and voting disks

- If you use a RAID device to store the Oracle Clusterware files, you must have a partition that has at least 560 MB of available space for the OCR and voting disk.

- If you use the redundancy features of Oracle Clusterware to provide high availability for the OCR and voting disk files, then:

- One file system must have 560 MB of available space for the primary OCR and a voting disk.

- A second file system must have 560 MB of available space for the OCR mirror and a voting disk.

- A third file system must have at least 500 MB for a voting disk (500 MB is the smallest disk size that can be used by OCFS).

- If you intend to store Oracle Clusterware binaries on OCFS, then you must have at least 650 MB of available space on one of the file systems.

Note:
Oracle does not recommend the use of a shared Oracle Clusterware home directory because it creates a single point-of-failure, and removes the ability to perform rolling upgrades of Oracle Clusterware.

The total required volume size listed in the previous paragraph is cumulative. For example, to store all OCR and voting disk files on a shared file system that does not provide redundancy at the hardware level (external redundancy), you should have at least 1.4 GB of storage available over a minimum of three volumes (two separate volume locations for the OCR and OCR mirror, and one voting disk on each volume). If you use a file system that provides data redundancy, then you need only one physical disk with 560 MB of available space to store the OCR and voting disk files.

Note:
If you are upgrading from a previous release of Oracle Clusterware, and the existing OCR and voting disk files are not 280 MB in size, you do not need to change the size of the OCR or voting disks before performing the upgrade.

3.4.2 Creating Required Partitions for Oracle Clusterware on Shared File Systems

The OCR and voting disk files are stored in the OCFS data file directory (datafile_disk\cdata\clustername) where datafile_disk is the OCFS partition and clustername is the name of your cluster.

Oracle Universal Installer (OUI) does not suggest a default location for the Oracle Cluster Registry (OCR) or the Oracle Clusterware voting disk. If you choose to create these files on a file system, then perform the steps described in this section to set up the unformatted partitions on the shared disks for OCFS. Windows refers to unformatted partitions as logical drives. If you need more information about creating partitions, then refer to the Windows online help from within the disk administration tools.

1. Run Windows Disk Management from one node to create an extended partition. Use a basic disk: dynamic disks are not supported.

2. If you plan to store the Oracle Clusterware binaries on OCFS, create at least two logical partitions: one for the Oracle Clusterware home and one for the Oracle Clusterware files.
You do not need to create separate partitions for the OCR and voting disk if you plan to use OCFS. OCFS creates individual files for the OCR and voting disk.

The number of partitions used for OCFS affects performance. Therefore, you should create the minimum number of partitions needed for the OCFS option you choose.

3. If your file system is not based on a RAID system, then you must provide storage space for creating redundant Oracle Clusterware files. Create an extended partition and logical partition for each additional file system or disk drive that will be used by Oracle Clusterware. The additional storage used for Oracle Clusterware files should not share any hardware device or disk to avoid single points of failure.

To create the required partitions, perform the following steps:

1. From one of the existing nodes of the cluster, run the Windows disk administration tool as follows:

- Click Start, then select Settings, Control Panel, Administrative Tools, and then Computer Management

- Expand the Storage folder to Disk Management. Use a basic disk with a Master Boot Record (MBR) partition style as an extended partition for creating partitions.

2. Right click inside an unallocated part of an extended partition and choose Create Logical Drive. A wizard presents pages for configuring the logical drive. Select the select logical drive radio button and click Next.

3. Enter the size that you want for the partition and click Next.

4. Choose the option "Do not assign a drive letter or path", click Next, and then choose the option "Do not format this partition". Click Finish on the last page of the wizard.

5. Repeat steps 2 through 4 for the second partition and any additional partitions. An optimal configuration is one partition for the Oracle home and one partition for Oracle Clusterware files, or a single partition for both the OCR and voting disk.

6. If you are preparing drives on a Windows 2003 system, then you should restart all nodes in the cluster after you have created the logical drives.

7. Check all nodes in the cluster to ensure that the partitions are visible on all the nodes and to ensure that none of the Oracle partitions have drive letters assigned. If any partitions have drive letters assigned, then remove them by performing these steps:

- Right-click the partition in the Windows disk administration tool

- Select "Change Drive Letters and Paths..." from the menu

- Click Remove in the "Change Drive Letter and Paths" window

3.5 Configuring Storage for Oracle Clusterware Files on Raw Devices

To use raw devices, you must create a separate partition for each file. If you are using three raw devices to store the Oracle Clusterware files, then:

- Create two partitions on the first raw device: one 280 MB partition for the voting disk and one 280 MB partition for the OCR.

- Create two partitions on the second raw device: one 280 MB partition for the voting disk and one 280 MB partition for the OCR mirror.

- Create a single partition on the third raw device for the voting file.

If you are not using OCFS or ASM for your data files, then you must also create raw partitions for your database files as described in "Configuring Logical Volumes or Unformatted Partitions".

To create and configure raw volumes or partitions, use the disk administration tools provided by the operating system or third party vendors. The following administration tools are provided by the operating system:

- Disk Management snap-in to manage disks.
To access this tool, type diskmgmt.msc at the command prompt. Alternatively, from the Start menu, select Programs, then Administrative Tools, then Computer Management. Then select the Disk Management node in the Storage tree.

- A command line tool to manage disks.
To access this tool, type diskpart.exe at the command prompt.

Note:
If you need to download the diskmgmt.msc tool, consult Microsoft documentation on the Microsoft Web site

http://www.microsoft.com/

See Also:
The online help or documentation for the administration tool you are using

You can use the diskpart tool command create partition to create primary or extended partitions, or create logical drives. The following example uses the diskpart tool to create a 280 MB extended partition on disk 100. In this syntax, diskpart.exe is the command line tool for managing disks.

c:\> diskpart.exe
DISKPART> select disk 100
DISKPART> create partition extended size=280

Optionally, mount the partition on an NTFS folder instead of assigning a drive letter:

DISKPART> assign mount=C:\mnt\raw_1

Note:
Be aware of the following restrictions for partitions:

- You cannot use primary partitions for storing Oracle Clusterware files while running the OUI to install Oracle Clusterware as described in Chapter 5, "Installing Oracle Clusterware". You must create logical drives inside extended partitions for the disks to be used by Oracle Clusterware files and Oracle ASM.

- With 32-bit Windows, you cannot create more than four primary disk partitions for each disk. One of the primary partitions can be an extend partition, which can then be subdivided into multiple logical partitions.

- You can assign mount points only to primary partitions and logical drives.

- You must create logical drives inside extended partitions for the disks to be used by Oracle Clusterware files and Oracle ASM.

- Oracle recommends that you limit the number of partitions you create on a single disk to prevent disk contention. Therefore, you may prefer to use extended partitions rather than primary partitions.

For these reasons, you might prefer to use extended partitions for storing Oracle software files and not primary partitions.

4 Configuring Oracle Real Application Clusters Storage

This chapter describes the Oracle Real Application Clusters (Oracle RAC) storage configuration tasks that you must complete before you start Oracle Universal Installer (OUI). This chapter includes information about the following tasks:

- Preliminary Shared Disk Preparation

- Reviewing Storage Options for Oracle Database and Recovery Files

- Configuring Disks for Automatic Storage Management

- Configuring Logical Volumes or Unformatted Partitions

- Requirements for Files Managed by Oracle

4.1 Preliminary Shared Disk Preparation

Complete the following steps to prepare shared disks for storage:

- Disabling Write Caching

- Enabling Automounting for Windows 2003

4.1.1 Disabling Write Caching

You must disable write caching on all disks that will be used to share data between the nodes in your cluster. Perform these steps to disable write caching:

1. Click Start, then click Settings, then Control Panel, then Administrative Tools, then Computer Management, then Device Manager, and then Disk drives.

2. Expand the Disk drives and double-click the first drive listed.

3. Under the Disk Properties tab for the selected drive, uncheck the option that enables the write cache.
For Windows 2000, select the option Optimize for quick removal on the Policies tab.

4. Double-click each of the other drives that will be used by Oracle Clusterware and Oracle RAC and disable write caching as described in the previous step.

Caution:
Any disks that you use to store files, including database files, that will be shared between nodes, must have write caching disabled.

4.1.2 Enabling Automounting for Windows 2003

If you are using Windows 2003, then you must enable disk automounting, depending on the Oracle products that you are installing and on other conditions. You must enable automounting when using:

- Raw, or unformatted, partitions for Oracle Real Application Clusters (Oracle RAC)

- Cluster file system for Oracle RAC

- Oracle Clusterware

- Unformatted partitions for single-node database installations

- Logical drives for Automatic Storage Management (ASM)

To enable automounting:

1. Enter the following commands at a command prompt:

c:\> diskpart
DISKPART> automount enable
Automatic mounting of new volumes enabled.

2. Type exit to end the diskpart session

3. Repeat steps 1 and 2 for each node in the cluster.

4. When you have prepared all of the cluster nodes in your Windows 2003 system as described in the previous steps, restart all of the nodes.

Note:
All nodes in the cluster must have automatic mounting enabled in order to correctly install Oracle RAC and Oracle Clusterware. Oracle recommends that you enable automatic mounting before creating any logical partitions for use by the database, ASM, or the Oracle Cluster File System.
You must restart each node after enabling disk automounting. After it is enabled and the node is restarted, automatic mounting remains active until it is disabled.

4.2 Reviewing Storage Options for Oracle Database and Recovery Files

This section describes supported options for storing Oracle Database software and database files. It includes the following sections:

- Overview of Oracle Database and Recovery File Storage Options

- General Storage Considerations

- Guidelines for Placing Oracle Data Files on a File System

- Guidelines for Placing Oracle Recovery Files on a File System

- After You Have Selected Disk Storage Options

4.2.1 Overview of Oracle Database and Recovery File Storage Options

There are four ways to store Oracle Database and recovery files on shared disks:

- ASM (database files only): ASM is an integrated, high-performance database file system and disk manager for Oracle files. Because ASM requires an Oracle Database instance, it cannot contain Oracle software, but you can use ASM to manage database and recovery files. If you are using Oracle Standard Edition and Real Application Clusters (Oracle RAC), then you must use ASM to store all the database files.

- Oracle Cluster File System (OCFS): Note that if you intend to use OCFS for your database files, then you should create partitions large enough for the all the database and recovery files when you create partitions for use by Oracle Database.

Note:
If you want to have a shared Oracle home directory for all nodes, then you must use OCFS.

- Direct Network File Systems (NFS): You can configure Oracle Database to access NFS V3 servers directly using an Oracle internal Direct NFS client.

- Raw storage: If you choose to use raw storage instead of ASM or OCFS for storing data files, then you must manually manage the disk space with Microsoft Windows disk management tools or by tools provided by third party vendors. Also, you must create individual raw volumes or partitions for every shared database file that uses raw storage.

The storage option that you choose for recovery files can be the same as or different from the option that you choose for the database files. However, you cannot use raw storage to store recovery files.

4.2.2 General Storage Considerations

For all installations, you must choose the storage options that you want to use for Oracle Clusterware files and Oracle Database files. If you want to enable automated backups during the installation, then you must also choose the storage option that you want to use for recovery files (the flash recovery area). You do not have to use the same storage option for each file type.

For single-instance Oracle Database installations using Oracle Clusterware for failover, you must use OCFS, ASM, or shared raw disks if you do not want the failover processing to include dismounting and remounting the disks containing your database files.

The following table shows the storage options supported for storing Oracle Clusterware files, Oracle Database files, and Oracle Database recovery files. Oracle Clusterware files include the Oracle Cluster Registry (OCR) and the voting disk. Oracle Database files include data files, control files, redo log files, the server parameter file, and the password file.

Note:
For the most up-to-date information about supported storage options for Oracle RAC installations, refer to the Certify pages on the OracleMetaLink Web site:

https://metalink.oracle.com

Storage Option

File Types Supported

Oracle Clusterware

Database

Recovery

ASM

No

Yes

Yes

Cluster file system (OCFS)

Yes

Yes

Yes

Shared raw storage

Yes

Yes

No

Direct NFS

No

Yes

Yes

Use the following guidelines when choosing the storage options that you want to use for each file type:

- If you meet all of the requirements listed for the chosen storage options, then you can choose any combination of the supported storage options for each file type.

- Oracle recommends that you choose ASM as the storage option for database and recovery files.

- For Standard Edition cluster installations, ASM is the only supported storage option for database or recovery files.

- If you intend to use ASM with Oracle RAC, and you are configuring a new ASM instance, then you must ensure that your system meets the following conditions:

- All nodes on the cluster have Oracle Clusterware release 1 (11.1) installed

- Any existing ASM instance on any node in the cluster is shut down

- If you intend to upgrade an existing Oracle RAC database, or an Oracle RAC database with ASM instances, then you must ensure that your system meets the following conditions:

- The Oracle RAC database or Oracle RAC database with ASM instance is running on the node from which the Oracle Universal Installer (OUI) and Database Configuration Assistant (DBCA) is run

- The Oracle RAC database or Oracle RAC database with ASM instance is running on the same nodes that you intend to make members of the new cluster installation. For example, if you have an existing Oracle RAC database running on a three node cluster, then you must install the upgrade on all three nodes. You cannot attempt to upgrade only 2 nodes of the cluster.

4.2.3 Guidelines for Placing Oracle Data Files on a File System

If you decide to place the Oracle data files on OCFS, then use the following guidelines when deciding where to place them:

- You can choose either a single cluster file system or more than one cluster file system to store the data files:

- If you want to use a single cluster file system, then choose a cluster file system on a physical device that is dedicated to the database.
For best performance and reliability, choose a RAID device or a logical volume on more than one physical device and implement the stripe-and-mirror-everything methodology, also known as SAME.

- If you want to use more than one cluster file system, then choose cluster file systems on separate physical devices or partitions that are dedicated to the database.
This method enables you to distribute physical I/O and create separate control files on different devices for increased reliability. It also enables you to fully implement Oracle Optimal Flexible Architecture (OFA) guidelines. To implement this method, you must choose either the Advanced database creation option or choose the Custom installation type during installation.

- If you intend to create a preconfigured database during the installation, then the cluster file system (or systems) that you choose must have at least 4 GB of free disk space.
For production databases, you must estimate the disk space requirement depending on the use you want to make of the database.

- For optimum performance, the cluster file systems that you choose should be on physical devices that are used only by the database.

Note:
You must not create an NTFS partition on a disk that you are using for OCFS.

- The default location suggested by Oracle Universal Installer for the database file directory is a subdirectory of the Oracle base directory. However, this default location is not appropriate for Oracle RAC production databases.

4.2.4 Guidelines for Placing Oracle Recovery Files on a File System

You must choose a location for recovery files prior to installation only if you intend to enable automated backups during installation.

If you choose to place the Oracle recovery files on a cluster file system, then use the following guidelines when deciding where to place them:

- To prevent disk failure from making the database files as well as the recovery files unavailable, place the recovery files on a cluster file system that is on a different physical disk from the database files.

Note:
Alternatively use an ASM disk group with a normal or high redundancy level for either or both file types, or use external redundancy.

- The cluster file system that you choose should have at least 3 GB of free disk space.
The disk space requirement is the default disk quota configured for the flash recovery area (specified by the DB_RECOVERY_FILE_DEST_SIZE initialization parameter).

If you choose the Custom installation type or the Advanced database configuration option, then you can specify a different disk quota value. After you create the database, you can also use Oracle Enterprise Manager to specify a different value.

See Also:
Oracle Database Backup and Recovery Basics for more information about sizing the flash recovery area.

- The default location suggested by Oracle Universal Installer for the recovery area directory is a subdirectory of the Oracle base directory. However, this default location is not appropriate for Oracle RAC production databases.

4.2.5 After You Have Selected Disk Storage Options

When you have determined your disk storage options, you must perform the following tasks in the following order:

1: Check for available storage with CVU

Refer to "Checking for Available Shared Storage with CVU" .

2: Configure storage for Oracle Database files and recovery files

- To use a OCFS for database or recovery file storage, refer to "Configuring Storage for Oracle Database Files on a Shared File System" to create volumes or partitions with sizes sufficient to store Oracle Database files and binaries.

- To use ASM for database or recovery file storage, refer to "Configuring Disks for Automatic Storage Management".

- To use raw devices (partitions) for database file storage, refer to "Configuring Logical Volumes or Unformatted Partitions".

- To use a network file system (NFS) for database file or recovery file storage, refer to "Configuring Direct NFS Storage for Data Files".

4.3 Configuring Storage for Oracle Database Files on a Shared File System

To use OCFS for your Oracle home and data files, at a minimum, the following partitions must exist before you run OUI to install Oracle Clusterware:

- 3 GB or larger partition for the Oracle home, if you want a shared Oracle home

- 3 GB or larger partition for the Oracle Database data files and recovery files

To use ASM for your database files, you only need to perform the actions related to creating partitions for the Oracle home and the Oracle Clusterware files.

Log in to Windows with Administrative privileges and perform the steps described in this section to set up the shared disk raw partitions for OCFS. Windows refers to raw partitions as logical drives. If you need more information about creating partitions, then refer to the Windows online help from within the disk administration tools.

1. Run Windows Disk Management from one node to create an extended partition. Use a basic disk: dynamic disks are not supported.

2. Create a partition for the Oracle Database data files and recovery files, and optionally create a second partition for the Oracle home.
The number of partitions used for OCFS affects performance. Therefore, you should create the minimum number of partitions needed for the OCFS option you choose.

Note:

Oracle supports installing the database into multiple Oracle Homes on a single system. This allows flexibility in deployment and maintenance of the database software. For example, it allows you to run different versions of the database simultaneously on the same system, or it allows you to upgrade specific database or Automatic Storage Management instances on a system without affecting other running databases.

However, when you have installed multiple Oracle Homes on a single system, there is also some added complexity introduced that you may need to take into account to allow these Oracle Homes to coexist. For more information on this topic, please see Note 460054.1 on OracleMetaLink.

To create the required partitions, perform the following steps:

1. From one of the existing nodes of the cluster, run the Windows disk administration tool as follows:

- Click Start, then select Settings, Control Panel, Administrative Tools, and then Computer Management

- Expand the Storage folder to Disk Management. Use a basic disk with a Master Boot Record (MBR) partition style as an extended partition for creating partitions.

2. Right click inside an unallocated part of an extended partition and choose Create Logical Drive. A wizard presents pages for configuring the logical drive. Select the select logical drive radio button and click Next.

3. Enter the size that you want for the partition and click Next.

4. Choose the option "Do not assign a drive letter or path", click Next, and then choose the option "Do not format this partition". Click Finish on the last page of the wizard.

5. Repeat steps 2 through 4 for the second and any additional partitions. An optimal configuration is one partition for the Oracle home and one partition for Oracle Database files.

6. If you are preparing drives on a Windows 2003 system, then you should restart all nodes in the cluster after you have created the logical drives.

7. Check all nodes in the cluster to ensure that the partitions are visible on all the nodes and to ensure that none of the Oracle partitions have drive letters assigned. If any partitions have drive letters assigned, then remove them by performing these steps:

- Right-click the partition in the Windows disk administration tool

- Select "Change Drive Letters and Paths..." from the menu

- Click Remove in the "Change Drive Letter and Paths" window

4.4 Configuring Disks for Automatic Storage Management

This section describes how to configure disks for use with ASM. Before you configure the disks, you must determine the number of disks and the amount of free disk space that you require. The following sections describe how to identify the requirements and configure the disks for ASM:

- General Steps for Configuring Automatic Storage Management

- Step 1: Identifying Storage Requirements for Automatic Storage Management

- Step 2 (Optional): Using an Existing Automatic Storage Management Disk Group

- Step 3: Creating DAS or SAN Disk Partitions for Automatic Storage Management

- Step 4: Manually Configuring Disks for Automatic Storage Management

4.4.1 General Steps for Configuring Automatic Storage Management

Follow these steps to configure ASM:

1. Identify your site's storage requirements.

2. Optionally, use an existing ASM disk group.

3. If you are creating a new ASM disk group, then create partitions for DAS or SAN disks.

4. Use one of the following methods to complete the ASM configuration:

- If you plan to install Oracle Database using interactive mode, then Oracle Universal Installer prompts you for the ASM disk configuration information during the installation.

- If you plan to install Oracle Database using noninteractive mode, then you must configure the disks manually before performing the installation.

4.4.2 Step 1: Identifying Storage Requirements for Automatic Storage Management

To identify the storage requirements for using ASM, you must determine how many devices and the amount of free disk space that you require. To complete this task, follow these steps:

1. Determine whether you want to use ASM for Oracle data files, recovery files, or both.

Note:
You do not have to use the same storage mechanism for data files and recovery files. One can use the file system, while the other uses ASM. If you plan to use ASM for both data files and recovery files, then you should create separate ASM disk groups for the data files and the recovery files.

If you plan to enable automated backups during the installation, then you can choose ASM as the storage mechanism for recovery files by specifying an ASM disk group for the flash recovery area. Depending how you choose to create a database during the installation, you have the following options:

- If you select an installation method that runs DBCA in interactive mode (for example, by choosing the Advanced database configuration option), then you can decide whether you want to use the same ASM disk group for data files and recovery files. You can also choose to use different disk groups for each file type. Ideally, you should create separate ASM disk groups for data files and recovery files.
The same choice is available to you if you use DBCA after the installation to create a database.

- If you select an installation type that runs DBCA in non-interactive mode, then you must use the same ASM disk group for data files and recovery files.

2. Choose the ASM redundancy level that you want to use for the ASM disk group.
The redundancy level that you choose for the ASM disk group determines how ASM mirrors files in the disk group, and determines the number of disks and amount of disk space that you require. The redundancy levels are as follows:

- External redundancy
An external redundancy disk group requires a minimum of one disk device. The effective disk space in an external redundancy disk group is the sum of the disk space in all of its devices.

Because ASM does not mirror data in an external redundancy disk group, Oracle recommends that you use only RAID or similar devices that provide their own data protection mechanisms as disk devices in this type of disk group.

- Normal redundancy
In a normal redundancy disk group, ASM uses two-way mirroring by default (except for the control file, which is mirrored three ways), to increase performance and reliability. A normal redundancy disk group requires a minimum of two disk devices, or two failure groups. The effective disk space in a normal redundancy disk group is half the sum of the disk space in all of its devices.

For most installations, Oracle recommends that you use normal redundancy disk groups.

- High redundancy
In a high redundancy disk group, ASM uses three-way mirroring to increase performance and provide the highest level of reliability. A high redundancy disk group requires a minimum of three disk devices (or three failure groups). The effective disk space in a high redundancy disk group is one-third the sum of the disk space in all of its devices.

While high redundancy disk groups do provide a high level of data protection, you must consider the higher cost of additional storage devices before deciding to use this redundancy level.

3. Determine the total amount of disk space that you require for the data files and recovery files.
Use the following table to determine the minimum number of disks and the minimum disk space requirements for the installation:

Redundancy Level

Minimum Number of Disks

Data Files

Recovery FIles

Both File Types

External

1

1.15 GB

2.3 GB

3.45 GB

Normal

2

2.3 GB

4.6 GB

6.9 GB

High

3

3.45 GB

6.9 GB

10.35 GB

If an ASM instance already exists on the system, then you can use an existing disk group to meet these storage requirements. If necessary, you can add disks to an existing disk group during the installation. The next set of procedures describes how to identify existing disk groups and determine the free disk space that they contain.

4. Optionally identify failure groups for the ASM disk group devices.

Note:
You need to complete this step only to use an installation method that runs DBCA in interactive mode. Do this if, for example, you choose the Custom installation type or the Advanced database configuration option. Other installation types do not enable you to specify failure groups.

If you intend to use a normal or high redundancy disk group, then you can further protect your database against hardware failure by associating a set of disk devices in a custom failure group. Failure groups define ASM disks that share a common potential failure mechanism. For more information about ASM failure groups, refer to Oracle Database Storage Administrator's Guide.

Note:
If you define custom failure groups, you must specify a minimum of two failure groups for normal redundancy disk groups and three failure groups for high redundancy disk groups.

5. If you are sure that a suitable disk group does not exist on the system, then install or identify appropriate disk devices to add to a new disk group. Use the following guidelines when identifying appropriate disk devices:

- All of the devices in an ASM disk group should be the same size and have the same performance characteristics.

- Do not specify two or more partitions on a single physical disk as ASM disks in the same disk group. ASM expects each device for a disk group to be on a separate physical disk.

- Although you can specify a logical volume as a device in an ASM disk group, Oracle does not recommend their use. Logical volume managers can hide the physical disk architecture, preventing ASM from optimizing I/O across the physical devices.

Tip:
As you progress through the following steps, make a list of the raw device names you intend to use and have it available during your database or ASM installation.

4.4.3 Step 2 (Optional): Using an Existing Automatic Storage Management Disk Group

To use ASM as the storage option for either database or recovery files, and an existing ASM disk group already exists, you have the following options, depending on the installation method that you select:

- If you select an installation method that runs DBCA in interactive mode (for example, by choosing the Advanced database configuration option for example), then you can decide whether you want to create a new disk group, or use an existing disk group.
The same choice is available to you if you use DBCA after the installation to create a database.

- If you select an installation type that runs DBCA in non-interactive mode, then you must choose an existing disk group for the new database; you cannot create a new disk group. However, you can add disk devices to an existing disk group if the existing disk group has insufficient free space for your requirements.

Note:
The ASM instance that manages the existing disk group can be running in a different Oracle home directory.

To determine whether an existing ASM disk group exists, or to determine whether there is sufficient disk space in a disk group, you can use Oracle Enterprise Manager, either Grid Control or Database Control. Alternatively, you can use the following procedure:

1. In the Services Control Panel, make sure that the OracleASMService+ASMn service, where n is the node number, has started.

2. Open a Windows command prompt and temporarily set the ORACLE_SID environment variable to specify the appropriate value for the ASM instance that you want to use.
For example, if the ASM SID is named +ASM1, then enter a setting similar to the following:

C:\> set ORACLE_SID = +ASM1

3. Use SQL*Plus to connect to the ASM instance as the SYS user with the SYSASM privilege and start the instance if necessary with a command similar to the following:

C:\> sqlplus /nolog
SQL> CONNECT SYS AS SYSASM
Enter password: sys_password
Connected to an idle instance.

SQL> STARTUP

4. Enter the following command to view the existing disk groups, their redundancy level, and the amount of free disk space in each disk group:

SQL> SELECT NAME,TYPE,TOTAL_MB,FREE_MB FROM V$ASM_DISKGROUP;

5. From the output, identify a disk group with the appropriate redundancy level and note the free space that it contains.

6. If necessary, install, or identify the additional disk devices required to meet the storage requirements listed in the previous section.

Note:
If you are adding devices to an existing disk group, then Oracle recommends that you use devices that have the same size and performance characteristics as the existing devices in that disk group.

4.4.4 Step 3: Creating DAS or SAN Disk Partitions for Automatic Storage Management

To use direct-attached storage (DAS) or storage area network (SAN) disks for ASM, each disk must have a partition table. Oracle recommends creating exactly one partition for each disk that encompasses the entire disk.

Note:
You can use any physical disk for ASM, as long as it is partitioned. However, you cannot use network-attached storage (NAS) or Microsoft dynamic disks.

Use Microsoft Computer Management utility or the command line tool diskpart to create the partitions. Ensure that you create the partitions without drive letters. After you have created the partitions, the disks can be configured.

See Also:
"Assigning Logical Names" for more information about using diskpart to create a partition

4.4.5 Step 4: Manually Configuring Disks for Automatic Storage Management

To use ASM with DAS or SAN devices, the disks must be stamped with a header. If you install Oracle Database in interactive mode, then Oracle Universal Installer configures the disks' headers during the installation process.

However, if you plan to install Oracle Database in noninteractive mode, then you need to configure the disks manually before installation either by using asmtoolg (GUI version) or using asmtool (command line version). You can also use these tools to reconfigure the disks after installation. The asmtoolg and asmtool utilities only work on partitioned disks; you cannot use ASM on unpartitioned disks.

The following section describes the asmtoolg and asmtool functions and commands.

4.4.5.1 Overview of asmtoolg and asmtool

The asmtoolg and asmtool tools associate meaningful, persistent names with disks to facilitate using those disks with ASM. ASM uses disk strings to operate more easily on groups of disks at once. The names that asmtoolg or asmtool create make this easier than using Windows drive letters.

All disk names created by asmtoolg or asmtool begin with the prefix ORCLDISK followed by a user-defined prefix (the default is DATA), and by a disk number for identification purposes. You can use them as raw devices in the ASM instance by specifying a name \\.\ORCLDISKprefixn, where prefix either can be DATA, or can be a value you supply, and where n represents the disk number.

To configure your disks with asmtoolg, refer to the section "Using asmtoolg (Graphical User Interface)". To configure the disks with asmtool, refer to the section "Using asmtool (Command Line)".

4.4.5.2 Using asmtoolg (Graphical User Interface)

Use asmtoolg, a graphical interface, to create device names; use asmtoolg to add, change, delete, and examine the devices available for use in ASM.

To add or change disk stamps:

1. In the installation media labeled Oracle Database 11g Release 1 (11.1), navigate to db\asmtool, and double-click asmtoolg.
If Oracle Database is already installed, then navigate to ORACLE_BASE\ORACLE_HOME\bin, and double-click asmtoolg.

2. Select the Add or change label option, and then click Next.
asmtoolg shows the devices available on the system. Unrecognized disks are labeled as a "Candidate device." Unformatted partitions are labeled as "Oracle raw device file." Stamped ASM disks are labeled as "Stamped ASM disk," and unstamped ASM disks are labeled as "Unstamped ASM disks." The tool also shows disks that are recognized by Windows as a file system (such as NTFS). These disks are not available for use as ASM disks, and cannot be selected. In addition, Microsoft Dynamic disks are not available for use as ASM disks.

If necessary, follow the steps under "Creating Partitions for Logical Volumes" to create disk partitions for the ASM instance.

3. On the Stamp Disks screen, select the disks to stamp.
For ease of use, ASM can generate unique stamps for all of the devices selected for a given prefix. The stamps are generated by concatenating a number with the prefix specified. For example, if the prefix is DATA, then the first ASM link name is ORCLDISKDATA0.

You can also specify the stamps of individual devices.

4. Optionally, select a disk to edit the individual stamp (ASM link name).

5. Click Next.

6. Click Finish.

To delete disk stamps:

1. Select the Delete labels option, then click Next.
The delete option is only available if disks exist with stamps. The delete screen shows all stamped ASM disks.

2. On the Delete Stamps screen, select the disks to unstamp.

3. Click Next.

4. Click Finish.

4.4.5.3 Using asmtool (Command Line)

asmtool is a command-line interface for stamping disks. It has the following options:

Option

Description

Example

-add

Adds or changes stamps. You must specify the hard disk, partition, and new stamp name. If the disk is a raw device or has an existing ASM stamp, then you must specify the -force option.
If necessary, follow the steps under "Creating Partitions for Logical Volumes" to create disk partitions for the ASM instance.

asmtool -add [-force]
\Device\Harddisk1\Partition1 ORCLDISKASM0
\Device\Harddisk2\Partition1 ORCLDISKASM2...

-addprefix

Adds or changes stamps using a common prefix to generate stamps automatically. The stamps are generated by concatenating a number with the prefix specified. If the disk is a raw device or has an existing ASM stamp, then you must specify the -force option.

asmtool -addprefix ORCLDISKASM [-force]
\Device\Harddisk1\Partition1
\Device\Harddisk2\Partition1...

-list

List available disks. The stamp, windows device name, and disk size in megabytes are shown. Some disks may be file systems, and cannot be stamped. If the disk is a raw device or has an existing ASM stamp, then you must specify the -force option.

asmtool -list [-force]

-delete

Removes existing stamps from disks.

asmtool -delete ORCLDISKASM0 ORCLDISKASM1...

Note:
For -add, -addprefix, and -delete, asmtool will notify any ASM instances on the local machine and other nodes in the cluster if available, to rescan the available disks.

4.5 Configuring Logical Volumes or Unformatted Partitions

If you have an array of disks managed by a logical volume manager, or have decided to use unformatted devices to store data files, you need to prepare the disks for use by Oracle Database.

This section contains the following topics:

- Creating Partitions for Logical Volumes

- Assigning Logical Names

- Creating the DBCA Raw Device Mapping File

4.5.1 Creating Partitions for Logical Volumes

You must create the following logical volumes or partitions prior to installing Oracle Database.

Number

Partition Size (MB)

Purpose and Sample Logical Volume Name

1

800

SYSTEM tablespace: dbname_system_raw_800m

1

400 + (Number of instances * 250)

SYSAUX tablespace: dbname_sysaux_raw_900m
This SYSAUX entry is for a two-node cluster.

1 for each instance

500

UNDOTBS1 tablespace: dbname_undotbs1_raw_500m

1

180

EXAMPLE tablespace: dbname_example_raw_180m

1

120

USERS tablespace: dbname_users_raw_120m

2 for each instance

120

Two online redo log files (where m is the thread number and n is the log number, 1 or 2): dbname_redom_n_raw_120m

2

110

First and second control files:
dbname_control[1|2]_raw_110m

1

250

TEMP tablespace: dbname_temp_raw_250m

1

5

Server parameter file (SPFILE): dbname_spfile_raw_5m

1

5

Password file: dbname_pwdfile_raw_5m

To create and configure logical volumes or partitions, use the disk administration tools provided by the operating system or third party vendors. The following administration tools are provided by the operating system:

- Disk Management snap-in to manage disks.
To access this tool, type diskmgmt.msc at the command prompt. Alternatively, from the Start menu, select Programs, then Administrative Tools, then Computer Management. Then select the Disk Management node in the Storage tree.

- Command line tool to manage disks.
To access this tool, type diskpart.exe at the command prompt.

Note:
If you need to download the diskmgmt.msc tool, consult Microsoft documentation on the Microsoft Web site

http://www.microsoft.com/

See Also:
The online help or documentation for the administration tool you are using

You can use the diskpart tool command create partition to create primary or extended partitions, or create logical drives. The following example uses the diskpart tool to create a 120 MB extended partition on disk 100. In this syntax, diskpart.exe is the command line tool for managing disks.

c:\> diskpart.exe
DISKPART> select disk 100
DISKPART> create partition extended size=120

Note:
Be aware of the following restrictions for partitions:

- You cannot use primary partitions for storing Oracle Clusterware files while running the OUI to install Oracle Clusterware as described in Chapter 5, "Installing Oracle Clusterware". You must create logical drives inside extended partitions for the disks to be used by Oracle Clusterware files and Oracle ASM.

- With 32-bit Windows, you cannot create more than four primary disk partitions for each disk. One of the primary partitions can be an extend partition, which can then be subdivided into multiple logical partitions.

- You can assign mount points only to primary partitions and logical drives.

- You must create logical drives inside extended partitions for the disks to be used by Oracle Clusterware files and Oracle ASM.

- Oracle recommends that you limit the number of partitions you create on a single disk to prevent disk contention. Therefore, you may prefer to use extended partitions rather than primary partitions.

For these reasons, you might prefer to use extended partitions for storing Oracle software files and not primary partitions.

4.5.2 Assigning Logical Names

After creating volumes, assign logical names for Oracle Database. You can assign names to partitions by using importSYMLinks from the command line, or by using Oracle Object Link Manager. To use Oracle Object Link Manager to create persistent symbolic links to the corresponding raw partitions, run the command CRS_home\bin\GUIOracleObjManager.exe

Note:
For Windows Server 2008, you must have administrator privileges and you must run commands from an Administrative command prompt to run executables that reside in the Oracle Clusterware home.

4.5.3 Creating the DBCA Raw Device Mapping File

Note:
You must complete this procedure only if you are using raw devices for database files. You do not specify the raw devices for the Oracle Clusterware files in the DBCA raw device mapping file.

To enable DBCA to identify the appropriate raw partition symbolic links for each database file, you must create a raw device mapping file, as follows:

1. Set the ORACLE_BASE environment variable to specify the Oracle base directory that you identified or created previously, as in this example:

C:\>set ORACLE_BASE = E:\oracle

2. Create a database subdirectory under the Oracle base directory as in this example:

C:\>mkdir E:\oracle\dbname

where dbname is the name of the database that you chose previously.

3. Change directory to the %ORACLE_BASE%\dbname directory.

4. Using any text editor, create a file called conf.txt. The file should have the following characteristics:

- Each line in the file must have the following format:

database_object_identifier = symbolic link name

- For your Oracle RAC database, the file should specify all the shared files for the Oracle RAC database being created. You should have created logical volumes or disk partitions for each of these files, for example:

- The data files for each tablespace

- At least one automatic undo tablespace data file

- The temporary tablespace tempfile

- At least two redo log files (redon_1, redon_2) for each instance where n is the instance number

- The system parameter file (SPFILE)

- The password file

- At least two control files (control1, control2)

Note:
In Windows, by default, \ represents the escape key. To enter a backslash as part of a script, you must enter it in as a string literal. This means that when configuring the mapping file, for Windows to read the mapping file with the path \\.\, you must enter the path as \\\\.\\. Windows reads this as "escape backslash escape backslash period escape backslash."

The following syntax example is for a mapping file for a two-instance Oracle RAC cluster:

system=\\\\.\\dbname_SYSTEM
sysaux=\\\\.\\dbname_SYSAUX
spfile=\\\\.\\dbname_SPFILE
users=\\\\.\\dbname_USERS
temp=\\\\.\\dbname_TEMP
undotbs1=\\\\.\\dbname_UNDOTBS1
undotbs2=\\\\.\\dbname_UNDOTBS2
control1=\\\\.\\dbname_CONTROL1
control2=\\\\.\\dbname_CONTROL2
redo1_1=\\\\.\\dbname_REDO1_1
redo1_2=\\\\.\\dbname_REDO1_2
redo2_1=\\\\.\\dbname_REDO2_1
redo2_2=\\\\.\\dbname_REDO2_2
example=\\\\.\\dbname_EXAMPLE
pwdfile=\\\\.\\dbname_pwdfile

5. Save the file and note the file name that you specified.

6. You may optionally set an environment variable, DBCA_RAW_CONFIG, to specify the full path to this file. For the Oracle base defined in Step 1, you would use the following command:

C:\>set DBCA_RAW_CONFIG=E:\oracle\dbname\conf.txt

4.6 Configuring Direct NFS Storage for Data Files

This section contains the following information about Direct NFS:

- About Direct NFS Storage

- Using the Oranfstab File with Direct NFS

- Mounting NFS Storage Devices with Direct NFS

- Specifying Network Paths with the Oranfstab File

- Enabling Direct NFS Client

- Disabling Direct NFS Client

- Checking NFS Buffer Size

4.6.1 About Direct NFS Storage

With Oracle Database 11g Release 1 (11.1), you can configure Oracle Database to access NFS V3 servers directly using an Oracle internal Direct NFS client. Some NFS file servers require NFS clients to connect using reserved ports. If your filer is running with reserved port checking, then you must disable it for Direct NFS to operate. To disable reserved port checking, consult your NFS file server documentation.

4.6.2 Using the Oranfstab File with Direct NFS

If you use Direct NFS, then you must create a new configuration file, oranfstab, to specify the options/attributes/parameters that enable Oracle Database to use Direct NFS. You must add the oranfstab file to the ORACLE_BASE\ORACLE_HOME\dbs directory.

For Oracle RAC installations, if you want to use Direct NFS, then you must replicate the oranfstab file on all of the nodes. You must also keep all of the oranfstab files synchronized on all nodes.

When the oranfstab file is placed in ORACLE_BASE\ORACLE_HOME\dbs, the entries in the file are specific to a single database. All nodes running an Oracle RAC database should use the same ORACLE_BASE\ORACLE_HOME\dbs\oranfstab file.

4.6.3 Mounting NFS Storage Devices with Direct NFS

Direct NFS determines mount point settings to NFS storage devices based on the configuration information in oranfstab. If Oracle Database is unable to open an NFS server using Direct NFS, then an error message is written into the Oracle alert and trace files indicating that Direct NFS could not be established.

The Oracle files on the NFS server that are served by the Direct NFS client can be accessed by way of a third party NFS client. The usual considerations for maintaining integrity of the Oracle files apply in this situation.

The database files accessed through Direct NFS Client should also be mounted using other means, such as CIFS or NFS. This ensures that the kernel input and output interface is able to access these files.

4.6.4 Specifying Network Paths with the Oranfstab File

Direct NFS can use up to four network paths defined in the oranfstab file for an NFS server. The Direct NFS client performs load balancing across all of the specified paths. If a specified path fails, then Direct NFS re-issues input and output commands over any remaining paths.

Note:
You can have only one active Direct NFS implementation for each instance. Using Direct NFS on an instance prevents the use of another Direct NFS implementation.

Use the following views for Direct NFS management:

- V$DNFS_SERVERS: Lists the servers that are accessed using Direct NFS.

- V$DNFS_FILES: Lists the files that are currently open using Direct NFS.

- V$DNFS_CHANNELS: Shows the open network paths, or channels, to servers for which Direct NFS is providing files.

- V$DNFS_STATS: Lists performance statistics for Direct NFS.

4.6.5 Enabling Direct NFS Client

To enable Direct NFS Clients, you must add an oranfstab file to ORACLE_BASE\ORACLE_HOME\dbs. When oranfstab is placed in this directory, the entries in this file are specific to one particular database. The Direct NFS Client searches for the mount point entries as they appear in oranfstab. Direct NFS uses the first matched entry as the mount point. Complete the following procedure to enable Direct NFS:

1. Create an oranfstab file with the following attributes for each NFS server that you want to access using Direct NFS:

- SERVER: The NFS server name.

- PATH: Up to four network paths to the NFS server, specified either by IP address, or by name, as displayed using the ifconfig command.

- EXPORT: The exported path from the NFS server. Use a UNIX-style path syntax.

- MOUNT: The local mount point for the NFS server. Use a WINDOWS-style path syntax.

The following is an example of an oranfstab file with two NFS server entries:

server: MyDataServer1
path: 132.34.35.12
path: 132.34.35.13
export: /vol/oradata1 mount: C:\ORACLE\ORADATA\ORCL

server: MyDataServer2
path: NfsPath1
path: NfsPath2
path: NfsPath3
path: NfsPath4
export: /vol/oradata2 mount: C:\ORACLE\ORADATA\ORCL2
export: /vol/oradata3 mount: C:\ORACLE\ORADATA\ORCL3

Note:
You can specify two optional parameters in oranfstab file:

- uid: The UNIX user ID to be used by Direct NFS

- gid: The UNIX group ID to be used by Direct NFS

The Direct NFS Client uses the uid or gid value to access all NFS servers listed in oranfstab. Direct NFS ignores a uid or gid value of 0.

Note:
The exported path from the NFS server must be accessible for read/write/execute by the user with the uid, gid specified in oranfstab or, if neither uid nor gid is listed, by the user with the uid:65534,gid:65534.

2. Oracle Database uses an ODM library, oranfsodm11.dll, to enable Direct NFS. To replace the standard ODM library, oraodm11.dll, with the ODM NFS library, oranfsodm11.dll, complete the following steps:

1. Change directory to ORACLE_BASE/ORACLE_HOME/bin.

2. Shutdown Oracle.

3. Enter the following commands:

copy oraodm11.dll oraodm11.dll.stub
copy /Y oranfsodm11.dll oraodm11.dll

4.6.6 Disabling Direct NFS Client

Use one of the following methods to disable the Direct NFS client:

- Remove the oranfstab file.

- Restore the stub oraodm10.dll file by reversing the process you completed in .

- Remove the specific NFS server or export paths in the oranfstab file.

4.6.7 Checking NFS Buffer Size

Direct NFS requires an NFS server supporting NFS read and write buffers of at least 16384 bytes. Direct NFS issues writes at wtmax granularity to the NFS server. Direct NFS cannot serve an NFS server with a wtmax of less than 16384. Oracle recommends that you use the value 32768.

Note:
If you remove an NFS path from oranfstab that Oracle Database is using, then you must restart the database for the change to be effective. In addition, the mount point that you use for the file system must be identical on each node.

See Also:
Your storage vendor documentation for additional information about NFS Buffer Size parameters

4.7 Requirements for Files Managed by Oracle

If you use OCFS or ASM for your database files, then your database will be created by default with files managed by Oracle Database. You may also elect to use files managed by Oracle if you choose the Custom installation type or the Advanced database creation option. If you use this feature, you need only specify the database object name instead of file names when creating or deleting database files.

Configuration procedures are required in order to enable Oracle Managed Files.

See Also:
"Using Oracle-Managed Files" in Oracle Database Administrator's Guide

5 Installing Oracle Clusterware

This chapter describes the procedures for installing Oracle Clusterware on Windows-based systems. The topics in this chapter are:

- Verifying Oracle Clusterware Requirements with CVU

- Preparing to Install Oracle Clusterware with OUI

- Installing Oracle Clusterware with OUI

- Formatting Drives to Use Oracle Cluster File System after Installation

Note:
If you are going to install Oracle Real Application Clusters, then this chapter describes phase one of the Oracle Database 11g Real Application Clusters (Oracle RAC) installation, the Oracle Clusterware installation. The second phase of an Oracle RAC installation, installing Oracle RAC, is described in Oracle Real Application Clusters Installation Guide for Microsoft Windows.

Note:
To install Oracle Clusterware on Windows Server 2008, use the Oracle Database 11g Release 1 (11.1.0.7.0) for Microsoft Windows Windows Server 2008 media. In addition, for Windows Server 2008, you must have administrator privileges and you must run commands from an Administrative command prompt to run executables that reside in the Oracle Clusterware home.

5.1 Verifying Oracle Clusterware Requirements with CVU

Using the following command syntax, start Cluster Verification Utility (CVU) to check system requirements prior to installing Oracle Clusterware:

mountpoint\runcluvfy.bat stage -pre crsinst -n node_list

In the preceding syntax, replace mountpoint with the path for the installation media and the variable node_list with the names of the nodes in your cluster, separated by commas.

For example, with the installation files in a stage directory on the C: drive, enter the following command for a cluster with nodes node1, node2, and node3:

c:\stage\db\Disk1\runcluvfy.bat stage -pre crsinst -n node1,node2,node3

The Cluster Verification Utility Oracle Clusterware stage check verifies the following:

- User Equivalence: User equivalence exists on all the specified nodes

- Node Reachability: All the specified nodes are reachable from the local node

- Node Connectivity: Connectivity exists between all the specified nodes through the public and private network interconnections

- Administrative Privileges: The oracle user has proper administrative privileges to install Oracle Clusterware on the specified nodes

- Shared Storage Accessibility: If specified, the Oracle Cluster Registry (OCR) device and voting disk are shared across all the specified nodes

- System Requirements: All system requirements are met for installing Oracle Clusterware software, including software packages, memory, swap directory space, temp directory space, and required users and groups

5.1.1 Troubleshooting Clusterware Setup for Windows

If the CVU report indicates that your system fails to meet the requirements for Oracle Clusterware installation, then use the topics in this section to correct the problem or problems indicated in the report, and run the CVU command again.

User Equivalence Check Failed

Cause: Failure to establish user equivalency across all nodes. This can be due to not providing the Administrative user on each node with the same password.

Action: Either the same local administrative username and password must be used on every node in a cluster, or the same domain user with local administrative privileges must be used on all nodes. If you use a domain user, then the domain user must be in the same domain on every node.

Node Reachability Check

Cause: One or more nodes in the cluster cannot be reached using TCP/IP protocol, through either the public or private interconnects.

Action: Use the command ping address to check each node address. When you find an address that cannot be reached, check your list of public and private addresses to make sure that you have them correctly configured. Ensure that the public and private network interfaces have the same interface names on each node of your cluster.

Administrative Privileges Check Failed

Cause: The administrative privileges required for installation are missing or incorrect.

Action: From the node where you intend to run OUI, verify that you have administrative privileges on the other nodes. To do this, enter the following command for each node that is a part of the cluster where nodename is the name of the node:

net use \\nodename\C$

If you cannot log on, then you must correct the user information on that node. You must use a domain user name, or the same user name and password on each node in a cluster. If you use a domain user name, then log on under a domain with a username and password that has local administrative privileges on each node. When you have corrected the path configuration information on the node, run the CVU check again.

5.2 Preparing to Install Oracle Clusterware with OUI

Before you install Oracle Clusterware, use the following checklist to ensure that you have all the information you will need during installation, and you have completed all tasks that must be done before starting to install Oracle Clusterware. Mark the check box for each task as you complete it, and write down the information needed, so that you can provide it during installation.

- Verify Cluster Privileges
Before running Oracle Universal Installer, from the node where you intend to run the Installer, verify that you have administrative privileges on the other nodes. To do this, enter the following command for each node that is a part of the cluster:

net use \\nodename\C$

where nodename is the node name.

- Shut Down Running Oracle Processes
If you are installing Oracle Clusterware on a node that already has a single-instance Oracle Database 11g release 1 (11.1) installation, then stop the existing ASM instances. After Oracle Clusterware is installed, start up the ASM instances again. When you restart the single-instance Oracle database, the ASM instances use the Cluster Synchronization Services (CSSD) Daemon from Oracle Clusterware instead of the CSSD daemon for the single-instance Oracle database.

You can upgrade some or all nodes of an existing Cluster Ready Services installation. For example, if you have a six-node cluster, then you can upgrade two nodes each in three upgrading sessions.Base the number of nodes that you upgrade in each session on the load the remaining nodes can handle. This is called a "rolling upgrade."

If a Global Services Daemon (GSD) from Oracle9i Release 9.2 or earlier is running, then stop it before installing Oracle Database 11g release 1 (11.1) Oracle Clusterware by running the following command:

gsdctl stop

from the Oracle Database home that is running the GSD.

Caution:
If you have an existing Oracle9i release 2 (9.2) Oracle Cluster Manager (Oracle CM) installation, then do not shut down the Oracle CM service. Shutting down the Oracle CM service prevents the Oracle Clusterware 11g release 1 (11.1) software from detecting the Oracle9i release 2 node list, and causes failure of the Oracle Clusterware installation.

Note:
If you receive a warning to stop all Oracle services after starting OUI, then run the command

localconfig delete

from the Oracle_home that is running CSS.

- Determine your cluster name, public node names, private node names, and virtual node names for each node in the cluster
If you install the clusterware during installation, then you are asked to provide a public node name and a private node name for each node. When you enter the public node name, use the primary host name of each node. In other words, use the name displayed by the hostname command but without any portion of the domain name that may be returned by the command.

In addition, ensure that the following are true:

- Determine a cluster name with the following characteristics:

- It must be globally unique throughout your host domain

- It must be at least one character long and less than 15 characters long

- It must consist of the same character set used for host names: hyphens (-) and single-byte alphanumeric characters (a to z, A to Z, and 0 to 9). If you use third-party vendor clusterware, then Oracle recommends that you use the vendor cluster name

- Determine a private node name or private IP address for each node. The private IP address is an address that is only accessible by the other nodes in this cluster. Oracle uses private IP addresses for inter-node, or instance-to-instance Cache Fusion traffic. Oracle recommends that you provide a name in the format public_hostname-priv, for example myclstr2-priv.

- Determine a virtual host name for each node. A virtual host name is a public node name that is used to reroute client requests sent to the node if the node is down. Oracle uses virtual IP addresses (VIPs) for client to database connections, so the VIP address must be publicly accessible. Oracle recommends that you provide a name in the format public_hostname-vip, for example myclstr2-vip.

Note:
The following is a list of additional information about node IP addresses:

- The IP addresses that you use for all of the nodes in the current installation process must be from the same subnet.

- OUI fills the default private and virtual host names in the format nodename-priv and nodename-vip. You must fill in the addresses during installation if your host names differ from the default ones.

- Host names, private names, and virtual host names are not domain-qualified. If you provide a domain in the address field during installation, then the OUI removes the domain from the address.

- Private IP addresses should not be accessible as public interfaces. Using public interfaces for Cache Fusion can cause performance problems.

- Determine the complete path for the raw devices or shared file systems, and set up the voting disk and Oracle Cluster Registry partitions
During installation, at the Cluster Configuration Storage page, you are asked to provide paths for two files that must be shared across all nodes of the cluster, either on a shared raw device, or a shared file system file:

- The Cluster Synchronization Services (CSS) voting disk is a partition that Oracle Clusterware uses to verify cluster node membership and status. Provide at least 280 MB of disk space for each voting disk.The Oracle Cluster Registry (OCR) contains cluster and database configuration information for the Oracle RAC database and for Oracle Clusterware, including the node list, and other information about cluster configuration and profiles. Provide at least 280 MB disk space for each Oracle Cluster Registry.

In addition, if you intend to use Oracle Cluster File System (OCFS) then you are prompted to indicate which of the available disks you want to format with OCFS, what format type you want to use, and to what drive letter the formatted OCFS disk is mounted.

Ensure that you create at least the minimum required drives for installation.

See Also:
Chapter 3 for information about the minimum raw device sizes

- Disconnect all non-persistent drives
Before starting the Oracle Clusterware installation on Windows platform, please make sure that you disconnect all nonpersistent drives that are temporarily mounted on all the nodes. Alternatively, if you want to access the shared drive, then make the shared drive persistent using the following command:

net use * \\servername\sharename /persistent: YES

5.3 Installing Oracle Clusterware with OUI

This section provides you with information about how to use Oracle Universal Installer (OUI) to install Oracle Clusterware. It contains the following sections:

- Running OUI to Install Oracle Clusterware

- Installing Oracle Clusterware Using a Cluster Configuration File

5.3.1 Running OUI to Install Oracle Clusterware

Perform the following procedures to install Oracle Clusterware with OUI. You can run OUI from a VNC session, or Terminal Services in console mode.

1. Log in to Windows with Administrative privileges and run the setup.exe command from the \Disk1 directory on the Oracle Database 11g release 1 (11.1) installation media.
The setup.exe command opens the OUI Select a Product to Install page. Select Oracle Clusterware and click Next.

2. After you click Next, the Specify Home Details page enables you to accept the displayed path name for the Oracle Clusterware products or select a different one. You may also accept default directory and path name for the location of your Oracle Clusterware home or browse for an alternate directory and destination. You must select a destination that exists on each cluster node that is part of this installation. Click Next to confirm your choices.

3. The installer verifies that your environment meets all of the minimum requirements for installing and configuring the products that you have chosen to install. The results are displayed on the Product-Specific Prerequisite Checks page. Verify and confirm the items that are flagged with warnings and items that require manual checks. After you confirm your configuration, OUI displays the Specify Cluster Configuration page.

Note:
If the check identifies an existing, local CSS, then you must first shut down the Oracle Database and ASM instances from the Oracle home where CSS is running, and then run the following command from the same Oracle home before you continue with the installation:

Oracle home\bin\localconfig delete

4. The Specify Cluster Configuration page contains predefined node information.
If OUI detects that your system has Oracle9i Release 2 clusterware, then the Specify Cluster Configuration page displays the existing node list. Otherwise, OUI displays the Specify Cluster Configuration page with local node only. You may need to modify the default private and virtual names for your environment.

Provide a cluster name if you do not wish to use the name provided by OUI. Note that the selected cluster name must be globally unique throughout the enterprise. In addition, the allowable character set for cluster names is the same as that for hostnames, that is single-byte alphanumeric characters (a to z, A to Z, and 0 to 9), and hyphens (-).

If you prefer, you may instead provide the cluster configuration information in a text file and provide that file name instead of completing the individual fields on the Specify Cluster Configuration page. See the following section, "Installing Oracle Clusterware Using a Cluster Configuration File", for details about cluster configuration files.

Note:
If you are upgrading your cluster or part of your cluster from Oracle9i release 2 Cluster Ready Services to Oracle Clusterware release 11g, then to ensure backward compatibility, OUI prevents you from changing the cluster name from the existing name by disabling the cluster name field.

Click Next after you have entered the cluster configuration information. This saves your entries and opens the Specify Network Interface Usage page.

5. In the Specify Network Interface Usage page OUI displays a list of cluster-wide interfaces. Use the drop-down menus on this page to classify each interface as Public, Private, or Do Not Use. You must classify at least one interface as Public and one as Private. Click Next when you have made your selections to open the Select Disk Formatting Options page.

6. On the Cluster Configuration Storage page, identify the disks that you want to use for the Oracle Clusterware files and, optionally, Oracle Cluster File System (OCFS) storage. Highlight each of these disks one at a time and click Edit to open the Cluster Configuration Storage page on which you can define the details for selected disks.

Note:
The OUI page described in this step displays logical drives from which you must make your selections. If you have a previous version of Oracle Clusterware installed on the system, and the installer detects an existing voting disk and OCR on a OCFS or raw partition, then you do not need to create new partitions for the OCR and Voting disks when upgrading to Oracle Clusterware 11g.

7. On the Cluster Configuration Storage page, designate whether you want to place a copy of the OCR, a copy of the voting disk, or a copy of both files (if CFS is selected) on the partition. If you plan to use CFS, then indicate whether you plan to store software, database files, or both software and database files on selected partition, and select an available drive letter to be used to mount the partition once formatted.

8. After you click Next, OUI displays a Summary page that shows the cluster node information, along with the space requirements and availability. Verify the installation that OUI is about to perform and click Finish.

9. When installation finishes successfully, click Exit.

At this point, you have completed the Oracle Clusterware installation.

To install Oracle Database 11g with Oracle RAC, refer to Oracle Real Application Clusters Installation Guide for Microsoft Windows. If you intend to use Oracle Clusterware without an Oracle RAC database, then refer to Oracle Database Installation Guide for Microsoft Windows.

5.3.2 Installing Oracle Clusterware Using a Cluster Configuration File

During the Oracle Clusterware installation, on the Specify Cluster Configuration page, you are given the option either of providing cluster configuration information manually, or of using a cluster configuration file. A cluster configuration file is a text file that you can create before starting OUI, which provides OUI with information about the cluster name and node names that it needs to configure the cluster.

Oracle suggests that you consider using a cluster configuration file if you intend to perform repeated installations on a test cluster, or if you intend to perform an installation on many nodes.

To create a cluster configuration file:

1. On the installation media, navigate to the directory Disk1\response.

2. Using a text editor, open the response file crs.rsp, and find the section CLUSTER_CONFIGURATION_FILE.

3. Follow the directions in that section for creating a cluster configuration file.

Note:
For Windows Server 2008, you must have administrator privileges and you must run commands from an Administrative command prompt to run executables that reside in the Oracle Clusterware home.

5.3.3 Silent Installation of Oracle Clusterware

Complete the following procedure to perform a noninteractive (silent) installation:

1. On the installation media, navigate to the directory Disk1\response.

2. Using a text editor, open the response file crs.rsp. Follow the directions in each section, and supply values appropriate for your environment.

3. Use the following command syntax to run OUI in silent mode:

setup.exe -silent -reponseFile path_to_your_reponse_file

For example:

C:\ setup.exe -silent -responseFile \download\oracle\Disk1\response\mycrs.rsp

Note:
For Windows Server 2008, you must have administrator privileges and you must run commands from an Administrative command prompt to run executables that reside in the Oracle Clusterware home.

5.4 Formatting Drives to Use Oracle Cluster File System after Installation

If you install Oracle Database 11g with Oracle RAC, and later you want to install OCFS, then run the ocfsformat.exe command from the crs_home\cfs directory using the following syntax:

crs_home\cfs\OcfsFormat /m drive_letter /c clustersize [/v volume_label] [/f]

Where:

- /m drive_letter is the mountpoint for the file system which you want to format with OCFS. On Windows, provide a drive letter corresponding to the logical drive.

- clustersize is the size of the cluster in kilobytes

- volume_label is an optional volume label

Note:
The clustersize is essentially the block size. Recommended values are 1024 (1 MB) if the OCFS partition is to be used for Oracle datafiles and 4 (4 KB) if the OCFS partition is to be used for the ORACLE_HOME.

6 Oracle Clusterware Post-Installation Procedures

This chapter describes how to complete the required post-installation tasks after you have installed Oracle Clusterware 11g Release 1 (11.1). It contains the following sections:

- Backing Up the Voting Disk after Installation

- Downloading and Installing Patches

- Required Tasks to Install Oracle RAC Database Release 9.2

6.1 Backing Up the Voting Disk after Installation

After your Oracle Clusterware installation is complete, and after you are sure that your system is functioning properly, make a backup of the contents of the voting disk using ocopy.exe which resides on the Oracle Database home.

Also make a backup of the voting disk contents after you complete any node additions or node deletions, and after running any deinstallation procedures.

6.2 Downloading and Installing Patches

Refer to the OracleMetaLink Web site for required patch updates for your installation. To download required patch updates:

1. Use a Web browser to view the OracleMetaLink Web site:
https://metalink.oracle.com

2. Log in to OracleMetaLink.

Note:
If you are not an OracleMetaLink registered user, then click Register for MetaLink and register.

3. On the main OracleMetaLink page, click Patches & Updates.

4. On the Patches & Update page, click Advanced Search.

5. On the Advanced Search page, click the search icon next to the Product or Product Family field.

6. In the Search and Select: Product Family field, select Database and Tools in the Search list field, enter RDBMS Server in the text field, and click Go.
RDBMS Server appears in the Product or Product Family field. The current release appears in the Release field.

7. Select your platform from the list in the Platform field, and at the bottom of the selection list, click Go.

8. Any available patch updates appear under the Results heading.

9. Click the number of the patch that you want to download.

10. On the Patch Set page, click View README and read the page that appears. The README page contains information about the patch set and how to apply the patches to your installation.

11. Return to the Patch Set page, click Download, and save the file on your system.

12. Use the unzip utility provided with Oracle Database 10g to uncompress the Oracle patch updates that you downloaded from OracleMetaLink. The unzip utility is located in the $ORACLE_HOME/bin directory.

13. Refer to "Deinstalling Oracle Clusterware from Windows Environments" for information about how to stop database processes in preparation for installing patches.

6.3 Required Tasks to Install Oracle RAC Database Release 9.2

To install an Oracle9i RAC database on Oracle Clusterware 11g (11.1), perform the following configuration steps to prevent permission errors:

1. Obtain a listing of the nodes that are currently configured with Oracle Clusterware.
On any node in the cluster, run a command using the following syntax, where CRS_home is the Oracle Clusterware home:

CRS_home\bin\olsnodes

For example:

D:\> app\crs\bin\olsnodes

2. Create the OracleCRSToken_username for the Oracle9i software owner on each node in the cluster. After the service is created, change the ownership for the GSD resource to the Oracle9i software owner.
If you are using a domain user, then run the following command on only one node in the cluster using the specified syntax, where CRS_home is the Oracle Clusterware home, 92_user_domain is the domain of the Oracle9i software owner, and 92_username is the user name of the Oracle9i software owner:

CRS_home\bin\crsuser add 92_user_domain\92_username

Note:
When specifying a domain user, you must you the format domain\username instead of the format username@domain

If you are using a local Administrator user, then run the following command on each node in the cluster using the specified syntax, where CRS_home is the Oracle Clusterware home, 92_username is the user name of the Oracle9i software owner, and nodename is the name of the node on which the service is being configured:

CRS_home\bin\crsuser add nodename\92_username

3. On any node, run the following command for each node in the cluster, where nodename is the name of the node you are configuring:

- Domain users

CRS_home\bin\crs_setperm ora.nodename.gsd -o 92_user_domain\92_username

- Local Adminsitrator users

CRS_home\bin\crs_setperm ora.nodename.gsd -o 92_username

You must repeat this command until it has been run one time for each node in your cluster.

4. Enable and start the GSD daemon on all nodes in the cluster.
On any node in the cluster, run commands using the following syntax, where CRS_home is the Oracle Clusterware home:

CRS_HOME\bin\srvctl enable nodeapps -g
CRS_HOME\bin\srvctl start nodeapps -all

For example, if the Oracle Clusterware home is D:\app\grid, then enter the following commands:

D:\app\grid\bin> srvctl enable nodeapps -g
D:\app\grid\bin> srvctl start nodeapps -all

A Troubleshooting the Oracle Clusterware Installation Process

This appendix provides troubleshooting information for installing Oracle Clusterware. The topics in this appendix are:

- General Installation Issues

- Oracle Clusterware Install Actions Log Errors and Causes

See Also:
Oracle Real Application Clusters Administration and Deployment Guide for additional information about Oracle Clusterware and Oracle RAC configuration and deployment.

A.1 General Installation Issues

The following is a list of examples of types of errors that can occur during installation:

Nodes unavailable for selection from the OUI Node Selection screen

Cause: Oracle Clusterware is either not installed, or the Oracle Clusterware services are not up and running.

Action: Install Oracle Clusterware, or review the status of your Oracle Clusterware. Consider restarting the nodes, as doing so may resolve the problem.

Node nodename is unreachable

Cause: Unavailable IP host

Action: Attempt the following:

1. Run the command ipconfig /all. Compare the output of this command with the contents of the C:\WINNT\system32\drivers\etc\hosts file to ensure that the node IP is listed.

2. Run the command nslookup to see if the host is reachable.

Time stamp is in the future

Cause: One or more nodes has a different clock time than the local node. If this is the case, then you may see output similar to the following:

time stamp 2005-04-04 14:49:49 is 106 s in the future

Action: Ensure that all member nodes of the cluster have the same clock time.

A.2 Oracle Clusterware Install Actions Log Errors and Causes

The following is a list of potential errors in the installActions.log:

- PRIF-10: failed to initialize the cluster registry
Configuration assistant "Oracle Private Interconnect Configuration Assistant" failed

- Step 4: Starting up Oracle Clusterware stack on all nodes
node1 service OracleCSService in improper STOPPED state, err(997)

node2 service OracleCSService in improper STOPPED state, err(997)

- Step 4: Starting up Oracle Clusterware stack on all nodes
node1 service OracleCSService in improper STOPPED state, err(0)

node2 service OracleCSService in improper STOPPED state, err(997)

- Step 4: Starting up Oracle Clusterware stack on all nodes
oracletest1 failed to startup service OracleEVMService, err(1053)

- Step 1: checking status of Oracle Clusterware cluster
Step 2: configuring OCR repository

ignoring upgrade failure of ocr(-1073740972)

failed to configure Oracle Cluster Registry with CLSCFG, ret -1073740972

Each of these error messages can be caused by one of the following issues:

A.2.1 The OCFS format is not recognized on one or more of the remote cluster nodes

If you are using OCFS for your OCR and Voting disk partitions, then:

1. Leave the OUI window in place.

2. Restart the second node, and any additional nodes.

3. Retry the assistants

A.2.2 Timing issue with start of the OracleCSService:

If you think the cause may be a timing issue, then:

1. Leave the OUI window in place.

2. Start the OracleCSService manually on all nodes. You may also need to start the OracleObjectService manually.

3. Retry the assistants

A.2.3 You are on a Windows 2003 system, and Automount of new drives is not enabled:

If this is true, then:

For Oracle RAC on Windows 2003, you must issue the following commands on all nodes:

diskpart
automount enable

If you are already failing at the configuration assistants and this has not yet been run on all nodes in the cluster, you will need to clean up your Oracle Clusterware install, issue this command on all nodes, reboot all nodes, and start the Oracle Clusterware install again.

A.2.4 You have entered a period in one of the node names during Oracle Clusterware install

Periods (".") are not permitted in node names. Instead, use a hyphen ("-").

To resolve a failed installation, remove traces of the Oracle installation, and reinstall with a permitted node name.

A.2.5 Node1 failed to startup service OracleEVMService, err(1053)

The installation requires dll: MSVCP60.DLL. Copying MSVCP60.DLL to c:\winnt\system32 should allow EVM to start.

A.2.6 Step 1: checking status of Oracle Clusterware clusterStep 2: configuring OCR repositoryignoring upgrade failure of ocr(-1073740972)

This error indicates that the user that is performing installation does not have Administrator privileges.

B Deinstalling Oracle Clusterware

This appendix describes how to deinstall Oracle Clusterware in a Windows environment.

B.1 Deinstalling Oracle Clusterware from Windows Environments

Deinstall each Oracle Database 11g Real Application Clusters home as described in Oracle Real Application Clusters Installation Guide for Microsoft Windows. Then complete the de-installation by removing Oracle Clusterware software using one of the following procedures:

- Deinstalling Oracle Clusterware with No Previous Cluster Software Versions

Note:
For Windows Server 2008, you must have administrator privileges and you must run commands from an Administrative command prompt to run executables that reside in the Oracle Clusterware home.

B.1.1 Deinstalling Oracle Clusterware with No Previous Cluster Software Versions

Perform the following steps to de-install Oracle 11g Oracle Clusterware software from a Windows environment:

1. Stop and remove the Oracle Clusterware node applications on each node that is associated with the Oracle home that you are deleting. Do this by running the following command for all of the nodes that are affected by the deletion of the Oracle home:

srvctl stop nodeapps -n nodename

where nodename is the node name. Repeat this command for each node in the cluster.

Then remove Oracle Clusterware node applications by running the following commands:

svrctl remove nodeapps -n nodename

where nodename is the node name. Repeat this command for each node in the cluster, responding to any operating-system prompts to confirm your operations for each node.

2. Click Start and navigate to Settings, then to Control Panel, then to Administrative Tools, and then to Services. Stop the following services:

- OracleRemExecService

- OracleClusterVolumeService

- OracleObjectService

- OracleCRService

- OracleCSService

- OracleEVMService

3. If you have services with names such as OracleCRSTokenname, then remove them by running the following command:

crsuser remove user_name

where user_name is a user name.

4. Start OUI. On the Select a Product page, click Deinstall Products to display the list of installed products. Select the Oracle Clusterware home you want to de-install.

5. Shut down and restart each node that is a member of your cluster.

6. If you are not using a cluster file system, then on each node, use Windows Explorer to delete the Oracle directory, its subdirectories, and their contents.

C Configuring Raw Devices for Oracle Database

This appendix provides additional information about configuring raw devices to deploy Oracle Database with Oracle Real Application Clusters. You must configure raw devices if you do not use Automatic Storage Management (ASM) or an Oracle Cluster File System. The topics in this appendix are:

- Support for Raw Devices

- Raw Devices Required by Database Configuration Assistant

C.1 Support for Raw Devices

Oracle Database supports the use of raw devices for Oracle files on Windows 2000 and Windows Server 2003. You can partition a raw device to store data and control files. You can also use the entire raw device to store data. You can create partitions on Windows by using the Disk Management utility, Diskmgmt.msc. To access this utility:

1. Click Start.

2. Select Run... from the program list.

3. In the Run dialog box, type diskmgmt.msc.

C.2 Raw Devices Required by Database Configuration Assistant

If you want to use Database Configuration Assistant (DBCA) to create a database on raw storage, that is, without using ASM or an Oracle Cluster File System, then you must configure raw devices as described in this section. These devices are in addition to the OCR and voting disk required to install Oracle Clusterware. Create these devices before running Oracle Universal Installer to install Oracle Database 11g software. DBCA cannot create an Oracle RAC database unless you have properly configured the following devices:

- Four raw devices for four tablespace data files

- At least two raw devices for control files

- One raw device for each instance for its own tablespace for automatic undo management

- At least two raw devices for redo log files for each instance

- One raw device for the server parameter file

Note:
Each instance has its own redo log files, but all instances in a cluster share the control files and data files. In addition, each instance's online redo log files must be readable by all other instances to enable recovery.

C.2.1 Planning Your Raw Device Creation Strategy

If you are planing to install Oracle RAC, then before you install, you should create enough partitions of specific sizes to support your database, and also leave a few spare partitions of the same size for future expansion. For example, if you have space on your shared disk array, then select a limited set of standard partition sizes for your entire database. Partition sizes of 100 MB, 500 MB, and 1 GB are suitable for most databases. Also create a few very small and a few very large spare partitions that are, for example, 10 MB and perhaps 5 GB or greater in size. Based on your plans for using each partition, determine the placement of these spare partitions by combining different sizes on one disk, or by segmenting each disk into same-sized partitions.

Note:
Ensuring that there are spare partitions enables you to perform emergency file relocations or additions if a tablespace data file becomes full.

D How to Perform Oracle Clusterware Rolling Upgrades

This appendix describes how to perform Oracle Clusterware rolling upgrades. A rolling upgrade process allows one or more nodes in the cluster to have a different software version than the other nodes in the cluster for a short period of time. During the time period where cluster nodes have different software versions, the cluster is fully functional, but some administrative functions are disabled until the upgrade has completed, for example, adding new cluster resources. Once the rolling upgrade process starts, the time it takes to complete the upgrade of all the nodes in the cluster should not exceed 24 hours.

Because you must stop database processes before initiating an Oracle Clusterware process, this chapter also includes information about how to stop processes in Oracle Real Application Clusters (Oracle RAC) databases.

Note:
You can use the procedures in this chapter to prepare to perform rolling upgrades of Oracle Clusterware from any Oracle Clusterware 10g release 10.2 or Oracle Clusterware 11g installation to the latest patch update for that version. For example, you can use these procedures to prepare to upgrade from Oracle Clusterware 10.2.0.1 to 10.2.0.3.

This appendix contains the following topics:

- Back Up the Oracle Software Before Upgrades

- Restrictions for Clusterware Upgrades to Oracle Clusterware 11g

- Verify System Readiness for Patches and Upgrades

- How to Perform Oracle Clusterware Rolling Upgrades for Patches

- How to Perform Rolling Upgrades From an Earlier Release to 11g

D.1 Back Up the Oracle Software Before Upgrades

Before you make any changes to the Oracle software, whether you intend to upgrade or patch part of the database or clusterware, or all of your cluster installation, Oracle recommends that you create a backup of the Oracle software.

D.2 Restrictions for Clusterware Upgrades to Oracle Clusterware 11g

To upgrade existing Oracle Clusterware 10g release 2 or Oracle Cluster Ready Services 10g release 1 installations to Oracle Clusterware 11g, you must first upgrade the existing installations to the most recent patch update. The following sections provide information about Oracle Clusterware upgrades.

D.2.1 Overview of Upgrading from Oracle Clusterware 10g Release 2 to Oracle Clusterware 11g

Complete the following procedure to prepare an existing Oracle Clusterware release 10.2 for rolling upgrade to Oracle Clusterware release 11g:

1. Upgrade the Oracle Clusterware home (or CRS home) to the latest patch set for Oracle Clusterware release 10.2 (for example, 10.2.0.3 or higher). Upgrading the database Oracle home is not required.
If you attempt a rolling upgrade to Oracle Clusterware 11g release 1 (11.1) and your current Oracle Clusterware installation has not been upgraded to at least Oracle Clusterware 10g release 10.2.0.3, then a prerequisite check failure is reported by the installer.

If you are not using the rolling upgrade method to upgrade Oracle Clusterware to release 11.1, then you can use any Oracle Clusterware 10g release 10.2 version.

2. Review the release notes for Oracle Clusterware 11g and support notes for any additional requirements or patches.

3. Upgrade Oracle Clusterware from release 10.2 to release 11.1.

D.2.2 Overview of Upgrading from Oracle Cluster Ready Services 10g Release 1 to Oracle Clusterware 11g

Complete the following procedure to prepare an existing Oracle Cluster Ready Services release 10.1 for rolling upgrade to Oracle Clusterware release 11g:

1. Upgrade the Oracle Cluster Ready Services 10g release 1 home (or CRS home) to release 10.1.0.3 or higher. Upgrading the database Oracle home is not required.

2. Review the release notes for Oracle Clusterware 11g and support notes for any additional requirements or patches.

3. Upgrade Oracle Cluster Ready Services 10g release 1 to Oracle Clusterware 11g release 1 (11.1).

D.3 Verify System Readiness for Patches and Upgrades

If you are completing a patch update of your database or clusterware, then after you download the patch software, and before you start to patch or upgrade your database, review the Patch Set Release Notes that accompany the patch to determine if your system meets the system requirements for the operating system and the hardware platform.

Use the Cluster Verification Utility to assist you with system checks in preparation for starting a database patch or upgrade.

See Also:
Oracle Database Upgrade Guide

D.4 How to Perform Oracle Clusterware Rolling Upgrades for Patches

To perform a rolling upgrade, complete all of the following steps in sequence:

- Copy Patch Software to the Primary Upgrade Node

- Stop All Oracle Processes on Upgrade Nodes

- Start OUI and Complete the Upgrade Processes on Upgrade Nodes

Note:
To perform rolling upgrades, the existing Oracle Clusterware home directory, sometimes referred to as the CRS home, must be located on local directories on the node. You cannot perform rolling upgrades on a shared Oracle Clusterware home directory.
Also note that Oracle does not support attempting to add additional nodes to a cluster during a rolling upgrade.

D.4.1 Copy Patch Software to the Primary Upgrade Node

Download the patch software to the primary node in the cluster (the node where you performed the initial Oracle Clusterware and Oracle Database installation). Review the patch set Readme to confirm that your system meets the system requirements for the patch set updates, and complete any special instructions for particular environments or configurations.

To download patches, or download the patch note Readme:

1. Log in to My Oracle Support (https://support.oracle.com)

2. Click the Patches & Updates tab.

3. Search for the patch that you want to install.

4. Click the patch number to open the patch page
From this location, you can download the patch binary, download the patch Readme, and obtain other information regarding the patch update.

5. Download the patch set installation archive to a directory inside the Oracle base directory that meets the following requirements:

- It is not the existing Oracle home directory, or Oracle Clusterware home directory

- It is not under an existing Oracle home directory, or Oracle Clusterware home directory

6. Extract the patch set installation archive

D.4.2 Stop All Oracle Processes on Upgrade Nodes

During the rolling upgrade process, the CRS stack is shut down on each node. On each node on which you want to perform a rolling upgrade, before you upgrade Oracle Clusterware, you must shut down all other processes that use Oracle Clusterware, such as the Oracle RAC database instances and the Oracle ASM instances.

You can use this procedure to shut down one node, and perform upgrades one node at a time, or to shut down groups of nodes, and upgrade groups of nodes at the same time. To specify a group of nodes, use a node list with the -n parameter instead of a single node name.

Complete the following steps:

Note:
You must perform these steps in the order listed.

1. On each node you intend to upgrade, stop all listener and other processes, such as Oracle Enterprise Manager Database Control, that are managed by Oracle Clusterware or Oracle Cluster Ready Services or that access the Oracle Databases managed by Oracle Clusterware. These are the same processes that you must stop before adding additional products to Oracle Database 11g release 1 (11.1). However, when upgrading Oracle Clusterware or Oracle Cluster Ready Services, you must stop these processes regardless of whether you plan to modify the software in the associated Oracle Database home directories.

Note:
Before you shut down any processes that are monitored by Enterprise Manager Grid Control, set a blackout in Grid Control for the processes that you intend to shut down. This is necessary so that the availability records for these processes indicate that the shutdown was planned downtime, rather than an unplanned system outage.

2. Shut down all Oracle RAC instances on each node you intend to upgrade. To shut down Oracle RAC instances on individual nodes in the database, enter the following command, where db_name is the name of the database, and instance_name is the name of the instance:

C:>oracle_home\BIN\srvctl stop instance -d db_name -i instance_name

Repeat this process on each node of the cluster on which you intend to perform the rolling upgrade.

To shut down all Oracle RAC instances for a database, enter the following command, where db_name is the name of the database:

C:>oracle_home\BIN\srvctl stop database -d db_name

3. Shut down all Oracle ASM instances on each node on which you intend to perform the rolling upgrade.

Note:
If you shut down ASM instances, then you must first shut down all database instances on the nodes you intend to upgrade that use ASM, even if these databases run from different Oracle homes.

To shut down an ASM instance, enter the following command, where node is the name of the node where the ASM instance is running:

C:>oracle_home\BIN\srvctl stop asm -n node

4. Stop all Oracle Clusterware or Oracle Cluster Ready Services node applications on each node on which you intend to perform the rolling upgrade. To stop node applications running on a node, enter the following command, where node is the name of the node:

C:>oracle_home\BIN\srvctl stop nodeapps -n node

5. Log in as the Windows Administrator user and shut down the Oracle Clusterware process on each node on which you intend to perform the rolling upgrade by entering the following command:

C:>CRS_home\BIN\crsctl stop crs

6. Stop the Oracle ASM, Oracle Database, and CRS services from the Service Control Manager window.

D.4.3 Start OUI and Complete the Upgrade Processes on Upgrade Nodes

To complete the upgrade, use the Oracle Universal Installer (OUI) downloaded with the patch update or with the new software.

Complete the following steps:

1. Start the patch set OUI. At the Product Selection Page, click Next.

2. On the Specify Home Details window, select the Oracle Clusterware home directory, and click Next.

3. On the Specify Hardware Cluster Installation Nodes window, select the nodes where you want to perform the upgrade, and click Next.

4. Follow further instructions as directed from the OUI windows.

Repeat these steps for each node or group of nodes on which you are performing a rolling upgrade.

D.5 How to Perform Rolling Upgrades From an Earlier Release to 11g

Use the following procedure to upgrade Oracle Clusterware from an earlier release to a later release:

Note:
You cannot change the owner of the Oracle Clusterware home during an upgrade. You must use the same Oracle software owner that owns the existing Oracle Clusterware home.

1. Load the Oracle Clusterware 11g release 1 (11.1) installation media into a CD or DVD ROM drive, or stage locally on disk.

2. Complete operating system prerequisite checks on each of the nodes that you intend to upgrade, to ensure that they meet the system prerequisites for Oracle Clusterware 11g release 1 (11.1).

3. If necessary, perform patch upgrades of the previous release Oracle Clusterware or Oracle Cluster Ready Services software to the most recent patch version, as described in the preceding section, "How to Perform Oracle Clusterware Rolling Upgrades for Patches".

4. After you have completed any necessary patch upgrades, shut down processes on the nodes you intend to upgrade, as described in the preceding section, "Stop All Oracle Processes on Upgrade Nodes".

5. Ensure that you are logged in as the Windows Administrator user before running OUI. After you start OUI, provide information as prompted.

Note:
You can upgrade one node, a subset of nodes, or all nodes, depending on your upgrade plan. However, note the following information about starting upgrades with OUI:

- If this is the initial upgrade, then the node where OUI is running must be one of the set of nodes that is being upgraded.

- If this is the second or subsequent upgrade, then the node where OUI is running must be on a node that has not been upgraded.

6. On a node that has not already been upgraded, repeat Steps 4 and 5 for the second and subsequent installs of a rolling upgrade.

7. Ensure that the Oracle Process Monitor service (OPMD) starts when upgrading from Oracle Database 10.x to Oracle Database 11.1. To ensure this, after the installation is complete, run the following command from the CRSHOME/bin subdirectory:

opmd -install

Note:
it is possible that the OPMD has already been installed on the node. If that is the case you may see the error OpmdInstall: OPM service is already installed, please uninstall first. You can ignore this error, it indicates that no further work needs to be performed.

8. After completing the Oracle Clusterware upgrade, if you intend to upgrade Oracle Automatic Storage Management, or the Oracle Database software, complete preinstallation procedures for installing Oracle Database or Oracle RAC, as described in the Oracle Database Installation Guide for Microsoft Windows or Oracle Real Application Clusters Installation Guide for Microsoft Windows.

See Also:
Oracle Database Upgrade Guide for additional information about completing database upgrades

Index
A  B  C  D  E  F  H  I  J  L  M  N  O  P  R  S  T  U  V  W

Numerics

32-bit

Windows-based installation, 2

64-bit

Windows-based installation, 2

A

asmtool utility, 4.4.5.3

asmtoolg utility, 4.4.5.2

authentication support

preinstallation requirements, 2.9.1

Automatic Storage Management

storage option for data files, 4.2.2

Automatic Storage Management (ASM)

asmtool utility, 4.4.5.3

asmtoolg utility, 4.4.5.2

configuring disks, 4.4.1

DAS disks, 4.4.4

disk groups

recommendations for, 4.4.2

redundancy levels, 4.4.2

disks, supported, 4.4.4 failure groups, 4.4.2

characteristics, 4.4.2

examples, 4.4.2

identifying, 4.4.2

mirroring, 4.4.2 partition creation, 4.4.4 redundancy levels, 4.4.2 SAN disks, 4.4.4 space required for preconfigured database, 4.4.2 automount

enable, 3.1.2, 4.1.2

B

bind order

Windows network, 2.8.3.3

C

certification, hardware and software, 2.3

cluster configuration file, 5.3.2

Cluster Configuration page

Oracle Universal Installer, 5.3.1

Cluster Configuration Storage page

Oracle Universal Installer, 5.3.1

cluster file system

storage option for data files, 4.2.2

cluster name, 5.3.1

requirements for, 5.2

cluster nodes

private node names, 5.2

public node names, 5.2

virtual node names, 5.2

cluster privileges

verifying for OUI cluster installation, 2.10

Cluster Ready Services

upgrading, 5.2

Cluster Synchronization Services, 5.2 Cluster Verification Utility

difference between runcluvfy and cluvfy, 2.11.1

Oracle Clusterware configuration check, 5.1

shared storage area check, 3.3

user equivalency troubleshooting, 5.1.1

commands

cluvfy, 2.11.3

runcluvfy.bat, 2.11.1

configuring disks for ASM, 4.4.1 configuring raw devices, 4.5 control files

raw devices, 4.5.1

CSS, 5.2

OCCSD, 5.2

custom database

failure groups for ASM, 4.4.2

requirements when using Automatic Storage Management, 4.4.2

CVU. See Cluster Verification Utility

D

DAS (direct attached storage) disks, 4.4.4

data files

minimum disk space for, 4.2.3

recommendations for file system, 4.2.3

storage options, 4.2.2

data loss

minimizing with Automatic Storage Management, 4.4.2

Database Configuration Assistant

raw storage requirements, C.2

databases

Automatic Storage Management requirements, 4.4.2

de-install CRS software

Windows-based systems, B.1.1

de-install Oracle Clusterware software, B.1 device names

creating with asmtool, 4.4.5.3

creating with asmtoolg, 4.4.5.2

directory

database file directory, 4.2.3

DisableDHCPMediaSense parameter, 1.2, 2.8.1, 2.8.3.4 disk automount

enable, 3.1.2, 4.1.2

disk group

Automatic Storage Management, with ASM redundancy levels, 4.4.2

recommendations for Automatic Storage Management disk groups, 4.4.2

disk space

checking, 2.5

required partition sizes, 3.4.1

requirements for preconfigured database in ASM, 4.4.2

disk space requirements

Windows, 2.4.1

disks

configuring for Automatic Storage Management, 4.4.1

configuring raw devices, 4.5

supported for Automatic Storage Management, 4.4.4

domain user, 2.7

used in installation, 1.3

E

enable automount, 3.1.2, 4.1.2

enable disk automount, 3.1.2, 4.1.2

environment variables

TEMP and TMPDIR, 2.5

EXAMPLE tablespace

raw device for, 4.5.1

examples

Automatic Storage Management failure groups, 4.4.2

external redundancy

Automatic Storage Management redundancy level, 4.4.2

F

failure groups

Automatic Storage Management, 4.4.2

characteristics in Automatic Storage Management, 4.4.2

examples in Automatic Storage Management, 4.4.2

features, new, Preface file system

storage option for data files, 4.2.2

using for data files, 4.2.3

file systems

system requirements, 2.4.1

Windows system requirements, 2.4.1

files

control files

raw devices, 4.5.1

raw device mapping file

on Windows, 4.5.3

redo log files

raw devices, 4.5.1

server parameter file

raw devices, 4.5.1

SPFILE

raw devices, 4.5.1

Firefox, 2.3.1

and Oracle Application Express, 2.3.1

H

hardware certification, 2.3

high redundancy

Automatic Storage Management redundancy level, 4.4.2

I

installation

Automatic Storage Management requirements, 4.4.2

using cluster configuration file, 5.3.2

J

Java Runtime Environment

requirements, 2.4.1

Windows requirements, 2.4.1

L

local administrative user, 2.7

local administrator user, 1.3

local device

using for data files, 4.2.3

Logical Volume Manager (LVM)

recommendations for Automatic Storage Management, 4.4.2

logical volumes

M

mapping file

for raw devices

on Windows, 4.5.3

Media Sensing, 1.2, 2.8.1, 2.8.3.4

disabling, 1.2, 2.8.3.4

Microsoft Internet Explorer, 2.3.1

and Oracle Application Express, 2.3.1

mirroring ASM disk groups, 4.4.2 Mozilla, 2.3.1 My Oracle Support, 6.2 My Oracle Support site

about, 2.3, 2.3

accessing, 2.3, 2.3

N

Netscape Navigator, 2.3.1

network interface bind order, 2.8.3.3

new features, Preface

normal redundancy, Automatic Storage Management redundancy level, 4.4.2

NTFS

system requirements, 2.4.1

O

OCCSD, 5.2

OCFS

formatting drives for, 5.4

installation, 5.4

OCR

primary and mirror, 3.2.2

upgrading from previous release, 3.4.1

Windows, 3.5

operating systems, supported, 2.6 ORA_DBA group, 1.3, 2.1 Oracle Advanced Security

preinstallation requirements, 2.9.1

Oracle Application Express

supported browsers for, 2.3.1

Oracle Cluster Registry

see OCR

Oracle Clusterware

install with Oracle Universal Installer on Windows, 5.3.1

installation setup procedures for Windows, 5.3.1

installing, 5

installing with Oracle Universal Installer, 5.3

Oracle Clusterware home

Windows-based system, 5.3.1

Oracle Connection a Manager, 2.3.2.2 Oracle Database

data file storage options, 4.2.2

minimum disk space requirements, 4.2.3

requirements with Automatic Storage Management, 4.4.2

Windows Terminal Services support, 2.3.2.2

Oracle Enterprise Manager

preinstallation requirements, 2.9.2

Oracle Object Link Manager, 2.3.2.2 Oracle patch updates, 6.2 Oracle Services for Microsoft Transaction Server, 2.3.2.2 Oracle Universal Installer

and Oracle Clusterware, 5.3

Cluster Configuration page, 5.3.1

Cluster Configuration Storage page, 5.3.1

Specify File Locations page, 5.3.1

Specify Network Interface Usage page, 5.3.1

Summary page, 5.3.1

Welcome page, 5.3.1

OUI

see Oracle Universal Installer

P

partition creation for Automatic Storage Management disks, 4.4.4

partitions

sizing for Oracle Clusterware files, 1.1

using with Automatic Storage Management, 4.4.2

patch updates

download, 6.2

install, 6.2

My Oracle Support, 6.2

patch upgrades

rolling upgrades, D.4

post-installation

patch download and install, 6.2

preconfigured database

Automatic Storage Management disk space requirements, 4.4.2

requirements when using Automatic Storage Management, 4.4.2

preinstallation

raw device creation, C.2.1

requirements for Oracle Advanced Security, 2.9.1

requirements for Oracle Enterprise Manager, 2.9.2

privileges

verifying for OUI cluster installation, 2.10

R

RAID

using for Oracle data files, 4.2.3

RAID (Redundant Array of Independent Disks)

recommended ASM redundancy level, 4.4.2

raw device

for server parameter file, 4.5.1

for SPFILE, 4.5.1

raw devices

configuring, 4.5

creating a raw device mapping file

on Windows, 4.5.3

Database Configuration Assistant, C.2 for control files, 4.5.1 for EXAMPLE tablespace, 4.5.1 for redo log files, 4.5.1 for SYSAUX tablespace, 4.5.1 for SYSTEM tablespace, 4.5.1 for TEMP tablespace, 4.5.1 for UNDOTBS tablespace, 4.5.1 for USERS tablespace, 4.5.1 setting up, C.2.1 storage option for data files, 4.2.2 raw partitions

See raw devices

raw storage

Database Configuration Assistant requirements, C.2, C.2

Real Application Clusters

raw device setup, C.2.1

redo log files

raw devices for, 4.5.1

redundancy level

and space requirements for preconfigured database, 4.4.2

for Automatic Storage Management, 4.4.2

redundant array of independent disks

See RAID

required service packs, 2.6 requirements

for Java Runtime Environment, 2.4.1

for Java Runtime Environment on Windows, 2.4.1

for Oracle Enterprise Manager, 2.9.2

hardware certification, 2.3, 2.3

software certification, 2.3, 2.3

Web browser support, 2.3.1

Windows Remote Desktop Connection support, 2.3.2.2

Windows service packs, 2.6

rolling upgrade

of Oracle Database with patch, D.4

S

SAN (storage area network) disks, 4.4.4

Server Management (SRVM), 2.3.2.2

server parameter file

raw device, 4.5.1

service pack requirements

Windows, 2.6

service packs, 2.6 software certification, 2.3 Specify File Locations page

Oracle Universal Installer, 5.3.1

Specify Network Interface Usage page

Oracle Universal Installer, 5.3.1

SPFILE

raw devices, 4.5.1

storage area network disks, 4.4.4 Summary page

Oracle Universal Installer, 5.3.1

supported operating systems, 2.6 SYSAUX tablespace

for raw devices, 4.5.1

system requirements

for NTFS file systems, 2.4.1

for Windows file systems, 2.4.1

SYSTEM tablespace

for raw devices, 4.5.1

T

TEMP environment variable, 2.5

TEMP tablespace

for raw devices, 4.5.1

temporary directory on Windows, 2.5 temporary disk space

checking on Windows, 2.5

freeing on Windows, 2.5

tmp directory

checking space for Windows installation, 2.5

freeing space for Windows installation, 2.5

TMPDIR environment variable, 2.5 troubleshooting

user equivalency, 5.1.1

U

UNDOTBS tablespace

raw device, 4.5.1

upgrade

of Cluster Ready Services, 5.2

user

domain, 1.3

local Administrator, 1.3

user equivalence

testing, 5.1.1

USERS tablespace

raw device, 4.5.1

V

voting disks, 3.2.2

configuration of, 5.2

requirement of absolute majority of, 3.2.2

upgrading from a previous release, 3.4.1

Windows, 3.5

W

Web browser support, 2.3.1

Web browsers

Firefox, 2.3.1

Microsoft Internet Explorer, 2.3.1

Mozilla, 2.3.1

Netscape Navigator, 2.3.1

Welcome page

Oracle Universal Installer, 5.3.1

Windows

32-bit installation, 2

64-bit installation, 2

creating a raw device mapping file, 4.5.3

Windows Media Sensing, 2.8.1

disabling, 1.2, 2.8.3.4

Windows network bind order, 2.8.3.3 Windows Server 2008, Preface, 2.6, 2.7.1, 2.11.5, 4.5.2, 5, 5.3.2, 5.3.3 Windows Terminal Services

support, 2.3.2.2

Oracle Legal Notices

Copyright Notice

Copyright © 1994-2016, Oracle and/or its affiliates. All rights reserved.

License Restrictions Warranty/Consequential Damages Disclaimer

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

Warranty Disclaimer

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

Restricted Rights Notice

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

Hazardous Applications Notice

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

Third-Party Content, Products, and Services Disclaimer

This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Alpha and Beta Draft Documentation Notice

If this document is in preproduction status:

This documentation is in preproduction status and is intended for demonstration and preliminary use only. It may not be specific to the hardware on which you are using the software. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to this documentation and will not be responsible for any loss, costs, or damages incurred due to the use of this documentation.

Private Alpha and Beta Draft Documentation Notice

If this document is in private preproduction status:

The information contained in this document is for informational sharing purposes only and should be considered in your capacity as a customer advisory board member or pursuant to your beta trial agreement only. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described in this document remains at the sole discretion of Oracle.

This document in any form, software or printed matter, contains proprietary information that is the exclusive property of Oracle. Your access to and use of this confidential material is subject to the terms and conditions of your Oracle Master Agreement, Oracle License and Services Agreement, Oracle PartnerNetwork Agreement, Oracle distribution agreement, or other license agreement which has been executed by you and Oracle and with which you agree to comply. This document and information contained herein may not be disclosed, copied, reproduced, or distributed to anyone outside Oracle without prior written consent of Oracle. This document is not part of your license agreement nor can it be incorporated into any contractual agreement with Oracle or its subsidiaries or affiliates.

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.