Contents

Title and Copyright Information

Send Us Your Comments

Preface

Intended Audience

Documentation Accessibility

Related Documents

Part I Overview

1 Introduction

1.1 What is High Availability

1.1.1 High Availability Problems

1.1.2 High Availability Solutions

1.2 Oracle Application Server High Availability Concepts

1.2.1 Terminology

1.2.2 Oracle Application Server Base Architecture

1.2.3 Oracle Application Server High Availability Architectures

1.2.4 Choosing the Best High Availability Architecture

1.3 Organization of this Guide

1.4 High Availability Information in Other Documentation

2 Oracle Application Server High Availability Framework

2.1 Redundant Architectures

2.1.1 Oracle Application Server Active-Active Configurations: Oracle Application Server Clusters

2.1.2 Oracle Application Server Active-Passive Configurations: Oracle Application Server Cold Failover Clusters

2.2 High Availability Services in Oracle Application Server

2.2.1 Process Death Detection and Automatic Restart

2.2.1.1 Process Management with Oracle Process Manager and Notification Server

2.2.2 Configuration Management

2.2.2.1 Configuration Management with Distributed Configuration Management

2.2.3 State Replication

2.2.4 Server Load Balancing and Failover

2.2.4.1 Internal Load Balancing Mechanism Provided in Oracle Application Server

2.2.4.2 External Load Balancers

2.2.5 Backup and recovery

2.2.5.1 Oracle Application Server Backup and Recovery Tool

2.2.6 Disaster Recovery

2.2.6.1 Oracle Application Server Guard

Part II Middle-tier High Availability

3 Middle-tier High Availability

3.1.1 Active-Active

3.1.1.1 Oracle Application Server Web Cache Sub-Tier

3.1.1.2 Oracle HTTP Server Sub-Tier

3.1.1.3 Oracle Application Server Containers for J2EE Sub-Tier

3.1.2 Active-Passive

3.1.2.1 Oracle Application Server Cold Failover Cluster (Middle-Tier)

3.2 Highly Available Middle-tier Configuration Management Concepts

3.2.1 Oracle Application Server Clusters Managed Using DCM

3.2.1.1 What is a DCM-Managed OracleAS Cluster?

3.2.1.2 Oracle Application Server DCM Configuration Repository Types

3.2.2 Manually Managed Oracle Application Server Clusters

3.3 Middle-tier Backup and Recovery Considerations

3.4 Middle-tier Applications

3.4.1 Oracle Application Server Portal

3.4.1.1 Enabling Redundancy for OracleAS Portal

3.4.1.2 Configuring Load Balancer Routers for OracleAS Portal

3.4.1.3 Session Binding for Web Clipping Portlet

3.4.1.4 OracleAS Portal and OracleAS Web Cache

3.4.2 Oracle Application Server Wireless

3.4.2.1 OracleAS Wireless Clustering Architecture

3.4.3 OracleAS Integration B2B

3.4.4 Oracle Application Server Integration InterConnect

3.4.5 Oracle Business Intelligence Discoverer

3.4.5.1 Oracle Business Intelligence Discoverer Preferences Server

4 Managing and Operating Middle-tier High Availability

4.1 Middle-tier High Availability Configuration Overview

4.1.1 DCM-Managed Oracle Application Server Clusters

4.1.2 Manually Managed Oracle Application Server Clusters

4.2 Using DCM-Managed OracleAS Clusters

4.2.1 Creating DCM-Managed OracleAS Clusters

4.2.1.1 Associating An Instance With An OracleAS Database-based Farm

4.2.1.2 Associating An Instance With An OracleAS File-based Farm

4.2.1.3 Using the Application Server Control Console Create Cluster Page

4.2.2 Adding Instances To DCM-Managed OracleAS Clusters

4.2.3 Removing Instances from DCM-Managed OracleAS Clusters

4.2.4 Starting Stopping and Deleting DCM-Managed OracleAS Clusters

4.2.5 Configuring Oracle HTTP Server Options for DCM-Managed OracleAS Clusters

4.2.5.1 Using and Configuring mod_oc4j Load Balancing

4.2.5.2 Configuring Oracle HTTP Server Instance-Specific Parameters

4.2.5.3 Configuring mod_plsql With Real Application Clusters

4.2.6 Understanding DCM-Managed OracleAS Cluster Membership

4.2.6.1 How the Common Configuration is Established

4.2.6.2 Parameters Excluded from the Common Configuration: Instance-Specific Parameters

4.3 Availability Considerations for the DCM Configuration Repository

4.3.1 Availability Considerations for DCM-Managed OracleAS Cluster (Database)

4.3.2 Availability Considerations for DCM-Managed OracleAS Cluster (File-based)

4.3.2.1 Selecting the Instance to Use for a OracleAS File-based Farm Repository Host

4.3.2.2 Protecting Against The Loss of a Repository Host

4.3.2.3 Impact of Repository Host Unavailability

4.3.2.4 Impact of Non-Repository Host Unavailability

4.3.2.5 Updating and Checking the State of Local Configuration

4.3.2.6 Performing Administration on a DCM-Managed OracleAS Cluster

4.3.2.7 Best Practices for Repository Backups

4.3.2.8 Best Practices for Managing Instances In OracleAS File-based Farms

4.4 Using Oracle Application Server Clusters (OC4J)

4.4.1 Overview of OracleAS Cluster (OC4J) Configuration

4.4.2 Cluster-Wide Configuration Changes and Modifying OC4J Instances

4.4.2.1 Creating or Deleting OC4J Instances In An OracleAS Cluster (OC4J)

4.4.2.2 Deploying Applications On An OracleAS Cluster (OC4J)

4.4.2.3 Configuring Web Application State Replication With OracleAS Cluster (OC4J)

4.4.2.4 Configuring EJB Application State Replication With OracleAS Cluster (OC4J-EJB)

4.4.2.5 Configuring Stateful Session Bean Replication for OracleAS Cluster (OC4J-EJB)s

4.4.3 Configuring OC4J Instance-Specific Parameters

4.4.3.1 Configuring OC4J Islands and OC4J Processes

4.4.3.2 Configuring Port Numbers and Command Line Options

4.5 Using Oracle Application Server Clusters (Portal)

4.6 Using Oracle Application Server Clusters (Web Cache)

4.7 Managing OracleAS Cold Failover Cluster (Middle-Tier)

4.7.1 Managing Configuration and Deployment for OracleAS Cold Failover Cluster (Middle-Tier)

4.7.1.1 Configuration and Deployment Changes for OracleAS Cold Failover Cluster (Middle-Tier)

4.7.1.2 Backup and Recovery for OracleAS Cold Failover Cluster (Middle-Tier)

4.7.1.3 Using Application Server Control Console for OracleAS Cold Failover Cluster (Middle-Tier)

4.7.2 Managing Failover for OracleAS Cold Failover Cluster (Middle-Tier)

4.7.2.1 Manual Failover for OracleAS Cold Failover Cluster (Middle-Tier)

4.7.2.2 Manual Failover for the Virtual IP in OracleAS Cold Failover Cluster (Middle-Tier)

4.7.2.3 Manual Failover of Components for OracleAS Cold Failover Cluster (Middle-Tier)

4.7.2.4 Manual Failover of OracleAS Cluster (OC4J-JMS)

4.8 Managing Custom Processes With OPMN

4.9 Managing Oracle Application Server Middle-tier Upgrades

4.9.1 Upgrading Oracle Application Server Instances

4.9.2 Upgrading DCM-Managed OracleAS Clusters

4.9.3 Upgrading Stateful OC4J Applications

4.10 Using OracleAS Single Sign-On With OracleAS Cluster (Middle-Tier)

Part III OracleAS Infrastructure High Availability

5 Oracle Application Server Infrastructure High Availability

5.1 Oracle Application Server Infrastructure Overview

5.2 Oracle Application Server Infrastructure Components

5.2.1 Oracle Application Server Metadata Repository

5.2.1.1 When to Use Oracle Application Server Metadata Repository

5.2.2 Oracle Identity Management

5.2.2.1 Oracle Internet Directory

5.2.2.2 Oracle Application Server Single Sign-On

5.2.2.3 Oracle Delegated Administration Services

5.2.2.4 Oracle Directory Integration and Provisioning

5.2.2.5 Oracle Application Server Certificate Authority

5.2.3 Oracle HTTP Server

5.2.4 Oracle Application Server Containers for J2EE (OC4J)

5.2.5 Oracle Enterprise Manager 10g Application Server Control Console

5.3 High Availability Configurations for Infrastructure

5.3.1 Active-Active High Availability Solutions

5.3.1.1 OracleAS Cluster (Identity Management)

5.3.1.2 Distributed OracleAS Cluster (Identity Management)

5.3.2 Active-Passive High Availability Solutions

5.3.2.1 OracleAS Cold Failover Cluster (Infrastructure)

5.3.2.2 Distributed OracleAS Cold Failover Cluster (Infrastructure)

5.3.2.3 OracleAS Cold Failover Cluster (Identity Management)

5.3.2.4 Distributed OracleAS Cold Failover Cluster (Identity Management)

5.4 OracleAS Infrastructure Backup and Recovery Considerations

5.4.1 OracleAS Cold Failover Cluster

5.4.2 Oracle Identity Management

6 Managing and Operating Infrastructure High Availability

6.1 Managing Oracle Application Server Cluster (Identity Management)

6.1.1 Starting Stopping and Monitoring OracleAS Cluster (Identity Management)

6.1.1.1 Starting OracleAS Cluster (Identity Management)

6.1.1.2 Stopping OracleAS Cluster (Identity Management)

6.1.1.3 Monitoring OracleAS Cluster (Identity Management)

6.1.2 Configuring OracleAS Cluster (Identity Management)

6.1.2.1 Changing Configuration Files For OracleAS Cluster (Identity Management)

6.1.2.2 Configuring A Load Balancer For OracleAS Cluster (Identity Management)

6.1.3 Failover For OracleAS Cluster (Identity Management)

6.1.4 Backup and Recovery For OracleAS Cluster (Identity Management)

6.1.5 Using Application Server Control With OracleAS Cluster (Identity Management)

6.1.6 Additional Considerations For OracleAS Cluster (Identity Management)

6.2 Managing Oracle Application Server Cold Failover Cluster (Infrastructure)

6.2.1 Starting Stopping and Monitoring OracleAS Cold Failover Cluster (Infrastructure)

6.2.1.1 Starting OracleAS Cold Failover Cluster (Infrastructure)

6.2.1.2 Stopping OracleAS Cold Failover Cluster (Infrastructure)

6.2.1.3 Monitoring OracleAS Cold Failover Cluster (Infrastructure)

6.2.2 Configuring OracleAS Cold Failover Cluster (Infrastructure)

6.2.2.1 Changing Configuration For OracleAS Cold Failover Cluster (Infrastructure)

6.2.2.2 Configuring Virtual IPs For OracleAS Cold Failover Cluster (Infrastructure)

6.2.3 Failover For OracleAS Cold Failover Cluster (Infrastructure)

6.2.3.1 Failover For OracleAS Cold Failover Cluster (Infrastructure) For Solaris Systems

6.2.3.2 Failover For OracleAS Cold Failover Cluster (Infrastructure) For Windows Systems

6.2.3.3 Failover For OracleAS Cold Failover Cluster (Infrastructure) For Linux Systems

6.2.4 Backing Up and Recovering OracleAS Cold Failover Cluster (Infrastructure)

6.2.5 Using Application Server Control With OracleAS Cold Failover Cluster (Infrastructure)

6.3 Managing Oracle Application Server Cold Failover Cluster (Identity Management)

6.3.1 Starting Stopping and Monitoring OracleAS Cold Failover Cluster (Identity Management)

6.3.1.1 Starting OracleAS Cold Failover Cluster (Identity Management)

6.3.1.2 Stopping OracleAS Cold Failover Cluster (Identity Management)

6.3.1.3 Monitoring OracleAS Cold Failover Cluster (Identity Management)

6.3.2 Configuring OracleAS Cold Failover Cluster (Identity Management)

6.3.2.1 Changing Configuration For OracleAS Cold Failover Cluster (Identity Management)

6.3.2.2 Configuring Virtual IPs For OracleAS Cold Failover Cluster (Identity Management)

6.3.3 Failover For OracleAS Cold Failover Cluster (Identity Management)

6.3.3.1 Failover For OracleAS Cold Failover Cluster (Identity Management) On UNIX Systems

6.3.3.2 Failover For OracleAS Cold Failover Cluster (Identity Management) On Linux Systems

6.3.4 Backup and Recovery For OracleAS Cold Failover Cluster (Identity Management)

Part IV Disaster Recovery

7 Oracle Application Server Disaster Recovery

7.1 Oracle Application Server 10g Disaster Recovery Solution

7.1.1 Requirements

7.2 Preparing the OracleAS Disaster Recovery Environment

7.2.1 Planning and Assigning Hostnames

7.2.1.1 Physical Hostnames

7.2.1.2 Network Hostnames

7.2.1.3 Virtual Hostname

7.2.2 Configuring Hostname Resolution

7.2.2.1 Using Local Hostnaming File Resolution

7.2.2.2 Using DNS Resolution

7.3 Overview of Installing Oracle Application Server 10g Software

7.4 Overview of OracleAS Guard and asgctl

7.4.1 Overview of asgctl

7.4.2 OracleAS Guard Client

7.4.3 OracleAS Guard Server

7.4.4 asgctl Operations

7.4.5 OracleAS Guard Integration with OPMN

7.4.6 Supported OracleAS Disaster Recovery Configurations

7.4.7 Supported Oracle Application Server Releases and Operating Systems

7.5 OracleAS Guard Operations -- Standby Instantiation and Standby Synchronization

7.5.1 Configuring OracleAS Guard and Other Relevant Information

7.5.2 Standby Instantiation

7.5.3 Standby Synchronization

7.6 Runtime Operations -- OracleAS Guard Switchover and Failover Operations

7.6.1.1 Scheduled Outages

7.6.1.2 Unplanned Outages

7.6.2 Monitoring OracleAS Guard Operations and Troubleshooting

7.6.2.1 Verifying the Farm

7.6.2.2 Displaying the Current Operation

7.6.2.3 Displaying a List of Recent Operations

7.6.2.4 Stopping an Operation

7.6.2.5 Tracing Tasks

7.6.2.6 Writing Information About the Farm to a File

7.6.2.7 Shutting Down a Farm with a Problem

7.6.2.8 Starting Up a Shutdown Farm

7.6.2.9 Error Messages

7.7 Wide Area DNS Operations

7.7.1 Using a Wide Area Load Balancer

7.7.2 Manually Changing DNS Names

7.8 Using OracleAS Guard Command-Line Utility (asgctl)

7.8.1 Typical OracleAS Guard Session Using asgctl

7.8.1.1 Invoking the OracleAS Guard Client and Connecting to the OracleAS Guard Server

7.8.1.2 Getting Help

7.8.1.3 Specifying the Primary Database

7.8.1.4 Verifying the Farm

7.8.1.5 Instantiating the Farm at the Secondary Site

7.8.1.6 Synchronizing the Secondary Site with the Primary Site

7.8.1.7 Disconnecting from the OracleAS Guard Server and Exiting the OracleAS Guard Client

7.8.1.8 Creating and Executing an asgctl Script

7.8.2 Periodic Scheduling of OracleAS Guard asgctl Scripts

7.8.3 Submitting OracleAS Guard Jobs to the Enterprise Manager Job System

7.8.4 Specifying Different Credentials for Two or More Nodes

7.8.4.1 Setting asgctl Credentials

7.8.4.2 Specifying the Primary Database

7.8.5 Reference Section: OracleAS Guard asgctl Command-line Commands

instantiate farm to

set asg credentials

set new primary database

set primary database

switchover farm to

Part V Appendices

A Troubleshooting High Availability

A.1 Problems and Solutions

A.1.1 Cluster Configuration Assistant Fails During Installation

A.1.2 Oracle Ultra Search Configuration Assistant is Unable to Connect to Oracle Internet Directory During High Availability Infrastructure Installation

A.1.3 Unable to Perform Online Database Backup and Restore in OracleAS Cold Failover Cluster Environment

A.1.4 odisrv Process Does Not Failover

A.1.5 Oracle Ultra Search Web Crawler Does Not Failover

A.1.6 Unable to Restore OracleAS Metadata Repository to a Different Host

A.1.7 Cannot Connect to Database for Restoration (Windows)

A.1.8 Unpredictable Behavior from Oracle Application Server Cluster (Identity Management) Configuration

A.1.9 Wrong Name Specified for Load Balancer

A.1.10 OracleAS Disaster Recovery: Standby Site Not Synchronized

A.1.11 OracleAS Disaster Recovery: Failure to Bring Up Standby Instances After Failover or Switchover

A.1.12 OracleAS Disaster Recovery: Unable to Start Standalone OracleAS Web Cache Installations at the Standby Site

A.1.13 OracleAS Disaster Recovery: Standby Site Middle-tier Installation Uses Wrong Hostname

A.1.14 OracleAS Disaster Recovery: Failure of Farm Verification Operation with Standby Farm

A.1.15 OracleAS Disaster Recovery: Sync Farm Operation Returns Error Message

A.2 Need More Help?

B Manually Managed Oracle Application Server Cluster

B.1 Overview of Manually Managed OracleAS Cluster

B.1.1 Oracle Application Server Manually Managed Clusters

B.1.2 What Are Manually Managed OracleAS Clusters?

B.1.3 When Do I Need To Use A Manually Managed OracleAS Cluster

B.1.3.1 No Database Requirement for Manually Managed OracleAS Cluster

B.1.3.2 Tiered Deployment Requirement for Manually Managed OracleAS Cluster

B.1.3.3 Tiered Deployment With Security Requirement

B.2 Configuring Manually Managed OracleAS Cluster

B.2.1 Associating Oracle Application Server Instances Together

B.2.2 Configuring OC4J Instances for State Replication

B.2.2.1 Configuring State Replication for Web Applications

B.2.2.2 Configuring State Replication for EJB Applications

B.2.3 Configuring the J2EE Application Properties

B.2.4 Configuring Oracle HTTP Server for Failover and Load Balancing

B.2.4.1 Understanding mod_oc4j Request Routing

B.2.4.2 Identifying the Instance Names

B.2.4.3 Configuring mod_oc4j Request Routing

C OracleAS Guard Error Messages

C.1 DGA Error Messages

C.1.1 LRO Error Messages

C.1.2 Undo Error Messages

C.1.3 Create Template Error Messages

C.1.4 Switchover Physical Standby Error Messages

C.2 Duf Error Messages

C.2.1 Database Error Messages

C.2.2 Connection and Network Error Messages

C.2.3 SQL*Plus Error Messages

C.2.4 JDBC Error Messages

C.2.5 OPMN Error Messages

C.2.6 Net Services Error Messages

C.2.7 System Error Messages

C.2.8 Warning Error Messages

C.2.9 OracleAS Database Error Messages

C.2.10 OracleAS Farm Error Messages

C.2.11 OracleAS Backup and Restore Error Messages

C.2.12 OracleAS Guard Synchronize Error Messages

C.2.13 OracleAS Guard Instantiate Error Messages

D Manual Sync Operations

D.1 Manually Synchronizing Baseline Installation with Standby Site Without Using OracleAS Guard asgctl Command-line Utility

D.1.1 Manually Backing Up the Production Site

D.1.1.1 Shipping OracleAS Infrastructure Database Archive Logs

D.1.1.2 Backing Up Configuration Files (OracleAS Infrastructure and Middle Tier)

D.1.2 Manually Restoring to Standby Site

D.1.2.1 Restoring Configuration Files (OracleAS Infrastructure and Middle Tier)

D.1.2.2 Restoring the OracleAS Infrastructure Database - Applying Log Files

E Setting Up a DNS Server

F Secure Shell (SSH) Port Forwarding

F.1 SSH Port Forwarding

Index