Previous  Next    Open TOC in new window    View as PDF - New Window  Get Adobe Reader - New Window
Content starts here

Failover and System Reliability

This section describes features that support recovery from failure. It contains the following topics:


Understanding Failover

In general, failover is the ability of a product to detect the failure of a particular component and switch to a working replica of that component without losing functionality. Oracle Entitlements Server support two failover scenarios:


Assuring Runtime Failover for SSMs

OES security providers depend on data stores for authentication, authorization, and credential mapping. You can configure failover for these important cases:

Note that SSMs have no runtime dependency on the Administration Server.

Figure 2-1 Runtime Availability

Runtime Availability


Assuring Administrative Server Availability

Failover for administration functions can be achieved by installing a secondary, redundant Administration Server that will be used only when the primary becomes unavailable.

For example, consider the global deployment illustrated in Figure 2-2. The depicted enterprise has applications staged on servers in New York, Tokyo, and London. It has also deployed redundant Administration Servers in its New York and London data centers and provides a replicated database to store policies and entitlements information. Under normal conditions, administrators interact with the primary Administration Server in New York only. When policies are updated, the primary Administration Server pushes the changes to all SSMs in the global environment.

Figure 2-2 Administrative Availability (Working Normally)

Administrative Availability (Working Normally)

If the data center in New York goes down, as illustrated in Figure 2-3, the SSMs detect this failure and connect to the secondary Administration Server. The secondary Administration Server uses the primary database unless it becomes unavailable, in which case the server connects to the secondary database server (the replica).

Note that both the primary and secondary Administration Server can use either the primary or secondary database servers.

Figure 2-3 Administrative Availability (After Failure)

Administrative Availability (After Failure)

One benefit of the architecture is that even if all Administration Servers, including secondary Administration Servers go down (for maintenance or due to failure), there is no impact on the applications in production or on the security services provided by the SSMs.

For information on how to configure the Administration Server for failover, see Setting up a Failover Administration Server.


Failover Considerations for the Database Server

Figure 2-3 provides a logical view of failover functionality when the primary database server fails.

Because the database server contains all the configuration and security data used by the Administration Application to protect applications and resources, it must be highly available and reliable. This can be accomplished by implementing the recommendations of the database manufacturer (for example, through the use of clustering architecture or hot standby).

The number of redundant database servers you configure can vary; however, a minimum of two is recommended to maintain reliable services. It is up to the system administrator to set up database failover and configure data replication between the database instances.

There are two approaches for making sure that two instances of the OES database contain the same data:

Figure 2-4 illustrates the failover mechanism for the Administration Server using Oracle RAC.

Figure 2-4 Administration Servers using Oracle RAC

Administration Servers using Oracle RAC

Common methods of achieving high availability include performing periodic back-ups, using fault tolerant disks, and manually copying files whenever they are changed. This is also the case for any optional external data sources in use. A database backup can be used for database recovery in the case of disk failure.


Failover Considerations for SSMs

Use the Administration Console to configure failover support for database-related and LDAP authentication providers by specifying the secondary database or LDAP server, as described in the following sections.

Secondary Databases

A secondary database can be configured for Database Credential Mapping, Database Authentication, ASI Authorization, and ASI Role Mapper providers.

The ASI Authorization Provider contacts an external process to evaluate its authorization queries. If that process dies, the ASI Authorization provider denies access to all resources. This provider can be configured to contact the Administration database to retrieve subject attributes and group membership for use in authorization and delegation decisions. If the database connection fails, the provider connects to the configured secondary database and will also try to reconnect to the failed database after a configurable time-out. If all database connections fail and defined policies operate on user attributes and group membership, all access is denied.

Secondary LDAP Servers

A secondary LDAP server can be configured for Novell LDAP, Active Directory, iPlanet, and Open LDAP authenticators.

The NT Authenticator already supports multiple domain controllers. The WebLogic Authenticator, WebLogic Authorizer and WebLogic Role Mapper use WebLogic’s internal LDAP server as its data store. No support for a redundant source is required.


Failover Considerations for SCMs

When a SCM starts, it contacts the Administration Server to obtain and cache the most current configuration data. When configuration data is modified, the Administration Server pushes the updates to the SCM.

Failover for the SCM server is implemented as follows:


Setting up a Failover Administration Server

The following tasks must be performed to set up a secondary Administration Server to provide failover support:

  1. Install the secondary Administration Server on a separate machine as described in Install a Secondary Administration Server.
  2. Note: For complete instructions on installing Administration Servers, see the Administration Server Installation Guide.
  3. Initialize the secondary Administration Server’s trust stores as described in Initialize the Secondary Server Trust Stores.
  4. Enable periodic trust synchronization on the secondary Administration Server as described in Enable Trust Synchronization.
  5. Restart the secondary Administration Server (CP6 and higher).
  6. OPTIONAL (CP6 AND HIGHER): Set PD.secondaryEnabled = true in BEA_HOME/ales32-admin/config/ on the primary Administration Server. For more information, see

Install a Secondary Administration Server

The secondary Administration Server must installed on a separate machine and set up in the same manner as the primary Administration Server.

  1. Install the container to host Administration Server (WebLogic Server or Apache Tomcat).
  2. Run the Administration Server installation program to install the secondary Administration Server. When prompted, provide the information described in Table 2-1.
  3. Table 2-1 Installing the Secondary Administration Server
    Secondary Server URL
    Leave blank
    Database Configuration
    With the exception of the Install Database Schema property, specify the same information specified when the primary Administration Server was installed.
    CAUTION: Be sure to clear the Install Database Schema checkbox so that the schema is not installed.
    Key Protection Password Selection
    It is recommended that you select the Advanced Password Configuration option and then specify the passwords when prompted.
    The password entered do not have to match those on the primary server.
    If the Advanced Password Configuration option is used, the passwords can be used to decrypt SCM and SSM cache files, which may be useful for debugging.

When complete, initialize the secondary trust stores as described in the next section.

Initialize the Secondary Server Trust Stores

When the secondary Administration Server is installed, a set of unique certificates is generated for it. Before starting the server, you must synchronize its trust stores with those in use on the primary Administration Server.

To initialize the secondary Administration Server trust stores:

  1. On the secondary Administration Server, create the following directories:

  2. Copy the contents of the OES_SHARED/keys directory from the primary Administration Server to the secondary Administration Server machine into the OES_HOME/primary-shared-keys directory.
  3. From the secondary server’s /bin directory, execute initialize_backup_trust.bat|.sh. When prompted for the primary shared SSL directory, enter the path to the OES_HOME/primary-shared-keys directory.

When complete, enable trust synchronization as described in the next section.

Enable Trust Synchronization

Whenever a new SSM/SCM is enrolled — or an existing SSM/SCM is unenrolled — only the trust stores on the primary Administration Server are updated. Unless the secondary Administration Server’s trust store is also updated, problems may occur during failover because the secondary Administration Server will not trust the new SSMs. To alleviate this, whenever a new SSM is enrolled, run the initialize_backup_trust script file - unless the SSM is installed locally on an existing BEA that is already enrolled.

To insure that the trust stores on secondary Administration Server’s are current, the secondary server can be configured to perform periodic synchronization with the primary server’s trust stores.

Note: It is very important that the trust synchronization be enabled on the secondary Administration Server only.

To configure the trust synchronization on the secondary Administration Server:

  1. Start the secondary Administration Server and access the Administration Console.
  2. In the Administration Console, select Administration Console at the top of the navigation tree in the left pane.
  3. In the right pane, select the Failover tab.
  4. Select the Backup radio button. Then complete the fields as described in Figure 2-2 and click Apply.
  5. Table 2-2 Secondary Server’s Failover Tab
    Primary URL
    The URL of the primary Administration Server in the format:
    where <server_name> is the server name or IP address.

    Note: This the same URL used to access the primary server’s Administration Console.

    Name of administrator user (the default is admin).
    Enter Password
    Confirm Password
    Password of the administrator user (the default is password).
    Synchronization Interval
    Number of seconds between synchronization attempts
    This value depends on how frequently SSM or SCM instances are enrolled and un-enrolled with the primary Administration Server.

A sample completion of the Failover tab is shown in Figure 2-5.

Figure 2-5 Configuring a Backup Server in the Administration Console

Configuring a Backup Server in the Administration Console

  Back to Top       Previous  Next