Configuring Targets for Failover in Active/Passive Environments

F Configuring Targets for Failover in Active/Passive Environments

This section provides a general reference for Cloud Control administrators who want to relocate Cold Failover Cluster (CFC) targets from one existing Management Agent to another. Although the targets are capable of running on multiple nodes, these targets run only on the active node in a CFC environment.

CFC environments commonly use a combination of cluster software to provide a virtual host name and IP address along with interconnected host and storage systems to share information and provide high availability (HA) for applications. Automating failover of the virtual host name and IP, in combination with relocating the Enterprise Manager targets and restarting the applications on the passive node, requires the use of the Oracle Enterprise Manager command-line interface (EM CLI) and Oracle or third-party cluster software. Several Oracle partner vendors offer clusterware solutions in this area.

This chapter covers the following topics:

Target Relocation in Active/Passive Environments

With Oracle Enterprise Manager 13c, a single Oracle Management Agent running on each node in the cluster can monitor targets configured for active/passive high availability. Only one Management Agent is required on each of the physical nodes of the CFC cluster because, in case of a failover to the passive node, Enterprise Manager can move the HA monitored targets from the Management Agent on the failed node to another Management Agent on the newly activated node using a series of EMCLI commands. See the Oracle® Enterprise Manager Command Line Interface Guide manual for more information.

If your application is running in an active/passive environment, the clusterware brings up the applications on the passive node in the event that the active node fails. For Enterprise Manager to continue monitoring the targets in this type of configuration, the existing Management Agent needs additional configuration.

The following sections describe how to prepare the environment to automate and restart targets on the new active node. Failover and failback procedures are also provided.

Installation and Configuration

The following sections describe how to configure Enterprise Manager to support a CFC configuration using the existing Management Agents communicating with the Oracle Management Service processes:

Prerequisites

The following steps assume that the monitored targets have already been installed and configured for failover in a CFC.

Prepare the Active/Passive environments as follows:

Ensure the operating system clock is synchronized across all nodes of the cluster. (Consider using Network Time Protocol (NTP) or another network synchronization method.)
Install management agents on each node in the cluster using the physical hostname. Install the Management Agent on a local disk volume on each node in the cluster. Once installed, the Management Agents are visible in the Cloud Control console.
Install and configure EMCLI on each node in the CFC cluster. See the Oracle® Enterprise Manager Command Line Interface Guide for more information.
When a target is being relocated, ensure that the plug-in version and plug-in revision are the same on both the Management Agent of the failed node and the Management Agent of the newly activated node.

Configuration Steps

The following steps show how to configure Enterprise Manager to support a CFC configuration using the existing Management Agents that are communicating with the OMS processes. The example that follows is based on a configuration with a two-node cluster that has one failover group.

Configuration involves two steps:

Discovering Targets

After the Active / Passive targets have been configured, use the Add Targets Manually screens in the Cloud Control console to add the targets (such as database, listener, application server, and so on). This screen can be accessed by navigating to Setup | Add Target | Add Targets Manually. You should perform this step specifying the active node (the node that is currently hosting the target to be added).

Deploying Plug-ins

After the target has been added determine which plug-ins have been deployed on the agent for the active host. This can be found by navigating to the agent homepage and viewing the plug-ins tab in the Configuration region.

Figure F-1 Agent Home Page

Make a note of the Plug-ins that do not have the Only Discovery Contents box checked. These plug-ins need to be deployed on the agent of the passive node.

After determining which plug-ins are missing by looking at the Agent homepage of the passive node, deploy any missing plug-ins by navigating to Setup | Extensibility | Plug-ins, selecting the relevant plug-in and using the Deploy on Management Agent menu to deploy the plug-in.

Failover Procedure

To speed relocation of targets after a node failover, configure the following steps using a script that contains the commands necessary to automatically initiate a failover of a target. Typically, the clusterware software has a mechanism with which you can automatically execute the script to relocate the targets in Enterprise Manager. Also, see "Relocation Script" for a sample script.

Shut down the target services on the failed active node.

On the active node where the targets are running, shut down the target services running on the virtual IP.
If required, disconnect the storage for this target on the active node.

Shut down all the applications running on the virtual IP and shared storage.
Enable the target's IP address on the new active node.
If required, connect storage for the target on the currently active node.
Relocate the targets in Cloud Control using EM CLI.
To relocate the targets to the Management Agent on the new active node, run the EM CLI relocate_targets verb for each target type (such as a listener or application servers) that you must relocate after the failover operation.

Example:
```
emcli relocate_targets
-src_agent=<node 1>:3872 
-dest_agent=<node 2>:3872
-target_name=<database_name> 
-target_type=oracle_database
-copy_from_src 
-force=yes
```
In this example, port 3872 is the default port for the Management Agent. To find the appropriate port number for your configuration, use the value for the Agent URL parameter. You can determine this parameter by running the following command for the Management Agent:

emctl status agent

Note:

In case of a failover event, the source Agent may not be running. However, there is no need to have the source Management Agent running to accomplish the relocate operation. EM CLI is an OMS client that performs its relocate operations directly against the Management Repository.
Bring up all targets on the new active node.
From the Enterprise Manager console, ensure all relocated targets are up and running .

Failback Procedure

To return the HA targets to the original active node, or to any other cluster member node:

Repeat the steps in "Failover Procedure" to return the HA targets to the active node.
Verify the target status in the Enterprise Manager console.

EM CLI relocate_targets Parameters

As shown in Failover Procedure, you run the EM CLI relocate_targets verb for each target type that will be failed over to (or be switched over) during relocation operations. Table F-1 documents the verb parameters associated with this EM CLI verb.

Table F-1 relocate_targets Verb Parameters

EM CLI Parameter	Description
-src_agent	Management Agent on which the target was running before the failover occurred.
-dest_agent	Management Agent that will be monitoring the target after the failover.
-target_name	Name of the target to be failed over.
-target_type	Type of target to be failed over (internal Enterprise Manager target type). For example, the Oracle database (for a standalone database or an Oracle RAC instance), the Oracle listener for a database listener, and so on.
-copy_from_src	Use the same type of properties from the source Management Agent to identify the target. This is a MANDATORY parameter. Not supplying this parameter may result in the corruption of the target definition.
-force	Force dependencies (if needed) to failover as well.

Relocation Script

The following example shows a relocation script that can executed from a clusterware configuration when a failover operation occurs.

Before running the script:

Set up the Default Normal Host Credential with Normal Host Credential.
Set up the Target Preferred Credential of the database instance with the Normal Database Credential, SYSDBA Database Credential, and Database Host Credential.

Relocation Script Example

#! /bin/ksh
#get the status of the targets
 
emcli get_targets 
 -targets="db1:oracle_database;listener_db1:oracle_listener" 
 -noheader
 
  if [[ $? != 0 ]]; then exit 1; fi
 
# blackout the targets to stop false errors.  This blackout is set to expire in 30 minutes.
 
emcli create_blackout 
 -name="relocating active passive test targets" 
 -add_targets="db1:oracle_database;listener_db1:oracle_listener"
 -reason="testing failover"
 -schedule="frequency:once;duration:0:30"
 
  if [[ $? != 0 ]]; then exit 1; fi
 
# relocate the targets to the new host
 
emcli relocate_targets 
 -src_agent=host1.example.com:3872 
 -dest_agent=host2.example.com:3872 
 -target_name=db1 -target_type=oracle_database 
 -copy_from_src -force=yes 
 
  if [[ $? != 0 ]]; then exit 1; fi
 
emcli relocate_targets 
 -src_agent=host1.example.com:3872 
 -dest_agent=host2.example.com:3872 
 -target_name=listener_db1 
 -target_type=oracle_listener 
 -copy_from_src -force=yes 
 
  if [[ $? != 0 ]]; then exit 1; fi
 
 
# End the blackout and let the targets become visible
 
emcli stop_blackout 
 -name="relocating active passive test targets"
 
  if [[ $? != 0 ]]; then exit 1; fi
 
# Recheck the status of the targets
 
emcli get_targets 
 -targets="db1:oracle_database;listener_db1:oracle_listener" 
 -noheader
 
  if [[ $? != 0 ]]; then exit 1; fi