9 OracleAS Disaster Recovery

This chapter describes issues associated with OracleAS Disaster Recovery. It includes the following topics:

Section 9.1, "General Issues and Workarounds"

9.1 General Issues and Workarounds

This section describes general issues and workarounds. It includes the following topic:

Section 9.1.1, "Compatible ASG Releases for OracleAS Instances from Different Oracle Application Server Releases"
Section 9.1.2, "Compatible ASG Releases in an OracleAS Disaster Recovery Topology"
Section 9.1.3, "Adding an Instance from a Remote Client Adds an Instance on the Local Instance and Not on the Remote Instance"
Section 9.1.4, "Switchover Operation in an Asymmetric Topology Requires All Components to be Shutdown on Instances on the Primary Site that Do Not Have a Standby Peer"
Section 9.1.5, "ASG to Catch Array Overflow Exceptions in Queries to Primary"
Section 9.1.6, "The create standby Command Fails if the Redo Log Files Directories do not Exist at the Standby"
Section 9.1.7, "Corrupt Index Blocks in Metadata Repository Databases"

9.1.1 Compatible ASG Releases for OracleAS Instances from Different Oracle Application Server Releases

By default, when you install an Oracle Application Server instance using a particular release of Oracle Application Server, a particular release of Application Server Guard (ASG) is installed into the Oracle home for the instance. You also install ASG on standalone hosts on which external resources (such as an Oracle database) are located that you have included in your OracleAS Disaster Recovery topology.

Multiple releases of Application Server Guard (ASG) are available. It is possible (recommended) in some cases to upgrade the ASG release that was installed in an Oracle Application Server instance home when you installed that instance. To upgrade the ASG release in an Oracle Application Server instance home, download the ASG standalone kit for the recommended ASG release from the Oracle Technology Network (OTN), and then use that ASG standalone kit to install the recommended ASG release into the home. You also use the ASG standalone kit to install ASG on standalone hosts that are included in your OracleAS Disaster Recovery topology.

Use Table 9-1 and Table 9-2 to determine whether a particular ASG release is compatible when installed into an Application Server instance home for a particular Oracle Application Server release. The left column of the table shows the different ASG releases for which a standalone ASG installation kit is available. The remaining columns show different Oracle Application Server releases for which an Oracle Application Server instance can be created.

This list describes the meaning of the entries in Table 9-1 and Table 9-2:

N: This ASG release is not compatible with an instance from this Oracle Application Server release.
X: This ASG release cannot be installed into the Oracle home for an instance from this Oracle Application Server release.
Y-NR: This ASG release is compatible with an instance from this Oracle Application Server release, but Oracle recommends that you do not install this ASG release into the instance's Oracle home because another ASG release is recommended.
Y: This ASG release is compatible with an instance from this Oracle Application Server release. Oracle recommends you install this ASG release into the instance's Oracle home.

Table 9-1 shows the compatible ASG releases for Oracle Application Server instances from Oracle Application Server 10.1.2.0.2 through 10.1.3.3.

Table 9-1 Compatible ASG Releases for OracleAS Instances from Releases 10.1.2.0.2 Through 10.1.3.3

ASG Release	10.1.2.0.2 OracleAS Instance	10.1.2.1 OracleAS Instance	10.1.2.2 OracleAS Instance	10.1.3.0 OracleAS Instance	10.1.3.1 OracleAS Instance	10.1.3.2 OracleAS Instance	10.1.3.3 OracleAS Instance
10.1.2.0.2	Y-NR	X	X	N	N	N	N
10.1.2.2	Y	Y	Y	N	N	N	N
10.1.2.2.1 (ASG-only release)^Foot 1	Y	Y	Y	N	N	N	N
10.1.3.0	N	N	N	Y-NR	X	N	X
10.1.3.1	N	N	N	Y-NR	Y-NR	Y-NR	X
10.1.3.3	N	N	N	Y	Y	Y	Y

^Footnote 1This is the ASG release that was provided (installed by default) with the OracleAS 10.1.4.2 release. It is compatible with the OracleAS 10.1.2.x releases. There is no OracleAS 10.1.2.2.1 release.

For example, if you have an Oracle Application Server 10.1.3.1 instance and you want to know which ASG release to install in the instance home, you can use Table 9-1 to determine the following:

No ASG 10.1.2.x release is compatible with an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.0 release cannot be installed into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.1 release is compatible with an Oracle Application Server 10.1.3.1 instance, but Oracle recommends that you do not install the ASG 10.1.3.1 release into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.3 release is compatible with an Oracle Application Server 10.1.3.1 instance and Oracle recommends that you install the ASG 10.1.3.3 release into the Oracle home for an Oracle Application Server 10.1.3.1 instance.

Table 9-2 shows the compatible ASG releases for Oracle Application Server instances from Oracle Application Server 10.1.4.0 through 10.1.4.2.

Table 9-2 Compatible ASG Releases for OracleAS Instances from Releases 10.1.4.0 Through 10.1.4.2

ASG Release	10.1.4.0 OracleAS Instance^Foot 1	10.1.4.1 OracleAS Instance^Foot 2	10.1.4.2 OracleAS Instance^Foot 3
10.1.2.0.2	Y-NR	Y-NR	X
10.1.2.2	Y-NR	Y-NR	Y-NR
10.1.2.2.1 (ASG-only release)^Foot 4	Y	Y	Y
10.1.3.0	N	N	N
10.1.3.1	N	N	N
10.1.3.3	N	N	N

^Footnote 1ASG 10.1.2.0.2 is installed by default.

^Footnote 2ASG 10.1.2.0.2 is installed by default.

^Footnote 3ASG 10.1.2.2.1 is installed by default.

^Footnote 4This is the ASG release that was provided (installed by default) with the OracleAS 10.1.4.2 release. It is compatible with the OracleAS 10.1.2.x releases. There is no OracleAS 10.1.2.2.1 release.

9.1.2 Compatible ASG Releases in an OracleAS Disaster Recovery Topology

This chart shows which ASG release combinations are compatible in in an OracleAS Disaster Recovery topology. A topology is a collection of Oracle Application Server instance homes and standalone host homes that combine to comprise the OracleAS Disaster Recovery production site and standby site. Each Oracle Application Server home has a specific ASG release installed, either by default or by a standalone ASG installation. Since OracleAS Disaster Recovery operations are distributed, there is a collection of recommended ASG releases that should be installed across the topology. Example for the EDG deployment, the collection of Oracle homes should be upgraded to ASG version 10.1.3.3 or 10.1.2.2.x.

Use Table 9-3 to determine whether two ASG releases are compatible in an OracleAS Disaster Recovery topology. Find the first ASG release in the left column of the table and then find the second ASG release in one of the other columns of the table.

This list describes the meaning of the entries in Table 9-3:

Y-NR: The first ASG release is compatible with the second ASG release, but Oracle recommends that you do not use this ASG release combination in your topology.
Y: The first ASG release is compatible with the second ASG release. Oracle recommends that you use this ASG release combination in your topology.

Table 9-3 shows which ASG releases are compatible with other ASG releases.

Table 9-3 Compatible ASG Releases in a Topology

ASG Release	10.1.2.0.2	10.1.2.2	10.1.2.2.1	10.1.3.0	10.1.3.1	10.1.3.3
10.1.2.0.2	Y-NR	Y-NR	Y-NR	Y-NR	Y-NR	Y-NR
10.1.2.2	Y-NR	Y	Y	Y-NR	Y-NR	Y
10.1.2.2.1	Y-NR	Y	Y	Y-NR	Y-NR	Y
10.1.3.0	Y-NR	Y-NR	Y-NR	Y-NR	Y-NR	Y-NR
10.1.3.1	Y-NR	Y-NR	Y-NR	Y-NR	Y-NR	Y-NR
10.1.3.3	Y-NR	Y	Y	Y-NR	Y-NR	Y

9.1.3 Adding an Instance from a Remote Client Adds an Instance on the Local Instance and Not on the Remote Instance

When using the asgctl add instance command, the OracleAS Guard client must be run from a system that is already included in the topology.

For example, when an OracleAS Guard client is connected to the OracleAS Guard server that is to be added to an existing topology, the following error is returned:

ASG_IAS-15785: ERROR: The topology is missing the instance that exists in the home 
where the ASG server is running. 
You must first discover or add the instance in home

The workaround to this problem is to use an OracleAS Guard client from a system that is already included in the topology to perform the asgctl add instance command to add an instance to the topology.

9.1.4 Switchover Operation in an Asymmetric Topology Requires All Components to be Shutdown on Instances on the Primary Site that Do Not Have a Standby Peer

Prior to performing an asgctl switchover operation in an asymmetric topology for instances that do not have a standby peer, you must perform an opmnctl stopall command to shutdown all components on each of these ignored instances on the primary site.

When an XML policy file is in use for an asymmetric topology and has the <instanceList successRequirement ="Ignore" set for an instance, for example, as shown in the following example, then in a switchover operation OracleAS Guard ignores that instance:

.
.
.
<instanceList successRequirement = "Ignore">
  <instance>instance B</instance>
</instanceList>
.
.
.

OracleAS Guard, on a switchover operation, shuts down all components on the old primary site except for OracleAS Guard and OPMN and ignores instance B because the policy file specifies to do so. The switchover operation fails because all components are not shut down on the primary site, in this case instance B because the policy file specifies to ignore instance B on the primary site, which has no standby peer.

To workaround this problem, the OracleAS Disaster Recovery Administrator must perform an opmnctl stopall operation for all components on instance B prior to the switchover operation in order for the switchover operation to succeed in this asymmetric topology.

9.1.5 ASG to Catch Array Overflow Exceptions in Queries to Primary

The asgctl create standby database command may fail with an ASG_DUF-4950 error and an error message stack on the console similar to the following:

ASG_DUF-4950: An error occurred on host "myhost" with IP "1.1.1.1" and port "7890"
ASG_SYSTEM-100: 10
ASG_DUF-4900: An exception occurred on the server.
ASG_DGA-13009: Error during Create Physical Standby

In the duf_client.log file on the primary site DB node, the following messages would be found:

java.lang.ArrayIndexOutOfBoundsException: 10
at oracle.duf.DufJdbc.queryRedoLogInfo(DufJdbc.java:535)
at oracle.duf.DufDb$jdbc.queryRedoLogInfo(DufDb.java:4966)
at oracle.duf.DufDb$jdbc.access$2000(DufDb.java:4884)
at oracle.duf.DufDb.queryRedoLogInfo(DufDb.java:3439)

This problem occurs because ASG does not properly handle a database configured with 10 or more redo log files.

To avoid this problem, reduce the number of database redo log files to fewer than 10.

This problem is fixed in release 10.1.3.3.

9.1.6 The create standby Command Fails if the Redo Log Files Directories do not Exist at the Standby

The asgctl create standby database command fails if the redo log files directories on the production site do not exist at the standby.ASG expects the target redo log directory structure to be symmetrical with the redo log directory structure on the production site. If they do not exist in the same directories, a failure will result with the following output:

ASG_ORACLE-300: ORA-00301: error in adding log file '/PATH/redo010.log' - file cannot be created
ASG_ORACLE-300: ORA-27040: file create error, unable to create file
ASG_DUF-3700: Failed in SQL*Plus executing SQL statement:  ALTER DATABASE ADD STANDBY LOGFILE GROUP 10
'/PATH/redo010.log' SIZE 52428800 /* ASG_DGA */;.
ASG_DUF-3535: Failed to create standby redo log.
ASG_DUF-3535: Failed to create standby redo log.
ASG_DGA-13011: Error during Create Physical Standby: Finish-configure standby.
ASG_DUF-3027: Error while executing Creating physical standby database - finish phase at step - finish step.

To avoid this problem, create the target redo log file directories on the standby site in the same directories as the production site.

9.1.7 Corrupt Index Blocks in Metadata Repository Databases

After an ASG switchover or failover operation, metadata index block corruption may occur in a Disaster Recovery metadata repository database.

For a full description of the problem and an explanation of how to deal with it, refer to OracleMetLink note 386830.1 at:

http://metalink.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=386830.1