Oracle® Application Server Release Notes 10g Release 3 (10.1.3.2) for Microsoft Windows Part Number B32199-08 |
|
|
View PDF |
This chapter describes issues related to highly available topologies. This chapter contains the following issues:
This section describes configuration issues and their workarounds. It includes the following topics:
Section 11.1.2, "Compatible ASG Releases in an OracleAS Disaster Recovery Topology"
Section 11.1.4, "Supported Java Object Types for Session Attributes in Distributed Applications"
Section 11.1.5, "The create standby Command Fails with Standby Control File Errors"
Section 11.1.6, "ASG to Catch Array Overflow Exceptions in Queries to Primary"
Section 11.1.7, "Startup of Database on the Standby2 Fails after a Switchover"
Section 11.1.9, "Corrupt Index Blocks in Metadata Repository Databases"
Section 11.1.10, "Database SIDs Must be the Same for Database Peers at Primary and Standby Sites"
Section 11.1.12, "Workaround for ASG_DUF-3800 "Failed trying to connect to the OPMN Manager" Error"
Section 11.1.15, "Use Fully Qualified Path Names with the add instance Command"
Section 11.1.17, "Steps to Add a Database with OMF or ASM to ASG Topology"
By default, when you install an Oracle Application Server instance using a particular release of Oracle Application Server, a particular release of Application Server Guard (ASG) is installed into the Oracle home for the instance. You also install ASG on standalone hosts on which external resources (such as an Oracle database) are located that you want to include in your OracleAS Disaster Recovery topology.
Multiple releases of ASG are available. It is possible (recommended) in some cases to upgrade the ASG release that was installed in an Oracle Application Server instance home when you installed that instance. To upgrade the ASG release in an Oracle Application Server instance home, download the ASG standalone kit for the recommended ASG release from the Oracle Technology Network (OTN), and then use that ASG standalone kit to install the recommended ASG release into the home. You also use the ASG standalone kit to install ASG on standalone hosts that you want to include in your OracleAS Disaster Recovery topology.
Use Table 11-1 and Table 11-2 to determine whether a particular ASG release is compatible when installed into an Application Server instance home for a particular Oracle Application Server release. The left column of the table shows the different ASG releases for which an ASG standalone installation kit is available. The remaining columns show different Oracle Application Server releases for which an Oracle Application Server instance can be created.
This list describes the meaning of the entries in Table 11-1 and Table 11-2:
N: This ASG release is not compatible with an instance from this Oracle Application Server release.
X: This ASG release cannot be installed into the Oracle home for an instance from this Oracle Application Server release.
Y-NR: This ASG release is compatible with an instance from this Oracle Application Server release, but Oracle recommends that you do not install this ASG release into the instance's Oracle home because another ASG release is recommended.
Y: This ASG release is compatible with an instance from this Oracle Application Server release. Oracle recommends you install this ASG release into the instance's Oracle home.
Table 11-1 shows the compatible ASG releases for Oracle Application Server instances from Oracle Application Server 10.1.2.0.2 through 10.1.3.3.
Table 11-1 Compatible ASG Releases for OracleAS Instances from Releases 10.1.2.0.2 Through 10.1.3.3
ASG Release | 10.1.2.0.2 OracleAS Instance | 10.1.2.1 OracleAS Instance | 10.1.2.2 OracleAS Instance | 10.1.3.0 OracleAS Instance | 10.1.3.1 OracleAS Instance | 10.1.3.2 OracleAS Instance | 10.1.3.3 OracleAS Instance |
---|---|---|---|---|---|---|---|
10.1.2.0.2 |
Y-NR |
X |
X |
N |
N |
N |
N |
10.1.2.2 |
Y |
Y |
Y |
N |
N |
N |
N |
10.1.2.2.1 (ASG-only release)Foot 1 |
Y |
Y |
Y |
N |
N |
N |
N |
10.1.3.0 |
N |
N |
N |
Y-NR |
X |
N |
X |
10.1.3.1 |
N |
N |
N |
Y-NR |
Y-NR |
Y-NR |
X |
10.1.3.3 |
N |
N |
N |
Y |
Y |
Y |
Y |
Footnote 1 This is the ASG release that was provided (installed by default) with the OracleAS 10.1.4.2 release. It is compatible with the OracleAS 10.1.2.x releases. There is no OracleAS 10.1.2.2.1 release.
For example, if you have an Oracle Application Server 10.1.3.1 instance and you want to know which ASG release to install in the instance home, you can use Table 11-1 to determine the following:
No ASG 10.1.2.x release is compatible with an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.0 release cannot be installed into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.1 release is compatible with an Oracle Application Server 10.1.3.1 instance, but Oracle recommends that you do not install the ASG 10.1.3.1 release into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.3 release is compatible with an Oracle Application Server 10.1.3.1 instance and Oracle recommends that you install the ASG 10.1.3.3 release into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
Table 11-2 shows the compatible ASG releases for Oracle Application Server instances from Oracle Application Server 10.1.4.0 through 10.1.4.2.
Table 11-2 Compatible ASG Releases for OracleAS Instances from Releases 10.1.4.0 Through 10.1.4.2
ASG Release | 10.1.4.0 OracleAS InstanceFoot 1 | 10.1.4.1 OracleAS InstanceFoot 2 | 10.1.4.2 OracleAS InstanceFoot 3 |
---|---|---|---|
10.1.2.0.2 |
Y-NR |
Y-NR |
X |
10.1.2.2 |
Y-NR |
Y-NR |
Y-NR |
10.1.2.2.1 (ASG-only release)Foot 4 |
Y |
Y |
Y |
10.1.3.0 |
N |
N |
N |
10.1.3.1 |
N |
N |
N |
10.1.3.3 |
N |
N |
N |
Footnote 1 ASG 10.1.2.0.2 is installed by default.
Footnote 2 ASG 10.1.2.0.2 is installed by default.
Footnote 3 ASG 10.1.2.2.1 is installed by default.
Footnote 4 This is the ASG release that was provided (installed by default) with the OracleAS 10.1.4.2 release. It is compatible with the OracleAS 10.1.2.x releases. There is no OracleAS 10.1.2.2.1 release.
This chart shows which ASG release combinations are compatible in in an OracleAS Disaster Recovery topology. A topology is a collection of Oracle Application Server instance homes and standalone host homes that combine to comprise the OracleAS Disaster Recovery production site and standby site. Each Oracle Application Server home has a specific ASG release installed, either by default or by an ASG standalone installation. Since OracleAS Disaster Recovery operations are distributed, there is a collection of recommended ASG releases that should be installed across the topology. Example for the EDG deployment, the collection of Oracle homes should be upgraded to ASG version 10.1.3.3 or 10.1.2.2.x.
Use Table 11-3 to determine whether two ASG releases are compatible in an OracleAS Disaster Recovery topology. Find the first ASG release in the left column of the table and then find the second ASG release in one of the other columns of the table.
This list describes the meaning of the entries in Table 11-3:
Y-NR: The first ASG release is compatible with the second ASG release, but Oracle recommends that you do not use this ASG release combination in your topology.
Y: The first ASG release is compatible with the second ASG release. Oracle recommends that you use this ASG release combination in your topology.
Table 11-3 shows which ASG releases are compatible with other ASG releases.
Table 11-3 Compatible ASG Releases in a Topology
ASG Release | 10.1.2.0.2 | 10.1.2.2 | 10.1.2.2.1 | 10.1.3.0 | 10.1.3.1 | 10.1.3.3 |
---|---|---|---|---|---|---|
10.1.2.0.2 |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
10.1.2.2 |
Y-NR |
Y |
Y |
Y-NR |
Y-NR |
Y |
10.1.2.2.1 |
Y-NR |
Y |
Y |
Y-NR |
Y-NR |
Y |
10.1.3.0 |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
10.1.3.1 |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
10.1.3.3 |
Y-NR |
Y |
Y |
Y-NR |
Y-NR |
Y |
To run portlet producers or WebCenter applications in an OracleAS Cluster (or active-active) topology, you need to configure Java Object Cache on all the nodes in the OracleAS Cluster with a list of all the cluster nodes' IP addresses and port numbers so that it can share objects and coordinate across the cluster. You specify this list in the javacache.xml
files.
There are two javacache.xml
files:
Portlet producers use their own Java Object Cache, which can be configured in ORACLE_HOME/portal/conf/javacache.xml
.
WebCenter applications, unless otherwise specified, use the global Java Object Cache, which can be configured in ORACLE_HOME/javacache/admin/javacache.xml
.
To configure portlet producers and WebCenter applications in an OracleAS Cluster topology:
Edit both javacache.xml
files on each node in the OracleAS Cluster:
ORACLE_HOME/portal/conf/javacache.xml
ORACLE_HOME/javacache/admin/javacache.xml
Make these edits to the files:
Set the isDistributed
element to true
. You may need to create this element, if it does not already exist in the file.
In the discoverer
element, specify the IP addresses of all the nodes in the OracleAS Cluster, and specify also the discovery port. You may need to create this element, if it does not already exist in the file.
Example:
<communication> <isDistributed>true</isDistributed> <discoverer ip="apphost1_ip_address"> discovery-port="apphost1_discovery_port"/> <discoverer ip="apphost2_ip_address" discovery-port="apphost2_discovery_port"/> </communication>
Note:
All caches cooperating in the same cache system must specify exactly the same set of IP addresses and port numbers in the same order.Access the Application Server Control Console at http://hostname:port/em/ and log in as the oc4jadmin
user. The Cluster Topology page appears.
Restart the home
and OC4J_WebCenter
instances and any other instances that are running WebCenter applications.
For information on Java Object Cache, see the "Java Object Cache" chapter in the Oracle Containers for J2EE Services Guide.
For applications that you want to deploy and run in a clustered environment (that is, you explicitly added the <distributable/>
tag to the application's web.xml
file and you have configured the application for session replication), be aware that you can bind only the following Java types as attributes in a session:
Java primitives
Java objects of type java.io.Serializable
, javax.ejb.EJBObject
, and javax.ejb.EJBHome
The types listed above reflect the Java objects that can be serialized and deserialized using the Java object serialization mechanism.
If you use other types, you will get an error that looks something like this:
<MSG_TEXT>ServletException cause</MSG_TEXT> <SUPPL_DETAIL><![CDATA[java.lang.IllegalArgumentException: Only java.io.Serializable, javax.ejb.EJBObject and javax.ejb.EJBHome instances can be bound to a session in a distributable web-application, not: oracle.adfdemo.view.faces.portal.SkinsDemo@2acfa2 (class oracle.adfdemo.view.faces.portal.SkinsDemo) at com.evermind.server.http.EvermindHttpSession.setAttribute (EvermindHttpSession.java:143) at com.sun.faces.context.SessionMap.put(ExternalContextImpl.java:580) at com.sun.faces.application.ApplicationAssociate. createAndMaybeStoreManagedBeans(ApplicationAssociate.java:299) at com.sun.faces.el.VariableResolverImpl.resolveVariable (VariableResolverImpl.java:97) at oracle.adfinternal.view.faces.el.AdfFacesVariableResolver.resolveVariable (AdfFacesVariableResolver.java:40) at oracle.adfinternal.view.faces.model.VariableResolverUtils$JspResolver. resolveVariable(VariableResolverUtils.java:79) at com.sun.faces.el.impl.NamedValue.evaluate(NamedValue.java:145) at com.sun.faces.el.impl.ComplexValue.evaluate(ComplexValue.java:166) at com.sun.faces.el.impl.ExpressionEvaluatorImpl.evaluate (ExpressionEvaluatorImpl.java:263) at com.sun.faces.el.ValueBindingImpl.getValue(ValueBindingImpl.java:160)
On Windows, the asgctl create standby database
command may fail with the ORA-01665 error "control file is not a standby control file."
On Windows, during the create standby operation, the SPFILE is created under $ohome/dbs instead of $ohome/database.
After failing over the database and issuing the create standby
command, the old SPFILE still exists.
To avoid this problem, remove the SPFILE under $ohome/database on Windows before executing the create standby
command.
The asgctl create standby database
command may fail with an ASG_DUF-4950 error and an error message stack on the console similar to the following:
ASG_DUF-4950: An error occurred on host "myhost" with IP "1.1.1.1" and port "7890" ASG_SYSTEM-100: 10 ASG_DUF-4900: An exception occurred on the server. ASG_DGA-13009: Error during Create Physical Standby
In the duf_client.log file on the primary site DB node, the following messages would be found:
java.lang.ArrayIndexOutOfBoundsException: 10 at oracle.duf.DufJdbc.queryRedoLogInfo(DufJdbc.java:535) at oracle.duf.DufDb$jdbc.queryRedoLogInfo(DufDb.java:4966) at oracle.duf.DufDb$jdbc.access$2000(DufDb.java:4884) at oracle.duf.DufDb.queryRedoLogInfo(DufDb.java:3439)
This problem occurs because ASG does not properly handle a database configured with 10 or more redo log files.
To avoid this problem, reduce the number of database redo log files to fewer than 10.
This problem is fixed in release 10.1.3.3.
Sometimes after an ASG switchover operation has been performed, a database has the following problem:
SQL> startup; ORA-09925: Unable to create audit trail file Linux Error: 2: No such file or directory Additional information: 9925
When the asgctl create standby database
command is executed on Linux, ASG creates the initfile with audit_file_dest
set to $ohome/admin/<SID>/admin instead of $ohome/admin/<SID>/adump on the standby site. For RAC consisting of multiple nodes, one of the nodes has the admin directory created, while the other one does not.
To avoid this problem, issue the following command on the standby RAC node #2 before attempting any manual startups:
mkdir $ohome/admin/<SID>/admin
This problem is fixed in release 10.1.3.3.
The asgctl create standby database
command fails if the redo log files directories on the production site do not exist at the standby.ASG expects the target redo log directory structure to be symmetrical with the redo log directory structure on the production site. If they do not exist in the same directories, a failure will result with the following output:
ASG_ORACLE-300: ORA-00301: error in adding log file '/PATH/redo010.log' - file cannot be created ASG_ORACLE-300: ORA-27040: file create error, unable to create file ASG_DUF-3700: Failed in SQL*Plus executing SQL statement: ALTER DATABASE ADD STANDBY LOGFILE GROUP 10 '/PATH/redo010.log' SIZE 52428800 /* ASG_DGA */;. ASG_DUF-3535: Failed to create standby redo log. ASG_DUF-3535: Failed to create standby redo log. ASG_DGA-13011: Error during Create Physical Standby: Finish-configure standby. ASG_DUF-3027: Error while executing Creating physical standby database - finish phase at step - finish step.
To avoid this problem, create the target redo log file directories on the standby site in the same directories as the production site.
After an ASG switchover or failover operation, metadata index block corruption may occur in a Disaster Recovery metadata repository database.
For a full description of the problem and an explanation of how to deal with it, refer to OracleMetaLink note 386830.1 at:
http://metalink.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=386830.1
The SIDs must be the same for database peers at a primary site and standby site(s) in a Disaster Recovery topology.
Use all uppercase characters for database initialization parameters.
In the following example, the database initialization parameter, service, that is used in the archive log destination parameter is in all uppercase characters (SERVICE):
log_archive_dest_2="SERVICE=SIDM valid_for=(online_logfiles,primary_role) db_unique_name=SIDM"
But in the following example, the database initialization parameter, service, that is used in the archive log destination parameter is in lowercase characters (service):
log_archive_dest_2="service=SIDM valid_for=(online_logfiles,primary_role) db_unique_name="SIDM"
When the database initialization parameter is not in all uppercase characters, error messages similar to the following can occur during an instantiate topology
or sync topology
operation:
stajo05: -->ASG_DUF-4950: An error occurred on host "stajo05" with IP "140.87.25.33" and port "7890" stajo05: -->ASG_SYSTEM-100: String index out of range: -9 stajo05: -->ASG_DUF-3760: Failed to query archive log destination information. stajo05: -->ASG_IAS-15753: Error preparing to instantiate the topology on host "stajo05" stajo05: -->ASG_DUF-3027: Error while executing Instantiating each instance in the topology to standby topology at step - prepare step.
When using the 10.1.2.2 ASG standalone kit to update a 10.1.4.x Oracle home, ASG installed a copy of optic.jar in its component path that is not compatible with the optic.jar of the Oracle home. Although this will not affect the operation of the runtime system, it could potentially cause problems with some future ASG operations in certain configurations.
To prevent these potential problems:
Rename optic.jar in the $ORACLE_HOME/dsa/jlib directory to optic.jar.orig on all nodes of the topology.
Restart the ASG component, if started.
If the above workaround is not performed, you may receive the ASG_DUF-3800 error "Failed trying to connect to the OPMN Manager" error message during a switchover topology
operation if your Disaster Recovery configuration includes Oracle Application Server release 10.1.4.0.1 Identity Management and the standalone ASG kit for release 10.1.2.2.
The error messages received may be similar to:
"4-May 12:51:20 stamx12 152.68.64.214:7890(home/opt/maa/oracle/product/10.1.4/im) 4-May 12:51:20 Running opmnctl reload command: "/opt/maa/oracle/product/10.1.4/im/opmn/bin/opmnctl reload". 4-May 12:51:25 stamx11: -->ASG_DUF-4950: An error occurred on host "stamx11" with IP "140.87.21.201" and port "7890" stamx11: -->ASG_DUF-4950: An error occurred on host "stamx11" with IP "152.68.64.213" and port "7890" stamx11: -->ASG_DUF-3800: Failed trying to connect to the OPMN Manager. stamx11: -->ASG_DUF-3027: Error while executing Starting each instance in the topology at step - starting instance step. stamx11: -->ASG_DUF-3027: Error while executing Switchover each instance in the topology to standby topology at step - starting and resyncing instance step."
Use the same port for ASG on the primary site and standby site(s) to avoid error messages such as the following during a clone instance
operation:
3-May 15:45:43 >>clone instance prodsso1 to stbyinfra1 3-May 15:45:43 stamx11: -->ASG_DUF-4950: An error occurred on host "stamx11" with IP "140.87.21.201" and port "7890" stamx11: -->ASG_DUF-3601: Error connecting to server host 152.68.64.213 on port 7890 stamx11: -->ASG_DUF-3512: Error creating remote worker on node 152.68.64.213:7890.
The dsa.conf file contains ASG configuration information, and it is configured into the Application Server instance's backup/restore IP configuration. The dsa.conf file configuration is handled symmetrically between Application Server instances. Due to this, the dsa.conf file from a production site's instance will be synchronized to the corresponding standby site's instance.
The port numbers between the production and standby instance pairings should match for ASG.
On Windows systems, install version 5.0.2134.1 or higher of sc.exe (the services kit) under the C:\windows\system32 directory before performing cloning operations.
Otherwise, you may see errors similar to these during a cloning operation:
stajz09: -->ASG_DUF-4040: Error executing the external program or script. The error code is "255" stajz09: -->ASG_IAS-15689: Error running the backup script stajz09: -->ASG_IAS-15685: Failed to backup configuration data for instance "IM.asinfra.us.oracle.com" stajz09: -->ASG_DUF-3027: Error while executing Clone Instance at step - backup step. stajz09: -->ASG_DUF-3027: Error while executing Clone Topology at step - clone home step.
As a best practice, use fully qualified path names with the add instance
command.
The ASG clone topology
and clone instance
commands are not supported by DR configurations if there are a different number of Oracle Homes at the primary and standby hosts.
As part of the cloning operation, the Oracle Inventory for each host is cloned. Therefore, the assumption is that the Oracle Home configuration is symmetrical for any host that is being cloned.
For a full description of supported Disaster Recovery asymmetric topologies, refer to Section 5.1.3.2 of the Application Server High Availability Guide for release 10.1.3.2.0.
The asgctl create standby database
command is designed to automate the creation of simple standby databases. It does not support some database options, such as the OMF (Oracle Managed Files) or ASM (Automatic Storage Management) storage options. If you plan to use the create standby database
command to create a database at the standby site, create a new database instance on the primary site using DBCA (Database Configuration Assistant) without specifying the OMF or ASM storage options.
To create a new database instance with the OMF or ASM storage options, follow the instructions in the "Creating a Standby Database the Uses OMF or ASM" section of the Oracle Data Guard Concepts and Administration manual at:
http://www.oracle.com/technology/documentation/index.html
Then, after creating the database with the OMF or ASM storage options, use the asgctl add instance
command to add the instance to the Disaster Recovery topology, so that it can be included in Disaster Recovery operations, such as failover and instantiate topology operations. See Section A.1.23 "OracleAS Guard Add Instance Command Fails When Attempting to Add an Oracle RAC Database to the Topology" of the Oracle Application Server High Availability Guide for release 10.1.3.2 for more information about using the asgctl add instance
command.
The section describes documentation errata. It includes the following topics:
The Oracle Application Server High Availability Guide for 10.1.3.2 includes the "Site Switchover Operations" subsection in Section 5.12.1.1 "Scheduled Outages."
In step 5 in the numbered list in the "Site Switchover Operations" subsection, the following two list elements regarding RAC deployments should be added as list elements c and d:
For Oracle RAC Disaster Recovery deployments, shut down all the RAC instances and start up a single RAC node prior to the switchover.
After setting up the ORACLE_HOME, ORACLE_SID, and PATH variables on the database node, start up the database node:
DBHOME/bin/sqlplus /as sysdba SQL> startup
Section 5.14.2.2 "Configuration Procedure" of the Oracle Application Server High Availability Guide for release 10.1.3.2.0 includes incorrect information in Step 1.
The introductory paragraph before Step 1 says that the standby site uses a non-Real Application Clusters database.
However, Step 1 says to "Stop the Real Application Clusters database on the standby site."
The database on the standby site is a non-Real Applications Cluster database. Step 1 should read "Stop the non-Real Application Cluster database on the standby site."