Oracle® Application Server Release Notes 10g Release 3 (10.1.3.1.0) for HP-UX PA-RISC Part Number B32220-06 |
|
|
View PDF |
This chapter describes issues related to highly available topologies using the OracleAS High Availability. This chapter contains the following issues:
This section describes general issues and workarounds. It includes the following topic:
Section 15.1.2, "Compatible ASG Releases in an OracleAS Disaster Recovery Topology"
Section 15.1.5, "HTTP Server Configuration When Using a Server Load Balancer"
Section 15.1.6, "Problem Performing a Clone Instance or Clone Topology Operation"
Section 15.1.7, "OracleAS Guard Release 10.1.2.1.1 Cannot Be Used with Oracle RAC Databases"
Section 15.1.10, "Problem in an Oracle RAC-non Oracle RAC Environment with Naming Conventions"
Section 15.1.12, "GNU Tar is Required for ASG Clone Topology or Clone Instance Operations"
Section 15.1.13, "ASG Operations Fail if Multiple DB ORACLE_HOMEs Exist on the Same System"
Section 15.1.15, "Application Server Components Tested and Certified with OracleAS Guard"
Section 15.1.16, "ASG to Catch Array Overflow Exceptions in Queries to Primary"
Section 15.1.18, "Corrupt Index Blocks in Metadata Repository Databases"
Section 15.1.19, "Database SIDs Must be the Same for Database Peers at Primary and Standby Sites"
Section 15.1.21, "Workaround for ASG_DUF-3800 "Failed trying to connect to the OPMN Manager" Error"
Section 15.1.23, "Use Fully Qualified Path Names with the add instance Command"
Section 15.1.25, "Start ESB Repository before Starting ESB Runtime Instances"
By default, when you install an Oracle Application Server instance using a particular release of Oracle Application Server, a particular release of Application Server Guard (ASG) is installed into the Oracle home for the instance. You also install ASG on standalone hosts on which external resources (such as an Oracle database) are located that you want to include in your OracleAS Disaster Recovery topology.
Multiple releases of ASG are available. It is possible (recommended) in some cases to upgrade the ASG release that was installed in an Oracle Application Server instance home when you installed that instance. To upgrade the ASG release in an Oracle Application Server instance home, download the ASG standalone kit for the recommended ASG release from the Oracle Technology Network (OTN), and then use that ASG standalone kit to install the recommended ASG release into the home. You also use the ASG standalone kit to install ASG on standalone hosts that you want to include in your OracleAS Disaster Recovery topology.
Use Table 15-1 and Table 15-2 to determine whether a particular ASG release is compatible when installed into an Application Server instance home for a particular Oracle Application Server release. The left column of the table shows the different ASG releases for which an ASG standalone installation kit is available. The remaining columns show different Oracle Application Server releases for which an Oracle Application Server instance can be created.
This list describes the meaning of the entries in Table 15-1 and Table 15-2:
N: This ASG release is not compatible with an instance from this Oracle Application Server release.
X: This ASG release cannot be installed into the Oracle home for an instance from this Oracle Application Server release.
Y-NR: This ASG release is compatible with an instance from this Oracle Application Server release, but Oracle recommends that you do not install this ASG release into the instance's Oracle home because another ASG release is recommended.
Y: This ASG release is compatible with an instance from this Oracle Application Server release. Oracle recommends you install this ASG release into the instance's Oracle home.
Table 15-1 shows the compatible ASG releases for Oracle Application Server instances from Oracle Application Server 10.1.2.0.2 through 10.1.3.3.
Table 15-1 Compatible ASG Releases for OracleAS Instances from Releases 10.1.2.0.2 Through 10.1.3.3
ASG Release | 10.1.2.0.2 OracleAS Instance | 10.1.2.1 OracleAS Instance | 10.1.2.2 OracleAS Instance | 10.1.3.0 OracleAS Instance | 10.1.3.1 OracleAS Instance | 10.1.3.2 OracleAS Instance | 10.1.3.3 OracleAS Instance |
---|---|---|---|---|---|---|---|
10.1.2.0.2 |
Y-NR |
X |
X |
N |
N |
N |
N |
10.1.2.2 |
Y |
Y |
Y |
N |
N |
N |
N |
10.1.2.2.1 (ASG-only release)Foot 1 |
Y |
Y |
Y |
N |
N |
N |
N |
10.1.3.0 |
N |
N |
N |
Y-NR |
X |
N |
X |
10.1.3.1 |
N |
N |
N |
Y-NR |
Y-NR |
Y-NR |
X |
10.1.3.3 |
N |
N |
N |
Y |
Y |
Y |
Y |
Footnote 1 This is the ASG release that was provided (installed by default) with the OracleAS 10.1.4.2 release. It is compatible with the OracleAS 10.1.2.x releases. There is no OracleAS 10.1.2.2.1 release.
For example, if you have an Oracle Application Server 10.1.3.1 instance and you want to know which ASG release to install in the instance home, you can use Table 15-1 to determine the following:
No ASG 10.1.2.x release is compatible with an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.0 release cannot be installed into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.1 release is compatible with an Oracle Application Server 10.1.3.1 instance, but Oracle recommends that you do not install the ASG 10.1.3.1 release into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
The ASG 10.1.3.3 release is compatible with an Oracle Application Server 10.1.3.1 instance and Oracle recommends that you install the ASG 10.1.3.3 release into the Oracle home for an Oracle Application Server 10.1.3.1 instance.
Table 15-2 shows the compatible ASG releases for Oracle Application Server instances from Oracle Application Server 10.1.4.0 through 10.1.4.2.
Table 15-2 Compatible ASG Releases for OracleAS Instances from Releases 10.1.4.0 Through 10.1.4.2
ASG Release | 10.1.4.0 OracleAS InstanceFoot 1 | 10.1.4.1 OracleAS InstanceFoot 2 | 10.1.4.2 OracleAS InstanceFoot 3 |
---|---|---|---|
10.1.2.0.2 |
Y-NR |
Y-NR |
X |
10.1.2.2 |
Y-NR |
Y-NR |
Y-NR |
10.1.2.2.1 (ASG-only release)Foot 4 |
Y |
Y |
Y |
10.1.3.0 |
N |
N |
N |
10.1.3.1 |
N |
N |
N |
10.1.3.3 |
N |
N |
N |
Footnote 1 ASG 10.1.2.0.2 is installed by default.
Footnote 2 ASG 10.1.2.0.2 is installed by default.
Footnote 3 ASG 10.1.2.2.1 is installed by default.
Footnote 4 This is the ASG release that was provided (installed by default) with the OracleAS 10.1.4.2 release. It is compatible with the OracleAS 10.1.2.x releases. There is no OracleAS 10.1.2.2.1 release.
This chart shows which ASG release combinations are compatible in in an OracleAS Disaster Recovery topology. A topology is a collection of Oracle Application Server instance homes and standalone host homes that combine to comprise the OracleAS Disaster Recovery production site and standby site. Each Oracle Application Server home has a specific ASG release installed, either by default or by an ASG standalone installation. Since OracleAS Disaster Recovery operations are distributed, there is a collection of recommended ASG releases that should be installed across the topology. Example for the EDG deployment, the collection of Oracle homes should be upgraded to ASG version 10.1.3.3 or 10.1.2.2.x.
Use Table 15-3 to determine whether two ASG releases are compatible in an OracleAS Disaster Recovery topology. Find the first ASG release in the left column of the table and then find the second ASG release in one of the other columns of the table.
This list describes the meaning of the entries in Table 15-3:
Y-NR: The first ASG release is compatible with the second ASG release, but Oracle recommends that you do not use this ASG release combination in your topology.
Y: The first ASG release is compatible with the second ASG release. Oracle recommends that you use this ASG release combination in your topology.
Table 15-3 shows which ASG releases are compatible with other ASG releases.
Table 15-3 Compatible ASG Releases in a Topology
ASG Release | 10.1.2.0.2 | 10.1.2.2 | 10.1.2.2.1 | 10.1.3.0 | 10.1.3.1 | 10.1.3.3 |
---|---|---|---|---|---|---|
10.1.2.0.2 |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
10.1.2.2 |
Y-NR |
Y |
Y |
Y-NR |
Y-NR |
Y |
10.1.2.2.1 |
Y-NR |
Y |
Y |
Y-NR |
Y-NR |
Y |
10.1.3.0 |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
10.1.3.1 |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
Y-NR |
10.1.3.3 |
Y-NR |
Y |
Y |
Y-NR |
Y-NR |
Y |
When using the asgctl add instance
command, the OracleAS Guard client must be run from a system that is already included in the topology.
For example, when an OracleAS Guard client is connected to the OracleAS Guard server that is to be added to an existing topology, the following error is returned:
ASG_IAS-15785: ERROR: The topology is missing the instance that exists in the home where the ASG server is running. You must first discover or add the instance in home
The workaround to this problem is to use an OracleAS Guard client from a system that is already included in the topology to perform the asgctl add instance
command to add an instance to the topology.
Prior to performing an asgctl switchover operation in an asymmetric topology for instances that do not have a standby peer, you must perform an opmnctl stopall command to shutdown all components on each of these ignored instances on the primary site.
When an XML policy file is in use for an asymmetric topology and has the <instanceList successRequirement ="Ignore"
set for an instance, for example, as shown in the following example, then in a switchover operation OracleAS Guard ignores that instance:
. . . <instanceList successRequirement = "Ignore"> <instance>instance B</instance> </instanceList> . . .
OracleAS Guard, on a switchover operation, shuts down all components on the old primary site except for OracleAS Guard and OPMN and ignores instance B because the policy file specifies to do so. The switchover operation fails because all components are not shut down on the primary site, in this case instance B because the policy file specifies to ignore instance B on the primary site, which has no standby peer.
To workaround this problem, the OracleAS Disaster Recovery Administrator must perform an opmnctl stopall operation for all components on instance B prior to the switchover operation in order for the switchover operation to succeed in this asymmetric topology.
If you are using a Server Load Balancer to direct HTTP requests to multiple Oracle HTTP Server instances, Web access to some applications (such as the Application Server Control console and Oracle Web Services Manager) may be redirected to the physical HTTP Server hosts.
To ensure that redirected requests are always sent to the load balancer, configure an Oracle HTTP Server virtual host for the load balancer.
For example, if Oracle HTTP Server is listening on port 7777 and a load balancer called bigip.acme.com
is listening on port 80, then consider the following entry in the httpd.conf
file:
NameVirtualHost *:7777 <VirtualHost *:7777> ServerName bigip.us.oracle.com Port 80 ServerAdmin youyour.address RewriteEngine On RewriteOptions inherit </VirtualHost>
At the current time, the semantics of an asgctl clone topology operation will not clone databases that are outside of the Oracle Application Server home, thus only the default database installed into the Oracle Application Server home by some infrastructure installation types will be cloned. The asgctl create standby database command should be used by users not familiar with Oracle Data Guard.
OracleAS Guard version shipped with this release is 10.1.2.1.1. This version of OracleAS Guard cannot be used with Oracle RAC Databases. For all other purposes, this OracleAS Guard version is completely supported by Oracle.
To use OracleAS Guard with an Oracle RAC database, it is recommended to use Release 10.1.2.2 stand alone version of OracleAS Guard with this release. OracleAS Guard 10.1.2.2 version (with instructions) is available for download from Oracle OTN as an OracleAS Guard stand alone install, or please contact Oracle Support for further instructions.
When adding an Oracle RAC instance to the topology using the OracleAS Guard add instance command and OracleAS Guard could not find the user specified identifier, an inappropriate error message was returned. If the user had entered the database name rather that the Oracle instance SID, there was no indication that this was the problem.
Now if OracleAS Guard is unable to locate the oratab entry (on UNIX) or the system registry service (on Windows) for the user specified database identifier, the following ASG_SYSTEM-100 message now precedes the existing ASG_DUF-3554 message and both messages will be displayed to the console:
ASG_SYSTEM-100: An Oracle database is identified by its database unique name (db_name) ASG_DUF-3554: The Oracle home that contains SID <user specified identifier> cannot be found
Oracle recommends that you shut down the database on the standby site if it is up and running before issuing the asgctl create standby database command; otherwise, the following error is returned:
ASG_DGA-12500: Standby database instance "<instance_name>" already exists on host "<hostname>"
There is a problem with the naming conventions used in the Oracle RAC/non Oracle RAC environment. The asgctl set primary database command must be issued for both the primary and standby site within asgctl to define the service name mapping within OracleAS Guard before attempting an asgctl create standby database command; otherwise, the following error message is returned.
ASG_DUF-4902: Object not found in clipboard for key "orcl1keySourceDb".
An error ora-01671 will occur, when attempting to perform an asgctl create standby database operation from a database that is already in 'physical standby' state. An appropriate error message should be echoed indicating that a standby database is already running, rather than returning this error. This is a known issue.
When using the ASG clone topology or clone instance operations, the tar utility is utilized. The target system(s) of these operations must have a version of GNU tar in the default PATH of the system user account in which the standalone ASG install runs.
GNU tar can be obtained at the following location:
If you have multiple ORACLE_HOME directories on the same system in a disaster recovery setup, the set primary database
command fails with the following error:
prodnode1: -->ASG_DUF-4950: An error occurred on host "stama03v1" with IP "XXX.XX.XX.XXX" and port "7890" prodnode1: -->ASG_ORACLE-300: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor prodnode1: -->ASG_DUF-3700: Failed in SQL*Plus executing SQL statement: connect sys/******@rac as sysdba;. prodnode1: -->ASG_DUF-3502: Failed to connect to database orcl. prodnode1: -->ASG_DUF-3027: Error while executing at step - Default Step. . The database credentials have been set successfully, but they have not been validated
To work around this issue, make sure you have only one database on the system sharing the same inventory for a disaster recovery setup.
If you are running Oracle BPEL Process Manager in an OracleAS Disaster Recovery topology, you want to ensure that:
the Oracle BPEL Process Manager dehydration data stored in databases at the primary and standby sites are continuously synchronized
when a switchover operation occurs, Oracle BPEL Process Manager uses the database at the standby site.
To achieve this:
Store the dehydration data in a database.
Set up log archiving and set Oracle Data Guard protection of the standby database at the maximum availability mode (instead of at the maximum protection mode). This allow logs to be applied continuously at the standby site without shutting down the primary database if the standby database is taken offline.
Note:
You should not configure Oracle Data Guard to maximum protection mode because, by definition, this mode takes the primary database offline if the standby database is taken offline.By default, the ASG "create standby database" command configures the standby database in maximum performance mode. You need to change the mode to maximum availability, as shown in step 2 below.
Include the database in the OracleAS Disaster Recovery topology. You want this database to be monitored by OracleAS Guard, even if Oracle Application Server was not utilized to set up Oracle Data Guard. This ensures that the database and related services are failed over in coordination with the Oracle Application Server services.
To create a standby database and put it in maximum availability mode using the ASG "create standby database" command, perform these steps:
Create a database on the standby site and include it in the OracleAS Disaster Recovery topology. Here are two ways in which you can achieve this:
Install a database manually on the standby site and then run the ASG "instantiate topology" command on the primary site.
ASGCTL> instantiate topology to standbynode1
standbynode1
refers to the physical name of the standby node. The standby node has a corresponding node on the primary site.
For details on ASG commands and OracleAS Disaster Recovery, see the Oracle Application Server High Availability Guide.
OR
Run the ASG "create standby database" command on the primary node to create a standby database on the standby node. The command configures the primary database for maximum performance and defines log_archive_dest_
n
with the "LGRW SYNC AFFIRM" archive attributes for the standby database.
ASGCTL> create standby database orcl1 on standbynode1
orcl1
refers to the database name, and standbynode1
refers to the physical name of the standby node.
Upgrade the configuration to maximum availability mode.
On the primary database, run the following SQL command:
SQL> alter database set standby database to maximize availability;
On the standby database, run the following SQL command to place the standby database in managed recovery mode. (This places the standby database in a constant state of media recovery.)
Note:
Configuring the standby database for managed recovery is not a requirement of maximum availability, but it provides for shorter failover times.SQL> alter database recovery managed standby database [disconnect from session];
Add the optional "disconnect from session" part if you want to end the session after the command.
Notes:
In the steps above, the Oracle Data Guard protection mode of the primary database changes from maximum performance to maximum availability.
For details on the different Oracle Data Guard protection modes (maximum protection, maximum availability, and maximum performance), see the Oracle Data Guard Concepts and Administration guide.
Running the primary database in maximum availability mode may cause a "hang" waiting for an available online log file. A maximum availability primary database will not reuse an online log file until it has been archived to the standby database. This could happen if the standby database is taken offline for a long time.
Only data with same synchronization requirements should be stored in the same database.
For example, the Oracle BPEL Process Manager dehydration store and the OracleAS Portal data should be stored in separate databases because the synchronization objectives of Oracle BPEL Process Manager and OracleAS Portal are different. The synchronization objective of Oracle BPEL Process Manager dehydration store is to maintain consistency between the dehydration store and the BPEL process, while the synchronization objective of OracleAS Portal is to ensure that data and configuration maintained within the middle tier and database do not diverge.
Actions Performed by the "sync topology" Command
When you run the ASG "sync topology" command in this configuration, it performs the following:
Performs a log switch at the primary and ensures that the log is shipped and archived
Performs process management at the primary and standby sites
Encapsulates the incremental changes for all the data in the Oracle homes
Restores the standby peers to the configuration level of the primary
Propagates the changes to all standby instances
For standby databases:
With managed recovery running, the "sync topology" command simply reports the sync-scn and the current database-scn of the standby database. For this configuration, the standby database-scn is guaranteed to be beyond the sync-scn. ASG logs the sync scn level as it corresponds to the current scn level of the standby database.
Without managed recovery running, the "sync topology" command recovers the standby database to the sync-scn. It is equivalent to running the following command:
alter database recover managed standby database until change <sync-scn>
OracleAS Guard has been tested and certified with most of the components in the Oracle Application Server 10g SOA Release (the only components not verified with ASG are BAM and Registry.
For some specific recommendations for BPEL PM and ESB configuration with OracleAS Guard see the OracleAS Guard Release Notes.
The asgctl create standby database
command may fail with an ASG_DUF-4950 error and an error message stack on the console similar to the following:
ASG_DUF-4950: An error occurred on host "myhost" with IP "1.1.1.1" and port "7890" ASG_SYSTEM-100: 10 ASG_DUF-4900: An exception occurred on the server. ASG_DGA-13009: Error during Create Physical Standby
In the duf_client.log file on the primary site DB node, the following messages would be found:
java.lang.ArrayIndexOutOfBoundsException: 10 at oracle.duf.DufJdbc.queryRedoLogInfo(DufJdbc.java:535) at oracle.duf.DufDb$jdbc.queryRedoLogInfo(DufDb.java:4966) at oracle.duf.DufDb$jdbc.access$2000(DufDb.java:4884) at oracle.duf.DufDb.queryRedoLogInfo(DufDb.java:3439)
This problem occurs because ASG does not properly handle a database configured with 10 or more redo log files.
To avoid this problem, reduce the number of database redo log files to fewer than 10.
This problem is fixed in release 10.1.3.3.
The asgctl create standby database
command fails if the redo log files directories on the production site do not exist at the standby.ASG expects the target redo log directory structure to be symmetrical with the redo log directory structure on the production site. If they do not exist in the same directories, a failure will result with the following output:
ASG_ORACLE-300: ORA-00301: error in adding log file '/PATH/redo010.log' - file cannot be created ASG_ORACLE-300: ORA-27040: file create error, unable to create file ASG_DUF-3700: Failed in SQL*Plus executing SQL statement: ALTER DATABASE ADD STANDBY LOGFILE GROUP 10 '/PATH/redo010.log' SIZE 52428800 /* ASG_DGA */;. ASG_DUF-3535: Failed to create standby redo log. ASG_DUF-3535: Failed to create standby redo log. ASG_DGA-13011: Error during Create Physical Standby: Finish-configure standby. ASG_DUF-3027: Error while executing Creating physical standby database - finish phase at step - finish step.
To avoid this problem, create the target redo log file directories on the standby site in the same directories as the production site.
After an ASG switchover or failover operation, metadata index block corruption may occur in a Disaster Recovery metadata repository database.
For a full description of the problem and an explanation of how to deal with it, refer to OracleMetLink note 386830.1 at:
https://metalink.oracle.com/metalink/plsql/ml2_documents.showDocument?p_database_id=NOT&p_id=386830
The SIDs must be the same for database peers at a primary site and standby site(s) in a Disaster Recovery topology.
Use all uppercase characters for database initialization parameters.
In the following example, the database initialization parameter, service, that is used in the archive log destination parameter is in all uppercase characters (SERVICE):
log_archive_dest_2="SERVICE=SIDM valid_for=(online_logfiles,primary_role) db_unique_name=SIDM"
But in the following example, the database initialization parameter, service, that is used in the archive log destination parameter is in lowercase characters (service):
log_archive_dest_2="service=SIDM valid_for=(online_logfiles,primary_role) db_unique_name="SIDM"
When the database initialization parameter is not in all uppercase characters, error messages similar to the following can occur during an instantiate topology
or sync topology
operation:
stajo05: -->ASG_DUF-4950: An error occurred on host "stajo05" with IP "140.87.25.33" and port "7890" stajo05: -->ASG_SYSTEM-100: String index out of range: -9 stajo05: -->ASG_DUF-3760: Failed to query archive log destination information. stajo05: -->ASG_IAS-15753: Error preparing to instantiate the topology on host "stajo05" stajo05: -->ASG_DUF-3027: Error while executing Instantiating each instance in the topology to standby topology at step - prepare step.
When using the 10.1.2.2 ASG standalone kit to update a 10.1.4.x Oracle home, ASG installed a copy of optic.jar in its component path that is not compatible with the optic.jar of the Oracle home. Although this will not affect the operation of the runtime system, it could potentially cause problems with some future ASG operations in certain configurations.
To prevent these potential problems:
Rename optic.jar in the $ORACLE_HOME/dsa/jlib directory to optic.jar.orig on all nodes of the topology.
Restart the ASG component, if started.
If the above workaround is not performed, you may receive the ASG_DUF-3800 error "Failed trying to connect to the OPMN Manager" error message during a switchover topology
operation if your Disaster Recovery configuration includes Oracle Application Server release 10.1.4.0.1 Identity Management and the standalone ASG kit for release 10.1.2.2.
The error messages received may be similar to:
"4-May 12:51:20 stamx12 152.68.64.214:7890(home/opt/maa/oracle/product/10.1.4/im) 4-May 12:51:20 Running opmnctl reload command: "/opt/maa/oracle/product/10.1.4/im/opmn/bin/opmnctl reload". 4-May 12:51:25 stamx11: -->ASG_DUF-4950: An error occurred on host "stamx11" with IP "140.87.21.201" and port "7890" stamx11: -->ASG_DUF-4950: An error occurred on host "stamx11" with IP "152.68.64.213" and port "7890" stamx11: -->ASG_DUF-3800: Failed trying to connect to the OPMN Manager. stamx11: -->ASG_DUF-3027: Error while executing Starting each instance in the topology at step - starting instance step. stamx11: -->ASG_DUF-3027: Error while executing Switchover each instance in the topology to standby topology at step - starting and resyncing instance step."
Use the same port for ASG on the primary site and standby site(s) to avoid error messages such as the following during a clone instance
operation:
3-May 15:45:43 >>clone instance prodsso1 to stbyinfra1 3-May 15:45:43 stamx11: -->ASG_DUF-4950: An error occurred on host "stamx11" with IP "140.87.21.201" and port "7890" stamx11: -->ASG_DUF-3601: Error connecting to server host 152.68.64.213 on port 7890 stamx11: -->ASG_DUF-3512: Error creating remote worker on node 152.68.64.213:7890.
The dsa.conf file contains ASG configuration information, and it is configured into the Application Server instance's backup/restore IP configuration. The dsa.conf file configuration is handled symmetrically between Application Server instances. Due to this, the dsa.conf file from a production site's instance will be syncronized to the corresponding standby site's instance.
The port numbers between the production and standby instance pairings should match for ASG.
As a best practice, use fully qualified path names with the add instance
command.
The ASG clone topology
and clone instance
commands are not supported by DR configurations if there are a different number of Oracle Homes at the primary and standby hosts.
As part of the cloning operation, the Oracle Inventory for each host is cloned. Therefore, the assumption is that the Oracle Home configuration is symmetrical for any host that is being cloned.
For a full description of supported Disaster Recovery asymmetric topologies, refer to Section 5.1.3.2 of the Application Server High Availability Guide for release 10.1.3.2.0.
In a distributed ESB environment where ESB Repository Server and ESB Runtime Server are in different Oracle homes, you need to start the ESB Repository before starting the ESB runtime instances.
The URL of an Oracle Enterprise Service Bus (ESB) service endpoint looks similar to the following:
http://esb-runtime-server:7777/event/ESBSystemName/ESB_ServiceName
The /event
is the context root ESB runtime engine server, which is a J2EE application. Behind the ESB server, there can be one or more ESB systems, and each system can contain one or more ESB services. Furthermore, you can configure one ESB Server in one OC4J container to house ESBSystem1 and ESBSystem2, and another ESB Server in another OC4J container to house ESBSystem3 and ESBSystem4, and so on.
A problem arises in that these ESB servers are on the same OPMN cluster and behind the same HTTP Servers. A call to http://virtualurl:80/event/ESBSystem1/ESBServer1
may be routed by Oracle HTTP Server to an ESB Server that is not associated with the specified ESB system and an error may occur.
Workaround
You can use static routing to work around this problem. Edit the Oracle HTTP Server mod_oc4j.conf
file and add static mount points pointing to each specific OC4J that provides a service. For example, if OC4J instances 1 and 3 provide service 1, and OC4J instance 2 provides service 2, you would have mount directives like these:
Oc4jMount /event/ESBSystemName/ESB_Service1/* instance://ias_instance_1:home Oc4jMount /event/ESBSystemName/ESB_Service1/* instance://ias_instance_3:home Oc4jMount /event/ESBSystemName/ESB_Service2/* instance://ias_instance_2:home
Note:
For more information on mount point configurations, refer to the section on theOc4jMount
directive in Oracle HTTP Server Administrator's Guide 10g (10.1.3.1.0), Chapter 7, "Understanding Modules".This section describes configuration issues and their workarounds. It includes the following topics:
Section 15.2.4, "Connecting to an OracleAS Guard Server May Return an Authentication Error"
Section 15.2.5, "All emagents Must Be Shut Down Before Performing OracleAS Guard Operations"
Section 15.2.6, "Procedure to Patch a 10.1.2.0.0 Disaster Recovery Setup with a 10.1.2.1.0 Patchset"
Section 15.2.9, "Add Instance Adds an Instance to Topology with Empty Instancename"
Section 15.2.10, "Create Standby Fails with if Initiated on a Different ASGCTL Shell"
Section 15.2.11, "Heartbeat Failure After Failover in Alert Logs"
Section 15.2.12, "Create Standby Database Fails If Database Uses OMF Storage or ASM storage"
Section 15.2.13, "Database Already Exists Errors During Create Standby"
Section 15.2.14, "Steps to Add a Database with OMF or ASM to ASG Topology"
The asgctl shutdown topology command only handles non-database instances. Thus, in a repCA environment when OracleAS Guard detects an instance and determines it to be a repCa type database, its instance is ignored in a shutdown topology operation. Any repCA type database is considered to be managed outside of OracleAS Guard.
Therefore, within an environment where an MRCA database has been added to the topology, the database will not be handled by the asgctl shutdown topology command.
In a Disaster Recovery environment that involves Oracle RAC databases, after a switchback operation(switchover topology to <primary site>
), the database will be started up on only one of the Oracle RAC nodes by OracleAS Guard; however, the remaining Oracle RAC instances on the primary site must be started up manually.
After performing an asgctl add instance operation from a remote client, it was noted that the instance was being added to the local system rather than to the intended remote system.
As a workaround to this problem, the Disaster Recovery Administrator must first perform an asgctl discover topology operation on the local client system before attempting to perform an asgctl add instance operation to add an instance to the remote system.
When a user connects to an OracleAS Guard server and gets an authentication error even though the correct user name and password were entered, the user should try to put the following flag in the dsa.conf
file in the <ORACLE_HOME>
/dsa
directory and try the operation again: dsa_realm_override=1
.
Note that this DSA configuration file parameter is not documented in the "OracleAS Guard Configuration File Parameters" section of the OracleAS Guard Release Information readme.txt
file.
Before performing any OracleAS Guard operations, you must shut down the emagents. This operation is required for OracleAS Guard commands that recycle OracleAS services. You can issue the asgctl run
command in a script to perform this operation from within OracleAS Guard. See the OracleAS Disaster Recovery chapters in the Oracle Application Server High Availability Guide for more information.
Otherwise, for example you may get an ORA-01093: ALTER DATABASE CLOSE only permitted with no sessions connected
error message.
Shutting down emagents is only described for performing a switchover operation. However, it applies to all OracleAS Guard operations. The documentation will be updated in a future release.
Assuming you already have an existing Disaster Recovery Setup for a 10.1.2.0.0 production database, follow these conceptual steps to apply a 10.1.2.1.0 Disaster Recovery Patchset:
Break the Disaster Recovery setup. Perform an asgctl failover command.
Apply the patch 10.1.2.1.0.
Recreate the Disaster Recovery setup. Perform an asgctl create standby database command followed by an asgctl instantiate topology command. Alternatively, see the Oracle Data Guard documentation for more information about how to reestablish the standby database.
If you attempt to perform an asgctl instantiate topology operation immediately following an asgctl failover operation, an ORA-01665: control file is not a standby control file
error message is returned.
To work around this problem, you must first perform an asgctl create standby database command to create the standby database on the remote host. See Section 15.3.1, "Availability of a Previously Undocumented asgctl Command: create standby database" for more information about this previously undocumented asgctl command. Also see Section 15.2.6, "Procedure to Patch a 10.1.2.0.0 Disaster Recovery Setup with a 10.1.2.1.0 Patchset" for more information.
When you are running OracleAS Guard in an Oracle RAC environment, you should have only one Oracle RAC instance running while performing OracleAS Guard operations. Otherwise, an error will occur where the primary database will complain that it is mounted by more than one instance, which will prevent a shutdown.
For example, when performing an OracleAS Guard create standby database operation in an Oracle RAC environment with more than one Oracle RAC instance running, the following error will be seen:
ASGCTL> create standby database orcl1 on stanb06v3 . . . This operation requires the database to be shutdown. Do you want to continue? Yes or No y Database must be mounted exclusive stanb06v1: -->ASG_DUF-4950: An error occurred on host "stanb06v1" with IP "141.86.22.32" and port "7890" stanb06v1: -->ASG_DUF-3514: Failed to stop database orcl1.us.oracle.com. stanb06v1: -->ASG_DGA-13002: Error during Create Physical Standby: Prepare-primary processing. stanb06v1: -->ASG_DUF-3027: Error while executing Creating physical standby database - prepare phase at step - primary processing step.
When you create a new database instance using the Database Configuration Assistant (DBCA), the SID defaults to the database name. If you enter a name in the SID field other than the database name, and then later add this database to the Disaster Recovery topology, the instancename in the topology added is empty.
To avoid this problem, make sure the DBName (without domain) and DBSID are the same when creating the database for a Disaster Recovery setup on primary or standby sites.
The create standby
database command fails if initiated by ASG clients from any node other than the source primary node where the database resides. To avoid the problem, run the create standby
command from the same primary (source) node, where the database for the primary site resides.
For example, if you ran the create standby
command from the production database to the standby database where prodnode1 is the primary site database nodename and standbynode1 is its standby database nodename. The ASGCTL shell
should always be invoked and connected to prodnode1. If you try to run ASGCTL shell
from standbynode1 and connect to prodnode1, the create standby
command fails.
The following warning appears in the alert logs of the database after a failover scenario, where the new primary database fails to tnsping its remote database instance.
Errors in file c:\oracle\product\10.2.0\admin\orcl\udump\orcl1_rfs_1816.trc: ORA-16009: remote archive log destination must be a STANDBY database . Fri Sep 08 09:11:13 2006 Errors in file c:\oracle\product\10.2.0\admin\orcl\bdump\orcl1_arc1_496.trc: ORA-16009: remote archive log destination must be a STANDBY database . Fri Sep 08 09:11:13 2006 PING[ARC1]: Heartbeat failed to connect to standby 'orcl1_remote1'. Error is 16009. Fri Sep 08 09:11:50 2006 Redo Shipping Client Connected as PUBLIC -- Connected User is Valid RFS[67]: Assigned to RFS process 628 RFS[67]: Database mount ID mismatch [0x4342404d:0x4341ffb0] Fri Sep 08 09:11:50 2006 Errors in file c:\oracle\product\10.2.0\admin\orcl\udump\orcl1_rfs_628.trc: ORA-16009: remote archive log destination must be a STANDBY database . Redo Shipping Client Connected as PUBLIC -- Connected User is Valid RFS[68]: Assigned to RFS process 2488 RFS[68]: Database mount ID mismatch [0x4342404d:0x4341ffb0] Fri Sep 08 09:12:05 2006 Errors in file c:\oracle\product\10.2.0\admin\orcl\udump\orcl1_rfs_2488.trc: ORA-16009: remote archive log destination must be a STANDBY database . Fri Sep 08 09:12:14 2006 Errors in file c:\oracle\product\10.2.0\admin\orcl\bdump\orcl1_arc1_496.trc: ORA-16009: remote archive log destination must be a STANDBY database
To avoid these error messages in the alert logs, null the log_archive_dest_2 parameter using the following commands:
alter system set log_archive_dest_2='SERVICE=null LGWR ASYNC REOPEN=60'; alter system set log_archive_dest_state_2='defer';
The create standby database
command fails with ASG_ORACLE-300: ORA-01276 errors if the database storage option uses OMF (Oracle Managed Files) or ASM (Automatic Storage Management).
To work around the problem, create a new database instance using DBCA on the primary site with alternate storage options before running thecreate standby database
command.
If you run a create standby
command to overwrite an existing database, you get the following error messages:
Checking whether standby instance already exists proddnode1: -->ASG_DUF-4950: An error occurred on host "proddnode1" with IP "a.b.c.d" and port "7891" standbynode1: -->ASG_DUF-4950: An error occurred on host "standbynode1" with IP "e.f.g.h" and port "7891" standbynode1: -->ASG_DGA-12500: Standby database instance "db102" already exists on host "standbynode1". standbynode1: -->ASG_DGA-13001: Error during Create Physical Standby: Prepare-check standby. standbynode1: -->ASG_DUF-3027: Error while executing Creating physical standby database - prepare phase at step - check standby step.
Use the oradim -delete -sid <DBSID>
command on Windows, or remove the database entry from oratab on UNIX platforms to make sure entries in the standby site for the database do not exist. Now rerun the create standby database to overwrite any existing database successfully.
The asgctl create standby database
command is designed to automate the creation of simple standby databases. It does not support some database options, such as the OMF (Oracle Managed Files) or ASM (Automatic Storage Management) storage options. If you plan to use the create standby database
command to create a database at the standby site, create a new database instance on the primary site using DBCA (Database Configuration Assistant) without specifying the OMF or ASM storage options.
To create a new database instance with the OMF or ASM storage options, follow the instructions in the "Creating a Standby Database the Uses OMF or ASM" section of the Oracle Data Guard Concepts and Administration manual at:
http://www.oracle.com/technology/documentation/index.html
Then, after creating the database with the OMF or ASM storage options, use the asgctl add instance
command to add the instance to the Disaster Recovery topology, so that it can be included in Disaster Recovery operations, such as failover and instantiate topology operations. See Section A.1.23 "OracleAS Guard Add Instance Command Fails When Attempting to Add an Oracle RAC Database to the Topology" of the Oracle Application Server High Availability Guide for release 10.1.3.2 for more information about using the asgctl add instance
command.
This section describes documentation errata and omissions. It includes the following topics:
Section 15.3.1, "Availability of a Previously Undocumented asgctl Command: create standby database"
Section 15.3.3, "Additional Site Switchover Information for RAC Deployments"
Section 15.3.4, "Configuring Jgroups and OC4J Groups in BPEL Active-Active Topology"
The asgctl create standby database command is not documented. The following information describes this command in more detail.
The syntax for the asgctl create standby database command is as follows:
create standby database <database_name> on <remote_host>
<database_name>
is the primary database unique name used to create the standby database on the remote host system.
<remote_host>
is the name of the host system on which the standby database is to be created.
Oracle software and OracleAS Guard software are required to be installed on the node designated as <remote_host>
.The init.ora
parameter file generated for the standby database is configured assuming a non Oracle RAC enabled standby database. If the standby database is to be Oracle RAC enabled, the following initialization parameters must be defined appropriately:
cluster_database
cluster_database_instances
remote_listener
Users should use this command sparingly and only as needed.
In the Oracle Application Server High Availability Guide, section 5.2.2, "Oracle BPEL Process Manager in an Active-Active Topology", contains a reference to the "Oracle BPEL Process Manager Clustering" chapter in the Oracle BPEL Process Manager Administrator's Guide. This chapter is actually located in the Oracle BPEL Process Manager Installation Guide.
The Oracle Application Server High Availability Guide for 10.1.3.2 includes the "Site Switchover Operations" subsection in Section 5.12.1.1 "Scheduled Outages."
In step 5 in the numbered list in the "Site Switchover Operations" subsection, the following two list elements regarding RAC deployments should be added as list elements c and d:
For Oracle RAC Disaster Recovery deployments, shut down all the RAC instances and start up a single RAC node prior to the switchover.
After setting up the ORACLE_HOME, ORACLE_SID, and PATH variables on the database node, start up the database node:
DBHOME/bin/sqlplus /as sysdba SQL> startup
Section 5.2.2, "Oracle BPEL Process Manager in an Active-Active Topology," in Oracle Application Server High Availability Guide 10g Release 3 (10.1.3.1.0) does not include the required Jgroups and OC4J groups configuration recommendations. Refer to Oracle Application Server Enterprise Deployment Guide 10g Release 3 (10.1.3.3.0) for complete information on configuring Jgroups and OC4J groups.