Oracle® Beehive Administrator's Guide Release 1 (1.5) Part Number E14836-04 |
|
|
View PDF |
Using Oracle Data Guard, you can configure your Oracle Beehive deployment for disaster recovery, in which end-user traffic is diverted to a standby system in the event of a failure in the primary system.
This module contains the following topics:
This module assumes you are familiar with installing, configuring, and maintaining Oracle databases, Oracle Beehive, and Oracle Data Guard.
To configure a disaster recovery system for Oracle Beehive, your primary deployment can either be a single-instance Oracle Beehive server, or a deployment with multiple Oracle Beehive servers. Likewise, you can use either a single instance Oracle Database, or a multi-instance RAC deployment, as your primary database.
These configuration steps show how to take advantage of service relocation for a switchover or failover to a local or remote standby deployment.
Planned and Unplanned Outages
In a planned outage (switchover) scenario, you perform manual steps to reverse database roles between the standby system and the primary system, and then shut down the (former) primary system. This is useful to maintain uptime when performing maintenance on the primary system, such as replacing hardware, that requries the servers to be shut down.
In a unplanned outage (failover) scenario, a failure of the primary system is detected and the standby system is made primary. Depending on your Oracle Data Guard configuration, manual steps may be required, or the failover may be performed automatically.
Database-Only versus Full Deployment
In an Oracle Beehive database-only recovery deployment, you create a standby deployment of the database used by Oracle Beehive. This provides for reduced downtime for planned and unplanned primary database outages. In this scenario, you use your primary deployment's Application tiers with the standby database, so no switchover or failover of the Oracle Beehive Application tier occurs.
In a full deployment scenario, you create a standby deployment of both the database and the Application tier(s). In a switchover or failover, you start the Oracle Beehive Application tiers on the standby system, and then redirect user traffic to the standby Oracle Beehive deployment. This provides for reduced downtime for planned and unplanned outages of the entire primary deployment.
Example Deployments
Figure Figure 17-1 shows a typical full-deployment scenario. In this scenario, a deployment with multiple Application tiers and database instances is duplicated by the standby deployment.
Figure 17-1 Oracle Beehive Full-Deployment Disaster Recovery Scenario
Figure Figure 17-2 shows a typical database-only deployment. In this scenario, there is a standby database that duplicates the primary multi-node database, but there are no standby Application tiers. The primary Application tiers remain active and connect to the standby database. This example provides database redundancy and failover in the event of a failure in the primary database, but there is no provision for Application-tier failover.
Figure 17-2 Oracle Beehive Database-Only Recovery Scenario
To configure Oracle Beehive for use in these scenarios, perform the following procedures:
Having a record of the details of your primary and standby envronments will be useful during setup, configuration, and maintenance of your disaster-recovery configuration.
The following is the sample environment used in examples in this chapter. The example Primary deployment consists of a two-node RAC database and a Beehive deployment using two Application tiers. The Standby deployment duplicates this configuration.
Tip:
Refer back to this section as you read through the rest of this chapter. In all of the examples, you can use the example values to see what your actual values should be.Example Deployment
Bootstrap Service: A common service defined on all nodes, specifically used for service relocation. In this chapter, beehivedg
is the example common service name on all nodes
Primary:
RAC Nodes (and Virtual IP): These are defined in the DNS and/or in /etc/hosts
. In this chapter, pr_db1
(pr_db1_vip
) and pr_db2
(pr_db2_vip
) are the example primary database nodes
ORACLE_SID: In this chapter, pr_orcl1
and pr_orcl2
are the example ORACLE_SIDs
Application tiers: In this chapter, pr_app1
and pr_app2
are the example primary Application tiers
Standby:
See also:
MAA 10g Setup Guide: Creating a RAC Physical Standby Database for a RAC Primary DatabaseRAC Nodes (and Virtual IP): In this chapter, stby_db1
(stby_db1_vip
) and stby_db2
(stby_db2_vip
) are the example standby database nodes
ORACLE_SID: In this chapter, stby_orcl1
and stby_orcl2
are the example standby ORACLE_SIDs
Application tiers: In this chapter, stby_app1
and stby_app2
are the example standby Application tiers
IP Addresses: You may find it useful to make a note of the IP addresses for each database node. IP addresses can be used as a RAC configuration method, or as a part of the DNS and/or /etc/hosts
file.
To configure service relocation and database triggers, perform the steps in the following sections:
Perform the following steps to create managed services:
If you are using a RAC database deployment, add a CRS-managed bootstrap service for the primary and standby databases by running the following commands:
For the primary, run the following command on any primary RAC node:
srvctl add service -s beehivedg -d pr_orcl1 -r pr_orcl1,pr_orcl2
For the standby, run the following command on any standby RAC node:
srvctl add service -s beehivedg -d stby_orcl1 -r stby_orcl1,stby_orcl2
Note:
This service name must be different than the name used in the database parameterSERVICE_NAMES
. In this example, beehivedg
is the service name.If you are using a single-instance database deployment, create the relocation service using the dbms_service
package. In SQL*Plus, connect as SYS or SYSDBA and enter the following:
begin dbms_service.create_service(service_name => 'beehivedg', network_name => 'beehivedg' ); end; /
When a switchover or failover to a standby database occurs, service relocation and affinity services or other CRS-managed services are not automatically started. Perform the following steps to create a trigger to start up database-related services. (In a full deployment scenario, you do not need to perform this procedure.)
If you have a multiple instance deployment, create a script to start CRS services. Example 17-1 shows an example script, which includes using affinity services, for a multiple instance deployment. Example 17-2 shows a similar script which does not include using affinity services:
Note:
Instructions for setting up affinity services are included in the RAC post-install steps in the Oracle Beehive Installation Guide for your platform.Example 17-1 Example CRS-Managed Service Startup Script With Affinity Services StartupService.sh
#!/bin/ksh export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/11.1.0/db export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/ctx/lib export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/opmn/bin:/usr/local/bin:/bin:/usr/bin export TIMESTAMP=`date '+%y%m%d%H%M%s'` export LOGFILE=/tmp/StartupService_${TIMESTAMP}.log echo "Starting $0 at `date`" > $LOGFILE HOSTNAME=`hostname | awk -F. '{print $1}'` case $HOSTNAME in pr_db1) export DB_UNIQUE_NAME=bhha export ORACLE_SID=bhha1 export AFF_SVC=aff1 ;; pr_db2) export DB_UNIQUE_NAME=bhha export ORACLE_SID=bhha2 export AFF_SVC=aff2 ;; stby_db1) export DB_UNIQUE_NAME=bhhb export ORACLE_SID=bhhb1 export AFF_SVC=aff1 ;; stby_db2) export DB_UNIQUE_NAME=bhhb export ORACLE_SID=bhhb2 export AFF_SVC=aff2 ;; *) echo "Node $HOSTNAME not identified or not defined for database bhha " >> $LOGFILE exit ;; esac echo "`date` : Starting CRS managed database service $AFF_SVC" >> $LOGFILE srvctl start service -d $DB_UNIQUE_NAME -s $AFF_SVC -i $ORACLE_SID >> $LOGFILE 2>&1 srvctl start service -d $DB_UNIQUE_NAME -s beehivedg -i $ORACLE_SID >> $LOGFILE 2>&1 echo "`date` : Start of CRS managed database service $AFF_SVC complete" >> $LOGFILE
Example 17-2 Example CRS-Managed Service Startup Script Without Affinity Services StartupService.sh
#!/bin/ksh export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/11.1.0/db export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/ctx/lib export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/opmn/bin:/usr/local/bin:/bin:/usr/bin export TIMESTAMP=`date '+%y%m%d%H%M%s'` export LOGFILE=/tmp/StartupService_${TIMESTAMP}.log echo "Starting $0 at `date`" > $LOGFILE HOSTNAME=`hostname | awk -F. '{print $1}'` echo "`date` : Starting beehivedg database service" >> $LOGFILE srvctl start service -d $DB_UNIQUE_NAME -s beehivedg -i $ORACLE_SID >> $LOGFILE 2>&1 echo "`date` : Start of beehivedg database service complete" >> $LOGFILE
Create a database startup trigger on the primary database (Oracle Data Guard will subsequently replicate your trigger on the standby database). For example, for a multiple instance deployment:
CREATE OR REPLACE TRIGGER manage_services_start after startup on database DECLARE role VARCHAR(30); BEGIN SELECT DATABASE_ROLE INTO role FROM V$DATABASE; IF role = 'PRIMARY' THEN dbms_scheduler.create_job( job_name=>'start_dbsvc', job_type=>'executable', job_action=>'<ORACLE_HOME>/bin/StartupService.sh', enabled=>TRUE ); END IF; END; /
job_action
should specify the location of the script you created in Step 1
Replace <ORACLE_HOME>
with the full path of your Oracle Database Oracle home.
For a single instance deployment (in which you do not need a script):
CREATE OR REPLACE TRIGGER manage_services_start after startup on database DECLARE role VARCHAR(30); BEGIN SELECT DATABASE_ROLE INTO role FROM V$DATABASE; IF role = 'PRIMARY' THEN DBMS_SERVICE.START_SERVICE('beehivedg'); ELSE DBMS_SERVICE.STOP_SERVICE('beehivedg'); END IF; END; /
To test the script, you can restart the database (in a RAC deployment, restart the primary RAC node database). The related services should start automatically:
srvctl stop database -d pr_db1 srvctl start database -d pr_db1 srvctl status service -d pr_db1
You must configure the FAN ONS Publisher and Database Role Change trigger in order to enable database failover operations. (In a "full deployment" scenario, there is no switchover or failover betweeen database deployments, so you can skip this step.)
See Also:
Pages 11-13 of "Client Failover in Data Guard Configurations for Highly Available Oracle Databases: Oracle Database 10g Release 2" Oracle Maximum Availability Architecture White Paper, found at the following link on the Oracle Technology Network website:Configure the FAN ONS Publisher Configuration File in $ORACLE_HOME/dbs/cfo${ORACLE_SID}.ora
(on both the primary and the standby deployments).
For example (including affinity):
bhha peer=stby_orcl bhhb peer=pr_orcl bhha service=beehivedg location=pr_db1,pr_orcl1:pr_db2,pr_orcl2 bhha service=aff1 location=pr_db1,pr_orcl1 bhha service=aff2 location=pr_db2,pr_orcl2 bhhb service=beehivedg location=stby_db1,stby_orcl1:stby_db2,stby_orcl2 bhhb service=aff1 location=stby_db1,stby_orcl1 bhhb service=aff2 location=stby_db2,stby_orcl2
On all prirmary and standby RAC nodes, copy the $ORACLE_HOME/dbs/cfo${ORACLE_SID}.ora
file to each node and rename it with the appropriate ORACLE_SID
Create a database role change trigger:
CREATE OR REPLACE TRIGGER ons_JDBCpublish AFTER DB_ROLE_CHANGE ON DATABASE BEGIN dbms_scheduler.create_job( job_name=>'publish_events', job_type=>'executable', job_action=>'/u01/app/oracle/product/11.1.0/db/bin/cfo.sh', enabled=>TRUE ); END;
In a database-only deployment, configure your Oracle Beehive servers by performing the following configuration steps:
Note:
You only need to perform these steps on one Application tier.Update bootstrap connect_string to use service relocation:
Run beectl stop --all
Run the following command:
beectl modify_bootstrap_configuration --connect_string \ "(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=yes)(FAILOVER=on)\ (ADDRESS=(PROTOCOL=TCP)(HOST=pr_db1_vip.example.com)(PORT=1521))\ (ADDRESS=(PROTOCOL=TCP)(HOST=pr_db2_vip.example.com)(PORT=1521))\ (ADDRESS=(PROTOCOL=TCP)(HOST=stby_db1_vip.example.com)(PORT=1521))\ (ADDRESS=(PROTOCOL=TCP)(HOST=stby_db2_vip.example.com)(PORT=1521)))\ (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=beehivedg.us.oracle.com)))
Update the ONS entries by modifying the ONS setup to include the standby nodes:
Note:
6200 is the current setting forremoteport
in the $CRS_HOME/opmn/conf/ons.config
file and should also show in the $CRS_HOME/bin/onsctl
ping outputbeectl> modify_property --component _current_site:Database \ --name OnsNodeConfiguration \ --value pr_db1.example.com:6200 \ --value pr_db2.example.com:6200 \ --value stby_db1.example.com:6200 \ --value stby_db2.example.com:6200
Activate the Configuration on the first Beehive Application tier by running the following commands:
beectl activate_configuration beectl modify_local_configuration_files beectl start --all
One at a time, run the following commands on each remaining Application tier instance:
beectl stop -all beectl modify_local_configuration_files -restart_needed false beectl start --all
An Oracle Data Guard configuration consists of one database that functions in the primary role and one or more databases that function in the standby role. Typically, the role of each database does not change. However, if Data Guard is used to maintain service in response to a primary database outage, you must initiate a role transition between the current primary database and one standby database in the configuration.
A database operates in one of the following mutually exclusive roles: primary or standby. Data Guard enables you to change these roles dynamically by issuing SQL statements, or by using either of the Data Guard broker's interfaces.
See Also:
Chapter 1, "Oracle Data Guard Broker Concepts," of Oracle Data Guard BrokerOracle Data Guard supports the following role transitions:
Switchover
Allows the primary database to switch roles with one of its standby databases. There is no data loss during a switchover. After a switchover, each database continues to participate in the Data Guard configuration with its new role. Switchover is generally used for planned outages.
Failover
Changes a standby database to the primary role in response to a primary database failure. If the primary database was not operating in either maximum protection mode or maximum availability mode before the failure, some data loss may occur. If Flashback Database is enabled on the primary database, it can be reinstated as a standby for the new primary database once the reason for the failure is corrected. Failover is used when unplanned outages occur.
Performing a Switchover
To perform a database-only switchover operation follow the steps in MetaLink Note 751600.1.
To perform a switchover operation in a full-deployment scenario, perform the following steps:
Stop Oracle Beehive on the primary site application nodes
Follow the steps in MetaLink Note 751600.1
Start the Application tiers in your standby deployment
Once the standby Oracle Beehive system is up and running, you can redirect user traffic to the standby deployment
Performing a Failover
To perform a database-only switchover operation follow the steps in section 8.2.2, "Performing a Failover to a Physical Standby Database," in Chapter 8 of the Oracle Data Guard Concepts and Administration Guide.
To perform a switchover operation in a full-deployment scenario, perform the following steps:
Stop Oracle Beehive on the primary site application nodes
Follow the steps in section 8.2.2, "Performing a Failover to a Physical Standby Database," in Chapter 8 of the Oracle Data Guard Concepts and Administration Guide
Start the Application tiers in your standby deployment
Once the standby Oracle Beehive system is up and running, you can redirect user traffic to the standby deployment
This section includes the following topics
You can modify the outbound connect timeout.
See Also:
Page 5 of "Client Failover in Data Guard Configurations for Highly Available Oracle Databases: Oracle Database 10g Release 2" Oracle Maximum Availability Architecture White Paper, found at the following link on the Oracle Technology Network website:For example, to modify the outbound connect timeout to three seconds:
beectl modify_property --component _current_site:Database:DefaultNonXaPool \ --name ConnectionProperties \ --value oracle.net.ns.SQLnetDef.TCP_CONNTIMEOUT_STR:3000
Listener throttling is a performance adjustment you can make to the database listener that may be necessary for the standby database to operate efficiently during and after a switchover or failover. When the system initially moves from the primary to the standby, there will be a very high number of connections moved all at once to the standby system, which can cause "connection storm." Listener throttling is a method of controlling the number of simultaneous connections made to the database.
The appropriate value to use depends on the size of your deployment (number of concurrent users), and the capability of your database hardware.
See Also:
For complete details of this procedure, see Appendix E, "Listener Connection Rate Throttling," of "Optimizing Availability During Unplanned Outages Using Oracle Clusterware and RAC " Oracle Maximum Availability Architecture White Paper, found at the following link on the Oracle Technology Network website:The CONNECTION_RATE_<listener_name>
parameter indicates the connection rate per second for all addresses with RATE_LIMIT
set to YES
, yes
, or a specific rate.
In this example, 10 connections will be spawned per second only for the first address in the address list. The RATE_LIMIT
parameter can also specify the rate itself instead of only turning throttling on at the default rate. For example, RATE_LIMIT=5
on a particular address indicates a specific setting for just that address. When specifying a RATE_LIMIT
at the address level, you should not specify the CONNECTION_RATE_<listener_name>
property.
Note:
If both the global rate and a specific rate are specified, the global rate is the one enforced.Perform the following procedure to enable listener throttling on each database node:
Edit the listener.ora
file. In Example 17-3, only the first address has throttling enabled. Note that the setting of 10 is an example; the actual setting depends on the details of your environment
Example 17-3 Example listener.ora
File for Listener Throttling
LISTENER_PR_DB1 = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = pr_db1_vip)(PORT = 1521)(RATE_LIMIT=YES)(QUEUESIZE=1024)(IP = FIRST)) ) (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = <X.X.X.X>)(PORT = 1521)(IP = FIRST)) ) (ADDRESS_LIST = (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC)) ) ) ) CONNECTION_RATE_LISTENER_PR_DB1=30
Substitute the correct IP address for <X.X.X.X>
.
Implement the listener.ora
changes by running the following command:
lsnrctl reload