Setting Up a Tuxedo Application
This topic includes the following sections:
The Oracle Real Application Clusters (RAC) feature supports clustering of machines that utilize replicated Oracle database services accessing the same Oracle database. Oracle RAC provides the ability to concurrently access the same Oracle database from instances physically located on multiple Oracle server machines, and offers the ability to failover unsuccessful database instances to alternate locations.
However, specific support for Oracle RAC is required by the Transaction Monitor in order to take advantage of these replication and failover features in an XA transaction environment. This is because Oracle RAC does not allow the same database to be accessed from multiple RAC instances within the same XA transaction. In addition, Oracle 10gR1 (and earlier) require
Transaction Monitor involvement when prepared transactions failover from one RAC instance to another.
Tuxedo 9.1 provides Transaction Monitor support for Oracle RAC by allowing an administrator to specify lists of groups associated with different RAC instances, thereby allowing Tuxedo to ensure that groups associated with different instances of the same RAC database do not participate in the same transaction. The Tuxedo Oracle RAC support feature also provides a way for Tuxedo transaction manager server (TMS) processes to be notified of RAC failover events. Consequently, this allows the TMS to re-obtain a list of prepared transactions from Oracle as required for RAC failover recovery.
See Tuxedo 9.1 Installation Guide, BEA Tuxedo 9.1 Platform Data Sheets for specific platform software requirements.
Tuxedo support for Oracle RAC requires two steps:
The following command and environment variables are used to exclusively configure Tuxedo 9.1 for Oracle RAC support:
Oracle RAC does not allow the same database to be accessed from multiple RAC instances within the same XA transaction. In addition, Oracle 10gR1 (and earlier) require Transaction Monitor involvement when prepared transactions failover from one RAC instance to another.
The TUXRACGROUPS
environment variable is used to associate Tuxedo groups with specific instances of Oracle RAC configurations so that Tuxedo does not include groups from multiple instances of the same RAC configuration within the same XA transaction.
A single transaction cannot span multiple Oracle RAC instances. The groups that participate in a particular transaction are determined at the time the transaction is started. Each transaction is assigned to one particular instance of each RAC configuration such that the groups in each instance of a particular RAC configuration are assigned to an equal number of transactions.
Warning: The TUXRACGROUPS
environment variable must be set on all machines in a configuration, and must have the same sets of groups specified in the same order on all machines.
Warning: If this restriction is not followed, then inconsistent sets of groups can be included within a transaction. The coordinating group will notice the inconsistency at commit time, roll back the transaction, and send an error message to the userlog.
The TUXRACGROUPS
environment variable is used to define Oracle RAC group configurations. Its syntax: is as follows:
TUXRACGROUPS="G1,G2,,Gm;H1,H2,,Hn[;]:I1,I2,,Io;J1,J2,,Jp[;][:]"
Used to specify groups in the same instance of an Oracle RAC configuration. Multiple groups from a comma separated list can be used together in the same transaction.
Used to specify sets of groups in different instances of an oracle RAC configuration. Groups from different RAC instances from the same RAC database configuration cannot be used together in the same transaction.
Since the purpose of the TUXRACGROUPS
environment variable is to specify groups associated with different instances of the same Oracle RAC configuration, all applications using the TUXRACGROUPS
variable should have at least one semicolon in the environment variable value.
This section describes three different examples for defining Oracle RAC group configurations.
Figure 6-1 shows a simple Oracle RAC configuration.
Figure 6-1 (ORA1) Simple Configuration
In this example, there is one Oracle database, (ORA1), two Oracle RAC instances with 1 group per each instance.
The same transaction request to both GROUP1
and GROUP2
cannot be sent because they access database services through different instances that map to the same Oracle RAC database configuration.
TUXRACGROUPS="GROUP1;GROUP2:GROUP3;GROUP4,GROUP5"
Figure 6-2 shows an example of adding multiple groups to a single instance.
Figure 6-2 (ORA2) Single Oracle RAC Instance with Multiple Groups
In this example, there are two Oracle databases: ORA1
and ORA2
. ORA1
offers machine-specific services ORA1SITE1
and ORA1SITE2
, and ORA2
offers machine-specific services ORA2SITE1
and ORA2SITE2
.
The objective is to assign an approximately equal number of transactions (and configure the same services) to the groups associated with each instance of an Oracle RAC configuration.
The same transaction request to both GROUP1
and GROUP2
cannot be sent because they access database services through different instances that map to the same Oracle RAC database configuration. The same applies to GROUP3
and GROUP4
or GROUP3
GROUP5
, the same transaction cannot be sent to both these groups.
GROUP4
and GROUP5 both access the same database service of the same Oracle RAC database configuration, so these groups would be permitted together. GROUP1
and GROUP4
would be permitted together, because they access different RAC database configurations. If there is also a GROUP6
in this configuration, it would be permitted with any other group, because GROUP6
is not an Oracle RAC group.
Note: The number of groups in each Oracle RAC instance does not have to be the same.
The *GROUPS
and *SERVERS
section of the UBBCONFIG
file for such a configuration might look as follows:
Listing 6-1 Example of *GROUPS Section of UBBCONFIG File
*GROUPS
DEFAULT: TMSNAME=TMS_ORA TMSCOUNT=2
GROUP1 LMID=SITE1 GRPNO=1
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA1SITE1+SesTm=100+LogDir=.+MaxCur=5"
GROUP2 LMID=SITE2 GRPNO=2
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA1SITE2+SesTm=100+LogDir=.+MaxCur=5"
GROUP3 LMID=SITE1 GRPNO=3
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA2SITE1+SesTm=100+LogDir=.+MaxCur=5"
GROUP4 LMID=SITE2 GRPNO=4
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA2SITE2+SesTm=100+LogDir=.+MaxCur=5"
GROUP5 LMID=SITE2 GRPNO=5
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA2SITE2+SesTm=100+LogDir=.+MaxCur=5"
GROUP6 LMID=SITE1 GRPNO=6 TMSNAME=TMS_QM
OPENINFO="TUXEDO/QM:/home/myapplication/QUE:QSPACE"
Note: GROUP4
and GROUP5
have the same OPENINFO
strings, because they both use the same database service for the same database.
The specification of the OPENINFO
string for Oracle groups in the *GROUPS
section is the same as when using Oracle without RAC. For information on how to specify an OPENINFO
string for an Oracle group, refer to the Developing Applications with Oracle XA chapter in the Oracle Database Application Developer's Guide - Fundamentals.
Listing 6-2 Example of *SERVERS Section of UBBCONFIG File
*SERVERS
DEFAULT: RESTART=Y MAXGEN=5 REPLYQ=Y CLOPT="-A"
EMPLOYEE_SVR SRVGRP=GROUP1 SRVID=1
EMPLOYEE_SVR SRVGRP=GROUP2 SRVID=2
BANKING_SVR SRVGRP=GROUP3 SRVID=3
BANKING_SVR SRVGRP=GROUP4 SRVID=4
BANKING_SVR SRVGRP=GROUP5 SRVID=5
TUXRACGROUPS="GROUP11,GROUP12,GROUP13;GROUP21,GROUP22:GROUP3;GROUP4, GROUP5"
Figure 6-3 shows an example of adding multiple groups to multiple instances.
Figure 6-3 Multiple Oracle RAC Instances with Multiple Groups
This example is similar to the previous example — except that GROUP11
, GROUP12
, and GROUP13
are all associated with the first RAC instance of the first RAC configuration, and GROUP21
and GROUP22
are both associated with the second RAC instance.
If the first service call in a transaction in this configuration goes to GROUP12
, then it would be possible to send other service calls in this transaction to GROUP11
, GROUP12
, or GROUP13
, but not to GROUP21
or GROUP22
.
If a transactional service call is made to a service that is not advertised in any permitted groups but is available in one or more prohibited groups, the result is:
tperrno
is set to TPENOENT
tperrordetail
is set to the new value TPED_GROUP_FORBIDDEN
For each RAC configuration defined as part of the TUXRACGROUPS
environment variable, Tuxedo determines which RAC group(s) in that configuration participate in a particular transaction when that transaction is started.
TMS_rac_refresh(1)
, XARETRYDURATIONSECONDS
, and XARETRYINTERVAL
specifically handle transaction recovery issues.
TMS_rac_refresh(1)
is called when an Oracle RAC group fails over to an alternate group. TMS_rac_refresh(1)
should not be executed manually from the command line; the proper way to invoke TMS_rac_refresh(1)
is to use Oracle Fast Application Notification (FAN).
Note: For more details on configuring Oracle FAN, refer to Oracle 10g documentation.
The XARETRYDURATIONSECONDS
and XARETRYINTERVAL
environment variables are used to retry transaction recovery operations (xa_recover()
) as required by Oracle RAC.
Specifies the time interval during which the Tuxedo Transaction Manager Server (TMS) retries xa_recover()
operations when TMS_rac_refresh(1)
is called. If it is not set or set to 0, then xa_recover()
is performed once only.
A key process in configuring Tuxedo for Oracle RAC is setting up Oracle FAN to invoke TMS_rac_refresh(1)
with the appropriate group parameter on group failover. (More group parameter and group failover information is provided in Configuring Transaction Propagation.)
More information regarding Oracle FAN can be found in the
Workload Management with Oracle Real Application Clusters (PDF) White Paper
The following is an example of an Oracle FAN script.
Listing 6-3 Oracle FAN Script Example
//This File should be placed at ORA_CRS_HOME/racg/usrco//
------------------------------calout.sh-------------------------
#! /bin/ksh
#parse the event
AWK=awk
NOTIFY_EVENTTYPE=$1 # Event type is handled differently
for ARGS in $*
do
PROPERTY=\Qecho $ARGS|$AWK -F"=" '{print $1}'\Q
VALUE=\Qecho $ARGS|$AWK -F"=" '{print $2}'\Q
case ${PROPERTY} in
VERSION|version)NOTIFY_VERSION=$VALUE;;
SERVICE|service)NOTIFY_SERVICE=$VALUE;;
DATABASE|database)NOTIFY_DATABASE=$VALUE;;
INSTANCE|instance)NOTIFY_INSTANCE=$VALUE;;
HOST|host) NOTIFY_HOST=$VALUE ;;
STATUS|status) NOTIFY_STATUS=$VALUE;;
REASON|reason) NOTIFY_REASON=$VALUE;;
CARD|card) NOTIFY_CARDINALITY=$VALUE ;;
TIMESTAMP|timestamp) NOTIFY_LOGDATE=$VALUE;; # catch
event
??:??:??) NOTIFY_LOGTIME=$PROPERTY;; # catch event time
(hh24:mi:ss)
esac
done
#Set the REFRESH_DIR environment variable.
. /home/oracle/callout.env
#Make a log to record events.
FAN_LOGFILE=/home/oracle/app/products/10.1.0.3.0/db_1/calloutlog/\Qhostname\Q_upti
me.log
touch ${FAN_LOGFILE}
echo ${1} >>${FAN_LOGFILE}
#invoke the TMS_rac_refresh command.
if [ ${NOTIFY_EVENTTYPE} = "INSTANCE" -a ${NOTIFY_STATUS} = "down" ]
then
${REFRESH_DIR}/rac_refresh >> ${FAN_LOGFILE} 2>&1
fi
-----------------------------callout.sh end-----------------------------
-----------------------------callout.env----------------------------------
#! /bin/ksh
#TUXEDO and Oracle RAC server are not one the same machine.
export REFRESH_DIR=/tmp
-----------------------------callout.env end------------------------------
-----------------------------rac_refresh----------------------------------
#! /bin/ksh
#If TUXEDO and Oracle RAC server on different machine
. /home/oracle/callout.env
rsh -l ${LOGNAME} ${TUX_MASTER_MACHINE} ${REFRESH_DIR}/rac_refresh
>/tmp/run1.log 2>&1
rsh -l ${LOGNAME} ${TUX_NONMASTER_MACHINE}
${REFRESH_DIR}/rac_refresh >/tmp/run1.log 2>&1
#If TUXEDO and Oracle RAC server are on same machine
#set up environment variable
#export APPDIR=/tmp
#export ORACLE_HOME=/home/oracle/Ora10g
#export TUXDIR=/nfs/users/libo/r902/BJ/bld
#export PATH=.:${PATH}:${TUXDIR}/bin
#. $TUXDIR/tux.env
#export TUXCONFIG=${APPDIR} /tuxconfig
#invoke TMS_rac_refresh
#TMS_rac_refresh RACDBGRP1
#TMS_rac_refresh RACDBGRP3
-----------------------------rac_refresh end-----------------------------
Although the Tuxedo Oracle RAC environment variables can be initiated at the operating system command line, it is highly recommended that you use the ENVFILE
parameter specified in the *MACHINES
section of the UBBCONFIG
file to initiate these environment variables.
Apply the following syntax considerations when setting the environment variables for Oracle RAC.
ENVFILE,
which is the preferred method, quotation marks are not permitted around the environment variable value.