6 Introduction to Using Tuxedo with Oracle Real Application Clusters (RAC)
This chapter explains how to leverage advanced features provided through close integration between Oracle Tuxedo with Oracle RAC. Most of the features are only available Oracle Tuxedo Advanced Performance Pack license.
Features that are only available in Tuxedo Advanced Performance Pack | |
---|---|
Instance Awareness | |
XA Transaction Affinity | |
Common XID | |
Single Group Multiple Branches (SGMB) | |
Failover/Failback across Database Instances using Fast Application Notification (FAN) | |
Load Balancing across RAC Instances using Fast Application Notification (FAN) |
- Instance Awareness
- Using Tuxedo with XA Affinity
- Using Tuxedo with Common XID
- Using Tuxedo with Single Group Multiple Branches (SGMB)
- Using Tuxedo with Fast Application Notification (FAN)
- Using Tuxedo with Oracle Real Application Clusters (RAC)
See Also:
For more information, Using Oracle Tuxedo Advanced Performance Pack6.1 Instance Awareness
For the servers associated with Oracle Database, Tuxedo uses customized callback to retrieve Oracle Database instance information.
- XA Server
- Tuxedo uses customized callback to retrieve Oracle Database instance information by static callback registration for XA servers.
- Non-XA Server
- Tuxedo uses customized callback to retrieve Oracle Database
instance information by dynamic callback registration for non-XA
servers with the following requirements.
-
$TUXDIR/lib/tuxociucb.so.1.0
package should be deployed in$ORACLE_HOME/lib
. -
ORA_OCI_UCBPKG
environment variable must include the package name.
-
It is necessary to deploy the package into
$ORACLE_HOME/lib
before running a non-XA server that
needs instance awareness; otherwise, the server that uses dynamic
callback registration will fail.
Non-XA server tries to automatically set the environment
variable ORA_OCI_UCBPKG
when booting up. If the server
fails to do so, an error message will occur in ULOG
and the instance awareness will be disabled in the server.
6.2 Using Tuxedo with XA Affinity
This section contains the following topics.
6.2.1 Overview
Oracle RAC environments have a one-to-many relationship between database and instances. Servers or groups of a Tuxedo application running on an Oracle RAC may connect to different instances.
It is XA Affinity that ensures all database operations to connect the same RAC instance when possible (no matter if those operations are in one global transaction branch or in different branches of one global transaction) and automatically exchanges the affinity information between Tuxedo domain and WebLogic Server via WTC (WebLogic Tuxedo Connector).
Parent topic: Using Tuxedo with XA Affinity
6.2.2 XA Affinity Priority
Tuxedo server selection rules are listed as below, in high to low priority order.
-
GWTDOMAIN
Transaction routing - Oracle RAC routing for transaction affinity using RAC instances
- Service versioning
- Client/server affinity routing
- XA Affinity
- Load balance based on Oracle RAC LBA (Load Balancing Advisor)
- Tuxedo load balance
If XA Affinity is enabled, the Oracle RAC routing rule that
environment variable TUXRACGROUPS
specifies will be
disabled.
Note:
XA Affinity has no impact on domain routing. Only when a request arrives at a domain and starts to be routed to a service in that domain, XA Affinity affects server routing.Parent topic: Using Tuxedo with XA Affinity
6.2.3 XA Affinity Policy
Tuxedo selects the server that is associated with the same instance name, DB name, and service name. If this attempt fails, Tuxedo follows the following policies to find the server.
- Tuxedo tries to find the server which is associated with both the same DB name and the same service name. The server's group must not be involved in the current global transaction.
- If the above attempt fails, Tuxedo tries to find the server which is associated with both the same DB name and the same instance name.
- If the above attempt fails, Tuxedo tries to find the server which is associated with the same DB name.
- If the above attempt fails, Tuxedo finds the server according to Tuxedo normal load balance.
Note:
If Tuxedo finds multiple servers that are at the same priority, Tuxedo will find the server according to Tuxedo normal load balance.Parent topic: Using Tuxedo with XA Affinity
6.2.4 Prerequisites
6.2.4.1 Software Requirements
The feature can be run on all Oracle Tuxedo supported platforms, except for Oracle Tuxedo 32-bit on Microsoft Windows platforms.
For specific platform software requirements, refer to Oracle Tuxedo Platform Data Sheets.
Parent topic: Prerequisites
6.2.4.2 Installation Notes
- The Oracle Tuxedo must be 12 c Release 2 (12.1.3) or above.
- The Oracle Database must be 11.2.0.2.0 or above.
Parent topic: Prerequisites
6.2.5 Configurations
As long as the option XPP
in OPTIONS
of UBBCONFIG
*RESOURCES
section is specified, XA Affinity feature is enabled by default.
Note:
On Oracle Exalogic and Oracle SPARC SuperCluster platforms, theOPTIONS
parameter must be set to EECS
.
A new option, NO_XAAFFINITY
, is introduced to RMOPTIONS
of UBBCONFIG *RESOURCES
section to explicitly disable XA Affinity.
RMOPTIONS {[...|NO_XAAFFINITY],*}
The following Listing shows an example to configure EECS
; the following Listing depicts an example to explicitly disable XA Affinity.
Listing Example to Configure EECS
* RESOURCES
OPTIONS EECS
Listing Example to Explicitly Disable XA Affinity
* RESOURCES
OPTIONS EECS
RMOPTIONS NO_XAAFFINITY
This flag can also be specified in T_DOMAIN
class via TM_MIB
, when the tuxedo application is inactive. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.
Note:
XA Affinity requires Tuxedo servers to retrieve Oracle Database instance information. Users can query a server’s instance information through TuxedoTM_MIB
T_SERVER
class's TA_INSTSTR
field. For more information, see T_SERVER Class Definition.
Parent topic: Using Tuxedo with XA Affinity
6.2.6 Limitations
- Groups with MRM (multiple RM) are not supported.
- The max number of affinity context (database name+instance name+service name) in one transaction is 16.
- XA Affinity does not support multi-server single queue.
- XA Affinity does not support multi-threaded server.
- XA Affinity does not support cross-domain services.
Parent topic: Using Tuxedo with XA Affinity
6.3 Using Tuxedo with Common XID
This section contains the following topics.
6.3.1 Overview
In general, for global transactions, each participating group has its own transaction branch, and a distinguished transaction branch identifier (XID) identifies each branch. If a global transaction involves multiple groups, Oracle Tuxedo adopts two-phase commit on each branch, taking the first participating group as the coordinator.
However, with common XID feature in this release, Oracle Tuxedo shares the coordinator group transaction branch with all other groups within the same transaction. The groups that connect to the same Oracle Database instance through the same service use the coordinator branch directly. For those groups, XA committing operations are not required and are saved.
In the most extreme case, where all groups in a global
transaction use the coordinator branch directly, Oracle Tuxedo
adopts one-phase commit instead of two-phase commit; no
TLOG
is written.
Parent topic: Using Tuxedo with Common XID
6.3.1.1 Typical Scenario
Assume there are two groups GRP1
and
GRP2
in a Tuxedo application domain DOM1
.
Server SERV1
belongs to GRP1
and offers
service SVC1
while server SERV2
belongs
to GRP2
and offers service SVC2
. Both
SERV1
and SERV2
connect to the same
instance.
Then a native client begins a global transaction at first,
invokes SVC1
followed by SVC2
, and
commits the transaction.
If common XID functionality is enabled in the above case, Tuxedo
invokes one-phase commit on the transaction and no
TLOG
is written; otherwise, two-phase commit is
invoked.
Users are allowed to trace the above behaviors through TMTRACE
. Please refer to TMTRACE
for more information in the Section 5 - Files Formats, Data Descriptions, MIBs, and System Processes Reference
Parent topic: Overview
6.3.2 Prerequisites
6.3.2.1 Software Requirements
The feature can be run on all Oracle Tuxedo supported platforms, except for Oracle Tuxedo 32-bit on Microsoft Windows platforms.
For specific platform software requirements, refer to Oracle Tuxedo Platform Data Sheets.
Parent topic: Prerequisites
6.3.2.2 Installation Notes
- The Oracle Tuxedo must be 12c Release 2 (12.1.3) or above.
- The Oracle Database must be 11.2.0.2.0 or above.
Parent topic: Prerequisites
6.3.3 Configurations
As long as the option XPP
in OPTIONS
of UBBCONFIG * RESOURCES
section is specified, common XID feature is enabled by default. A new option, NO_COMMONXID
, is introduced to RMOPTIONS
of UBBCONFIG *RESOURCES
section to explicitly disable common XID.
RMOPTIONS {[...|NO_COMMONXID],*}
The following Listing shows an example to configure XPP
; the following Listing lists an example to explicitly disable common XID.
Listing Example to Explicitly Disable Common XID
* RESOURCES
OPTIONS XPP
RMOPTIONS NO_COMMONXID
This flag can also be specified in T_DOMAIN
class via TM_MIB
, when the tuxedo application is inactive. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.
Note:
Common XID requires Tuxedo servers to retrieve Oracle Database instance information. Users can query a server’s instance information through TuxedoTM_MIB
T_SERVER
class's TA_INSTSTR
field. For more information, see T_SERVER Class Definition.
Parent topic: Using Tuxedo with Common XID
6.3.4 Limitations
- Groups with MRM (multiple RM) are not supported.
- Multi-threaded servers do not provide instance information via
MIB
; however, common XID still performs well on server-dispatched threads. - In two-phase commit scenarios,
GWTDOMAIN
is always involved to do prepare and/or commit. - If the coordinator group is the group where
GWTDOMAIN
locates, common XID does not work.
Parent topic: Using Tuxedo with Common XID
6.4 Using Tuxedo with Single Group Multiple Branches (SGMB)
This section contains the following topics.
6.4.1 Overview
In previous releases, servers in the same participated group use
the same transaction branch in a global transaction. However the
transaction branch would fail if these serves connect to different
instances of a RAC. An XA error, XAER_AFFINITY
, will
be reported, meaning one branch cannot go through different
instances. For this reason, the RAC service used by a Tuxedo group
must be a singleton RAC service. A DTP service (if the DTP option,
-x
in srvctl modify service
, is
specified) or a service offered by only one instance could be a
singleton RAC service.
In this release, using different transaction branches on different instances in a single group will solve the issue. The Tuxedo group can then use non-singleton service and take advantage of its benefits, such as load balance.
Parent topic: Using Tuxedo with Single Group Multiple Branches (SGMB)
6.4.2 Prerequisites
Parent topic: Using Tuxedo with Single Group Multiple Branches (SGMB)
6.4.2.1 Software Requirements
The feature can be run on all Oracle Tuxedo supported platforms, except for Oracle Tuxedo 32-bit on Microsoft Windows platforms.
For specific platform software requirements, refer to Oracle Tuxedo Platform Data Sheets.
Parent topic: Prerequisites
6.4.2.2 Installation Notes
- The Oracle Tuxedo must be 12c Release 2 (12.1.3) or above.
- The Oracle Database must be 11.2.0.2.0 or above.
Parent topic: Prerequisites
6.4.3 Configurations
As long as the option EECS
in OPTIONS
of UBBCONFIG
*RESOURCES
section is specified, SGMB feature is enabled by default. A new option,
SINGLETON
, is introduced to RMOPTIONS
of UBBCONFIG
*RESOURCES
section to explicitly disable SGMB.
RMOPTIONS {[...|SINGLETON],*}
This option indicates all RAC services used in the domain are singleton, so the SGMB feature is not necessary to work.
The following Listing shows an example to configure EECS
; the following
Listing depicts an example to explicitly disable SGMB.
Listing Example to Explicitly Disable SGMB
* RESOURCES
OPTIONS EECS
RMOPTIONS SINGLETON
This flag can also be specified in T_DOMAIN
class via TM_MIB
, when the tuxedo application is inactive. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.
Note:
SGMB requires Tuxedo servers to retrieve Oracle Database instance information. Users can query a server’s instance information through TuxedoTM_MIB
T_SERVER
class's TA_INSTSTR
field. For more information, see T_SERVER Class Definition.
Parent topic: Using Tuxedo with Single Group Multiple Branches (SGMB)
6.4.4 Limitations
- Groups with MRM (multiple RM) are not supported.
- A transaction fails if more than 16 instances are involved in a single group.
- Read-Only optimization for XA does not work in a transaction if
the preferred reserved group is a multi-branch group. If
GWTDOMAIN
is not the coordinator, the preferred reserved group is the coordinator group; otherwise, the preferred reserved group is the participated group coming next in the coordinator domain. - Multi-threaded servers do not provide instance information through
MIB
; however, SGMB still performs well on server-dispatched threads.
Parent topic: Using Tuxedo with Single Group Multiple Branches (SGMB)
6.5 Using Tuxedo with Fast Application Notification (FAN)
This section contains the following topics.
6.5.1 Overview
Tuxedo uses Fast Application Notification (FAN) to
- Provide rapid failure detection.
- Remove invalid DB connections from Tuxedo server and create valid DB connection. If Tuxedo server cannot create valid DB connection, Tuxedo removes these servers from the routing list.
- Perform graceful shutdown for planned and unplanned Oracle RAC node outages.
- Adapt to changes in topology, such as adding or removing a node.
- Distribute runtime work requests to all active Oracle RAC instances, including those rejoining a cluster.
Parent topic: Using Tuxedo with Fast Application Notification (FAN)
6.5.2 Prerequisites
Parent topic: Using Tuxedo with Fast Application Notification (FAN)
6.5.2.1 Software Requirements
The feature can be run on all Oracle Tuxedo supported platforms, except for Oracle Tuxedo 32-bit on Microsoft Windows platforms.
For specific platform software requirements, refer to Oracle Tuxedo Platform Data Sheets.
Parent topic: Prerequisites
6.5.2.2 Installation Notes
- The Oracle Tuxedo must be 12c Release 2 (12.1.3) or above.
- The Oracle Database must be 11.2.0.2.0 or above.
- Using the connection steering requires Oracle client 11.2.0.2.0 or Oracle client 12.1.0.1.0 with a specific patch (contact Oracle Support for the patch, or higher release of Oracle client).
Parent topic: Prerequisites
6.5.3 Configurations
Parent topic: Using Tuxedo with Fast Application Notification (FAN)
6.5.3.1 Configurations on DB
DB configuration includes the following topics.
Parent topic: Configurations
6.5.3.1.1 ONS
On Oracle server side, ONS daemon must be enabled.
If Tuxedo is taken as a native client, ONS daemon on the client
side must also be enabled. The ONS daemon configuration file is
located in $ORACLE_HOME/opmn/conf/ons.config
. After
configuring ONS, start ONS daemon with onsctl
start
command. Please make sure that ONS daemon is
running all the time.
If Tuxedo is taken as a remote client, ONS daemon on the client side is not used. It is the preferred mode.
Note:
On the Oracle client side, if the Oracle version is lower than 12.1.0.1.0, ONS daemon must be enabled.Parent topic: Configurations on DB
6.5.3.1.2 Load Balancing Advisor (LBA)
The ONS may publish LBA about a service if the service has load balancing advisory goal. You can use -B option to specify the goal through srvctl
when creating or modifying the service.
Parent topic: Configurations on DB
6.5.3.1.3 TAF
If TAF is enabled, all Tuxedo servers can automatically do the reconnection by TAF; otherwise, only XA servers can automatically do the reconnection.
- XA server
-
OPENINFO
must includethreads=t
. - Non-XA server
- To monitor FAN event for the instance associated with the specific non-XA application server,
$TUXDIR/lib/tuxociucb.so.1.0
must be deployed in$ORACLE_HOME/lib
, and the name of this binary must be specified inORA_OCI_UCBPKG
environment variable.
Parent topic: Configurations on DB
6.5.3.2 Configurations on Tuxedo
It is required to configure TMFAN
server in UBBCONFIG *SERVERS
section and configure the option XPP
in OPTIONS
of UBBCONFIG *RESOURCES
section. A new option, NO_FAN
, is introduced to RMOPTIONS
of UBBCONFIG *RESOURCES
section to explicitly disable FAN.
RMOPTIONS {[...|NO_FAN],*}
The following Listing shows an example to configure XPP
and the following Listing depicts an example to explicitly disable FAN.
Listing Example to Explicitly Disable FAN
* RESOURCES
OPTIONS XPP
RMOPTIONS NO_FAN
This flag can also be specified in T_DOMAIN
class via TM_MIB
, when the tuxedo application is inactive. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.
Note:
FAN requires Tuxedo servers to retrieve Oracle Database instance information. Users can query a server’s instance information through TuxedoTM_MIB
T_SERVER
class's TA_INSTSTR
field. For more information, see T_SERVER Class Definition.Parent topic: Configurations
6.5.4 Limitations
- Groups with MRM (multiple RM) are unsupported.
- If the customized server is going to use OCI to connect Oracle
database,
OCI_NO_UCB
should not be set at OCI initialization time. - Load balance based on Oracle RAC LBA (Load Balancing Advisor) does not support multi-server single queue.
- Load balance based on Oracle RAC LBA (Load Balancing Advisor) does not support cross-domain services.
Parent topic: Using Tuxedo with Fast Application Notification (FAN)
6.6 Using Tuxedo with Oracle Real Application Clusters (RAC)
This section contains the following topics.
6.6.1 Overview
The Oracle Real Application Clusters (RAC) feature supports clustering of machines that utilize replicated Oracle database services accessing the same Oracle database. Oracle RAC provides the ability to concurrently access the same Oracle database from instances physically located on multiple Oracle server machines, and offers the ability to failover unsuccessful database instances to alternate locations.
However, specific support for Oracle RAC is required by the Transaction Monitor in order to take advantage of these replication and failover features in an XA transaction environment or to obtain optimal RAC performance. This is because Oracle 10g does not allow the same database to be accessed from multiple RAC instances within the same XA transaction.
Note:
Oracle 12c does allow the same database to be accessed from multiple RAC instances within the same XA transaction, but performance may be better if all accesses within a particular XA transaction occur from the same RAC instance.In addition, Oracle 10gR1 requires Transaction Monitor involvement when prepared transactions failover from one RAC instance to another.
Tuxedo provides Transaction Monitor support for Oracle RAC by allowing an administrator to specify lists of groups associated with different RAC instances. This allows Tuxedo to ensure that groups associated with different instances of the same RAC database do not participate in the same transaction. The Tuxedo Oracle RAC support feature also provides a way for Tuxedo transaction manager server (TMS) processes to be notified of RAC failover events which is required when using Oracle 10gR1.
Consequently, this allows the TMS to re-obtain a list of Oracle 10gR1 prepared transactions from Oracle as required for RAC failover recovery.
Note:
When using Oracle 10gR2, administrators should use an Oracle<b>
DTP Service</b>
to access the Oracle RAC system. This DTP service name should be specified in the OPENINFO string for the associated Tuxedo groups. Oracle 10gR2 verifies the service name, and migrates it to an alternate instance if required.
Note:
When using Oracle 12c or later release, the service name is transparently and automatically migrated to an alternate instance, if required, without any specific configuration.Parent topic: Using Tuxedo with Oracle Real Application Clusters (RAC)
6.6.2 Limitations
- Tuxedo supports Oracle RAC only when using Oracle 10g or later release, and does not support Oracle RAC when using Oracle 9i.
- In some instances, using Oracle RAC with the Dynamic XA switch enabled may generate a core dump and cause a system crash. Please contact Oracle Support directly if you encounter this issue and provide the following information:
- BUG 4644880 - Oracle bug fix identification number
- the patch set version for the 10g release you are using
Parent topic: Using Tuxedo with Oracle Real Application Clusters (RAC)
6.6.3 Software Requirements
For specific platform software requirements, refer to Oracle Tuxedo Platform Data Sheets.
Parent topic: Using Tuxedo with Oracle Real Application Clusters (RAC)
6.6.4 Configuring Tuxedo for Oracle RAC
Tuxedo support for Oracle RAC requires two steps:
The following command and environment variables are used to exclusively configure Tuxedo for Oracle RAC support:
-
Three environment variables
- TUXRACGROUPS (required for Oracle 10gR1 and 10gR2, optional for Oracle 11g and later releases)
- "XARETRYDURATIONSECONDS" (required only for Oracle 10gR1)
- "XARETRYINTERVAL" (required only for Oracle 10gR1)
- One Command
-
TMS_rac_refresh(1)
(required only for Oracle 10gR1)
-
Parent topic: Using Tuxedo with Oracle Real Application Clusters (RAC)
6.6.4.1 Configuring Transaction Propagation
Oracle 10gR1 does not allow the same database to be accessed from multiple RAC instances within the same XA transaction. In addition, Oracle 10gR1 requires Transaction Monitor involvement when prepared transactions failover from one RAC instance to another.
Oracle 10gR2 permits different RAC instances to operate on different transaction branches in RAC, but if transaction branches are on different instances, then they are loosely coupled and do not share locks. Also, for optimum commit performance, it is important to use only a single RAC instance within a given XA transaction.
For this reason, it is still important to associate an XA transaction with a single RAC instance in Oracle 10gR2. (For further information on using Oracle XA with RAC, refer to the "Developing Applications with Oracle XA" chapter in the Oracle Database Application Developer's Guide - Fundamentals.
The TUXRACGROUPS
environment variable is used to
associate Tuxedo groups with specific instances of Oracle RAC
configurations so that Tuxedo does not include groups from multiple
instances of the same RAC configuration within the same XA
transaction.
Note:
When using Oracle 10g, a single transaction should not span multiple Oracle RAC instances. The groups that participate in a particular transaction are determined at the time the transaction is started. Each transaction is assigned to one particular instance of each RAC configuration such that the groups in each instance of a particular RAC configuration are assigned to an equal number of transactions.Oracle 12c permits different RAC instances to operate on
different transaction branches in RAC, and if transaction branches
are on different instances, then they are tightly coupled and share
locks and resources. So the TUXRACGROUPS
environment
variable is not necessary when using Oracle 12c. This environment
variable still works in Oracle 12c and you can use it to associate
Tuxedo groups with specific instances of Oracle RAC
configurations.
- TUXRACGROUPS
- The
TUXRACGROUPS
environment variable specifies the groups that are associated with a particular RAC configuration, and will disallow sending service calls in the same transaction to two or more groups identified as different instances of the same RAC configuration.
WARNING:
If theTUXRACGROUPS
environment variable is used, it must be set on all machines in a configuration, and must have the same sets of groups specified in the same order on all machines.
If this restriction is not followed, then inconsistent sets of groups can be included within a transaction. The coordinating group will notice the inconsistency at commit time, roll back the transaction, and send an error message to the userlog.
- TUXRACGROUPS Syntax
- TUXRACGROUPS Examples
- Transaction Creation Behavior Using TUXRACGROUPS
- Data Dependent Routing Using TUXRACGROUPS
- Assigning Transactions to Special Oracle RAC Instances
- TUXRAGROUPS Transaction Use Cases
Parent topic: Configuring Tuxedo for Oracle RAC
6.6.4.1.1 TUXRACGROUPS Syntax
The TUXRACGROUPS
environment variable is used to define Oracle RAC group
configurations. Its syntax is as follows:
TUXRACGROUPS="G1,G2,…,Gm;H1,H2,…,Hn[;…]:I1,I2,…,Io;J1,J2,…,Jp[;…][:…]"
- Comma (,) separated list
- Used to specify groups in the same instance of an Oracle RAC configuration. Multiple groups from a comma separated list can be used together in the same transaction.
Note:
Typically, most users place all of the services associated with one database instance in a single group, therefore commas are not needed in theTUXRACGROUPS
value. - Semicolon (;) separated list
- Used to specify sets of groups in different instances of an oracle RAC configuration. Groups from different RAC instances from the same RAC database configuration cannot be used together in the same transaction.
- Colon (:) separated list
- Used to separate information about one Oracle RAC configuration from information about a different Oracle RAC configuration. The colon indicates that multiple Oracle RAC database configurations are totally independent of each other.
Note:
Typically, most users specify only one RAC database configuration, therefore colons are not needed in theTUXRACGROUPS
value.
Parent topic: Configuring Transaction Propagation
6.6.4.1.2 TUXRACGROUPS Examples
TUXRACGROUPS="G1;G2"
The following figure shows a simple Oracle RAC configuration.
In this example, there is one Oracle database, (ORA1), two Oracle RAC instances with 1 group per each instance.
The same transaction request to both GROUP1
and GROUP2
cannot be sent because they access database services through different instances that map to the same Oracle RAC database configuration.
Figure 6-1 (ORA1) Simple Configuration
TUXRACGROUPS="GROUP1;GROUP2:GROUP3;GROUP4,GROUP5"
The following Figure shows an example of adding multiple groups to a single instance.
In this example, there are two Oracle databases: ORA1
and ORA2
. ORA1
offers machine-specific services ORA1SITE1
and ORA1SITE2
, and ORA2
offers machine-specific services ORA2SITE1
and ORA2SITE2
. The objective is to assign an approximately equal number of transactions and configure the same services to the groups associated with each instance of an Oracle RAC configuration.
The same transaction request to both GROUP1
and GROUP2
cannot be sent because they access database services through different instances that map to the same Oracle RAC database configuration. The same applies to GROUP3
and GROUP4
or GROUP3
GROUP5
, the same transaction cannot be sent to both these groups.
GROUP4
and GROUP5
both access the same database service of the same Oracle RAC database configuration, so these groups would be permitted together. GROUP1
and GROUP4
would be permitted together, because they access different RAC database configurations. If there is also a GROUP6
in this configuration, it would be permitted with any other group, because GROUP6
is not an Oracle RAC group.
Note:
The number of groups in each Oracle RAC instance does not have to be the same.Figure 6-2 (ORA2) Single Oracle RAC Instance with Multiple Groups
The *GROUPS
and *SERVERS
sections of the UBBCONFIG
file for this configuration might look as follows in Listing:
Listing UBBCONFIG File *GROUPS and *SERVERS Sections Example
*GROUPS
DEFAULT: TMSNAME=TMS_ORA TMSCOUNT=2
GROUP1 LMID=SITE1 GRPNO=1
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA1SITE1+SesTm=100+LogDir=.+MaxCur=5"
GROUP2 LMID=SITE2 GRPNO=2
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA1SITE2+SesTm=100+LogDir=.+MaxCur=5"
GROUP3 LMID=SITE1 GRPNO=3
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA2SITE1+SesTm=100+LogDir=.+MaxCur=5"
GROUP4 LMID=SITE2 GRPNO=4
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA2SITE2+SesTm=100+LogDir=.+MaxCur=5"
GROUP5 LMID=SITE2 GRPNO=5
OPENINFO="ORACLE_XA:Oracle_XA+Acc=P/scott/tiger+SqlNet=ORA2SITE2+SesTm=100+LogDir=.+MaxCur=5"
GROUP6 LMID=SITE1 GRPNO=6 TMSNAME=TMS_QM
OPENINFO="TUXEDO/QM:/home/myapplication/QUE:QSPACE"
*SERVERS
DEFAULT: RESTART=Y MAXGEN=5 REPLYQ=Y CLOPT="-A"
EMPLOYEE_SVR SRVGRP=GROUP1 SRVID=1
EMPLOYEE_SVR SRVGRP=GROUP2 SRVID=2
BANKING_SVR SRVGRP=GROUP3 SRVID=3
BANKING_SVR SRVGRP=GROUP4 SRVID=4
BANKING_SVR SRVGRP=GROUP5 SRVID=5
Note:
GROUP4
and GROUP5
have the same OPENINFO
strings, because they both use the same database service from the same database. The specification of the OPENINFO
string for Oracle groups in the *GROUPS
section is the same as when using Oracle without RAC. For information on how to specify an OPENINFO
string for an Oracle group, refer to the Developing Applications with Oracle XA chapter in the Oracle Database Application Developer's Guide - Fundamentals.
TUXRACGROUPS="GROUP11,GROUP12,GROUP13;GROUP21,GROUP22:GROUP3;GROUP4, GROUP5"
The following figure shows an example of adding multiple groups to multiple instances.
This example is similar to the previous example — except that GROUP11
, GROUP12
, and GROUP13
are all associated with the first RAC instance of the first RAC configuration, and GROUP21
and GROUP22
are both associated with the second RAC instance.
If the first service call in a transaction in this configuration goes to GROUP12
, then it would be possible to send other service calls in this transaction to GROUP11
, GROUP12
, or GROUP13
, but not to GROUP21
or GROUP22
.
- the call fails
-
tperrno
is set toTPENOENT
-
tperrordetail
is set to the new value TPED_GROUP_FORBIDDEN
Figure 6-3 Multiple Oracle RAC Instances with Multiple Groups
For each RAC configuration defined as part of the TUXRACGROUPS
environment variable, Tuxedo determines which RAC group(s) in that configuration participate in a particular transaction when that transaction is started.
Parent topic: Configuring Transaction Propagation
6.6.4.1.3 Transaction Creation Behavior Using TUXRACGROUPS
Transactions are a pinned to Oracle RAC instances for as long as they exist. This is true independently, whether the call flow for such a transaction ever reaches a Tuxedo service associated with Oracle RAC or not.
There are two ways that transactions can be created:
- Transactions created in a group listed inside
TUXRACGROUPS
are pinned to the Oracle RAC instance configured viaTUXRACGROUPS
. - Transactions created in groups not listed inside
TUXRACGROUPS
are pinned to one of the available Oracle RAC instances in a load-balancing-like algorithm.
Parent topic: Configuring Transaction Propagation
6.6.4.1.4 Data Dependent Routing Using TUXRACGROUPS
Data dependent routing has been extended to support Oracle RAC
configurations. It is possible to define multiple groups for the
same routing range in the UBBCONFIG *ROUTING
section.
The following Listing shows an example of different Tuxedo groups
with the same range of values.
Listing Tuxedo Groups with Same Range Values
RANGES="1-5:GROUP1A, 1-5:GROUP1B, 6-10:GROUP2B, 6-10:GROUP2A, *:*"
In this example, GROUP1A
and GROUP1B
are responsible for the same data range and GROUP2A
and GROUP2B
are responsible for the same data range.
Tuxedo routes the service request to the group associated with the
Oracle RAC instance that the transaction belongs to.
Data dependent routing for transactional services offered in RAC groups achieves the desired result only if:
- Each Oracle RAC Instance configuration offers a service instance that can process each data value.
Since all but one of the instances in a RAC configuration are disallowed in a particular transaction, each data value must be specified for a service in each RAC instance. Otherwise, that data value will not be processed by any service in the RAC configuration for some transactions.
- Different service instances connected to the same Oracle RAC Instance process different data values.
If all data values are processed by the same set of service instances, then there is no need to use data dependent routing.
- Multiple
RANGES
entries for each routing value must be created for each RAC instance offering the service.If a routing was not configured for a special RAC instance a service calls for a transaction pinned to that Oracle RAC Instance will fail with
tperrno
set toTPENOENT
andtperror
detail set toTPED_GROUP_FORBIDDEN
.
When transactional routing occurs, any groups that are not permitted for the current transaction are ignored. The routing decision only considers:
- Groups associated with the allowable RAC instance.
- Groups not associated with a RAC configuration.
If routing is performed for a non-transactional request, all
groups can participate. The service is routed to the first group
matching the data value listed in the UBBCONFIG file
*ROUTING
section RANGES
field. All
non-transactional requests for a special range of values are
handled by one Oracle RAC instance only.
If routing is performed for a mixture of transactional and non-transactional requests, some applications may not require non-transactional request load balancing. You can vary the RAC instances listed first in your application for different data values so that non-transactional requests are balanced accordingly among services offered by different RAC instances.
There is no way to enforce load balancing between all groups associated with the same routing range for non-transactional requests. If you want to enforce one-by-one load balancing, try the following:
- Varying the RAC instance listed first for each data value so that each RAC instance occurs first for approximately equal amounts of data, or
- Calling an intermediate AUTOTRAN service (in the UBBCONFIG file
*SERVICES
section) to enforce that each service call is associated with a transaction.
Figure shows an example of routing transactional and non-transactional requests in an Oracle RAC configuration.
Figure 6-4 Routing Transactional/Non-Transactional Requests
The configuration shown in the example consists of 2 Oracle RAC
instances. If 1,000 transactions are created in a group not listed
in TUXRACGROUPS
, around 500 transactions will be
pinned to Oracle RAC instance 1 and can only access
GROUP1A
and GROUP2A
. The other 500
transactions will be pinned to Oracle RAC instance 2 and can only
access GROUP1B
and GROUP2B
.
The following Listing shows an example of how the
*SERVICES
and *ROUTING
sections of the
UBBCONFIG
file for this configuration might look:
Listing UBBCONFIG File *SERVICES and *ROUTING Sections Example
*SERVICES
DEPOSIT SRVGRP=GROUP1A ROUTING=MYROUTE
DEPOSIT SRVGRP=GROUP2A ROUTING=MYROUTE
DEPOSIT SRVGRP=GROUP1B ROUTING=MYROUTE
DEPOSIT SRVGRP=GROUP2B ROUTING=MYROUTE
*ROUTING
MYROUTE FIELD=”BRANCH_ID”
RANGES=”1-5:GROUP1A, 1-5:GROUP1B, 6-10:GROUP2B, 6-10:GROUP2A, *:*”
BUFTYPE=”FML32”
GROUP1A
and GROUP2A
belong to Oracle
RAC instance 1. GROUP1B
and GROUP2B
belong to Oracle RAC instance 2. Requests with a
BRANCH_ID
1 through 5 must be handled by
GROUP1A
or
. Requests
with a GROUP1B
BRANCH_ID
6 through 10 must be handled by
GROUP2A
or GROUP2B
.
For transactional requests, all transactions pinned to Oracle
RAC instance 1; branches 1-5 map to GROUP1A
and
branches 6-10 map to GROUP2A
. The other half is
assigned to Oracle RAC instance 2; branches 1-5 map to
GROUP1B
and branches 6-10 map to
GROUP2B
.
For non-transactional requests, branches 1-5 map to GROUP1A
, and branches 6-10 map to GROUP2B
. These are the first groups specified that match the respective routing ranges. Requests with an invalid BRANCH_ID
are mapped to any permitted group.
Note:
Oracle RAC instance 1 is specified first for one data range and RAC instance 2 is specified first for the other data range in an attempt to achieve some non-transactional load balancing between RAC instances.Parent topic: Configuring Transaction Propagation
6.6.4.1.5 Assigning Transactions to Special Oracle RAC Instances
You may want to split your environment into multiple machines. For example, you may want a Tuxedo domain with some machines only accessing Oracle RAC instance 1 and other machines only accessing Oracle RAC instance 2 in order to enforce regional independency if Tuxedo installations and Oracle RAC installations are distributed over different buildings. The environment may be configured so that as few as possible calls should be sent outside of a building.
The following Figure shows an example with, machine 1 serving
GROUP1A
and GROUP2A
; machine 2 serving
GROUP1B
and GROUP2B
. In addition, calls
might be made and transactions might be created from a Tuxedo
/Domain Gateway, for Tuxedo /WS clients, Tuxedo Native Clients,
Tuxedo /Q, or any server linked with another Resource Manager such
as MQ Series.
Whenever a request is sent, the transaction should be pinned to the local machine and avoid hopping between different machines as much as possible.
Figure 6-5 Assigning Transactions to Special Oracle RAC Instances
The following Listing shows a UBBCONFIG file example with two
physical machines, TUXM1
and TUXM2
,
running Tuxedo. Both machines have two groups connecting to an
Oracle RAC. Groups GROUP1A
and GROUP2A
are running on machine TUXM1
connecting to RAC
instance 1. Groups GROUP1B
and GROUP2B
are running on machine TUXM2
connecting to RAC
instance 2.
Listing UBBCONFIG File Example
*MACHINES
DEFAULT:
APPDIR="/path/to/appdir"
ENVFILE="/path/to/oracle.env"
TUXDIR="/path/to/tuxdir"
TUXCONFIG="/path/to/tuxconfig"
TLOGDEVICE="/path/to/TLOG"
"machine1" LMID=TUXM1
"machine2" LMID=TUXM2
*GROUPS
ADMGRPA LMID=TUXM1 GRPNO=10 OPENINFO=NONE
ADMGRPB LMID=TUXM2 GRPNO=20 OPENINFO=NONE
GROUP1A LMID=TUXM1 GRPNO=101 TMSNAME=TMS_ORA
OPENINFO="Oracle_XA:Oracle_XA+ACC=P/user/password+Sqlnet=ORA1SITE1+SesTm=1
00+LogDir=.+MaxCur=5"
GROUP1B LMID=TUXM2 GRPNO=102 TMSNAME=TMS_ORA
OPENINFO="Oracle_XA:Oracle_XA+ACC=P/user/password+Sqlnet=ORA1SITE2+SesTm=1
00+LogDir=.+MaxCur=5"
GROUP2A LMID=TUXM1 GRPNO=201 TMSNAME=TMS_ORA
OPENINFO="Oracle_XA:Oracle_XA+ACC=P/user/password+Sqlnet=ORA1SITE1+SesTm=1
00+LogDir=.+MaxCur=5"
GROUP2B LMID=TUXM2 GRPNO=202 TMSNAME=TMS_ORA
OPENINFO="Oracle_XA:Oracle_XA+ACC=P/user/password+Sqlnet=ORA1SITE2+SesTm=1
00+LogDir=.+MaxCur=5"
GROUP_TDOM_A LMID=TUXM1 GRPNO=301
GROUP_TDOM_B LMID=TUXM2 GRPNO=302
GROUP_CLIENT_A LMID=TUXM1 GRPNO=401 TMSNAME=TMS
GROUP_CLIENT_B LMID=TUXM2 GRPNO=402 TMSNAME=TMS
*SERVERS
DEFAULT: RESTART=Y MAXGEN=5 REPLYQ=Y CLOPT="-A"
TMSYSEVT SRVGRP="ADMGRPA" SRVID=10
TMUSREVT SRVGRP="ADMGRPA" SRVID=20
TMSYSEVT SRVGRP="ADMGRPB" SRVID=10 CLOPT="-A -- -S "
TMUSREVT SRVGRP="ADMGRPB" SRVID=20 CLOPT="-A -- -S "
EMPLOYEE_SVR SRVGRP=GROUP1A SRVID=1
EMPLOYEE_SVR SRVGRP=GROUP1B SRVID=2
BANKING_SVR SRVGRP=GROUP2A SRVID=3
BANKING_SVR SRVGRP=GROUP2B SRVID=4
DMADM SRVGRP="GROUP_TDOM_A" SRVID=100
GWADM SRVGRP="GROUP_TDOM_A" SRVID=110
GWTDOMAIN SRVGRP="GROUP_TDOM_A" SRVID=111 REPLYQ=Y RQADDR="GWGRP_M1"
GWADM SRVGRP="GROUP_TDOM_B" SRVID=110
GWTDOMAIN SRVGRP="GROUP_TDOM_B" SRVID=111 REPLYQ=Y RQADDR="GWGRP_M2"
Additionally, there is a group for administrative services, as
well as one group for Tuxedo /Domain gateways and one group for
native Tuxedo clients on both machines. All transactions are
created by GWTDOMAIN and native clients. Even if GWTDOMAIN and the
native Tuxedo clients never connect to an Oracle RAC directly, they
must be included in TUXRACGROUPS
as shown in the
following Listing to ensure that the opened transactions belong to
the correct RAC instance and are handled locally.
Note:
Native clients must settpinfo->grpname
to the local group to ensure the right behavior. For more information, see, "Avoiding Transactions Created by Tuxedo Native Clients Being Sent to a Remote Machine"Listing TUXGROUPS
TUXRACGROUPS="GROUP_TDOM_A,GROUP_CLIENT_A,GROUP1A,GROUP2A;GROUP_TDOM_B,
GROUP_CLIENT_B,GROUP1B,GROUP2B"
Parent topic: Configuring Transaction Propagation
6.6.4.1.6 TUXRAGROUPS Transaction Use Cases
- Dealing with Service Calls that are Made Outside of Transactions
- As long as no transaction is involved, Tuxedo will try to handle as many requests as possible on the local machine as long as the load allows and requests will only go to remote machines if no local services are idle according to the load balancing algorithm. Summarized this means one does not have to care about requests sent to remote machines if all services are available on all machines.
- Avoiding Transactions Created by a Group Handling an External Resource Manager Being Sent to a Remote Machine
- If you have a Tuxedo server built with another RM such as
MQSeries or another database, you can force newly started
transactions to be pinned to your local machine by including this
group into the
TUXRACGROUPS
environment variable as well. - Avoiding Transactions Created by GWTDOMAIN Being Sent to a Remote Machine?
- Create on local Tuxedo /Domain Gateway on each machine. Set the
TUXRACGROUPS
environment variable as shown in the following Listing . - Avoiding Transactions created by TMQFORWARD Being Sent to a Remote Machine
- Create a local Tuxedo /Q configuration on each machine. Set
your
TUXRACGROUPS
environment variable as shown in the following Listing. - Avoiding Transactions Created by Tuxedo Native Clients Being Sent to a Remote Machine
- You can also bind native clients to a special server group. You just have to build the client using the command
buildclient -r <RM_of_the_group> -f <source_file> -o <binary_file>
and initiatetpinit()
with the group name that you want to use. - Avoiding Sending Transactions Created by Tuxedo /WS Clients to Remote Machines
- The
grpname
value must be the NULL string(0-length string)
for Workstation clients. You cannot set any group name and you cannot pin /WS clients to special groups.tpbegin()
inside the Tuxedo /WS clients is always unspecified and the opened transaction is distributed in equal parts over all RAC instances.
Parent topic: Configuring Transaction Propagation
6.6.4.2 Configuring Transaction Recovery
TMS_rac_refresh(1)
,
XARETRYDURATIONSECONDS
, and
XARETRYINTERVAL
specifically handle transaction
recovery issues.
TMS_rac_refresh(1)
is called when an Oracle RAC
group fails over to an alternate group.
TMS_rac_refresh(1)
must not be executed
manually from the command line; the proper way to invoke
TMS_rac_refresh(1)
is to use Oracle Fast Application
Notification (FAN).
Note:
For more details on configuring Oracle FAN, refer to Oracle 10g documentation.The XARETRYDURATIONSECONDS
and
XARETRYINTERVAL
environment variables are used to
retry transaction recovery operations (xa_recover()
)
as required by Oracle RAC.
- XARETRYDURATIONSECONDS
- Specifies the time interval during which the Tuxedo Transaction
Manager Server (TMS) retries
xa_recover()
operations whenTMS_rac_refresh(1)
is called. If it is not set or set to 0, thenxa_recover()
is performed once only. - XARETRYINTERVAL
- Specifies the interval in seconds that
xa_recover()
operations are retried during theXARETRYDURATIONSECONDS
interval. TheXARETRYINTERVAL
value is relevant only ifXARETRYDURATIONSECONDS
is set to a value greater than 0.
- Configuring Oracle 10g Fast Application Notification (FAN)
- Configuring Transaction Recovery for Oracle 10gR2
- Configuring Transaction Recovery for Oracle 12c and Above
- Specifying Environment Variables in the UBBCONFIG File
Parent topic: Configuring Tuxedo for Oracle RAC
6.6.4.2.1 Configuring Oracle 10g Fast Application Notification (FAN)
A key process in configuring Tuxedo for Oracle RAC is setting up Oracle FAN to invoke TMS_rac_refresh(1)
with the appropriate group parameter on group failover. (More group parameter and group failover information is provided in Configuring Transaction Propagation.)
More information regarding Oracle FAN can be found in the Workload Management with Oracle Real Application Clusters (PDF) White Paper
Parent topic: Configuring Transaction Recovery
6.6.4.2.2 Configuring Transaction Recovery for Oracle 10gR2
For Oracle 10gR2, it is much simpler to configure transaction recovery. The database services specified in the OPENINFO
string for each group associated with Oracle RAC must be declared in Oracle as DTP services.
For example, in the following Listing, GROUP1
accessed Oracle via service ORA1SITE1
and GROUP2
accessed Oracle via service ORA1SITE2
. In Oracle 10gR2, service ORA1SITE1
must be declared with DTP=TRUE
, with preferred instance SITE1
, and with available instance SITE2
. Service ORA1SITE2
must be declared with DTP=TRUE
, with preferred instance SITE2
, and with available instance SITE1
. A similar process must be followed for groups GROUP3
, GROUP4
, and GROUP5
.
By declaring different preferred instances, the application will be able to get the benefit of load balancing during normal operation when both instances are available.
The setting of the TUXRACGROUPS
environment variable ensures that different instances of the RAC configuration are not combined in the same transaction in order to obtain optimal performance. If one of the RAC instances goes down, Oracle transfers the DTP service to the non-preferred instance while maintaining transactional integrity.
When using Oracle 10gR2 DTP services, it is not necessary and is not recommended to configure Oracle FAN, use TMS_rac_refresh(1)
or set the XARETRYDURATIONSECONDS
or XARETRYINTERVAL
environment variables.
Parent topic: Configuring Transaction Recovery
6.6.4.2.3 Configuring Transaction Recovery for Oracle 12c and Above
For Oracle Database release 12c and above, no specific configuration is required; transaction recovery is transparent.
Parent topic: Configuring Transaction Recovery
6.6.4.2.4 Specifying Environment Variables in the UBBCONFIG File
Although the Tuxedo Oracle RAC environment variables can be
initiated at the operating system command line, it is highly
recommended that you use the ENVFILE
parameter
specified in the *MACHINES
section of the
UBBCONFIG
file to initiate these environment
variables.
Apply the following syntax considerations when setting the environment variables for Oracle RAC.
- When Tuxedo environment variables are set using
ENVFILE,
which is the preferred method, quotation marks are not permitted around the environment variable value. - If environment variables are set at the command line, quotation marks are required if environment variable values contain characters that could be interpreted as special by the command line interpreter. An example of a special character is a semicolon.
- Ensure that the Tuxedo Oracle RAC environment variables are set consistently on all nodes in a RAC configuration.
See Also:
buildtms(1)
in Section 1 - CommandsUBBCONFIG(5)
in Section 5 - File Formats, Data Descriptions, MIBs, and System Processes Reference- About Transactions
- Configuring Your ATMI Application to Use Transactions
- Writing Global Transactions in Programming an Oracle Tuxedo ATMI Application Using C
- Oracle Real Application Clusters Home Page
- Oracle Application Server Adapters for Tuxedo
- Best Practices for Using XA with RAC
Parent topic: Configuring Transaction Recovery