Oracle Tuxedo/Oracle Exalogic Users Guide

     Previous  Next    Open TOC in new window    View as PDF - New Window  Get Adobe Reader - New Window
Content starts here

Oracle Tuxedo/Oracle Exalogic
Users Guide

This chapter contains the following topics:

 


Overview

This section contains the following topics

Scope

This document will introduce all special features for Exalogic platform, and describe the instruction on how to configure and run Oracle Tuxedo with these features.

Prerequisites

Before install Oracle Tuxedo, ensure set Oracle Exalogic machine properly that include storage and networking. For more information, see Oracle Exalogic Machine Owner's Guide.

Terminology

Feature Summary

Table 1 lists the Exalogic supported Oracle Tuxedo features.

Table 1 Exalogic Supported Oracle Tuxedo Features
Feature Name
Oracle Tuxedo Version
Direct Cross Node Communication Leveraging RDMA
Oracle Tuxedo 11gR1 (11.1.1.3.0)
Self-tuning Lock Mechanism
Oracle Tuxedo 11gR1 (11.1.1.3.0)
Oracle Tuxedo SDP Support
Oracle Tuxedo 11gR1 (11.1.1.3.0)

Direct Cross Node Communication Leveraging RDMA

In previous releases, messages between local client and remote server must go through bridge. For example, first the message will be sent to local bridge through IPC queue, next the local bridge sends it to remote bridge through network, then the remote bridge sends the message to server’s IPC queue, finally the server retrieves the message from its IPC queue, so the Bridge will become a bottleneck under high concurrency. In this release, if Direct Cross Node Communication Leveraging RDMA is enabled in the TUXCONFIG file, the local client and remote server can skip Bridge, and transfer message directly.

Self-Tuning Lock Mechanism

In previous Oracle Tuxedo Release, the tuxedo process will try to lock the bulletin board at user level for SPINCOUNT times before blocking on a UNIX semaphore. Typically it can get high performance if a proper SPINCOUNT is specified to make the process getting the lock at user level. Different load in an Oracle Tuxedo application may need different proper SPINCOUNT that the user must find it himself. In this release if Self-tuning Lock Mechanism is enabled, the Oracle Tuxedo will find the proper SPINCOUNT automatically under kinds of load at run time.

Oracle Tuxedo SDP Support

This feature enables Oracle Tuxedo component those are using BSD Socket APIs can leverage the advantages of SDP network protocol provided by Exalogic (such as high bandwidth, low latency as well as reduced CPU involvement).

 


Oracle Tuxedo installation

This section contains the following topics:

Configuration for Exalogic

Before the Oracle Tuxedo installation, you should understand the current state of the Exalogic environment.

It is assumed that you have completed all tasks described in the Oracle Exalogic Machine Owner's Guide, which discusses your data center site preparation, Oracle Exalogic machine commissioning, initial networking configuration including IP address assignments, and initial setup of the Sun ZFS Storage 7320 appliance.

This section contains the following topics:

Network

Run the Oracle OneCommand tool to complete the following tasks (as described in "Initial Configuration of an Exalogic Machine Using Oracle OneCommand" in the Oracle Exalogic Machine Owner's Guide).

Storage

The initial configuration of the Sun ZFS Storage 7320 appliance on your Oracle Exalogic machine is completed at the time of manufacturing. For more information about default shares (Exported File Systems), see the “Default Storage Configuration" section in the Oracle Exalogic Machine Owner's Guide.

After completing this initial configuration, you can proceed to create custom shares as needed.

Choosing Oracle Tuxedo Home

We recommend that you can install the Oracle Tuxedo product binaries in one of the shares on Sun ZFS Storage 7320 appliance locations, so you can run Oracle Tuxedo on any Exalogic nodes by one binary copy.

Notes: The share, which is a shared file system, must be accessible by all compute nodes. You can create a local user account for each node, and ensure it has the same uid and gid (to avoid permission access issues), or create NIS accounts for users.
Note: Oracle Tuxedo must be installed in a different directory if you want to develop the Oracle Tuxedo plug-in interface with different implementation for each Oracle Tuxedo.

Start installation

The installation procedure is the same as previous Oracle Tuxedo releases. For more information, see Installing the Oracle Tuxedo System.

Console-Mode Installation

The following is a sample for Console-Mode installation on Exalogic, assume we will install Oracle Tuxedo on the node named slce04cn01, and the shared directory "/home/oracle" is available.

  1. Log in to slce04cn01 as the Oracle Tuxedo administrator.
  2. Go to the directory where you downloaded the installer and invoke the installation procedure by entering the following command:

    prompt>sh./tuxedo111130_64_Linux_01_x86.bin-i console

    The Choose Locale screen is displayed.

  3. In the Choose Locale screen, enter 1, which is associated with English.
  4. The Introduction screen is displayed.

  5. In the Introduction screen, press <ENTER> to continue.
  6. The Choose Install Set screen is displayed.

  7. In the Choose Install Set screen, enter 1, which is associated with Full Install.
  8. The Choose Oracle Home screen is displayed.

  9. In the Choose Oracle Home screen, enter 1, which is associated with Create new Oracle Home.
The Specify a new Oracle Home directory screen is prompted.
  1. Enter your Oracle Home directory. Be sure to use the full pathname when specifying the new Oracle Home directory.
  2. The Oracle Home should be on the shared file system on the Sun Storage 7000 Unified Storage System, and can be accessible by all compute nodes in the Oracle Exalogic machine.

    For example, enter /home/oracle as the Tuxedo home directory.

    The Choose Product Directory screen is displayed.

  3. In the Choose Product Directory screen, enter 2, which is associated with Use Current Selection.
  4. The Install Samples (Y/N) is prompted.

  5. Enter Y to install the samples.
  6. The Pre- Installation Summary screen is displayed.

  7. In the Pre- Installation Summary screen, press <ENTER> to continue.
  8. The Ready To Install screen is displayed.

  9. In the Ready To Install screen, press <ENTER> to install.
  10. The Installing screen is displayed.

  11. In the Installing screen, no user input is required.
  12. When it finishes, the Configure tlisten Service screen is displayed.

  13. In the Configure tlisten Service screen, enter a tlisten password of your choice. Your password must be a string of alphanumeric characters in clear-text format that is no more than 80 characters in length. Then Verify your password.
  14. Note: A common password is required for all the nodes in an Oracle Tuxedo application (domain) to communicate successfully. For this reason, you must use the same password when you install Oracle Tuxedo on more than one compute nodes for a single application.

    The SSL Installation Choice screen is displayed.
  15. In the SSL Installation Choice screen, you can choose to enter 1, which is associated with YES (This is not mandatory for the installation).
  16. The Enter Your LDAP Settings for SSL Support screen is displayed.

  17. In the Enter Your LDAP Settings for SSL Support screen, input your LDAP Service Name, LDAP PortID, LDAP BaseObject and LDAP Filter File Location (this is not mandatory for the installation).
  18. The Installation Complete screen is displayed.

  19. In the Installation Complete screen, press <ENTER> to exit the installer.

 


Oracle Tuxedo Configuration

This section introduces the basic Oracle Tuxedo feature configuration on Exalogic. For more information, see the Oracle Tuxedo 11g Release 1 (11.1.1.3.0) Release Notes and Setting Up an Oracle Tuxedo Application.

Direct Cross Node Communication Leveraging RDMA

Using Direct Cross Node Communication Leveraging RDMA requires UBBCONFIG file configuration.

UBBCONFIG File

Direct Cross Node Communication Leveraging RDMA is only supported under MP mode. To enable this feature, you must specify both EXALOGIC and RDMA in OPTIONS, otherwise the message goes through the Bridge.

There are four attributes for Direct Cross Node Communication Leveraging RDMA in the *MACHINES section.

After enable RDMA option in the *RESOURCES section. Attribute “TYPE” of *MACHINES section cannot be set, since by default, any machines in MP mode should be Exalogic machine (with the same type) to support RDMA feature.

You can also get/change the configuration via TM_MIB. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.

Listing 1 shows a example UBBCONFIG File with Direct Cross Node Communication Leveraging RDMA enabled.

Listing 1 UBCONFIG File Example with Direct Cross Node Communication Leveraging RDMA Enabled
*RESOURCES
IPCKEY		87654

MASTER site1,site2
MAXACCESSERS		40
MAXSERVERS		40
MAXSERVICES		40
MODEL		MP
OPTIONS		LAN,EXALOGIC,RDMA
LDBAL		Y

*MACHINES
slce04cn01		LMID=site1
		APPDIR="/home/oracle/tuxedo11gR1/samples/atmi/simpapp"
              TUXCONFIG="/home/oracle/tuxedo11gR1/samples/atmi/simpapp/tuxconfig"
		TUXDIR="/home/oracle/tuxedo11gR1"
		UID=601
		GID=601
      RDMADAEMONIP=”192.168.10.1”
      RDMADAEMONPORT=9800
      RDMAQSIZE=65536
      RDMAQENTRIES=64

slce04cn02 LMID=site2
		         APPDIR="/home/oracle/tuxedo11gR1/samples/atmi/simpapp/slave"
		              TUXCONFIG="/home/oracle/tuxedo11gR1/samples/atmi/simpapp/slave/tuxconfig"
		TUXDIR="/home/oracle/tuxedo11gR1"
		UID=601
		GID=601
      RDMADAEMONIP=”192.168.10.2”
      RDMADAEMONPORT=9800
      RDMAQSIZE=65536
      RDMAQENTRIES=64

*GROUPS
GROUP1
	LMID=site1		GRPNO=1		OPENINFO=NONE
GROUP2
	LMID=site2		GRPNO=2		OPENINFO=NONE

*NETWORK
site1	NADDR="//slce04cn01:5432"
	NLSADDR="//slce04cn01:5442"

site1 NADDR="//slce04cn02:5432"
	NLSADDR="//slce04cn02:5442"

*SERVERS
DEFAULT:
		CLOPT="-A"

simpserv SRVGRP=GROUP2 SRVID=3

*SERVICES
TOUPPER

Self-Tuning Lock Mechanism

To enable Self-tuning Lock Mechanism, the options EXALOGIC and SPINTUNING must be specified in the UBBCONFIG file *RESOURCE section. Whenever the feature enabled, SPINCOUNT in *MACHINE cannot be set.

Two other optional attributes are added in *MACHINES section:

For more information, see UBBCONFIG(5) and UBBCONFIG(5) Additional Information, Example 2 Self-Tuning Lock Mechanism Configuration, in File Formats, Data Descriptions, MIBs, and System Processes Reference.

You can also set the configuration via TM_MIB.For more information, see TM_MIB(5) in File Formats, Data Descriptions, MIBs, and System Processes Reference.

Listing 2 shows a UBBCONFIG file example with Self-Tuning Lock Mechanism enabled.

Listing 2 UBBCONFIG File Example with Self-Tuning Lock Mechanism Enabled
*RESOURCES
IPCKEY                 123456
DOMAINID           simpapp
MASTER                ALLEN
MAXACCESSERS 10
MAXSERVERS      5
MAXSERVICES     10
MODEL          SHM
LDBAL           N
OPTIONS         EXALOGIC,SPINTUNING

*MACHINES
ALLENHOST       LMID="ALLEN"
APPDIR="/home/allen/Workspace/Tuxedo10/simpdir"
TUXCONFIG="/home/allen/Workspace/Tuxedo10/simpdir/tuxconfig"
TUXDIR="/home/allen/Software/OraHome/tuxedo11gR1"
SPINTUNING_FACTOR=1000
SPINTUNING_MINIDLECPU=20
*GROUPS
GROUP1
LMID=ALLEN GRPNO=1 OPENINFO=NONE

*SERVERS
DEFAULT:
CLOPT="-A"
simpserv   SRVGRP=GROUP1  SRVID=1

*SERVICES
TOUPPER

Oracle Tuxedo SDP Support

To enable Oracle Tuxedo SDP Support, you must specify EXALOGIC for OPTIONS in * RESOURCE section, and set the relevant configuration in UBBCONFIG file or DMCONFIG file.

You can also get/change the configuration via TM_MIB. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.

This section covers the following configurations:

MP

According to the requirements, MP should work inside IB clusters, i.e., both master and slave machines are inside IB cluster, so only consider SDP and IPoIB are used inside IB cluster, and in bootstrap phase, tmboot, tlisten, bsbridge and bridge are using Socket API to communicate with each other.

GWTDOMAIN

If the node running GWTDOMAIN has multiple network interfaces (multi-homed) with multiple IP addresses, it’s better to use explicit IP address when configuring GWTDOMAIN in DMCONFIG file instead of host name. Typically, every Exalogic node has at least two types of network interface, i.e., IB interface and Ethernet interface, in order to facilitate to demonstrate how to configure GWTDOMAIN, just presume IB interface is bound to IP address IB_IP, and Ethernet interface with IP address ETH_IP.

Functionally, GWTDOMAIN acts as both server and client in role, as server, it will listen on a configured IP address and port number in the DMCONFIG file to accept connection request from other GWTDOMAIN, as client, it will initiate connection request to other GWTDOMAIN by policy configured in the DMCONFIG file.

WSL

/WS client

JSL

WTC

To enable SDP connection between WTC and Oracle Tuxedo, do the following steps:

  1. Specify the NWAddr of the WTC service Local/Remote Access Points as follows:
  2. sdp://IB_IP:port

    It is the same as the GWTDOMAIN NWADDR configuration in the DMCONFIG file.

  3. Add additional Java Option “-Djava.net.preferIPv4Stack=true” to the java command line to start up WLS server.
Notes: If the WTC access point has SSL enabled, after configuring for the SDP, the SSL configuration is ignored.
Note: Only Weblogic Server 12c (12.1.1) and higher can connect to Oracle Tuxedo via SDP. For more information, see Enable IPv4 for SDP transport, NWAddr attribute for WTC local Tuxedo Domain configuration, and NWAddr attribute for WTC remote Tuxedo Domain configuration.

Database

The Oracle Tuxedo system uses the X/Open XA interface for communicating with the various resource managers. The XA Standard is widely supported in all the major database vendor products.

You can use SDP (Sockets Direct Protocol) for Oracle Database invocations. There is no special requirement for Oracle Tuxedo application.

Please configure the database to support InfiniBand, as described in Configuring SDP Protocol Support for Infiniband Network Communication to the Database Server in the Oracle Database Net Services Administrators Guide.

Note: The following SDP parameters affect performance when sending large data blocks to the database.

For example, you can set the two parameters in “/etc/modprobe.conf” on the server node as follows:

options ib_sdp sdp_zcopy_thresh=0 recv_poll=0

Oracle Tuxedo File Transfer Default Directory Name

Before starting Oracle Tuxedo, ensure that there is a shared directory for all Exalogic nodes when Direct Cross Node Communication Leveraging RDMA is enabled. Make sure that access permissions are properly set.

The default name is /u01/common/patches/tuxtmpfile, you can also set your own directory using the EXALOGIC_SHARED_PATH environment variable. It is used for Oracle Tuxedo file transfer. When the EMSQ is full, or the message size exceeds the queue size, Oracle Tuxedo puts this message into a temporary file under the /u01/common/patches/tuxtmpfile directory, and sends notification directly to the remote process queue. The remote process can then get the file as long as it receives the notification.

Choosing APPDIR

If the SECURITY is not set in the UBBCONFIG file, you can deploy your Oracle Tuxedo application to a shared directory. Before booting the Oracle Tuxedo application, ensure the following parameters are set correctly in the UBBCONFIG file:

Listing 13 shows a UBBCONFIG file shared APPDIR example.

Listing 13 UBBCONFIG File Shared APPDIR
*MACHINES
slce04cn01 LMID=site1
          APPDIR="/home/oracle/tuxapp"
          TUXCONFIG="/home/oracle/tuxapp/tuxconfig_cn01"
          TUXDIR="/home/oracle/tuxedo11gR1"
ULOGPFX="/ home/oracle/tuxapp /ULOG_cn01"
          RDMADAEMONIP="192.168.10.1"
          RDMADAEMONPORT=9800
          RDMAQSIZE=1048576
          RDMAQENTRIES=1024

slce04cn02 LMID=site2
          APPDIR=" home/oracle/tuxapp"
          TUXCONFIG=" home/oracle/tuxapp/tuxconfig_cn02"
          TUXDIR="/home/oracle/tuxedo11gR1"
ULOGPFX="/home/oracle/tuxapp /ULOG_cn02"
          RDMADAEMONIP="192.168.10.2"
          RDMADAEMONPORT=9800
          RDMAQSIZE=1048576
          RDMAQENTRIES=1024

If the SECUTIRY is set in UBBCONFIG file, you must configure a different APPDIR for your application. You must have a copy for each node.

limits.conf

Set the memlock in /etc/securitylimits.conf properly. Use the following formula to get the minimum value:

[Msgq_daemon shared memory size]*2 + MAXACCESSERS *14 000 kb

Msgq_daemon shared memory size: The size of shared memory allocated by Msgq_daemon. For more information, see Start/Stop tux_msgq_monitor

MAXACCESSERS: An attribute in the UBBCONFIG file.

For example:

Msgq_daemon shared memory size: 200*1024*1024 bytes

MAXACCESSERS: 100

The minimum memlock value is 1853030400 bytes.

In /etc/security/limits.conf, set memlock las follows:

* hard memlock 1853030

* soft memlock 1853030

EXALOGIC_MSGQ_CACHE_SIZE

Each Oracle Tuxedo thread has an EMSQ runtime cache; the default entry number is 32. You can change it between 32 and 2048 using the environment EXALOGIC_MSGQ_CACHE_SIZE variable before the Oracle Tuxedo application starts. Increasing the number can improve Oracle Tuxedo performance, but Msgq_daemon consumes more shared memory.

Performance can be improved using this environment variable when one process sends messages to many queues. Note the following examples:

 


Running Oracle Tuxedo

There is a difference running Oracle Tuxedo on a non-Exalogic platform if Direct Cross Node Communication Leveraging RDMA is enabled. The tux_msgq_monitor must be started before booting an Oracle Tuxedo application. This section includes the following topics:

Start/Stop tux_msgq_monitor

Assistant Tools

Shell scripts for start/stop Oracle Tuxedo

There are some shell scripts that simplify the startup/shutdown procedure. Using these tools, you can only run one command to start/stop both tux_msgq_monitor and an Oracle Tuxedo application. Before running these commands, ensure the environment variables TUXCONFIG, LD_LIBRARY_PATH and APPDIR are set properly.

For example, on the master node you can start/stop Oracle Tuxedo as follows:

On slave node, there are two shell scripts:

Tool for Computing the Shared Memory Size

When Msgq_daemon starts, it allocates a shared memory. You can get the memory size by using tmloadcf -c ubb, as shown in Listing 14.

Listing 14 UBBCONFIG File *MACHINES Section
*MACHINES
ex03	LMID=site1

      RDMADAEMONIP="192.168.10.1"
      RDMADAEMONPORT=9800
      RDMAQSIZE=100000
      RDMAQENTRIES=100
MAXACCESSERS=100

ex03_1 LMID=site2

      RDMADAEMONIP="192.168.10.2"
      RDMADAEMONPORT=9800
      RDMAQENTRIES=1000
MAXACCESSERS=200

ex04 LMID=site3
      RDMADAEMONIP="192.168.10.3"
      RDMADAEMONPORT=9800
      RDMAQSIZE=100000
      RDMAQENTRIES=100
MAXACCESSERS=200
MAXSERVERS=100

ex04_1 LMID=site4

      RDMADAEMONIP="192.168.10.4"
      RDMADAEMONPORT=9800
      RDMAQSIZE=1000000
      RDMAQENTRIES=1000
MAXACCESSERS=100

Run command tmloadcf -c ubb, get the output shown in Listing 15

Listing 15 tmloadcf -c ubb Output Example
Ipc sizing (minimum /T values only) ...
                  Fixed Minimums Per Node
SHMMIN: 1
SHMALL: 1
SEMMAP: SEMMNI
                  Variable Minimums Per Node
SEMUME,           A                             SHMMAX
                        SEMMNU,           *                               *



Node SEMMNS SEMMSL SEMMSL SEMMNI MSGMNI MSGMAP SHMSEG RCDMSZ
-----       -----  -----    ----   -------  -----   ------  ------  ------
ex03        126      15 120 A + 2 26 52 1178K 220M
ex04        221      28     220   A + 1      26      52   1340K    340M
ex04_1      121      15     120   A + 1      26      52   1178K   1300M
ex03_1      221      28     220   A + 1      25      50   1340K   2500M

RCDMSZ increases linearly when any of following items configured in UBB increases:

Note: You can adjust the size using this method only when setting EXALOGIC_MSGQ_CACHE_SIZE at the machine level.

 


Running Oracle Tuxedo on OVM

There are no special requirements for Oracle Tuxedo running on OVM.

 


SALT and TMA on Exalogic

Before installing SALT 11gR1 11.1.1.2.0 and TMA 11gR1 11.1.1.2.0 on Exalogic, you must run preinstall first, as follows.

preinstall <Oracle Home>
<Oracle Home>: The directory you specified when Oracle Tuxedo installation.

After the installation, run postinstall, for example:

postinstall <Oracle Home >
<Oracle Home>: The directory you specified when Oracle Tuxedo installation.

 


Migration

There is no special requirements if you do not use the Exalogic features. For more information, see Oracle Tuxedo Interoperability Guide.

Note: Oracle Tuxedo 11g Release 1 (11.1.1.3.0) does not support hot upgrading from previous Oracle Tuxedo releases if any of the Exalogic features are enabled.

  Back to Top       Previous  Next