![]() |
![]() |
![]() |
![]() |
![]() |
•
• Before install Oracle Tuxedo, ensure set Oracle Exalogic machine properly that include storage and networking. For more information, see Oracle Exalogic Machine Owner's Guide.
• Oracle Home: An Oracle Home directory serves as the central support directory for all Oracle products installed on the target system.
• Compute Node: A physical machine in an Exalogic rack that is meant for running an Oracle Fusion Middleware instance at any given time.
• Shared Storage: Shared storage refers to the Sun ZFS Storage 7320 appliance that is accessible by all compute nodes in the Oracle Exalogic Machine. All compute nodes in the Exalogic machine can access this storage appliance simultaneously for both read and write operations.
• UBBCONFIG: Each Oracle Tuxedo domain is controlled by a configuration file in which installation-dependent parameters are defined. The text version of the configuration file is referred to as UBBCONFIG. The UBBCONFIG file for a Oracle Tuxedo domain contains all the information necessary to boot the application.
• TUXCONFIG: The TUXCONFIG file is a binary version of the UBBCONFIG file. It is created by running the tmloadcf(1) command, which parses UBBCONFIG and loads the binary TUXCONFIG file to the location referenced by the TUXCONFIG environment variable.
• XA: The eXtended Architecture (XA) is the standard-defined interface between the Transaction Manager and the Resource Manager. XA allows program control of Resource Managers that are involved in distributed transactions
• InfiniBand: It is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable.
• EOIB: Ethernet over InfiniBand.Table 1 lists the Exalogic supported Oracle Tuxedo features.
Oracle Tuxedo 11gR1 (11.1.1.3.0) Oracle Tuxedo 11gR1 (11.1.1.3.0) Oracle Tuxedo 11gR1 (11.1.1.3.0)
•
• Run the Oracle OneCommand tool to complete the following tasks (as described in "Initial Configuration of an Exalogic Machine Using Oracle OneCommand" in the Oracle Exalogic Machine Owner's Guide).
• The initial configuration of the Sun ZFS Storage 7320 appliance on your Oracle Exalogic machine is completed at the time of manufacturing. For more information about default shares (Exported File Systems), see the “Default Storage Configuration" section in the Oracle Exalogic Machine Owner's Guide.The installation procedure is the same as previous Oracle Tuxedo releases. For more information, see Installing the Oracle Tuxedo System.The following is a sample for Console-Mode installation on Exalogic, assume we will install Oracle Tuxedo on the node named slce04cn01, and the shared directory "/home/oracle" is available.
1. Log in to slce04cn01 as the Oracle Tuxedo administrator.
3. In the Introduction screen, press <ENTER> to continue.For example, enter /home/oracle as the Tuxedo home directory.
15. In the Installation Complete screen, press <ENTER> to exit the installer.This section introduces the basic Oracle Tuxedo feature configuration on Exalogic. For more information, see the Oracle Tuxedo 11g Release 1 (11.1.1.3.0) Release Notes and Setting Up an Oracle Tuxedo Application.
• Direct Cross Node Communication Leveraging RDMA is only supported under MP mode. To enable this feature, you must specify both EXALOGIC and RDMA in OPTIONS, otherwise the message goes through the Bridge.There are four attributes for Direct Cross Node Communication Leveraging RDMA in the *MACHINES section.The IP address where the Msgq_daemon is bound. It must be configured, and must be an IPoIB address (not an Ethernet based IP address). You should configure one Msgq_daemon for one logic machine.The port number where Msgq_daemon listens on. It must be configured.After enable RDMA option in the *RESOURCES section. Attribute “TYPE” of *MACHINES section cannot be set, since by default, any machines in MP mode should be Exalogic machine (with the same type) to support RDMA feature.You can also get/change the configuration via TM_MIB. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.Listing 1 shows a example UBBCONFIG File with Direct Cross Node Communication Leveraging RDMA enabled.MODEL MPOPTIONS LAN,EXALOGIC,RDMATo enable Self-tuning Lock Mechanism, the options EXALOGIC and SPINTUNING must be specified in the UBBCONFIG file *RESOURCE section. Whenever the feature enabled, SPINCOUNT in *MACHINE cannot be set.Two other optional attributes are added in *MACHINES section:
• SPINTUNING_FACTOR: Controls the tuning target. More CPUs may be used with a larger value under a heavy load; higher performance may be gained.
• SPINTUNING_MINIDLECPU: Specifies the CPU idle time.For more information, see UBBCONFIG(5) and UBBCONFIG(5) Additional Information, Example 2 Self-Tuning Lock Mechanism Configuration, in File Formats, Data Descriptions, MIBs, and System Processes Reference.You can also set the configuration via TM_MIB.For more information, see TM_MIB(5) in File Formats, Data Descriptions, MIBs, and System Processes Reference.Listing 2 shows a UBBCONFIG file example with Self-Tuning Lock Mechanism enabled.OPTIONS EXALOGIC,SPINTUNINGTo enable Oracle Tuxedo SDP Support, you must specify EXALOGIC for OPTIONS in * RESOURCE section, and set the relevant configuration in UBBCONFIG file or DMCONFIG file.You can also get/change the configuration via TM_MIB. For more information, see File Formats, Data Descriptions, MIBs, and System Processes Reference.
•
•
•
• According to the requirements, MP should work inside IB clusters, i.e., both master and slave machines are inside IB cluster, so only consider SDP and IPoIB are used inside IB cluster, and in bootstrap phase, tmboot, tlisten, bsbridge and bridge are using Socket API to communicate with each other.Prefixed “sdp:” to the network address, and the network address must be an IPoIB address. Listing 3 shows a UBBCONFIG file SDP example.Listing 3 UBBCONFIG File SDP Configuration ExampleSITE1
NADDR="sdp://IB_IP: 9003"
NLSADDR="sdp://IB_IP: 3050”
SITE2
NADDR="sdp://IB_IP: 9003"
NLSADDR="sdp://IB_IP: 3050"Using the IPoIB address as the network address.
Listing 4 shows a UBBCONFIG file IPoIB configuration example.Prefixed “sdp:” to the network address, and the network address must be an IPoIB address, as shown in Listing 5.Prefixed “sdp:” to the network address, and the network address must be an IPoIB address as shown in Listing 6.Prefixed “sdp:” to the network address, and the network address must be an IPoIB address as shown in Listing 9.Prefixed “sdp:” to network address for WSNADDR, and the network address must be an IPoIB address, for example:Using the IPoIB address as the network address for WSNADDR, for example:Prefixed “sdp:” to the network address, and the network address must be an IPoIB address as shown in Listing 11.It is the same as the GWTDOMAIN NWADDR configuration in the DMCONFIG file.
2. Add additional Java Option “-Djava.net.preferIPv4Stack=true” to the java command line to start up WLS server.Only Weblogic Server 12c (12.1.1) and higher can connect to Oracle Tuxedo via SDP. For more information, seeEnable IPv4 for SDP transport, NWAddr attribute for WTC local Tuxedo Domain configuration, and NWAddr attribute for WTC remote Tuxedo Domain configuration.Please configure the database to support InfiniBand, as described in Configuring SDP Protocol Support for Infiniband Network Communication to the Database Server in the Oracle Database Net Services Administrators Guide.
Note:
• recv_poll
The receiving poll time before going to sleep waiting for data. You can improve the write performance by decreasing the recv_poll value. It is recommended to set it to zero.
• sdp_zcopy_thresh
The threshold for zero copy sending data. If the messages are longer than the sdp_zcopy_thresh value, it causes the user buffer to be pinned and the data sent directly from the original buffer. This results in decreased CPU use. The default value is 64K; it is recommended to set it to zero.For example, you can set the two parameters in “/etc/modprobe.conf” on the server node as follows:options ib_sdp sdp_zcopy_thresh=0 recv_poll=0The default name is /u01/common/patches/tuxtmpfile, you can also set your own directory using the EXALOGIC_SHARED_PATH environment variable. It is used for Oracle Tuxedo file transfer. When the EMSQ is full, or the message size exceeds the queue size, Oracle Tuxedo puts this message into a temporary file under the /u01/common/patches/tuxtmpfile directory, and sends notification directly to the remote process queue. The remote process can then get the file as long as it receives the notification.
• Listing 13 shows a UBBCONFIG file shared APPDIR example.Listing 13 UBBCONFIG File Shared APPDIRSet the memlock in /etc/securitylimits.conf properly. Use the following formula to get the minimum value:[Msgq_daemon shared memory size]*2 + MAXACCESSERS *14 000 kbMsgq_daemon shared memory size: The size of shared memory allocated by Msgq_daemon. For more information, see Start/Stop tux_msgq_monitorMAXACCESSERS: An attribute in the UBBCONFIG file.Msgq_daemon shared memory size: 200*1024*1024 bytesThe minimum memlock value is 1853030400 bytes.Each Oracle Tuxedo thread has an EMSQ runtime cache; the default entry number is 32. You can change it between 32 and 2048 using the environment EXALOGIC_MSGQ_CACHE_SIZE variable before the Oracle Tuxedo application starts. Increasing the number can improve Oracle Tuxedo performance, but Msgq_daemon consumes more shared memory.
• If there are 40 remote Oracle Tuxedo servers providing the same service and clients call the service 100 times, setting EXALOGIC_MSGQ_CACHE_SIZE to a value equal to or greater than 40 on the client improves performance.
• If there are 50 WSHs, and each WSH receives response messages from the same remote server, setting EXALOGIC_MSGQ_CACHE_SIZE to a value equal to or greater than 50 on the server environment improves performance.There is a difference running Oracle Tuxedo on a non-Exalogic platform if Direct Cross Node Communication Leveraging RDMA is enabled. The tux_msgq_monitor must be started before booting an Oracle Tuxedo application. This section includes the following topics:
• Start tux_msgq_monitortux_msgq_monitor functions as the Msgq_daemon default monitor. It is responsible for starting Msgq_daemon and checking it’s running status. When Msgg_daemon terminates abnormally, it restarts using tux_msgq_monitor.tux_msgq_monitor should be started before booting the Oracle Tuxedo application. One monitor can only serve one Oracle Tuxedo application on one logical machine.Before starting tux_msgq_monitor, ensure the environment variables TUXCONFIG, LD_LIBRARY_PATH and TUXDIR are properly set. Run tux_msgq_monitor as follows:
Table 2 tux_msgq_monitor: Options IPoIB address, it must be identical to the value of RDMADAEMONIP in UBBCONFIG file. Port number, it must be identical to the value of RDMADAEMONPORT in UBBCONFIG file. Shared memory size allocated by Msgq_daemon, you can get it by “tmloadcf –c”. For more information, see Tool for Computing the Shared Memory Size. Shared memory key used by Msgq_daemon.When tux_msgq_monitor is running, there will be two active processes named as tux_msgq_monitor and Msgq_daemon.After starting tux_msgq_monitor successfully, you can boot the Oracle Tuxedo application.
• Shutdown tux_msgq_monitor
Note: ipcrm –m shmid -- the shared memory created by Msgq_daemon.There are some shell scripts that simplify the startup/shutdown procedure. Using these tools, you can only run one command to start/stop both tux_msgq_monitor and an Oracle Tuxedo application. Before running these commands, ensure the environment variables TUXCONFIG, LD_LIBRARY_PATH and APPDIR are set properly.
• This script starts up tux_msgq_monitor, executes tmboot to start the Oracle Tuxedo application, and starts tlisten if option "-l" specified.Stops both the Oracle Tuxedo application and tux_msgq_monitor.
Note: In MP mode, for startup, you should run the commands in the following sequence:
1. tmlisten_start.sh on all the slave nodes.
2. tmboot.sh on the master node
For shutdown, you should run commands in the following sequence:
1. tmshut.sh on the master node
2. tmlisten_stop.sh on all the slave nodes.When Msgq_daemon starts, it allocates a shared memory. You can get the memory size by using tmloadcf -c ubb, as shown in Listing 14.Listing 14 UBBCONFIG File *MACHINES SectionListing 15 tmloadcf -c ubb Output ExampleAfter getting the RCDMSZ, you can adjust the actual size according to the following runtime factors:MAXSERVERS: A UBBCONFIG file attribute.RDMAQSIZE: A UBBCONFIG file attribute.If the EXALOGIC_MSGQ_CACHE_SIZE environment variable is set, at most 3* EXALOGIC_MSGQ_CACHE_SIZE* MAXSERVERS* RDMAQSIZE more bytes of shared memory should be allocated.For example, if MAXSERVERS is 100, RDMAQSIZE is 100000 and processes on machine ex04 start/stop frequently, you should configure the shared memory size with at most 1300M (340M+960M) instead of 340M.The number of machines in the UBBCONFIG file is considered in calculating RCDMSZ. If new machines are added using MIB, you should increase the size linearly.In some scenarios, Msgq_daemon needs to allocate more shared memory when increasing queue cache entry number using EXALOGIC_MSGQ_CACHE_SIZE. For example, if Oracle Tuxedo process start/stop or tpinit/tpterm frequently. Then you should consider adjusting the initial Msgq_daemon shared memory size, otherwise the message transfer may fail.The default value is 32 for each Oracle Tuxedo thread, and tmloadcf -c always assumes the value of 32. If you change the default value, adjust the Msgq_daemon shared memory size using the following method:MEMSZ: The current shared memory size calculated by above method.CACHENUM: The maximum entry queue cache number for each Oracle Tuxedo thread.
Note: You can adjust the size using this method only when setting EXALOGIC_MSGQ_CACHE_SIZE at the machine level.