BEA TSAM (Tuxedo System and Application Monitor), is a BEA Tuxedo add-on product. Tuxedo is widely used by enterprises that develop and use in mission-critical applications. It acts as the infrastructure layer in distributed computing environments. The complexity of Tuxedo and the applications running on top of it makes performance measurement extremely complex.
BEA TSAM monitors the major performance sensitive areas of a Tuxedo-supported enterprise computing environment. It can be used to monitor real-time performance bottlenecks and business data fluctuations, determine service models, and provide notification when pre-defined thresholds are violated.
BEA TSAM Features
The following is a list of BEA TSAM features:
Tracking a Tuxedo system call transmissions. Each monitored call is assigned a unique ID and is propagated along a call path tree. TSAM is able to track calls across multiple machines and domains.
Real-time call path tree tracking of a monitored request is displayed and the performance metrics for each step are available.
Call pattern summarization based on historical call tracking data.
Monitoring a particular Tuxedo service, checking its response time, IPC queue length and execution status. The data can be queried using recent or historical data.
Gathers Tuxedo GWTDOMAIN and BRIDGE overall throughput, graphically displaying the business data flow curve.
Tracking transactions with XA API specifications. Displays execution status and time used on each XA call. TSAM helps diagnose global distributed transactions.
The BEA TSAM Manager console provides the capability to create "Alert" definitions that generate events when predefined thresholds are reached. The events can be posted to Tuxedo and received by Tuxedo Event Broker subscribers.
Programming APIs that retrieve meta data packaged in a monitored call. Helps developers make application decisions dynamically.
Flexible monitoring controls. The sampling can be based on interval or ratio and the monitoring can be turned on or off dynamically without restarting application.
Plug-in mechanism for performance metrics collection at the Tuxedo infrastructure level. It provides great integration capability between TSAM and other third-party products.
Powerful plug-in-level event triggers without sending raw metrics data to the BEA TSAM Manager. It supports flexible FML boolean expression to achieve advanced event trigger conditions. Events can be posted to the Tuxedo Event Broker and/or the BEA TSAM Manager.
Scalable Tuxedo- side server monitoring design to meet small, middle and large Tuxedo runtime environments.
J2EE based solution. Easy to deploy, configure and use. It is a pure Web-based solution. The TSAM Console can be accessed anywhere using a compatible Web browser.
BEA TSAM Components
BEA TSAM includes two components:
BEA TSAM Agent: Performs Tuxedo-side data collection.
BEA TSAM Manager: Performs data storage, aggregation, computing and representation.
BEA TSAM Agent
The BEA TSAM Agent handles all Tuxedo-side back-end logic. It works in conjunction with the BEA TSAM Manager, and includes the following sub-components:
BEA TSAM Framework: A Tuxedo-side facility that defines and controls performance metrics collection behavior. It uses the Tuxedo traditional interface and can be easily integrated into an existing Tuxedo management suite.
BEA TSAM Plug-in: An extensible mechanism invoked by the BEA TSAM Framework. The BEA TSAM Agent provides default plug-ins to send data to the LMS (Local Monitor Server), and then to the BEA TSAM Manager. The default plug-in also checks event triggers, and generates events if needed.
You can develop your own plug-ins for additional data processing. A customized plug-in can be linked to an existing plug-in chain, or replace the default plug-in.
LMS (Local Monitor Server): The LMS is a Tuxedo system server. The BEA TSAM default plug-in sends data to the LMS. The LMS then passes the data to the BEA TSAM Manager in HTTP/XML message format.
The BEA TSAM Manager is built on standard J2EE technology. It includes following components:
BEA TSAM Data Server: Data server that accepts data from the LMS and stores the data in the database. It is a standard J2EE application.
BEA TSAM Console: The BEA TSAM presentation layer. It is a standard J2EE Web application and can be accessed via a compatible Web browser. After logging on to the BEA TSAM Console, you have access to full BEA TSAM functionality.
Apache Derby Database: The BEA TSAM Manager ships with an Apache Derby evaluation copy as the default database solution. Oracle is also supported using a pre-packaged SQL script (Oracle databases are not included with the BEA TSAM Manager).
Note:
The BEA TSAM Manager ships with Apache Tomcat as the J2EE application server used by the BEA TSAM Data Server and BEA TSAM Console.
Tuxedo is typically used by a client program (not necessarily a Tuxedo client process) that calls a service to perform a business computing logic scenario. The service implementation is completely transparent to the caller. This type of middleware transparency provides many benefits for development, deployment, and system administration. However, from a monitoring perspective, it is difficult for the end user or administrator to figure out what happens "behind the scene". BEA TSAM call path monitoring helps to alleviate this problem.
Call Path Tree Definition
A simple Tuxedo application call triggers a set of service invocations. The involved services constitute a tree (call path tree"). A call path tree strictly defines the following factors:
What type of services are involved to perform the initial service request.
The service invocation depth (that is, the depth of the call path tree).
The service invocation sequence. For example, client A calls SVC1. SVC1 calls SVC2 and SVC3.
Call transportation. The edge (how information is sent and received) of a call path tree represents the transportation information from caller to service provider. It could be an IPC queue, BRIDGE connection or DOMAIN connection. The elapse time used for each transportation is also recorded.
Monitoring Initiator
A "monitoring initiator" is a process that "initiates" tracking a call path tree. The process can be a Tuxedo client, application server, or client proxy server (WSH/JSH). A typical scenario is when a tpcall/tpacall is invoked by the monitoring initiator; call path monitoring begins. All the back-end services involved in this call are displayed on the call path tree representation in the BEA TSAM Console.
Note:
Currently only tpcall/tpacall can trigger a call path monitoring. Other communication models are not supported.
A Tuxedo application server performs two functions:
All sub-calls made in the service implementation are a part of the call path tree started by the original monitoring initiator (if the incoming request is already monitored).
It is a monitoring initiator with calls made in the service routine according to the monitoring policy definition.
Service Monitoring
Service monitoring focuses on pure Tuxedo service execution status. It does not care about call correlation, as call path monitoring does. Service monitoring can be used with call path monitoring together or performed independently.
System Server Monitoring
Tuxedo has two important system servers: BRIDGE and GWTDOMAIN. BRIDGE connects multiple Tuxedo machines in a Tuxedo domain. GWTDOMAIN connects one Tuxedo domain with another. The system server monitoring tracks message throughput, pending sent messages, and awaiting reply messages on each network link.
Transaction Monitoring
A critical use of Tuxedo is transaction monitoring. Tuxedo coordinates activities in a distributed transaction with an XA compliant resource manager, such as a database. BEA TSAM transaction monitoring tracks each XA call triggered in a transaction allowing you to clearly identify where a global distributed transaction is bottle necked.
Monitoring Policy
Monitoring policy controls monitoring behavior.
On or Off. BEA TSAM monitoring can be dynamically turned on or off (for a specific type or several kinds of monitoring) for a particular Tuxedo component. The Tuxedo component can be a particular server, group, or machine.
Interval-Based Monitoring. Monitoring is initiated based on specific time intervals. For example, call path monitoring. An interval-based monitoring policy can specify that the call path is tracked in 60-second intervals.
Ratio-Based Monitoring. Monitoring is initiated by the number of executions. For example, service monitoring. A ratio that is set to 5 indicates that every 5 executed services are monitored. For call path monitoring, a ratio set to 5 indicates that every 5 tpcall/tpacall calls are monitored.
Flexibility to Reduce Monitoring Performance Impact. BEA TSAM monitoring control enables you to configure the monitoring policy based on your application size, load and network activity.
Performance Metrics
BEA TSAM performance metrics are listed as follows:
Correlation ID: A unique identifier that represents a call path tree. It is generated by the monitoring initiator plug-in. It uses the following format:
Note:
DOMAINID:MASTERHOSTNAME:IPCKEY LMID PROCESSNAME PID TID COUNTER
Listing 1 shows an example of a Correlation ID. The monitored call is started by the program "bankclient" with process ID 8089 and thread ID 1 on machine "SITE1" on Tuxedo domain "TUXDOM1". The master is "bjsol18" and IPCKEY in TUXCONFIG is "72854".
Listing 1 Correlation ID Example
TUXDOM1:bjsol18:72854 SITE1 bankclient 8089 1 99
Service Name: The name of a Tuxedo Service.
Location: The string to identify the process who sends out the performance metrics.
IPC Queue Length: The message number in an IPC queue.
IPC Queue ID: Tuxedo identifier of an IPC queue.
Execution Time: The time used in a Tuxedo service or XA call execution, it is in millisecond level.
Wait Time: The time used of a message in the transportation stage.
Message Size: The Tuxedo message size.
Execution Status: The tpreturn service return code. It is defined by Tuxedo ATMI interface.
Call Flags: The flags passed to tpcall/tpacall in Tuxedo ATMI interface.
Call Type: tpcall, tpacall, or tpforward.
Elapse Time: The time elapsed sine the a call is monitored.
GTRID: Tuxedo global transaction ID.
Pending Message Number: The number of messages which are delivered to Tuxedo network layer and waiting for being sent.
Message Throughput: The total message number and volume accumulated in system server monitoring intervals.
Waiting Reply Message Number: The number of requests in GWTDOMAIN are waiting reply from remote domain.
XA Code: The XA call return code in transaction monitoring.
XA Name: The XA call name.
BEA TSAM Use Cases
BEA TSAM is built on top of Tuxedo and has unique service, call, and transaction tracking capabilities. Enterprise organization usually have many widely distributed services deployed and one client request that requires complex back-end service coordination to perform the processes.
It can be difficult for an administrator to figure out what exactly is happening during these interactions. BEA TSAM call path monitoring helps to alleviate this problem.
The followings are FAQs will help you to better understand how BEA TSAM works with your applications:
Enabling call path monitoring for a Tuxedo client or application server allows you to find out all the information behind a simple tpcall/tpacall. The tracking points span multiple machines and multiple domains. You can clearly see the following information in the call path tree:
The service invocation hierarchy that supports your call
The transmission cost for each message flow step, from IPC Queue to Network
The execution status of each service involved
The call type and call flags of all the intermediate calls
The waiting time in queue and response time for each service
The end-to-end response time
What about my services?
Service monitoring enables you to measure your service response time, IPC queue length, and execution status. Service monitoring provides the following information:
Service Execution Status Summary.
BEA TSAM tells you how many service executions succeeded or failed recently or during a period of time. BEA TSAM also computes the average response time. These are important factors in measuring the quality of your services.
Service Activity Trends.
BEA TSAM also displays your services activity trends. It tells you what the peek time is and when the services requests are low.
Is my network busy?
BEA TSAM allows you to monitor the network connection attached to your local domain gateways. You can easily find which link is busy and its data fluctuation trend. You have more in-depth understanding of the business data flow model between departments and organizations.
Who participates in my transaction?
BEA TSAM monitors the transaction XA calls. Transaction participants are listed on the transaction monitoring page. For a large distributed transaction, a slow branch can result in the entire transaction being slowly completed. BEA TSAM lets you know who the transaction participants are, and how much time is used during XA calls.
Solving Application Performance Problems
Why is the service response time is slow recently?
Turn on the call path monitoring for a particular call to investigate the following:
How much network-side time is used
Which services are the most time-consuming point in the call path tree
Is the service routed to a remote machine or a domain
Is it client wait time a reply problem?
My back-end services failed, but I don't know which one.
Turn on call path monitoring. You can find the service execution status for this call.
How many kinds of call paths are in my application?
Turn on call path monitoring using an adequate sampling policy. BEA TSAM will tell you how many call paths (a "call pattern") exist in your application.
Why is my global distributed transaction completed slowly?
Turn on BEA TSAM transaction monitoring. You can see the execution time used by the transaction participants.
I want to correlate local transactions with remote transactions.
Turn on BEA TSAM transaction monitoring for all involved processes and GWTDOMAIN. The BEA TSAM Console shows you the transaction mapping between local and remote transactions.
I want to know what is the peak time that my local domain uses resources from the remote domain, and how busy it is.
Use BEA TSAM system server monitoring on the GWTDOMAIN. BEA TSAM records the information for you, and shows you the throughput trends.
Can I check program request information?
Turn on call path monitoring with the proper monitoring policy and then use "tpgetcallinfo". The following information is provided.
The timestamp when the request leaves the caller
The timestamp when the request comes into to the server IPC queue
The client IP address (workstation client, GWWS client)
The monitoring initiator process, tpgetcallinfo(), can also tell you the total time used.
Improving Application Performance
Are my services too fine grained?
In some cases, too many services supporting a request may add to performance overhead. Use call path tree to investigate. The service number and the tree depth are key analysis factors.
Are my services deployed properly?
Some services are called more frequently than others. Use call path monitoring to gather the information, and re-consider the service deployment. It is best to have the most used services located on the local machine and LAN. Services across domain services should be used carefully.
Do I have too many servers configured?
BEA TSAM provides a central view of your Tuxedo applications with multiple domain support. Using BEA TSAM Console allows you to easily see how many domains, machines, servers and services are configured.
I want to be notified when a service execution fails or the response time exceeds a pre-defined threshold
You have two ways to do this. One is using BEA TSAM plug-in level event trigger; the other is to define alerts using the BEA TSAM Manager console.
BEA TSAM Plug-in Event Trigger.
It supports the FML boolean expression for what Tuxedo components you want to check and what metric conditions you want to trigger an event. It does not require performance metrics to be sent to the BEA TSAM Manager because the evaluation is done bythe BEA TSAM default plug-in.
BEA TSAM Manger Console Alert Management.
The BEA TSAM Manger Console allows you to define an alert with required conditions. When the threshold is reached, BEA TSAM generates the events. The events can also be posted to Tuxedo Event Brokers.
Quick Start
To add BEA TSAM functionality to an existing Tuxedo application, do the following steps:
For more information, see Policy Management in "BEA TSAM Console User Guide."
· Typical Monitoring Policy
Go to BEA TSAM > Administration > Policy Management, click "Create" to enter the "Policy Specification" page. Input "tsampolicy" in the "Name:" input field.
· Monitor call path initiated from a particular server
a. Click "New" button to show the "Policy Definition" page.
b. In the left panel "Tuxedo Component", select the "Domain", "Machine","Group" and "Server" in the drop down lists.
c. In the right panel "Policy Management Definition" select the "Enable" checkbox of "Call path"
d. Click "Add" button
e. Select that policy entry just created in "Monitoring Policy Set" table and click "Enable" button.
· Check BEA TSAM Agent results from Tuxedo
If you want to monitor a call path from a particular client process, you must use the TSAM Agent TMMONITOR environment variable for that client.
· Monitor services of a particular server
Same steps as call path monitoring policy set, except you must select the Services "Enable" check box.
· Monitor a Domain Gateway
Same steps as call path monitoring policy set but
- Select GWTDOMAIN you want in the "Tuxedo Component" panel
- Select System Servers "Enable" check box.
· Monitor XA calls in transaction for a particular group
Same steps as call path monitoring policy set but
- Select the GROUP you want in the "Tuxedo Component" panel
- Select the Transaction "Enable" check box
Start to Monitor Tuxedo
Login to TSAM Console, and start to monitor Tuxedo system and application.
Go to TSAM > Call Path to monitor call path.
Go to TSAM > Service to monitor service.
Go to TSAM > System Server to monitor system server.
