BEA WebLogic Enterprise 4.2    Developer Center     

        HOME   |   SITE MAP   |   SEARCH   |   CONTACT   |   GLOSSARY  |   PDF FILES |   WHAT'S NEW 
 
        Administration    |   TABLE OF CONTENTS   |   PREVIOUS TOPIC   |   NEXT TOPIC   |   INDEX 

Monitoring a Running System


As an administrator, you must ensure that once your application is up and running, it meets (and continues to meet) the performance, availability, and security requirements your company has set for it. To perform this task, you need to monitor the resources (such as shared memory), activities (such as transactions), and potential problems (such as security breaches) in your configuration, and take any corrective actions that are necessary.

To help you meet this responsibility, the WLE and BEA TUXEDO systems provides tools that enable you to oversee both system events and application events. This chapter explains how to use these tools to keep your application performing fast, well, and securely.

Specifically, this chapter discusses the following topics:

Overview of System and Application Data

This section describes the types of data available for monitoring a running system and explains how to use that data.

Components and Activities for Which Data Is Available

Your WLE or BEA TUXEDO system maintains parameter settings and generates statistics for the following system components:

Where the Data Resides

To ensure that you have the information necessary for monitoring your system, the WLE or BEA TUXEDO system provides the following three data repositories:

This chapter describes the data stored in the UBBCONFIG file and in the Bulletin Board, and provides instructions for monitoring that data. For a description of the log files, see Chapter 13, "Monitoring Log Files."

How You Can Use the Data

The administrative data provided by your WLE or BEA TUXEDO system allows you to monitor a multitude of potential trouble areas on your system. For example, this data allows you to:

Moreover, you can set up your system so that it is able to use the statistics in the Bulletin Board to make decisions and to modify system components dynamically, without your help. With proper configuration, your system may be able to do tasks such as the following (when indicated by Bulletin Board statistics):

Thus, by monitoring the administrative data for your system, you can prevent and resolve problems that threaten the performance, availability, and security of your application.

Types of Data

Two types of administrative data are available on every running WLE and BEA TUXEDO system: static and dynamic.

Static Data

Static data about your configuration consists of configuration settings that you assign when you first configure your system and application. These settings are never changed without intervention (either in real-time or through a program you have provided). Examples include system-wide parameters (such as the number of machines being used) and the amount of IPC resources (such as shared memory) allocated to your system on your local machine. Static data is kept in the UBBCONFIG file and in the Bulletin Board.

At times you will need to check the static data about your configuration. For example:

Dynamic Data

Dynamic data about your configuration consists of information that changes in real time, that is, while an application is running. For example, the load (the number of requests sent to a server) and the state of various configuration components (such as servers) change frequently. Dynamic data is kept in the Bulletin Board.

You will need to check the dynamic data about your configuration frequently. For example:

Monitoring Methods

To monitor a running application, you need to keep track of the dynamic aspects of your configuration and sometimes check the static data. Thus, you need to be able to watch the Bulletin Board on an ongoing basis and consult the UBBCONFIG file when necessary. The WLE and BEA TUXEDO system provides the following methods of doing both tasks, as shown in this table.

You Can Use the . . . By . . . For Instructions, See . . .

tmadmin command

Entering commands after a prompt

This chapter

AdminAPI

Using the MIB (and the commands described in this chapter) to write programs that monitor your run-time application

Chapter 21, "Event Broker/Monitor (BEA TUXEDO System)"

BEA TUXEDO Reference manual, section 5

BEA TUXEDO Web-based GUI

Using a graphical interface

The Help accessed directly from the GUI

Which method is best for you? The answer depends on your answers to several questions.

If you decide to monitor your system at run time through the tmadmin command interpreter, continue reading; this chapter describes tmadmin and explains how to use it.

Using the tmadmin Command Interpreter

This section provides the following information:

What is tmadmin?

The tmadmin command is an interpreter for 50 commands that let you view and modify a Bulletin Board and its associated entities.

Note: tmadmin is supported on UNIX and Windows NT platforms.

How might you want to use tmadmin to modify your system while it is running? Consider the following sample scenario. Suppose you want to check the current values for all the parameters listed in the Bulletin Board, such as maximum number of servers and services. You can do this by running the tmadmin command, bbparms.

How a tmadmin Session Works

  1. A tmadmin session starts when you (the administrator) enter the tmadmin command at a shell prompt. The shell prompt($) is replaced by the tmadmin prompt (>) which is used until you quit tmadmin.

    $ tmadmin [operating_mode_option]
    >

    You can request one of three operating modes on the command line: the default mode (which allows you to view and change the Bulletin Board and associated entities), read-only mode (-r), or configuration mode (-c).

  2. tmadmin verifies that the configuration is running. If the configuration is not running the following message is displayed:

    No bulletin board exists. Entering boot mode
    >

  3. tmadmin checks the TUXCONFIG and TUXOFFSET environment variables to get the location and offset at which the configuration file has been loaded. (Be sure you have defined these variables before beginning a tmadmin session.)

  4. tmadmin enters the Bulletin Board in one of the following three states, depending on which operating mode you have requested.

  5. The > prompt is displayed on your screen and you enter a tmadmin command.

    Not all tmadmin commands are available on every machine at all times. Which commands are available depends on several factors:

Summary of tmadmin Options

Whenever you start a tmadmin session, you have a choice of operating modes for that session: read-only mode, configuration mode, or the default operating mode. In addition, you can generate a report of the WLE or BEA TUXEDO version and license numbers.

Read-only Mode

In this mode, you can view the data in the Bulletin Board, but you cannot make any changes. The advantage of working in read-only mode is that your administrator process is not tied up by tmadmin; the tmadmin process attaches to the Bulletin Board as a client, leaving your administrator slot available for other work.

To start a tmadmin session in read-only mode, specify the -r option on the command line:

$ tmadmin -r
Configuration Mode

In this mode, you can view the data in the Bulletin Board and, if you are the BEA TUXEDO application administrator, you can make changes. You can start a tmadmin session in configuration mode on any machine, including an inactive machine. On most inactive machines, configuration mode is required. (The only inactive machine on which you can start a tmadmin session without requesting configuration mode is the MASTER machine.)

To start a tmadmin session in configuration mode, specify the -c option on the command line:

$ tmadmin -c
Default Operating Mode

If you want to view and change Bulletin Board data during a tmadmin session, you must:

  1. Have administrator privileges (that is, your effective UID and GID must be those of the administrator).

  2. Invoke the command interpreter without any options:

    $ tmadmin
Version number and license number report

To find out which version of the WLE or BEA TUXEDO system you are running and to get the license number for it, specify the -v option on the command line:

$ tmadmin -v

After displaying the version and license numbers, tmadmin exits, even if you have specified -c or -r in addition to -v. When -v is requested, all other options are ignored.

tmadmin Meta-commands

The tmadmin command interpreter is equipped with a set of meta-commands, commands that help you use tmadmin. Table 15-1 lists the tmadmin meta-commands.

Note: The tables and examples in this chapter include the abbreviated forms of the tmadmin command names.

Table 15-1 tmadmin Meta-commands

Use This Command Or its Abbreviation To

default

d

Set defaults for arguments of other commands

dump

du

Download the current Bulletin Board into a file

echo

e

Display input command lines

help

h

Display command list or command syntax

paginate

page

Pipe output of commands to a pager

quit

q

Terminate the session

verbose

v

Show output in verbose mode (a toggle key)

!shlcmd

(n/a)

Escape to the shell and run the specified shell command

!!

(n/a)

Repeat the previous shell command

<RETURN>

(n/a)

Repeat the last tmadmin command

Default

The default meta-command (d) lets you set and unset defaults for the following frequently used parameters for most tmadmin commands: group name, server ID, machine, user name, client name, queue address, service name, device blocks, device offset, and UDL configuration device path. (For details, see the tmadmin(1) reference page in the BEA TUXEDO Reference Manual.)

Note: You cannot assign defaults to any parameters for the boot and shutdown commands.

Once defaults are set, they remain in effect until the session ends or until the parameters are reset to different values. The remainder of this section provides a list of instructions for checking, setting, and unsetting defaults:

Optional versus Required Arguments

Most tmadmin commands require explicit information about the resource on which the command is to act. Required arguments can always be specified on the command line, and can often be set via the default command, as well. tmadmin reports an error if the required information is not available from either source.

Some tmadmin statistical commands interpret unspecified default parameters as all.

Running tmadmin Commands

This section provides the basic procedure for running tmadmin commands. Commands for doing specific monitoring tasks through tmadmin are provided in the section "Monitoring a Running System with tmadmin" in this chapter.

Note: For complete details about tmadmin, see the tmadmin(1) reference page in the BEA TUXEDO Reference Manual.

Perform the following steps to run the tmadmin commands.

  1. Make sure the TUXCONFIG and TUXOFFSET environment variables have been set.

  2. Enter tmadmin in the appropriate operating mode.

  3. When the tmadmin session prompt (>) is displayed, enter your first tmadmin command. Specify, on the command line, how much information from the Bulletin Board you want to have displayed.

    For example: bbparms -v

    For example: bbparms -t

  4. After viewing the output of your first tmadmin command, continue entering tmadmin commands until you are ready to end the session.

  5. End the tmadmin session by entering:

    quit

Monitoring a Running System with tmadmin

Table 15-2 provides a list of potential problems that you might want to check while monitoring your run-time system, along with a list of the tmadmin commands that enable you to perform such a check. The table also suggests follow-up actions you might take if the tmadmin command you run generates a particular type of output.

Note: For a comprehensive list of the tmadmin commands, see the tmadmin(1) reference page in the BEA TUXEDO Reference Manual.

.
Table 15-2 Commands for Monitoring Tasks

To Determine
Whether . . .
Run this Command . . . If . . . Then . . .

Any servers are stalled in a service

$ tmadmin -r
> printserver

The Current Service and Request fields do not change

The server is spending excessive time on the current service.

In a development environ- ment, the server might be stalled in an infinite loop; you may want to stop it.

The load distribution is appropriate

$ tmadmin -r
> printserver

The values in the Load Done field are not reasonably similar

Check the layout of the MSSQs and the data-dependent routing.

If the current servers have too heavy a load, you may want to boot more servers.

A particular service is doing any work

$ tmadmin -r
> printservice

The value in the Requests Completed field is 0

Data-dependent routing may be preventing requests from being sent to that server for that service. You can:

Any clients are inactive

$ tmadmin -r
> printclient

Tell the client-via a broadcast message-to exit

The work is distributed in such a way that it is flowing smoothly through the system

$ tmadmin -r
> printqueue

Some queues are always heavy and others are not

Check the arrangement of services within servers, data-dependent routing, and/or queue organization.

A client is tying up a connection and preventing a server from doing any work for another client

$ tmadmin -r
> printconn

A client is maintaining control of a connection and is not issuing any requests

  1. Suspend the client by using the client MIB. (We recommend using the BEA Administration Console for this task.)

  2. Terminate the client.

The network is stable

$ tmadmin -r
> printnet

A machine is no longer connected

You may want to:

  1. Partition the machine (that is, take it off the network).

  2. Resolve the problem.

  3. Reconnect the machine.

You must manually commit or abort a transaction

$ tmadmin -r
> printtrans

For example, the status is TMGDECIDED

The first phase of the two-phase commit has completed successfully. This means you must find out why the second phase cannot be completed.

For example, you may find that the coordinating TMS cannot complete the transaction because a participating site has gone down.

Your operating system resources (such as shared memory and semaphores) on a local machine are sufficient

$tmadmin -r
> bbsread

You do not have sufficient resources in the operating system

Increase the IPC resources (semaphores, shared memory segments, and so on) in the operating system.

You want to keep the current values for system-wide parameters (in the RESOURCES section of your UBBCONFIG file)

$ tmadmin -r
> bbparms

You do not have sufficient resources for your application

  1. Stop the application.

  2. Configure additional IPC resources (assuming you have enough available) by increasing the values of relevant parameters (such as MAXSERVERS and MAXCLIENTS) in the RESOURCES section of the configuration file.

  3. Re-boot the application.

Example: Output from tmadmin Commands

This section provides examples of output from the following tmadmin monitoring commands:

printqueue Output

The following output from the printqueue command lets you check the distribution of work in the bankapp application.

printqueue [qaddress]

tmadmin - Copyright © 1987-1990 AT&T; 1991-1993 USL. All rights reserved.

>pq

a.out Name Queue Nam # Svrs Wk Q'd # Queued Ave. Len Machine

TLR

TLR

28706

28706

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

TMS_SQL

TMS_SQL

BANKB1_T

BANKB1_T

2

2

0

0

0

0

0.0

0.0

SITE1

SITE1

TLR

TLR

24946

24946

1

1

0

0

0

0

0.1

0.1

SITE1

SITE1

BAL

BAL

8533

8533

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

BAL

BAL

24915

24915

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

BTADD

BTADD

28897

28897

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

XFER

XFER

4380

4380

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

XFER

XFER

28840

28840

1

1

100

100

0

0

1.0

1.0

SITE1

SITE1

TLR

TLR

12519

12519

1

1

100

100

2

2

0.0

0.0

SITE1

SITE1

BBL

BBL

24846

24846

1

1

0

0

2

2

0.0

0.0

SITE1

SITE1

ACCT

ACCT

71

71

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

TMS_SQL

TMS_SQL

BANKB3_T

BANKB3_T

2

2

0

0

0

0

0.0

0.0

SITE1

SITE1

BAL

BAL

28958

28958

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

ACCT

ACCT

254

254

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

BTADD

BTADD

12310

12310

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

XFER

XFER

16494

16494

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

TMS_SQL

TMS_SQL

BANKB2_T

BANKB2_T

2

2

0

0

0

0

0.0

0.0

SITE1

SITE1

BTADD

BTADD

8430

8430

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

ACCT

ACCT

24641

24641

1

1

0

0

0

0

0.0

0.0

SITE1

SITE1

Note: By default, information is supplied for all queues. If you want your output to be limited to information about only one queue, specify the address for the desired queue.

The output of this command includes the following information:

In the Column Labeled . . . You See . . .

a.out Name

The name of the executable to which the queue is connected

Queue Name

The symbolic queue name (set to either the RQADDR parameter of UBBCONFIG or a randomly chosen value)

# Svrs

The number of servers connected to the queue

Wk Q'd

The load factor of all requests currently queued

# Queued

The actual number of requests queued

Ave. Len

The average queue length

Note: Not available in MP mode.

Machine

The LMID of the machine on which the queue is located

printconn Data

The following (verbose) output from the printconn command shows that the client process has:

printnet Command Output

This section shows the output from the following procedure.

  1. The printnet command was run. (The output shows the number of messages sent and received by both sites.)

  2. The BRIDGE process at SITE2 was stopped.

  3. The printnet command was re-entered. (The output shows that SITE2 is no longer connected to the master machine, SITE1.)

    printnet [-m machine_list]

    tmadmin - Copyright © 1987-1990 AT&T; 1991-1993 USL. All rights reserved.

    > echo
    Echo now on.

    > pnw
    SITE1 Connected To: msgs sent msgs received
    SITE2 100103

    SITE2 Connected To: msgs sent msgs received
    SITE1 104 101

    > pnw
    SITE1 Connected To: msgs sent msgs received

    Could not retrieve status from SITE2

    >

printtrans Command Output

The printtrans command reports statistics only for transactions that are currently in progress, specifically, statistics on the number of rollbacks, commits, and aborts that have been executed on your machine, group, or server.

This section shows the output produced by running the printtrans command in terse and verbose modes:

Case Study: Monitoring Run-time bankapp

This section presents a sample configuration for a multiprocessor (MP) version of the bankapp application. This section also shows the output that was returned when the local IPC resources and system-wide parameters were checked by running the appropriate tmadmin commands.

Configuration File for bankapp

For this case study, we will use the configuration defined in the UBBCONFIG file shown in Listing 15-3.

Listing 15-3 UBBCONFIG File for bankapp (MP Version)
#Copyright (c) 1997, 1998 BEA Systems, Inc.
#All rights reserved
*RESOURCES
IPCKEY 80952
UID 4196
GID 601
PERM 0660
MAXACCESSERS 40
MAXSERVERS 35
MAXSERVICES 75
MAXCONV 10
MAXGTT 20
MASTER SITE1,SITE2
SCANUNIT 10
SANITYSCA
12
BBLQUERY 180
BLOCKTIME 30
DBBLWAIT 6
OPTIONS LAN,MIGRATE
MODEL MP
LDBAL Y
#
*MACHINES
mchn1 LMID=SITE1
TUXDIR="/home/tuxroot"
APPDIR="/home/apps/bank"
ENVFILE="/home/apps/bank/ENVFILE"
TLOGDEVICE="/home/apps/bank/TLOG"
TLOGNAME=TLOG
TUXCONFIG="/home/apps/bank/tuxconfig"
TYPE="3B2"
ULOGPFX="/home/apps/bank/ULOG"
wgs386 LMID=SITE2
TUXDIR="/home2/tuxroot"
APPDIR="/home2/apps/bank"
ENVFILE="/home2/apps/bank/ENVFILE"
TLOGDEVICE="/home2/apps/bank/TLOG"
TLOGNAME=TLOG
TUXCONFIG="/home2/apps/bank/tuxconfig"
TYPE="386"
ULOGPFX="/home2/apps/bank/ULOG"
#
*GROUPS
DEFAULT: TMSNAME=TMS_SQL TMSCOUNT=2
# For NT/Netware, :bankdb: becomes ;bankdb;
BANKB1 LMID=SITE1 GRPNO=1
OPENINFO="TUXEDO/SQL:/home/apps/bank/bankdl1:bankdb:readwrite"
BANKB2 LMID=SITE2 GRPNO=2
OPENINFO="TUXEDO/SQL:/home2/apps/bank/bankdl2:bankdb:readwrite"

*NETWORK
SITE1 NADDR="//mach1.beasys.com:1900"
BRIDGE="/dev/tcp"
NLSADDR="//mach1.beasys.com:1900"
SITE2 NADDR="//mach386.beasys.com:1900"
BRIDGE="/dev/tcp"
NLSADDR="//mach386.beasys.com:1900"
*SERVERS
#
DEFAULT: RESTART=Y MAXGEN=5 REPLYQ=Y CLOPT="-A"

TLR SRVGRP=BANKB1 SRVID=1 RQADDR=tlr1 CLOPT="-A -- -T 100"
TLR SRVGRP=BANKB1 SRVID=2 RQADDR=tlr1 CLOPT="-A -- -T 200"
TLR SRVGRP=BANKB2 SRVID=3 RQADDR=tlr2 CLOPT="-A -- -T 600"
TLR SRVGRP=BANKB2 SRVID=4 RQADDR=tlr2 CLOPT="-A -- -T 700"
XFER SRVGRP=BANKB1 SRVID=5
XFER SRVGRP=BANKB2 SRVID=6
ACCT SRVGRP=BANKB1 SRVID=7
ACCT SRVGRP=BANKB2 SRVID=8
BAL SRVGRP=BANKB1 SRVID=9
BAL SRVGRP=BANKB2 SRVID=10
BTADD SRVGRP=BANKB1
BTADD SRVGRP=BANKB2 SRVID=12
AUDITC SRVGRP=BANKB1 SRVID=13 CONV=Y MIN=1 MAX=10
BALC SRVGRP=BANKB1 SRVID=24
BALC SRVGRP=BANKB2 SRVID=25
#

*SERVICES
DEFAULT: LOAD=50 AUTOTRAN=N
WITHDRAWAL PRIO=50 ROUTING=ACCOUNT_ID
DEPOSIT PRIO=50 ROUTING=ACCOUNT_ID
TRANSFER PRIO=50 ROUTING=ACCOUNT_ID
INQUIRY PRIO=50 ROUTING=ACCOUNT_ID
CLOSE_ACCT PRIO=40 ROUTING=ACCOUNT_ID
OPEN_ACCT PRIO=40 ROUTING=BRANCH_ID
BR_ADD PRIO=20 ROUTING=BRANCH_ID
TLR_ADD PRIO=20 ROUTING=BRANCH_ID
ABAL PRIO=30 ROUTING=b_id
TBAL PRIO=30 ROUTING=b_id
ABAL_BID PRIO=30 ROUTING=b_id
TBAL_BID PRIO=30 ROUTING=b_id
ABALC_BID PRIO=30 ROUTING=b_id
TBALC_BID PRIO=30 ROUTING=b_id

*ROUTING
ACCOUNT_ID FIELD=ACCOUNT_ID
BUFTYPE="FML"
RANGES="10000-59999:BANKB1,
60000-109999:BANKB2,
*:*"
BRANCH_ID FIELD=BRANCH_ID
BUFTYPE="FML"
RANGES="1-5:BANKB1,
6-10:B

Output from Checking the Local IPC Resources

To check the local IPC resources for this configuration, a tmadmin session was started, and the bbsread command was run. The output of bbsread is shown in Listing 15-4.

Listing 15-4 bbsread Output
SITE1> bbsread

IPC resources for the bulleti
board o
machine SITE1:
SHARED MEMORY: Key: 0x1013c38
SEGMENT 0:
ID: 15730
Size: 36924
Attached processes: 12
Last attach/detach by: 4181

This semaphore is the system semaphore
SEMAPHORE: Key: 0x1013c38
Id: 15666
| semaphore | current | last | # waiting |
| number | status | accesser | processes |
|----------------------------------------------|
| 0 | free | 4181 | 0 |
|------------|----------|----------|-----------|
This semaphore set is part of the user-level semaphore
SEMAPHORE: Key: IPC_PRIVATE
Id: 11572
| semaphore | current | last | # waiting |
| number | status | accesser | processes |
|----------------------------------------------|
| 0 | locked | 4181 | 0 |
| 1 | locked | 4181 | 0 |
| 2 | locked | 4181 | 0 |
| 3 | locked | 4181 | 0 |
| 4 | locked | 4181 | 0 |
| 5 | locked | 4181 | 0 |
| 6 | locked | 4181 | 0 |
| 7 | locked | 4181 | 0 |
| 8 | locked | 4181 | 0 |
| 9 | locked | 4181 | 0 |
| 10 | locked | 4181 | 0 |
| 11 | locked | 4181 | 0 |
| 12 | locked | 4181 | 0 |
| 13 | locked | 4181 | 0 |
|------------|----------|----------|-----------|

Note: The display is the same with verbose mode on or off.

Output from Checking System-wide Parameter Settings

To check the current values of the system-wide parameters for this configuration, we started a tmadmin session and ran the bbparms command. The output of bbparms is shown in Listing 15-5.

Listing 15-5 Sample bbparms Output
> bbparms
Bulleti
Board Parameters:
MAXSERVERS: 35
MAXSERVICES: 75
MAXACCESSERS: 40
MAXGTT: 20
MAXCONV: 10
MAXBUFTYPE: 16
MAXBUFSTYPE: 32
IPCKEY: 35384
MASTER: SITE1,SITE2
MODEL: MP
LDBAL: Y
OPTIONS: LAN,MIGRATE
SCANUNIT: 10
SANITYSCAN: 12
DBBLWAIT: 6
BBLQUERY: 180
BLOCKTIME: 30

Note: The display is the same with verbose mode on or off.


Copyright © 1999 BEA Systems, Inc. All Rights Reserved.
Required browser version: Netscape Communicator version 4.0 or higher, or Microsoft Internet Explorer version 4.0 or higher.
Last update: July 05, 1999.