This topic includes the following sections:
The Oracle Tuxedo /Q administrator has three primary areas of responsibility, which are:
Close cooperation with the application developers and programmers is a must; the configuration and the queue attributes must reflect the requirements of the application.
A brief example of the use of the queued message facility is distributed with the software and is described in A Sample Application.
Three servers are provided with the Oracle Tuxedo /Q component. One is the Transaction Manager Server (TMS),
TMS_QM, for the Oracle Tuxedo /Q resource manager. That is, it manages global transactions for the queued message facility. It must be defined in the
GROUPS section of the configuration file.
The other two, TMQUEUE(5) and TMQFORWARD(5), provide services to users. They must be defined in the
SERVERS section of the configuration file.
The application can also create its own queue servers, if the functionality of
TMQFORWARD does not fully meet the needs of the application. For example, the administrator might want to have a special server to dequeue messages moved to the error queue.
The application can also choose peer-to-peer communication. In this case, the application communicates with other applications, or a client communicates with other clients, by not using any forwarding server.
In addition to the standard requirements of a group name tag and a value for
GRPNO (see UBBCONFIG(5) for details), there must be a server group defined for each queue space the application will use. The
OPENINFO parameters need to be set. Here are examples:
TMS_QM is the name for the transaction manager server for Oracle Tuxedo /Q. In the
TUXEDO/QM is the literal name for the resource manager as it appears in
$TUXDIR/udataobj/RM. The values for <
device_name> and <
queue_space_name> are instance-specific and must be set to the pathname for the universal device list and the name associated with the queue space, respectively. These values are specified by the Oracle Tuxedo administrator using qmadmin(1).
|Note:||The chronological order of these specifications is not critical. The configuration file can be created either before or after the queue space is defined. The important thing is that the configuration must be defined and queue space and queues created before the facility can be used.|
There can be only one queue space per
GROUPS section entry. The
CLOSEINFO parameter is not used.
The following example is taken from the reference page for TMQUEUE(5).
TMQUEUEGRP1 GRPNO=1 TMSNAME=TMS_QM
TMQUEUEGRP2 GRPNO=2 TMSNAME=TMS_QM
The TMQUEUE(5) reference page gives a full description of the
SERVERS section of the configuration file, but there are some points worth additional emphasis here.
TMQUEUE recognizes a
timeout option when specified after the double dash (- -) in the
CLOPT parameter. This timeout value affects only operations begun within the server if it finds that a transaction is not in effect, in other words, either the client called tpenqueue(3c) or tpdequeue(3c) without first calling tpbegin(3c) or it began a transaction and called
tpdequeue() with the
TPNOTRAN flag set to exclude the queue request from the client's transaction. The default for
timeout is 30 seconds. If a
tpdequeue request is received with the
flags set to
TPETIME error will be returned if the wait exceeds
There is potential confusion among queue space names, queue names, and service names. The first place you are apt to encounter the confusion is in the specification of the message queue server:
TMQUEUE. When specifying this server in the configuration file you can use the
-s flag of the
CLOPT parameter to name the queue space served by a given instance of the server, which is the same as saying it is a service advertised by the function:
TMQUEUE. In an application that uses only one queue space, it is not necessary to specify the
CLOPT -s option; it will default to
TMQUEUE:TMQUEUE. If the application requires more than a single queue space, the names of the queue spaces are included as arguments to the
-s option in the
SERVERS section entry for the queued message server.
An alternative way of making this specification is to rebuild the message queue server, using buildserver(1), and name the queue spaces with the similar sounding
-s option. This has the result of fixing, or hardcoding, the service names in the server executable.
# The following two specifications are equivalent:
TMQUEUE SRVGRP="TMQUEUEGRP1" SRVID=1000 RESTART=Y GRACE=0 \
buildserver -o TMQUEUE -s myqueuespace:TMQUEUE -r TUXEDO/QM \
TMQUEUE SRVGRP="TMQUEUEGRP1" SRVID=1000 RESTART=Y GRACE=0 \
The preceding discussion described the specification of services (that is, queue space names) in the message queue server. This capability can be used to bring about data-dependent routing of queued messages such that the message is queued for processing by a service within a specific group depending on a value in a field of the message buffer. To do this the same queue space name is specified in two different groups and a routing specification is made part of the configuration file to govern the group where the message is queued. The following example is taken from the TMQUEUE(5) reference page (the queue space name has been changed).
TMQUEUEGRP1 GRPNO=1 TMSNAME=TMS_QM
TMQUEUEGRP2 GRPNO=2 TMSNAME=TMS_QM
TMQUEUE SRVGRP="TMQUEUEGRP1" SRVID=1000 RESTART=Y GRACE=0 \
TMQUEUE SRVGRP="TMQUEUEGRP2" SRVID=1000 RESTART=Y GRACE=0 \
MYROUTING FIELD=ACCOUNT BUFTYPE="FML" \
TMQUEUE supports all of the standard ATMI buffer types. If your application needs to add other types, it can be done by copying
$TUXDIR/tuxedo/tuxlib/types/tmsypesw.c, adding an entry for your special buffer types, making sure to leave the final line null, and using the revised file as input to a buildserver(1) command. An example of the
buildserver command is shown on the TMQUEUE(5) reference page.
You can also use the
-s option of the
buildserver command to associate additional service names with
TMQUEUE as an alternative to specifying them in the server
CLOPT parameter (see above).
You can assign a subtype to a buffer using the tpalloc(3c) subtype parameter and later extract the subtype using the tptypes(3c) function. This gives you the ability to assign a type to data without having to create an entirely new user-defined ATMI buffer type. This is especially useful for buffers containing character arrays (
CARRAY) or strings (
The third system-supplied server included with the Oracle Tuxedo /Q component is TMQFORWARD(5). This is the server that takes messages from specified queues, passes them along to Oracle Tuxedo servers via tpcall(3c), and handles associated reply messages. The full description of how the server is defined in the configuration file can be found on the TMQFORWARD(5) reference page, but the topics that follow bring out some points that are worth additional emphasis.
TMQFORWARD is referred to as a server and each instance used by an application must be defined in the
SERVERS section of the configuration file, but it has characteristics that set it apart from ordinary servers. For example:
An instance of
TMQFORWARD is tied to a queue space through the server group with which it is associated, specifically through the third field in the
OPENINFO statement for the group. In the topics that follow we will examine other key parameters, especially
CLOPT parameters that come after the double dash.
A required parameter is
queuename. . . This parameter specifies the queue(s) to be checked by this instance of the server.
queuename is a NULL-terminated string of up to 127 characters; it is the same as the name of the application service that will process the message once it has been taken off the queue by
TMQFORWARD. It is also the name that a programmer specifies as the second argument of tpenqueue(3c) or tpdequeue(3c) when preparing to call the message queue server,
TMQFORWARD does its work within a transaction that it begins and ends. The
trantime option is available to specify the length of time in seconds before the transaction is timed out. The transaction is begun when
TMQFORWARD finds a message on the queue it is checking; it is committed after a reply has been enqueued either to the reply queue or the failure queue, so the transaction encompasses calling the service that processes the message and receiving a reply. The default is 60 seconds.
TMQFORWARD is booted it periodically checks the queue to which it is assigned. If it finds the queue empty, it pauses for
idletime seconds before checking again. If a value is not specified, the default is 30 seconds; a value of 0 says to keep checking the queue constantly, which can be wasteful of CPU resources if the queue is frequently empty.
-e option is specified, the server will shut itself down gracefully (and will create a user log message) when it finds the queue empty. This behavior may be used to your advantage in connection with the threshold command that you can specify for a queue. There is a more complete discussion about the
-e option and the threshold command in Creating Queue Spaces and Queues.
When a service request fails after being called by
TMQFORWARD, the transaction is rolled back, and the message is put back on the queue for a later retry (up to a limit of retries specified for the queue). The
-d option adds the following refinement: if the failed service returns a non-NULL reply, the reply (and its associated
tpurcode) are put on a failure queue (if one is associated with the message and the queue exists) and the original request message is deleted. Also with the
-d option, if the original request message is to be deleted at the same time as the retry limit configured for the queue is reached, the original request message is put into the error queue.
The rationale behind this option is that rather than blindly retrying, the originating client can be coded to examine the failure message and determine whether further attempts are reasonable. It provides a way of handling a failure that is due to some inherently reasonable condition (for example, a record is not found because the account does not exist).
Customized application buffer types can be added to the type switch and incorporated into
TMQFORWARD with the buildserver(1) command. It should be noted, however, that when you customize
TMQFORWARD it is an error to specify service names with a
We have described configuration parameters in terms of
UBBCONFIG parameters. However, it should be noted that the specifications in the
SERVERS sections can also be added to the
TUXCONFIG file of a running application by using
tmconfig(1) (see tmconfig, wtmconfig(1)). Of course, the group and the servers will have to be booted once they have been defined.
This topic covers three of the qmadmin(1) commands that are used to establish the resources of the Oracle Tuxedo /Q component. The
APPQ_MIB Management Information Base provides an alternative method of administering Oracle Tuxedo /Q programmatically. See the APPQ_MIB(5) reference page for more information on the MIB.
Most of the key commands of
qmadmin have positional parameters. If the positional parameters (those not specified with a dash (
-) preceding the option) are not specified on the command line when the command is invoked,
qmadmin prompts you for the required information.
The universal device list (UDL) is a VTOC file under the control of the Oracle Tuxedo system. It maps the physical storage space on a machine where the Oracle Tuxedo system is run. An entry in the UDL points to the disk space where the queues and messages of a queue space are stored; the Oracle Tuxedo system manages the input and output for that space. If the queued message facility is installed as part of a new Oracle Tuxedo installation, the UDL is created by tmloadcf(1) when the configuration file is first loaded.
Before you create a queue space, you must create an entry for it in the UDL. The following is an example of the commands:
# First invoke the /Q administrative interface, qmadmin
# The QMCONFIG variable points to an existing device where the UDL
# either resides or will reside.
# Next create the device list entry
crdl /dev/rawfs 50 500
# The above command sets aside 500 physical pages beginning at block # 50
# If the UDL has no previous entries, offset (block number) 0 must # be used
If you are going to add an entry to an existing Oracle Tuxedo UDL, the value for the
QMCONFIG variable must be the same pathname specified in
TUXCONFIG. Once you have invoked
qmadmin, it is recommend that you run a
lidl command to see where space is available before creating your new entry.
A queue space makes use of IPC resources; when you define a queue space you are allocating a shared memory segment and a semaphore. As noted above, the easiest way to use the command is to let it prompt you. (You can also use the
T_APPQSPACE class of the APPQ_MIB(5) to create a queue space.) The sequence looks like this:
Queue space name: myqueuespace
IPC Key for queue space: 230458
Size of queue space in disk pages: 200
Number of queues in queue space: 3
Number of concurrent transactions in queue space: 3
Number of concurrent processes in queue space: 3
Number of messages in queue space: 12
Error queue name: errq
Initialize extents (y, n [default=n]):
Blocking factor [default=16]: 16
The program insists that you provide values for all prompts except the final three. As you can see, there are defaults for the last two; while you will almost certainly want to name an error queue, you are not required to. If you provide a name here, you must create the error queue with the
qcreate command. If you choose not to name an error queue, bear in mind that messages that normally would be moved to the error queue (for example, when a retry limit is reached), are permanently lost.
The program does not prompt you to specify the size of the area to reserve in shared memory for storing non-persistent messages for all queues in the queue space. When you require non-persistent (memory-based) messages, you must specify the size of the memory area on the qspacecreate command line with the
The value for the IPC key should be picked so as not to conflict with your other requirements for IPC resources. It should be a value greater than 32,768 and less than 262,143.
The size of the queue space, the number of queues, and the number of messages that can be queued at one time all depend on the needs of your application. Of course, you cannot specify a size greater than the number of pages specified in your UDL entry. In connection with these parameters, you also need to look ahead to the queue capacity parameters for an individual queue within the queue space. Those parameters allow you to (a) set a limit on the number of messages that can be put on a queue, and (b) name a command to be executed when the number of enqueued messages on the queue reaches the threshold. If you specify a low number of concurrent messages for the queue space, you may create a situation where your threshold on a queue will never be reached.
To calculate the number of concurrent transactions, count each of the following as one transaction:
If your client programs begin transactions before they call
tpenqueue, increase the count by the number of clients that might access the queue space concurrently. The worst case is that all clients access the queue space at the same time.
For the number of concurrent processes count one for each
TMQFORWARD server in the group that uses this queue space and one for a fudge factor.
You can choose to initialize the queue space as you use the
qspacecreate command, or you can let it be done by the
qopen command when you first open the queue space.
Each queue that you intend to use must be created with the
qcreate command. You first have to open the queue space with the
qopen command. If you do not provide a queue space name,
qopen will prompt for it. (You can also use the
T_APPQ class of the APPQ_MIB(5) to create a queue.)
The prompt sequence for
qcreate looks like the following:
Queue name: service1
Queue order (priority, time, fifo, lifo): fifo
Out-of-ordering enqueuing (top, msgid, [default=none]): none
Retries [default=0]: 2
Retry delay in seconds [default=0]: 30
High limit for queue capacity warning (b for bytes used, B for blocks used,
% for percent used, m for messages [default=100%]): 80%
Reset (low) limit for queue capacity warning [default=0%]: 0%
Queue capacity command:
No default queue capacity command
Queue 'service1' created
You can skip all of these prompts (except the prompt for the queue name); if you do not provide a name for the queue, the program displays a warning message and prompts again. For the other parameters, the program provides a default and displays a message that specifies the default.
The program does not prompt you for a default delivery policy and memory threshold options. The default delivery policy option allows you to specify whether messages with no specified delivery mode are delivered to persistent (disk-based) or non-persistent (memory-based) storage. The memory threshold option allows you to specify values used to trigger command execution when a non-persistent memory threshold is reached. To use these options, you must specify them on the qcreate command line with
Messages are put into the queue based on the order specified by this parameter and dequeued from the top of the queue unless selection criteria are applied to the dequeuing operation. If
priority, expiration, and/or
time are chosen as queue order criteria, then messages are inserted into the queue according to values in the
TPQCTL structure. A combination of sort criteria may be specified with the most significant criteria specified first. Separate multiple criteria with commas (
lifo (which are mutually exclusive) are specified, they must be the last value specified. The sequence in which parameters are specified determines the sort criteria for the queue. In other words, a specification of
priority, fifo would say that the queue should be arranged by message priority and that within messages of equal priority they should be dequeued on a first in, first out basis.
If the administrator enables out-of-order enqueues; that is, if
msgid are specified at the prompt, programmers can specify (via values in the
TPQCTL structure of a
tpenqueue call) that a message is to be put at the top of the queue or ahead of the message identified by
msgid. Give this option some thought; once the choice is made you have to destroy and recreate the queue to change it.
Normal behavior for a queued message facility is to put a message back on the queue if the transaction that dequeues it is rolled back. It will be dequeued again when it reaches the top of the queue. You can specify the number of retries that should be attempted and also a time delay between retries. Note that when a dequeued message is put back on the queue for retry, queue order specifications are, in effect, suspended for
Retry delay seconds. During this time, the message is unavailable for any dequeuing operation.
The default for the number of retries is 0, which means that no retries are attempted. When the retry limit is reached, the system moves the message to the error queue for the queue space, assuming an error queue has been named and created. If the error queue does not exist the message is discarded.
The delay time is expressed in seconds. When message queues are lightly populated so that a message restored to the queue reaches the top almost immediately, you can save CPU cycles by building in a delay factor. Your general policy on retries should be based on the experience of your particular application. If you have a fair amount of contention for the service associated with a given queue, you may get a lot of transient problems. One way to deal with them is to specify a large number of retries. (The number is strictly subjective, as is the time between retries.) If the nature of your application is such that any rolled back transaction signals a failure that is never going to go away, you might want to specify 0 retries and move the message immediately to the error queue. (This is very much like what happens when you specify the -
d option for
TMQFORWARD; the only difference is that a non-zero length failure message must be received for
TMQFORWARD automatically to drop the message from the queue.)
There are three parameters of the
qcreate command that can be used to partially automate the management of a queue. The parameters set a high and low threshold figure (it can be expressed as bytes, blocks, messages or percent of queue capacity) and allow you to specify a command that is executed when the high threshold is reached. (Actually, the command is executed once when the high threshold is reached, but not again until the low threshold is reached first prior to the high threshold.)
The following are two examples of ways the parameters can be used:
High limit for queue capacity warning (b for bytes used, B for blocks used, % for percent used, m for messages [default=100%]): 80%
Reset (low) limit for queue capacity warning [default=0%]: 10%
Queue capacity command: /usr/app/bin/mailme myqueuespace service1
This sequence sets the upper threshold at 80% of disk-based queue capacity and specifies a command to be executed when the queue is 80% full. The command is a script you have created that sends you a mail message when the threshold is reached. (
service1 are hypothetical arguments to your command.) Presumably, once you have been informed that the queue is filling up you can take action to ease the situation. You do not get the warning message again unless the queue load drops to 10% of capacity or below, and then rises again to 80%. You can also set thresholds and specify commands for the management of non-persistent (memory-based) queue capacity using the
-n option of the qcreate command.
|Note:||If you are working on a Windows machine, see Windows Standard I/O for additional information about configuring commands within a
The second example is somewhat more automated and takes advantage of the -
e option of the
High limit for queue capacity warning (b for bytes used, B for blocks used, % for percent used, m for messages [default=100%]): 90%
Reset (low) limit for queue capacity warning [default=0%]: 0%
Queue capacity command: tmboot -i 1002
This sequence assumes that you have configured a reserve
TMQFORWARD server for the queue in question with
SRVID=1002 and have included the -
e option in its
CLOPT parameter. (It also assumes that the server is not booted or, if booted, has shut itself down as a result of finding the queue empty.) When the queue reaches 90% capacity, the
tmboot command is executed to boot the reserve server. The -
e option causes the server to shut itself down when the queue is empty. You have set the low threshold to 0% so as not to kick off unnecessary
tmboot commands for a server that is already booted.
The default values for the three options are 100%, 0%, and no command.
The discussion above about creating a queue and providing parameters for its operation was written from the viewpoint of creating a queue for messages to be processed by a service of the same name. A queue may also be used for other purposes as well, such as peer-to-peer communication. The parameters for creating a queue are the same regardless of its use. The
TPQCTL structure used when a message is enqueued to a service queue includes fields to specify a reply queue and a failure queue.
TMQFORWARD detects the success or failure of the tpacall(3c) it makes to the requested service and, if these queues have been created by the administrator, enqueues the reply accordingly. If no reply or failure queue exists, the success or failure response message from the service is dropped leaving the originating client with no information about the outcome of the queued request. Even if there is no reply message from the service, if a reply queue exists, a zero-length message is enqueued there by
TMQFORWARD to inform the originating client of the outcome.
When creating a reply or a failure queue, bear in mind that in most cases messages are dequeued from these queues by a client process looking for information about an earlier enqueued request. Since the most common way of dequeuing such messages is by the
msgid (message identifier) or
corrid (correlation identifier) associated with the message—as opposed to taking a message off the top of the queue—the queue ordering criteria are less significant. In this case,
fifo is probably sufficient. The
retry delay parameters have no significance for reply queues; just take the defaults. The
queue capacity thresholds and commands are likely to be useful on reply queues, and the recommended usage is to alert the administrator so that he or she can intervene.
An error queue is a system queue. One of the
qspacecreate prompts asks for the name of the error queue for the queue space. When you have actually created an error queue of the name specified, the system uses it as a place to move messages from the service queue that have reached their retry limit. The management of the error queue is up to the administrator who can either handle the messages manually through commands of
qmadmin or can set up an automated way of handling them through the
APPQ_MIB MIB. The
queue capacity parameters can be used, but all of the other
qcreate parameters, with the exception of
qname, are not useful for the error queue.
|Note:||We recommend against using the same queue as both an error queue and a service failure queue; doing so makes it more difficult to cleanly manage the application and could lead to clients trying to access the administrator's area.|
TMQFORWARD handle encrypted message buffers without decrypting them. However, there are situations where the /Q component needs to decrypt enqueued message buffers, as described in with /Q” in Using Security in ATMI Applications.
As mentioned in the “
tpdequeue() operation has the side effect of destroying an encrypted queued message if the invoking process does not hold a valid decryption key. Thus, application programmers need to open a decryption key for a process before the process calls
tpdequeue() to retrieve an encrypted message; otherwise, the message will be lost.
For information on opening a decryption key, seeand Encrypted Messages” in Using Security in ATMI Applications.
This topic covers some things the queue administrator may have to do from time to time to keep a queue space operating efficiently.
If you find you need more disk storage for a queue space, you can add it with the
qaddext command of qmadmin(1). (You can also use the
TA_MAXPAGES attribute of the
T_APPQSPACE class of APPQ_MIB(5) to add extents.) The
qmadmin command takes the queue space name and a number of pages as arguments. The pages come from extents defined in the UDL for the device in your
QMCONFIG variable. The queue space must be inactive; you can use the exclamation point to execute a command outside of
qmadmin to shut down the associated server group. For example:
> !tmshutdown -g TMQUEUEGRP1
> qaddext myqueue 100
The queue space must be closed;
qmadmin closes it for you if you try to add extents to it while it is open. All non-persistent messages currently in the queue space are lost when the
qaddext command is issued and completes successfully.
A convenient command to use to back up a queue space is the UNIX command
dd. Shut down the associated server group first. The command lines should look like this:
tmshutdown -g TMQUEUEGRP1
dd if=<qspace_device_file> of=<output_device_filename>
For other options, see
dd(1) in a UNIX system reference manual.
This same command can be used to migrate the queue space to a machine of the same architecture, although you may need to start the command sequence with a
chdl command to provide a new device name if the present name does not exist on the target machine.
Similar archival techniques are available on server platforms that do not have the
dd command. First, shut down the group containing the queue space you want to back up or migrate. Then, use an archival tool to save the queue space device to a file or other medium that may then be used as a backup or used to move the queue space to another server.
If you need to move a queue space to a machine with a different architecture (primarily byte order), the procedure is more complex. Create and run an application program to dequeue all messages from all queues in the queue space and write them out in machine-independent format. Then enqueue the messages in the new queue space.
Messages dequeued and forwarded using
TMQFORWARD are executed within a global transaction because the operation crosses group boundaries. If the messages are executed by servers that are not associated with an RM or that do not run within a global transaction, they should have a server group with
TMSNAME=TMS (for the NULL XA interface).
The global transaction begun by
TMQFORWARD when it dequeues a message for execution is terminated by a
tpcommit(). The administrator can set the
CMTRET parameter in the configuration file to control whether the transaction commits when it is logged or when it is complete. (See the discussion of
CMTRET in the
RESOURCES section of the UBBCONFIG(5) reference page.)
Handling transaction timeout requires cooperation between the queue administrator and the programmer developing client programs that dequeue messages. When tpdequeue(3c) is called with the
flags argument set to include
TMQUEUE server will wait for a message to arrive on a queue before returning to the caller. The number of seconds before it times out is based on the following:
If a message is not immediately available when using
TMQUEUE requires an action resource so that
TMQUEUE may service other requests. You may want to increase the number of actions the queue space may handle concurrently. Use the
actions option to the
qspacechange commands. This option specifies the number of additional actions that can be handled concurrently. When a waiting operation is encountered and additional actions are available, the blocking operation is set aside until it can be satisfied. If no actions are available, the call to
TMQFORWARD server attempts to forward messages to a service that is not available, the situation can develop where the retry limit for the queue may be reached. The message is then moved to the error queue (if one exists). To avoid this situation the administrator should either shut the
TMQFORWARD server down or set the retry count higher.
When a message is moved to the error queue it is no longer associated with the original queue. If errors are going to be handled by the administrator moving the message back to the service queue when the service is known to be available, then the queue name may be stored as part of the
corrid in the
TPQCTL structure so the queue name is associated with the message.
In order to carry out a command that you have configured within a
qmadmin() session, such as the
qchange ... Queue capacity command described in Using Queue Capacity Limits, the Windows
CreateProcess() function spawns a child process as a
DETACHED PROCESS. This type of process does not have an associated console for standard input/output. Therefore, for instance, if you use standard DOS syntax to set the
qchange ... Queue capacity command to run a built-in DOS command (such as
date) and then pipe or redirect the standard output to a file, the file will be empty when the command completes.
As an example of resolving this problem, suppose that for the
qchange ... Queue capacity command you want to capture
date information in a file using command
date /t > x.out. To accomplish this task interactively, you would proceed as follows:
go through all the setups... the threshold queue capacity warning,
and so on
> "Queue capacity command: " cmd /c date /t > x.out
To accomplish this task from a command file, say
.cmd, you would add the command
date /t > x.out to
.cmd and then proceed as follows:
go through all the setups... the threshold queue capacity warning,> "Queue capacity command: "
and so on