This chapter deals with the use of the ATMI C language functions for enqueuing and dequeuing messages: The BEA TUXEDO programmer coding client or server programs for the queued message facility should be familiar with the C language binding to the BEA TUXEDO ATMI. General guidance on BEA TUXEDO programming is available in the BEA TUXEDO Programmer's Guide. Detailed pages on all the ATMI functions are in Section 3c of the BEA TUXEDO Reference Manual.
The calls used to place a message on a BEA TUXEDO System/Q queue can originate in any client or server process associated with the application. The list includes:
tpenqueue(3c) and tpdequeue(3c), plus some ancillary functions.
Prerequisite Knowledge
Where Requests Can Originate
The coverage of BEA TUXEDO System/Q programming in this chapter reflects the illustration in Chapter 1, or at least the left-hand portion of it. In that figure a client (or a process acting in the role of a client) queues a message by calling The illustration in Chapter 1 goes on to show the queued message being dequeued by the server A brief example of the use of the queued message facility is distributed with the software and is described in Appendix A, "A Sample Application."
The syntax for When a There are some important arguments to control the operation of The entry for server group The example shown on the manual page for Within a queue space, message queues are named according to application services that process the requests. Emphasis on the Default Case
tpenqueue(3c) and specifying a queue space available through the TMQUEUE server. The client later retrieves a reply via a tpdequeue call to TMQUEUE.
TMQFORWARD and sent to an application server for processing (via tpcall). When a reply to the tpcall is received, TMQFORWARD enqueues the reply message. Since a major goal of TMQFORWARD is to provide an interface between the queue space and existing application services, it does not require further application coding. For that reason, this chapter concentrates on the client-to-qspace side.
Enqueuing Messages
tpenqueue is as follows.
#include <atmi.h>
int tpenqueue(char *qspace, char *qname, TPQCTL *ctl,
char *data, long len, long flags)tpenqueue call is issued it tells the system to store a message on the queue identified in qname in the space identified in qspace. The message is in the buffer pointed to by data and has a length of len. By the use of bit settings in flags the system is informed how the call to tpenqueue is to be handled. Further information about the handling of the enqueued message and replies is provided in the TMQCTL structure pointed to by ctl.
Command Line Arguments, tpenqueue(3)
tpenqueue(3c). Let's look at some of them.
tpenqueue(): the qspace Argument
qspace identifies a queue space previously created by the administrator. When a server is defined in the SERVERS section of the configuration file, the service names it offers are aliases for the actual queue space name (which is specified as part of the OPENINFO parameter in the GROUPS section). For example, when your application uses the server TMQUEUE, the value pointed at by the qspace argument is the name of a service advertised by TMQUEUE. If no service aliases are defined, the default service is the same as the server name, TMQUEUE. In this case the configuration file can include:
TMQUEUE
SRVGRP = QUE1 SRVID = 1
GRACE = 0 RESTART = Y CONV = N
CLOPT = "-A"
or
CLOPT = "-s TMQUEUE"QUE1 has an OPENINFO parameter that specifies the resource manager, the pathname of the device and the queue space name. The qspace argument in a client program can then look like this:
if (tpenqueue("TMQUEUE", "STRING", (TPQCTL *)&qctl,
(char *)reqstr, 0,0) == -1) {
Error checking
}TMQUEUE(5) shows how an alias for service names can be included when the server is built and specified in the configuration file. The sample program in Appendix A, "A Sample Application," also specifies an alias service name.
tpenqueue(): the qname Argument
qname is a pointer to such a value; an exception in which qname is not an application service is described later in the chapter.
tpenqueue(): the data and len Arguments
data points to a buffer that contains the message to be processed. The buffer must be one that was allocated with a call to tpalloc(3c). len gives the length of the message. Some BEA TUXEDO buffer types (such as FML) do not require len to be specified; in such cases, the argument is ignored. data can be NULL; when it is, len is ignored and the message is enqueued with no data portion.
tpenqueue(): the flags Arguments
flags values are used to tell the BEA TUXEDO system how the tpenqueue call is handled; the following are valid flags:
TPNOTRAN
TPNOBLOCK
tperrno set to TPEBLOCK. When the flag is not set the call blocks until the condition subsides; it fails if a blocking or transaction timeout occurs (TPETIME).
TPNOTIME
TPSIGRSTRT
tperrno to TPGOTSIG.
The third argument to tpenqueue() is a pointer to a structure of type TPQCTL. The TPQCTL structure has members that are used by the application and by the BEA TUXEDO system to pass parameters in both directions between application programs and the queued message facility. The client that calls tpenqueue sets flags to mark members the application wants the system to fill in. The structure is also used by tpdequeue; some of the members do not come into play until the application calls that function. The complete structure is shown in Listing 3-1.
Listing 3-1 The tpqctl_t Structure
#define TMQNAMELEN 15
#define TMMSGIDLEN 32
#define TMCORRIDLEN 32
struct tpqctl_t { /* control parameters to queue primitives */
long flags; /* indicates which of the values are set */
long deq_time; /* absolute/relative time for dequeuing */
long priority; /* enqueue priority */
long diagnostic; /* indicates reason for failure */
char msgid[TMMSGIDLEN]; /* id of message before which to queue */
char corrid[TMCORRIDLEN]; /* correlation id used to identify message */
char replyqueue[TMQNAMELEN+1]; /* queue name for reply message */
char failurequeue[TMQNAMELEN+1]; /* queue name for failure message */
CLIENTID cltid; /* client identifier for originating client */
long urcode; /* application user-return code */
long appkey; /* application authentication client key */
};
typedef struct tpqctl_t TPQCTL;
The following is a list of valid bits for the flags parameter controlling input information for tpenqueue.
TPNOFLAGS
TPNOFLAGS.
TPQTOP
TPQBEFOREMSGID
msgid field. This request may not be granted depending on whether or not the queue was configured to allow overriding the queue ordering to put a message ahead of another by msgid. TPQTOP and TPQBEFOREMSGID are mutually exclusive flags. Assumes a prior (successful) call with TPQMSGID set.
TPQTIME_ABS
deq_time field. The deq_time is an absolute time value as generated by time(2) or mktime(3C), if they are available in your UNIX operating system, or gp_mktime(3c), provided with the BEA TUXEDO system. The value set in the deq_time field is the number of seconds since 00:00:00 UTC, January 1,1970. TPQTIME-ABS can be overridden and the message dequeued immediately by MSGID or CORRID.
TPQTIME_REL
deq_time specifies the number of seconds to delay after the transaction completes before the submitted request should be processed. TPQTIME_REL can be overridden and the message dequeued immediately by MSGID or CORRID. TPQTIME_ABS and TPQTIME_REL are mutually exclusive flags.
TPQPRIORITY
priority. The priority must be in the range 1 to 100, inclusive.
TPQCORRID
corrid is available when a request is dequeued with tpdequeue(3c). This identifier accompanies any reply or failure message that is queued so an application can correlate a reply with a particular request. The entire value should be initialized such that the value can be matched at a later time. This can be done, for example, by padding with null characters to the full 32-character size.
TPQREPLYQ
replyqueue is associated with the queued message. Any reply to the message will be queued to the named queue within the same queue space as the request message. This string must be NULL-terminated (maximum 15 characters in length). If a reply is generated for the service and a reply queue is not specified or the reply queue does not exist, the reply is dropped.
TPQFAILUREQ
failurequeue field is associated with the queued message. If a failure occurs when executing the enqueued message, a failure message will go to the named queue within the same queue space as the original request message. This string must be NULL-terminated (maximum 15 characters in length).
Additionally, the urcode element of TPQCTL can be set with a user-return code. This value will be returned to the application that calls tpdequeue(3c) to dequeue the message.
On output from tpenqueue, the following elements may be set in the TPQCTL structure:
long flags; /* indicates which of the values are set */
char msgid[32]; /* id of enqueued message */
long diagnostic; /* indicates reason for failure */
An additional setting of the flags parameter requests output information from tpenqueue. If this flag bit is turned on when tpenqueue is called, then the associated element in the structure is populated if available and the bit remains set. If the value is not available, tpenqueue completes with the flag bit turned off.
TPQMSGID
tpenqueue was successful, the message identifier will be stored in msgid.
If the call to tpenqueue fails and tperrno is set to TPEDIAGNOSTIC, a value indicating the reason for failure is returned in diagnostic. The possible values are:
QMEINVAL]
QMEBADRMID]
QMENOTOPEN]
QMETRAN]
TPNOTRAN flag and an error occurred trying to start a transaction in which to enqueue the message.
QMEBADMSGID]
QMESYSTEM]
QMEOS]
QMENOTA]
QMEPROTO]
QMEBADQUEUE]
QMENOSPACE]
The remaining members of the control structure are not used on input to tpenqueue.
If the administrator in creating a queue allows tpenqueue calls to override the order of messages on the queue, you have two mutually exclusive ways to use that capability. You can specify that the message is to be placed at the top of the queue by setting flags to TPQTOP or you can specify that it be placed ahead of a specific message by setting flags to TPQBEFOREMSGID and setting msgid to the ID of the message you wish to precede. This assumes that you saved the message-ID from a previous call in order to be able to use it here. Your administrator must tell you what the queue supports; it can be created to allow either or both of these overrides, or to allow neither.
If the queue was created with priority as a queue ordering parameter, you can set a value in priority to specify the dequeuing priority for the message. The value must be in the range 1 to 100; the higher the number the higher the priority. If priority was not one of the queue ordering parameters, setting a priority here has no effect.
A queue can be created with time as a queue ordering parameter. When this is the case, you can specify in deq_time either an absolute time for the message to be dequeued or a time relative to the enqueuing transaction. You set flags to either TPQTIME_ABS or TPQTIME_REL to say how the value should be treated.
BEA TUXEDO System/Q provides a function, gp_mktime(3c), that is used to convert a date and time provided in a tm structure to the number of seconds since January 1, 1970. The value is returned in time_t, a typedef'd long. To set an absolute time for the message to be dequeued (we are using 12:00 noon, December 9, 1992), do the following.
tm structure.
#include <stdio.h>
#include <time.h>
static char *const wday[] = {
"Sunday", "Monday", "Tuesday", "Wednesday",
"Thursday", "Friday", "Saturday", "-unknown-"
};
struct tm time_str;
/*...*/
time_str.tm_year = 1992 - 1900;
time_str.tm_mon = 12 - 1;
time_str.tm_mday = 9;
time_str.tm_hour = 12;
time_str.tm_min = 0;
time_str.tm_sec = 1;
time_str.tm_isdst = -1;gp_mktime to produce a value for deq_time and set the flags to indicate
an absolute time is being provided.
#include <atmi.h>
TPQCTL qctl;
if ((qctl->deq_time = (long)gp_mktime(&time_str)) == -1) {
/* check for errors */
}
qctl->flags = TPQTIME_ABStpenqueue.
if (tpenqueue(qspace, qname, qctl, *data,*len,*flags) == -1) {
/* check for errors */
}
If you want to specify a relative time for dequeuing, for example, nnn seconds after the completion of the enqueuing transaction, place the number of seconds in deq_time and set flags to TPQTIME_REL.
Messages are always enqueued within a transaction; the only question is, within whose transaction? There are two choices. If caller of tpenqueue is in transaction mode and TPNOTRAN is not set, then the enqueuing is done within the caller's transaction. The caller knows for certain from the success or failure of tpenqueue whether the message was enqueued or not. If the call succeeds, the message is guaranteed to be on the queue. If the call fails, the transaction is rolled back, including the part where the message was placed on the queue.
If caller of tpenqueue is not in transaction mode or if TPNOTRAN is set, the message is enqueued in a separate transaction. If the call to tpenqueue returns success, the message is guaranteed to be on the queue. If the call to tpenqueue fails with a communication error or with a transaction or blocking timeout, the caller is left in doubt about whether the failure occurred before or after the message was enqueued.
Note that specifying TPNOTRAN while the caller is not in transaction mode has no meaning.
The syntax for tpdequeue is as follows:
#include <atmi.h>
int tpdequeue(char *qspace, char *qname, TPQCTL *ctl, \
char **data, long *len, long flags)
When this call is issued it tells the system to dequeue a message from the qname queue in space named qspace. The message is placed in a buffer (originally allocated by tpalloc(3c)) at the address pointed to by *data. len points to the length of the data. If len is 0 on return from tpdequeue, the message had no data portion. By the use of bit settings in flags the system is informed how the call to tpdequeue is to be handled. The TPQCTL structure pointed to by ctl carries further information about how the call should be handled.
There are some important arguments to control the operation of tpdequeue(3c). Let's look at some of them.
qspace identifies a queue space previously created by the administrator. When a server is defined in the SERVERS section of the configuration file, the service names it offers are aliases for the actual queue space name (which is specified as part of the OPENINFO parameter in the GROUPS section). For example, when your application uses the server TMQUEUE, the value pointed at by the qspace argument is the name of a service advertised by TMQUEUE. If no service aliases are defined, the default service is the same as the server name, TMQUEUE. In this case the configuration file can include:
TMQUEUE
SRVGRP = QUE1 SRVID = 1
GRACE = 0 RESTART = Y CONV = N
CLOPT = "-A"
or
CLOPT = "-s TMQUEUE"
The entry for server group QUE1 has an OPENINFO parameter that specifies the resource manager, the pathname of the device and the queue space name. The qspace argument in a client program can then look like this:
if (tpdequeue("TMQUEUE", "REPLYQ", (TPQCTL *)&qctl,
(char **)&reqstr, &len,TPNOTIME) == -1) {
Error checking
}The example shown on the manual page for TMQUEUE(5) shows how alias service names can be included when the server is built and specified in the configuration file. The example in Appendix A, "A Sample Application," also specifies an alias service name.
Reply queue names in a queue space need to be agreed upon within the application. The administrator creates a reply queue (and often an error queue) in the same manner a message queue is created. qname is a pointer to the name.
The arguments have a different flavor than they do on tpenqueue. *data points to the address of a buffer where the system is to place the message being dequeued. When tpdequeue is called, it is an error for its value to be NULL.
When tpdequeue returns, len points to a long that carries information about the length of the data retrieved. If it is 0, it means that the reply had no data portion. This can be a legitimate and successful reply in some applications; receiving even a 0 length reply can be used to show successful processing of the enqueued request. If you wish to know whether the buffer has changed from before the call to tpdequeue, save the prior length and compare it to len.
flags values are used to tell the BEA TUXEDO system how the tpdequeue call is handled; the following are valid flags:
TPNOTRAN
TPNOBLOCK
tperrno set to TPEBLOCK. When the flag is not set the call blocks until the condition subsides; it fails if a blocking or transaction timeout occurs (TPETIME). This blocking condition does not include blocking on the queue itself if the TPQWAIT option in flags is specified.
TPNOTIME
TPNOCHANGE
data is not allowed to change. By default, if a buffer is received that differs in type from the buffer pointed to by *data, then *data's buffer type changes to the received buffer's type so long as the receiver recognizes the incoming buffer type. That is, the type and subtype of the received buffer must match the type and subtype of the buffer pointed to by *data.
TPSIGRSTRT
tperrno to TPGOTSIG.
The third argument to tpdequeue() is a pointer to a structure of type TPQCTL. The TPQCTL structure has members that are used by the application and by the BEA TUXEDO system to pass parameters in both directions between application programs and the queued message facility. The client that calls tpdequeue sets flags to mark members the application wants the system to fill in. As described earlier, the structure is also used by tpenqueue; some of the members only apply to that function. The entire structure is shown in Listing 3-1.
On input to tpdequeue, the following elements may be set in the TPQCTL structure:
long flags; /* indicates which of the values are set */
char msgid[32]; /* id of message to dequeue */
char corrid[32]; /* correlation identifier of message to dequeue */
The valid flags on input to tpdequeue are:
TPNOFLAGS
TPQGETBYMSGID
msgid be dequeued. The message identifier would be one that was returned by a prior call to tpenqueue. This option cannot be used with the TPQWAIT option.
TPQGETBYCORRID
corrid be dequeued. The correlation identifier would be one that the application specified when enqueuing the message with tpenqueue. This option cannot be used with the TPQWAIT option.
TPQWAIT
Following is a list of valid bits for the flags parameter controlling output information from tpdequeue. If the flag bit is turned on when tpdequeue is called, then the associated element (see Listing 3-1) in the structure is populated if available and the bit remains set. If the value is not available, the flag bit is turned off after tpdequeue completes.
TPQPRIORITY
priority.
TPQMSGID
tpdequeue was successful, the message identifier will be stored in msgid.
TPQCORRID
tpdequeue was successful and the message was queued with a correlation identifier, the value will be stored in corrid. Any reply to a queue must have this correlation identifier.
TPQREPLYQ
replyqueue. Any reply to the message should go to the named reply queue within the same queue space as the request message.
TPQFAILUREQ
failurequeue. Any failure message should go to the named failure queue within the same queue space as the request message.
If the call to tpdequeue failed and tperrno is set to TPEDIAGNOSTIC, a value indicating the reason for failure is returned in diagnostic. The valid codes for diagnostic include those shown above for tpenqueue and the following additional codes:
QMENOMSG]
QMEINUSE]
When tpdequeue is called with flags set to TPQWAIT, the TMQUEUE server may be blocked waiting for a message to come onto the queue. The amount of time it is blocked can be controlled by the transaction timeout value set by the caller in tpbegin(3c) or by the -t option in the CLOPT parameter of the TMQUEUE server (if the transaction is started in the server). To avoid blocking tpenqueue calls that also use the TMQUEUE server, it may be desirable to configure two or more TMQUEUE servers (or MSSQ sets) offering different service names for the same queue space. It could be set up so that all enqueue and nonwaiting dequeue operations use one set of TMQUEUE servers and all waiting dequeue operations use the second set.
In considering how best to handle errors in dequeuing it is helpful to differentiate between errors encountered by TMQFORWARD as it attempts to dequeue a message to forward to the requested service and errors that occur in the service that processes the request. This subject was discussed in Chapter 1, but is repeated here in the context of writing application programs.
By default, if a message is dequeued within a transaction and the transaction is rolled back, then (if the retry parameter is greater than 0) the message ends up back on the queue and can be dequeued and executed again. It may be desirable to delay for a short period before retrying to dequeue and execute the message, allowing the transient problem to clear (for example, allowing for locks in a database to be released by another transaction). Normally, a limit on the number of retries is also useful to ensure that an application flaw doesn't cause significant waste of resources. When a queue is configured by the administrator, both a retry count and a delay period (in seconds) can be specified. A retry count of 0 implies that no retries are done. After the retry count is reached, the message is moved to an error queue that is configured by the administrator for the queue space. If the error queue is not configured, then messages that have reached the retry count are simply deleted. Messages on the error queue must be handled by the administrator who must work out a way of notifying the originator that meets the requirements of the application. This kind of handling is almost transparent to the originating program that put the message on the queue. There is a virtual guarantee that once a message is successfully enqueued it will be processed according to the parameters of tpenqueue and the attributes of the queue. Notification that a message has been moved to the error queue should be a rare occurrence in a system that has properly tuned its queue parameters.
A failure queue (normally, different from the queue space error queue) may be associated with each queued message. This queue is specified on the enqueuing call as the place to put any failure messages. The failure message for a particular request can be identified by an application-generated correlation identifier that is associated with the message when it is enqueued.
The default behavior of retrying until success (or a predefined limit) is quite appropriate when the failure is caused by a transient problem that is later resolved, allowing the message to be handled appropriately.
There are cases where the problem is not transient. For example, the queued message may request operating on an account that does not exist (and the application is such that it won't come into existence within a reasonable time period if at all). In this case, it is desirable not to waste any resources by trying again. If the application programmer or administrator determines that failures for a particular operation are never transient, then it is simply a matter of setting the retry count to zero, although this will require a mechanism to constantly clear the queue space error queue of these messages (for example, a background client that reads the queue periodically). More likely, it is the case that some problems will be transient (for example, database lock contention) and some problems will be permanent (for example, the account doesn't exist) for the same service.
In the case that the message is processed (dequeued and passed to the application via a tpcall) by TMQFORWARD, there is no mechanism in the information returned by tpcall to indicate whether a TPESVCFAIL error is caused by a transient or permanent problem.
As in the case where the application is handling the dequeuing, a simple solution is to return success for the service, that is, tpreturn with TPSUCCESS, even though the operation failed. This allows the transaction to be committed and the message removed from the queue. If reply messages are being used, the information in the buffer returned from the service can indicate that the operation failed and the message will be enqueued on the reply queue. The rcode argument of tpreturn can also be used to return application specific information.
In the case where the service fails and the transaction must be rolled back, it is not clear whether or not TMQFORWARD should execute a second transaction to remove the message from the queue without further processing. By default, TMQFORWARD will not delete a message for a service that fails. TMQFORWARD's transaction is rolled back and the message is restored to the queue. A command line option may be specified for TMQFORWARD that indicates that a message should be deleted from the queue if the service fails and a reply message is sent back with length greater than 0. The message is deleted in a second transaction. The queue must be configured with a delay time and retry count for this to work. If the message is associated with a failure queue, the reply data will be enqueued to the failure queue in the same transaction as the one in which the message is deleted from the queue.
If your application expects to receive replies to queued messages, here is a procedure you may want to follow.
TPSUCCESS on a logical failure and return an
explanatory code in the rcode argument of tpreturn.
Fill in the values for TPQCORRID TPQREPLYQ
TPQFAILUREQ TPQMSGIDcorrid, replyqueue and failurequeue before issuing the call. On return from the call, save corrid.
tpdequeue to check for a reply, specify the reply queue in the
qname argument and set flags to turn on the bits for the following flags:
Use the saved correlation identifier to populate TPQCORRID TPQREPLYQ
TPQFAILUREQ TPQMSGID
TPQGETCORRIDcorrid before issuing the call. If the call to tpdequeue fails and sets tperrno to TPEDIAGNOSTIC, then further information is available in diagnostic. If you receive the error code QMENOMSG, it means that no message was available for dequeuing.
tpdequeue. This time have qname point to the name of the
failure queue and set flags to turn on the bits for the following flags:
Populate TPQCORRID TPQREPLYQ
TPQFAILUREQ TPQMSGID
TPQGETBYCORRIDcorrid with the correlation identifier. When the call returns, check len to see if data has been received and check urcode to see if the service has returned a user return code.
Sequential processing of messages can be achieved by having one service enqueue a message for the next service in the chain before its transaction is committed. The originating process can track the progress of the sequence with a series of tpdequeue calls to the reply_queue, if each member uses the same correlation-ID and returns a 0 length reply.
Alternatively, word of the successful completion of the entire sequence can be returned to the originator by using unsolicited notification. To make sure that the last transaction in the sequence ended with a tpcommit, a job step can be added that calls tpnotify using the client identifier that is carried in the TPCTL structure returned from tpdequeue or in the TPSVCINFO structure passed to the service. The originating client must have called tpsetunsol to name the unsolicited message handler being used.
In all of the foregoing discussion of enqueuing and dequeuing messages there has been an implicit assumption that the queues were being used as an alternative form of request/response processing. It may have occurred to you that the message itself does not have to be a service request and you would be correct. The queued message facility can be used equally as effectively to transfer data from one process to another.
If it suits your application to use BEA TUXEDO System/Q for this purpose, have the administrator create a separate queue and code your own receiving program for dequeuing messages from that queue.