BEA MessageQ
Installation and Configuration Guide for OpenVMS

MessageQ message recovery services (MRS) enables an application to send a message to a target application and guarantee its delivery despite an application, system, or network failure. A message is recoverable if MessageQ writes it to a message recovery journal prior to delivery. MessageQ has two message recovery journals controlled by the MRS Server:

• Store and Forward file (SAF) --- for outgoing recoverable messages
• Destination Queue file (DQF) --- for incoming recoverable messages

In addition, the MessageQ message recovery system provides the following auxiliary journal files (controlled by the JRN Server):

• Dead Letter Journal (DLJ) --- a disk-based mechanism for storing undeliverable messages for recovery under user or program control.
• Post Confirmation Journal (PCJ) --- a file containing successfully delivered recoverable messages.

If a recoverable message cannot be delivered to the target application, the MRS Server writes it to the SAF or DQF file depending upon the delivery interest point selected when the message was sent. DQF and SAF files are dynamically created and deleted as needed by the recovery system (MRS_SERVER process).

After a message is written to the SAF or DQF file, the MRS Server writes a copy of the message into the message queue, where the receiver program can access it. After the receiver program has completed processing the message, it either reads the next recoverable message, thereby performing an implicit confirmation (if the queue is configured for this option), or calls the pams_confirm_msg service to explicitly instruct MRS to delete the message recovery file.

The size of SAF and DQF files is fixed and is set by the MRS parameter AREA_SIZE. The MRS Server creates a new SAF or DQF file each time the file fills up. MRS attempts to empty SAF files by periodically attempting to reach the corresponding remote queue. The MRS Server creates a separate set of SAF files for each remote queue. The maximum number of SAF files that can be created is set using the NUM_SAF_AREAS parameter. The maximum number of DQF files that can be created is set using the NUM_DQF_AREAS parameter.

In support of large messages, 4 MB, the journal files may be automatically extended by the MRS and JRN Servers in order to store a message that will not fit into the "fixed" size. This prevents a large message from being split across multiple journal files.

Both DQF and SAF files can be closed and the stream of messages in the file redirected to other targets. This can be in response to a message to the MRS Server from a suitably privileged user program, or activated from the Utilities menu (DMQ$MGR_UTILITY) as described in Chapter 11 .

6.2.2 Using Auxiliary Journal Files

The DLJ and PCJ journals are served by a separate server process, JRN_SERVER, that maintains a history of all journal filename changes in a disk file. This history is displayed by the DIR and DUMP commands of DMQ$MGR_UTILITY to make it easy to find an old journal.

The PCJ and DLJ journals consist of the files having the following format:

dev:[dir]name_xxxxx.ext;ver

where xxxxx is a sequence number assigned by the JRN Server, where ext is either PCJ or DLJ, and ver is a version number assigned by the JRN Server. The journals use the sequence number to tie multiple journal files from the same stream together. The version number is used the distinguish different journal streams.

Example 6-1 Example Journal History

 
$type DMQ$MRS:DMQ_0099_00040_DLJ.JH 
   1-DEC-1997 12:58:45.70 DMQ$MRS:MRS_0099_00040_*.DLJ;1 
   6-JAN-1998 10:41:01.27 DMQ$MRS:MRS_0099_00040_*.DLJ;2 
 
$type DMQ$MRS:DMQ_0099_00040_PCJ.JH;1 
   1-DEC-1997 12:58:47.80 DMQ$MRS:MRS_0099_00040_*.PCJ;1 
   6-JAN-1998 10:42:06.40 DMQ$MRS:MRS_0099_00040_*.PCJ;2 
   6-JAN-1998 10:42:12.97 DMQ$MRS:MRS_0099_00040_*.PCJ;3 
    

A Dead Letter Journal can be configured for each MessageQ message queuing group. On MessageQ for OpenVMS systems, DLJ files are created automatically by the JRN Server when a message queuing group is configured with JRN enabled.

To use the Dead Letter Journal, the sender program uses the pams_put_msg service specifying the appropriate delivery mode argument and PDEL_UMA_DLJ as the [uma] argument. Messages that are forced to take the UMA because they cannot be delivered are written to the dead letter journal of the sender's group. Both recoverable and non-recoverable messages can specify the DLJ UMA.

The DLJ file provides a backup mechanism for recovering messages that could not be delivered to their target queue. On OpenVMS systems, an application can provide recovery under program control by opening the DLJ, reading each message in the file and attempting redelivery by using the pams_open_jrn, pams_read_jrn, and pams_close_jrn services.

6.2.2.2 Using the Post Confirmation Journal

Recoverable messages that are processed by the receiver program can be written to the post confirmation journal of the target group. On OpenVMS systems, the receiver program uses the [force _j] argument of the pams_confirm_msg service to control whether successfully delivered messages are journaled as follows:

• PDEL_DEFAULT_JRN --- Enables writing the message to the PCJ file if the PCJ is set enabled.
• PDEL_FORCE_JRN --- Enables writing to a journal file regardless of whether the group was configured with journaling enabled.
• PDEL_NO_JRN --- Disables journaling regardless of the PCJ settings.

The default journaling action can be set under program control by the MRS_SET_PCJ message to the MRS Server process. The current PCJ file can also be closed and a new one opened by the same message.

6.2.2.3 Retrieving Journaled Messages

To retrieve journaled messages on MessageQ for OpenVMS systems, use the pams_open_jrn, pams_read_jrn, and pams_close_jrn services to open, read, and close the DLJ or PCJ files. Each journal file is assigned a journal handle by the pams_open_jrn routine. This handle is passed to pams_read_jrn and pams_close_jrn to identify the journal file.

Each successive journal read operation returns the next message. Arguments for pams_read_jrn are similar to arguments for pams_get_msg with the following additional message data that identifies the target of the message:

• Message queuing group and queue address of the target
• Confirmation value used by target application, PCJ only
• Date and Time message was written to file.

Once a message is retrieved from a journal file, it can be reprocessed, or reinserted into the message flow with the pams_put_msg service.

6.3 Starting MRS and JRN Servers

To use the message recovery services, set the following parameters in the profile section of the DMQ$INIT.TXT file (Example 6-2 ):

• Start the MRS Server by setting the symbol
  ENABLE_MRS to YES
• Start the JRN Server by setting the symbol
  ENABLE_JRN to YES

Example 6-2 is an excerpt from the profile section (refer to Chapter 3 for more information on MessageQ startup).

Example 6-2 DMQ$INIT.TXT Profile Section Enable Parameters

----------------------------------------------------------------- 
%PROFILE      ***** Profile Parameters ***** 
* 
... 
ENABLE_MRS  YES ! Enable MessageQ
 Message Recovery Services 
ENABLE_JRN  YES ! Enable Dead letter / Post Confirmation journal 
... 
* 
%EOS 
----------------------------------------------------------------- 

The following topics are covered in this section:

• Setting Parameters for MRS and JRN
• How MessageQ Manages Destination Queue Files (DQFs)
• Specifying the Location of MRS Files

Control the operation of MRS and JRN Servers by setting the parameters in the MRS/JRN initialization section of DMQ$INIT.TXT. Example 6-3 shows typical MRS and JRN initialization entries in the DMQ$INIT.TXT file.

Example 6-3 MRS and JRN Server Initialization Section

%MRS   ******* MRS/JRN Servers Initialization Section ************ 
* 
AREA_SIZE             512  ! disk blks per file (min:128, max:16384, def:512) 
NUM_DQF_AREAS        1000  ! min:100, max:1000000,    def:1000 
NUM_SAF_AREAS        1000  ! min:  0, max:1000000,    def:1000 
NUM_PCJ_AREAS        1000  ! min:  0, max:1000000,    def:1000 
NUM_DLJ_AREAS        1000  ! min:  0, max:1000000,    def:1000 
NUM_MESSAGES          512  ! min:128, max:2147483647, def:512 
NUM_QUEUES            128  ! min:128, max:2147483647, def:128 
CACHE_PERCENTAGE       90  ! % rcv quota for MRS msgs (min:1, max:100, def:90) 
* 
USE_HIGH_WATER_MARK   YES  ! checkpt MRS sizing params to disk    (YES/NO) 
LOAD_MRS_CTRS         YES  ! init recoverable msg ctrs on startup (YES/NO) 
RCVR_ONLY_CONFIRM     YES  ! limit confirms to receiving process  (YES/NO) 
* 
XGRP_JRN_CTRL          NO  ! allow JRN cntrl msgs from other groups (YES/NO) 
REDELIVERY_TIMER       10  ! integer seconds (min:0, max:5000, def:10) 
* 
PCJ_FILENAME    DMQ$MRS:MRS_%bg.PCJ  ! char[64] - %bg is a macro that 
DLJ_FILENAME    DMQ$MRS:MRS_%bg.DLJ  ! char[64] - expands to bus_group 
* 
%EOS 

Table 6-1 describes the MRS and JRN Server initialization parameters (fields).

Table 6-1 MRS/JRN Initialization Fields

Field	Description
AREA_SIZE	Number of disk blocks in a recovery file area. Minimum = 128, maximum = 16,384, default = 512
NUM_DQF_AREAS	The number of areas that can hold local recoverable messages. Minimum = 100, maximum = 1000000, default = 1000
NUM_SAF_AREAS	The number of areas that can hold remote recoverable messages. Minimum = 0, maximum = 1000000, default = 1000
NUM_PCJ_AREAS	The number of areas that can hold Post Confirmation Journal files. Minimum = 0, maximum = 1000000, default = 1000
NUM_DLJ_AREAS	The number of areas that can hold Dead Letter Journal files. Minimum = 0, maximum = 1000000, default = 1000
NUM_MESSAGES	Sets the number of messages (both remote and local) that MRS can retain in memory. Carefully review this setting and set the value of this parameter to the most realistic number possible. When this parameter is exhausted, the MRS Server will stop and allocate more memory and then continue processing. As the MRS Server grows, performance degrades. Minimum = 128; maximum = 2147483647; default = 512
NUM_QUEUES	Sets the number of queues that MRS can track. Minimum = 128; maximum = 2147483647; default = 128
CACHE_PERCENTAGE	The receive quota to determine the number of messages that MRS maintains queued in the message pool to a given target queue. Minimum = 1, maximum = 100, default = 90 We recommend that you always set CACHE_PERCENTAGE to less than 100. This parameter operates as follows: When a message queue is attached, enough recoverable messages are delivered to match or exceed this limit. When the limit is attained, no more recoverable messages are delivered until enough recoverable messages are confirmed by the target to drop below the limit. While recoverable message delivery is stopped due to the limit, new recoverable messages are written to the DQF file to be delivered in order of receipt. Nonrecoverable messages are delivered as usual.
USE_HIGH_WATER_MARK	The parameter that controls automatic MRS sizing on the disk (YES/NO). Set USE_HIGH_WATER_MARK = YES to size MRS from the largest values saved from previous runs. Set USE_HIGH_WATER_MARK = NO if sizing MRS from largest previous size causes problems at startup and causes MRS to purge its checkpoint database.
LOAD_MRS_CTRS	The parameter that initiates recoverable messsage counting prior to startup (YES/NO).
RCVR_ONLY_CONFIRM	The parameter that limits confirmations to the receiving process only (YES/NO).
XGRP_JRN_CTRL	The parameter that allows JRN cntrl msgs to come from other groups (YES/NO).
REDELIVERY_TIMER	A value that specifies the time interval, in seconds, between attempts to deliver a recoverable message to a target that has exceeded its receive quota. This value is also used for redelivery attempts when a MessageQ buffer pool is exhausted. Minimum = 0; maximum = 5000; default = 10 (seconds)
PCJ_FILENAME	The default postconfirmation journal file (PCJ) name (64 characters). It is in effect until another name is set via a SET_PCJ message. The default value is DMQ$MRS:MRS_ bbbb_ ggggg.PCJ, where bbbb is the bus ID and ggggg is the group ID.
DLJ_FILENAME	The default DLJ filename (64 characters). It remains in effect until another name is set using a SET_PCJ message. The default value is DMQ$MRS:MRS_ bbbb_ ggggg.DLJ, where bbbb is the bus ID and ggggg is the group ID.

Note:

After you set the MRS configuration parameters, you must restart the queuing group for these values to take effect. Use the Restart/Shutdown COM_SERVER option on the MessageQ main menu to restart the group.

Each permanent queue that receives recoverable messages requires a destination queue file located in the directory referred to by the logical name DMQ$MRS. DQF files are dynamically created and deleted as needed by the recovery system. Dynamic creation of DQF files provides the following benefits:

• No system management input is required to start receiving recoverable messages.
• You do not have to make any calculations of how much disk space to assign to each queue. Instead, systems are sized on the amount of disk space available, and how much of that space is to be allotted to recoverable messaging.

  ---

  Note:

  
  After the pool of disk space that is allotted to recoverable messaging is exhausted, the recovery system forces the Undeliverable Message Action (UMA) to occur for each inbound message until the space becomes available.

  ---

  
  The destination queue file is usually larger than the queue. Therefore, if the MRS Server wrote each message in the DQF to the target queue, it would overfill. To make sure that the MRS Server does not overfill the target queue, set the MRS parameter CACHE_PERCENTAGE to a value that is smaller than 100 and specify queue quotas for each queue that accepts recoverable messages.

Choose a common directory where the DQF files and SAF files will be written. Considerations include contention with other files used by your programs (database, logs, and so on) and projected available space on the disk. The location is specified by the logical name DMQ$MRS. Define DMQ$MRS as follows in the DMQ$BOOT command procedure file:

$ define/table=DMQ$LNM_TBL/EXEC/nolog DMQ$MRS directory_spec 

The default location is DMQ$DISK:[DMQ$V40.MRS_bbbb_ggggg], where bbbb is the 4-digit bus ID and ggggg is the 5-digit group ID.

For the cases when disk usage/loading is intense, MessageQ supports a list of directories located on separate devices, such as

$ define/table=DMQ$LNM_TBL/EXEC/nolog DMQ$MRS disk1:[mrsdir], 
-disk2:[mrsdir], diskn:[mrsdir] 

The MRS file system is split up into a variable number of fixed-sized areas. These areas are dynamically created before they are used and are assigned as needed.

6.5.1 Sizing the Amount of Recovery File Space

File creation occurs asynchronously ahead of use, based on the information accumulated on past loads. These files are known as FREE files. The minimum and maximum FREE files used can be controlled via the logicals DMQ$MRS_FREE_FILE_MIN and DMQ$MRS_FREE_FILE_MAX in the DMQ$SET_SERVER_LOGICALS.COM file in the DMQ$USER area for a specific group. The default minimum FREE files is ten (10). To disable the use of FREE files set the DMQ$MRS_FREE_FILE_MIN to zero (0). The sender program should not usually experience delays due to dynamic file creation; however, under heavy load, delays may occasionally occur.

The total disk space taken up by journals can be controlled by the parameters listed in Table 6-2 .

Table 6-2 Determining Journal File Size

Journal Parameter	Definition
AREA_SIZE	This parameter determines the size in disk blocks for each journal file. The default is 512. This size may need to be increased for performance reasons to cut down on the number of file creations required.
NUM_DQF_AREAS	This parameter determines the maximum number of Destination Queue files that can be held by a group. Therefore, the maximum amount of the disk space taken by DQF files is equal to AREA_SIZE * NUM_DQF_AREAS. The minimum allowed is 100.
NUM_SAF_AREAS	This parameter determines the maximum number of Store and Forward files that can be held by a group. Therefore, the maximum amount of the disk space taken by SAF files is equal to AREA_SIZE * NUM_SAF_AREAS. A setting of zero (0) will disable Store and Forward Journaling.
NUM_DLJ_AREAS	This parameter determines the maximum number of Dead Letter Journal files that can be held by a group. Therefore, the maximum amount of the disk space taken by DLJ files is equal to AREA_SIZE * NUM_DLJ_AREAS. A setting of zero (0) will disable Dead Letter Journaling.
NUM_PCJ_AREAS	This parameter determines the maximum number of Post Confirmation Journal files that can be held by a group. Therefore, the maximum amount of the disk space taken by PCJ files is equal to AREA_SIZE * NUM_PCJ_AREAS. A setting of zero (0) will disable Post Confirmation Journaling.

6.5.2 Sizing the Journal File Chunk Size

The journal file chunk size defaults to 512 bytes. This value is controlled by the DMQ$MRS_CHUNK_SIZE logical in the DMQ$SET_SERVER_LOGICALS.COM file in the groups DMQ$USER area. This value can be controlled for performance tuning. A chunk size of 512 bytes can hold up to a 480 byte message including the message header. Each chunk has a 32 byte record header. It is recommended that the chunk size reflect the average message size, but cannot exceed 32767 bytes. The chunk size will be block aligned by MRS.

6.5.3 Sizing MRS Server In-Memory Data Structures at Startup

The NUM_QUEUES and NUM_MESSAGES fields are used to set the initial size of the key MRS data structures when the MRS Server starts up.

NUM_QUEUES sets the number of queues (both remote and local) that MRS can keep track of. NUM_MESSAGES sets the number of messages that MRS can track. Both these parameters expand dynamically as needed.

Both NUM_QUEUES and NUM_MESSAGES can be automatically sized to the largest values from previous runs based on the setting of the USE_HIGH_WATER_MARK parameter. This allows MRS to allocate space more efficiently than if it expands its data structures several times. You can control the sizing by setting these parameters in the MRS and JRN initialization table.

While the MRS Server is running, it might request additional space to hold messages until confirmed.

Note:

If the value for maximum outgoing messages is too high, the MRS Server will not be able to allocate enough virtual memory without altering PGFLQUO for the MRS Server or the SYSGEN parameter VIRTUALPAGECNT. Large DQFs can also boost MRS memory allocation requests beyond PGFLQUO.

The operation of reading all recoverable messages and building an in-memory index does not scale well as the number of recoverable messages increases. To address this, recoverable files do not have to be scanned prior to use. This, however, means that reliable counts of the number of recoverable messages are not normally available immediately when the group starts up.

A switch setting is provided to enable accurate counts on startup, in the case that accurate counts of messages are required prior to starting messaging.

Set LOAD_MRS_CTRS to NO to reduce startup time by not counting messages.

Set LOAD_MRS_CTRS to YES to have MRS count all messages prior to startup.

6.6 MRS Internal Operation Tracing at Startup

MRS tracing can be turned on or off and redirected dynamically. You should not enable MRS internal operation tracing unless instructed to do so by BEA Systems Support. Tracing slows down the operation of the MRS Server and consumes a significant amount of disk space. Refer to Chapter 11 for detailed information on process and error tracing.

6.7 Confirming Message Removal from the Recovery System

In MessageQ , messages are removed from the recovery system through implicit or explicit confirmation. To explicitly confirm messages, the receiver program uses the pams_confirm_msg service. The action of the pams_confirm_msg service is determined by:

• CONFIRM_STYLE queue attribute (from the queue configuration table (QCT))
• DMQ$BLOCKING_CONFIRM logical name
• RCVR_ONLY_CONFIRM parameter (from the MRS and JRN initialization table)

The CONFIRM_STYLE attribute from the Queue Configuration Table (QCT) controls how the messages can be confirmed. The QCT is described in Chapter 5 . This section describes the CONFIRM_STYLE attribute only.

Valid CONFIRM_STYLE attribute settings are:

Setting	Description	Use
II	Implicit, In-order confirmation	If the confirm mode is set to [II], then each call to pams_get_msg or pams_get_msgw for an attached queue confirms the recoverable message from the previous call. Therefore, no special coding needs to be done in the application to correctly process recoverable messages. The messages are confirmed in the order they are dequeued. This is the most efficient way to confirm messages.
EI	Explicit, In-order confirmation	Explicit confirmation [EI] is useful when several messages need to be collected and then a single transaction applied using data from all the messages. The pams_confirm_msg service is used after all the information has been received and correctly applied. Setting the confirm mode to EI allows such a behavior. The messages must be confirmed in the order they are dequeued. This is the most efficient way to confirm messages.
EO	Explicit, Out-of-order confirmation	Same as above. However, several messages may be dequeued and then confirmed in any order by a call to pams_confirm_msg. This can be useful when handling multiple simultaneous transactions from several sources.

Note:

A period [.] specifies the default which is EO.

The logical name DMQ$BLOCKING_CONFIRM is used to control whether or not pams_confirm_msg waits for a response from the recovery system.

DMQ$BLOCKING_CONFIRM can be set in the process table to YES or NO; the default is YES. The messaging rate for draining recoverable messages from a DQF journal can be increased significantly by setting the logical name DMQ$BLOCKING_CONFIRM to NO.

This causes pams_confirm_msg to release control to the application immediately after the confirmation request is made to the recovery system instead of blocking for a response that indicates that the message has been deleted.

Note:

Set DMQ$BLOCKING_CONFIRM to NO only if the application can tolerate the occasional redelivery of a message in the case where a failure/restart occurs after a confirmation request is made, but before it is served.

DMQ$BLOCKING_CONFIRM can be set for a single process by defining the name in the process logical name table. It can also be set group-wide, by defining the name in DMQ$LNM_TBL.

6.7.3 Using RCVR_ONLY_CONFIRM Parameter

The RCVR_ONLY_CONFIRM parameter in the MRS initialization section of the group initialization file can be set (YES or NO) to limit message confirmations to the receiving process only.

MessageQ Selective Broadcast Services (SBS) enable applications to send a message to many receiver programs using a single program call. The sender program uses the standard pams_put_msg service to send a message to a set of queues registered to receive broadcast messages. The SBS Server also maintains the queue availability database used by the AVAIL/UNAVAIL services.

This version of MessageQ for OpenVMS includes the following new features:

• SBS provides an important enhancement to Ethernet multicasting called the Retransmission Protocol (RP). Each universal Multipoint Outbound Target (MOT) that supports Ethernet multicasting can be configured with an option to retry transmission of cross-group messages in the event of delivery problems. Messages sent to a MOT that is configured for retransmission are stored in the SBS retransmission list after they are broadcast. The size of the retransmission list is configured in the %SBS section of the group initialization file. These parameter sets the maximum number of messages stored by the SBS Server to fulfill retransmission requests in the event of message delivery failures.
• SBS supports a new and improved message API which is RISC aligned and feature enhanced. The API includes expanded rule based broadcast selection critera and status reporting.
• SBS contains support for large (4 MB) messages using the standard message transport (TCP/IP or DECnet), however direct Ethernet multicasting is limited to a maximum message size of 32000 bytes.
• SBS introduces the COMM_SERVICE to the %SBS section of the DMQ$INIT file. The COMM_SERVICE is intended to prioritize and support the future segmentation of broadcast streams.
• SBS supports dual rail Ethernet configurations for greater broadcast reliability.

Note:

In this version of MessageQ for OpenVMS, SBS includes a new set of message-based services, which obsoletes the old message interfaces. However, applications that currently use the prior versions of SBS will continue to run in a Version 4.0A MessageQ for OpenVMS environment. The older, obsolete, message interfaces are not supported on non-OpenVMS SBS platforms such as Unix or Windows NT.