|
|
|
|
|
|
Configuring Message Recovery
BEA MessageQ Message Recovery Services (MRS) provide a mechanism for guaranteed message delivery by storing messages on disk and automatically attempting redelivery until the message is received by the target system. On BEA MessageQ for OpenVMS systems, message recovery is provided by two servers:
The following topics are covered in this chapter:
How Message Recovery Services Work
In BEA MessageQ, the COM Server process starts up subsidiary servers, depending upon the contents of the Profile section (see Setting Servers and Configuring Groups in the Profile Section). The server which handles recoverable messaging is the MRS Server. It is started by setting ENABLE_MRS to YES. The server which handles auxiliary journal files is the Journal (JRN) Server. It is started by setting ENABLE_JRN to YES.
BEA MessageQ message recovery services (MRS) enable an application to send a message to a target application and guarantee its delivery despite an application, system, or network failure. A message is recoverable if BEA MessageQ writes it to a message recovery journal prior to delivery. BEA MessageQ has two message recovery journals that are controlled by the MRS Server:
In addition, the BEA MessageQ message recovery system provides the following auxiliary journal files (controlled by the JRN Server):
Using Recoverable Journal Files
If a recoverable message cannot be delivered to the target application, the MRS Server writes it to the SAF or DQF file, depending upon the delivery interest point selected when the message was sent. DQF and SAF files are dynamically created and deleted as needed by the recovery system (MRS_SERVER process).
After a message is written to the SAF or DQF file, the MRS Server writes a copy of the message into the message queue, where the receiver program can access it. After the receiver program has completed processing the message, it either reads the next recoverable message, thereby performing an implicit confirmation (if the queue is configured for this option), or calls the pams_confirm_msg service to explicitly instruct MRS to delete the message recovery file.
The size of SAF and DQF files is fixed and is set by the MRS parameter AREA_SIZE. The MRS Server creates a new SAF or DQF file each time the file fills up. MRS attempts to empty SAF files by periodically attempting to reach the corresponding remote queue. The MRS Server creates a separate set of SAF files for each remote queue. The maximum number of SAF files that can be created is set using the NUM_SAF_AREAS parameter. The maximum number of DQF files that can be created is set using the NUM_DQF_AREAS parameter.
In support of large messages, up to 4 MB, the journal files may be automatically extended by the MRS and JRN Servers in order to store a message that will not fit into the "fixed" size. This prevents a large message from being split across multiple journal files.
Both DQF and SAF files can be closed and the stream of messages in the file redirected to other targets. This can be in response to a message to the MRS Server from a suitably privileged user program, or activated from the Utilities menu (DMQ$MGR_UTILITY) as described in Using BEA MessageQ System Management Utilities.
Using Auxiliary Journal Files
The DLJ and PCJ journals are served by a separate server process called JRN_SERVER that maintains a history of all journal filename changes in a disk file. This history is displayed by the DIR and DUMP commands of DMQ$MGR_UTILITY to make it easy to find an old journal.
The PCJ and DLJ journal filename formats are defined in the %MRS section of DMQ$INIT.TXT as follows.
PCJ_FILENAME DMQ$MRS:MRS_%bg.PCJ ! char[64] - %bg is a macro that
DLJ_FILENAME DMQ$MRS:MRS_%bg.DLJ ! char[64] - expands to bus_group
BEA MessageQ adds a sequence and version number to the file names for file management purposes. The resulting default PCJ and DLJ file names appear as follows on disk:
DMQ$MRS:MRS_bbbb_ggggg_xxxxxx.PCJ;ver
DMQ$MRS:MRS_bbbb_ggggg_xxxxxx.DLJ;ver
In these filenames, bbbb is the bus number, gggggg is the group number, xxxxxx is a sequence number assigned by the JRN server, and ver is an OpenVMS file version number assigned by the JRN server. The journals use the sequence number to tie multiple journal files from the same stream together. The version number is used to distinguish between different journal streams.
Note: The file extension is user-defined. By default, the .PCJ and .DLJ extensions are used.
The application does not have to account for the internal sequence number (xxxxxx) added to the file names when using the pams_open_jrn API call.
Listing 5-1 shows a sample journal history. Note that the internal sequence number is not displayed in the listing.
Listing 5-1 Example Journal History
$type DMQ$MRS:DMQ_0099_00040_DLJ.JH
1-DEC-1997 12:58:45.70 DMQ$MRS:MRS_0099_00040_*.DLJ;1
6-JAN-1998 10:41:01.27 DMQ$MRS:MRS_0099_00040_*.DLJ;2
$type DMQ$MRS:DMQ_0099_00040_PCJ.JH;1
1-DEC-1997 12:58:47.80 DMQ$MRS:MRS_0099_00040_*.PCJ;1
6-JAN-1998 10:42:06.40 DMQ$MRS:MRS_0099_00040_*.PCJ;2
6-JAN-1998 10:42:12.97 DMQ$MRS:MRS_0099_00040_*.PCJ;3
Using the Dead Letter Journal
A Dead Letter Journal (DLJ) can be configured for each BEA MessageQ message queuing group. On BEA MessageQ for OpenVMS systems, DLJ files are created automatically by the JRN Server when a message queuing group is configured with JRN enabled.
To use the DLJ, the sender program uses the pams_put_msg service specifying the appropriate delivery mode argument and PDEL_UMA_DLJ as the uma argument. Messages that are forced to take the UMA because they cannot be delivered are written to the DLJ of the sender's group. Both recoverable and nonrecoverable messages can specify the DLJ UMA.
The DLJ file provides a backup mechanism for recovering messages that could not be delivered to their target queue. On OpenVMS systems, an application can provide recovery under program control by opening the DLJ, reading each message in the file and attempting redelivery by using the pams_open_jrn, pams_read_jrn, and pams_close_jrn services.
Using the Post Confirmation Journal
Recoverable messages that are processed by the receiver program can be written to the post confirmation journal (PCJ) of the target group. On OpenVMS systems, the receiver program uses the force_j argument of the pams_confirm_msg service to control whether successfully delivered messages are journaled as follows:
The default journaling action can be set under program control by the MRS_SET_PCJ message to the MRS Server process. The current PCJ file can also be closed and a new one opened by the same message.
Retrieving Journaled Messages
To retrieve journaled messages on BEA MessageQ for OpenVMS systems, use the pams_open_jrn, pams_read_jrn, and pams_close_jrn services to open, read, and close the DLJ or PCJ files. Each journal file is assigned a journal handle by the pams_open_jrn routine. This handle is passed to pams_read_jrn and pams_close_jrn to identify the journal file.
Each successive journal read operation returns the next message. Arguments for pams_read_jrn are similar to arguments for pams_get_msg with the following additional message data that identifies the target of the message:
After a message is retrieved from a journal file, it can be reprocessed, or reinserted into the message flow using the pams_put_msg service.
Starting MRS and JRN Servers
To use the message recovery services, set the following parameters in the profile section of the DMQ$INIT.TXT file as shown in Listing 5-2:
Listing 5-2 is an excerpt from the profile section (see Defining the Message Queuing Environment, for more information on BEA MessageQ startup).
Listing 5-2 DMQ$INIT.TXT Profile Section Enable Parameters
-----------------------------------------------------------------
%PROFILE ***** Profile Parameters *****
*
...
ENABLE_MRS YES ! Enable MessageQ Message Recovery Services
ENABLE_JRN YES ! Enable Dead letter / Post Confirmation journal
...
*
%EOS
-----------------------------------------------------------------
Configuring MRS and JRN Servers
The following topics are covered in this section:
Setting Parameters for MRS and JRN
You can control the operation of MRS and JRN Servers by setting the parameters in the MRS/JRN initialization section of DMQ$INIT.TXT. Listing 5-3 shows typical MRS and JRN initialization entries in the DMQ$INIT.TXT file.
Listing 5-3 MRS and JRN Server Initialization Section
%MRS ******* MRS/JRN Servers Initialization Section ************
*
AREA_SIZE 512 ! disk blks per file (min:128, max:16384, def:512)
NUM_DQF_AREAS 1000 ! min:100, max:1000000, def:1000
NUM_SAF_AREAS 1000 ! min: 0, max:1000000, def:1000
NUM_PCJ_AREAS 1000 ! min: 0, max:1000000, def:1000
NUM_DLJ_AREAS 1000 ! min: 0, max:1000000, def:1000
NUM_MESSAGES 512 ! min:128, max:2147483647, def:512
NUM_QUEUES 128 ! min:128, max:2147483647, def:128
CACHE_PERCENTAGE 90 ! % rcv quota for MRS msgs (min:1, max:100, def:90)
*
USE_HIGH_WATER_MARK YES ! checkpt MRS sizing params to disk (YES/NO)
LOAD_MRS_CTRS YES ! init recoverable msg ctrs on startup (YES/NO)
RCVR_ONLY_CONFIRM YES ! limit confirms to receiving process (YES/NO)
*
XGRP_JRN_CTRL NO ! allow JRN cntrl msgs from other groups (YES/NO)
REDELIVERY_TIMER 10 ! integer seconds (min:0, max:5000, def:10)
*
PCJ_FILENAME DMQ$MRS:MRS_%bg.PCJ ! char[64] - %bg is a macro that
DLJ_FILENAME DMQ$MRS:MRS_%bg.DLJ ! char[64] - expands to bus_group
*
%EOS
Table 5-1 describes the MRS and JRN Server initialization parameters (fields). Some parameters may need to be changed to accomodate large messages (up to 4MB). See Configuring BEA MessageQ for Large Messages in Sizing and Tuning the BEA MessageQ Environment, for more information.
Table 5-1 MRS/JRN Initialization Fields
|
Field |
Description |
|
AREA_SIZE |
Number of disk blocks in a recovery file area. |
|
NUM_DQF_AREAS |
The number of areas that can hold local recoverable messages. |
|
NUM_SAF_AREAS |
The number of areas that can hold remote recoverable messages. |
|
NUM_PCJ_AREAS |
The number of areas that can hold Post Confirmation Journal files. |
|
NUM_DLJ_AREAS |
The number of areas that can hold Dead Letter Journal files. |
|
NUM_MESSAGES |
Sets the number of messages (both remote and local) that MRS can retain in memory. Carefully review this setting and set the value of this parameter to the most realistic number possible. When this parameter is exhausted, the MRS Server will stop and allocate more memory and then continue processing. As the MRS Server grows, performance degrades. |
|
NUM_QUEUES |
Sets the number of queues that MRS can track. |
|
CACHE_PERCENTAGE |
The receive quota that determines the number of messages MRS maintains queued in the message pool to a given target queue. Minimum = 1, maximum = 100, default = 90. Note: We recommend that you always set This parameter operates as follows:
|
|
USE_HIGH_WATER_MARK |
The parameter that controls automatic MRS sizing on the disk (YES/NO). Set USE_HIGH_WATER_MARK = YES to size MRS from the largest values saved from previous runs. Set USE_HIGH_WATER_MARK = NO if sizing MRS from largest previous size causes problems at startup and causes MRS to purge its checkpoint database. |
|
LOAD_MRS_CTRS |
The parameter that initiates recoverable messsage counting prior to startup (YES/NO). |
|
RCVR_ONLY_CONFIRM |
The parameter that limits confirmations to the receiving process only (YES/NO). |
|
XGRP_JRN_CTRL |
The parameter that allows JRN control messages to come from other groups (YES/NO). |
|
REDELIVERY_TIMER |
A value that specifies the time interval, in seconds, between attempts to deliver a recoverable message to a target that has exceeded its receive quota. This value is also used for redelivery attempts when a BEA MessageQ buffer pool is exhausted. |
|
PCJ_FILENAME |
The default postconfirmation journal file (PCJ) name (64 characters). It is in effect until another name is set via a SET_PCJ message. The default value is DMQ$MRS:MRS_bbbb_ggggg.PCJ, where bbbb is the bus ID and ggggg is the group ID. |
|
DLJ_FILENAME |
The default DLJ filename (64 characters). It remains in effect until another name is set using a SET_PCJ message. The default value is DMQ$MRS:MRS_bbbb_ggggg.DLJ, where bbbb is the bus ID and ggggg is the group ID. |
Note: After you set the MRS configuration parameters, you must restart the queuing group for these values to take effect. Use the Restart/Shutdown COM_SERVER option on the BEA MessageQ main menu to restart the group.
How BEA MessageQ Manages Destination Queue Files (DQFs)
Each permanent queue that receives recoverable messages requires a destination queue file located in the directory referred to by the logical name DMQ$MRS. DQF files are dynamically created and deleted as needed by the recovery system. Dynamic creation of DQF files provides the following benefits:
Note: After the pool of disk space that is allotted to recoverable messaging is exhausted, the recovery system forces the Undeliverable Message Action (UMA) to occur for each inbound message until the space becomes available.
The destination queue file is usually larger than the queue. Therefore, if the MRS Server wrote each message in the DQF to the target queue, it would overfill. To make sure that the MRS Server does not overfill the target queue, set the MRS parameter CACHE_PERCENTAGE to a value that is smaller than 100 and specify queue quotas for each queue that accepts recoverable messages.
Specifying the Location of MRS Files
Choose a common directory where the DQF files and SAF files will be written. You should consider which other files are used by your programs (database, logs, and so on) and project available space on the disk. The location is specified by the logical name DMQ$MRS. Define DMQ$MRS as follows in the DMQ$BOOT command procedure file:
$ define/table=DMQ$LNM_TBL/EXEC/nolog DMQ$MRS directory_spec
The default location is DMQ$DISK:[DMQ$V50.MRS_bbbb_ggggg], where bbbb is the 4-digit bus ID and ggggg is the 5-digit group ID.
In cases where disk usage/loading is intense, BEA MessageQ supports a list of directories located on separate devices, for example:
$ define/table=DMQ$LNM_TBL/EXEC/nolog DMQ$MRS disk1:[mrsdir],
-disk2:[mrsdir], diskn:[mrsdir]
Sizing MRS File Space
The MRS file system is divided into a variable number of fixed-sized areas. These areas are dynamically created before they are used and are assigned as needed.
Sizing the Amount of Recovery File Space
File creation occurs asynchronously before files are used, based on the information accumulated on past loads. These files are known as FREE files. The minimum and maximum FREE files used can be controlled via the logicals DMQ$MRS_FREE_FILE_MIN and DMQ$MRS_FREE_FILE_MAX in the DMQ$SET_SERVER_LOGICALS.COM file in the DMQ$USER area for a specific group. The default minimum FREE files is ten (10). To disable the use of FREE files, set the DMQ$MRS_FREE_FILE_MIN to zero (0). The sender program should not usually experience delays due to dynamic file creation; however, under heavy load, delays may occasionally occur.
The total disk space taken up by journals can be controlled by the parameters listed in Table 5-2.
Table 5-2 Determining Journal File Size
|
Journal Parameter |
Definition |
|
AREA_SIZE |
This parameter determines the size in disk blocks for each journal file. The default is 512. This size may need to be increased for performance reasons to cut down on the number of file creations required. |
|
NUM_DQF_AREAS |
This parameter determines the maximum number of Destination Queue files (DQF) that can be held by a group. Therefore, the maximum amount of the disk space taken by DQF files is equal to AREA_SIZE * NUM_DQF_AREAS. The minimum allowed is 100. |
|
NUM_SAF_AREAS |
This parameter determines the maximum number of Store and Forward (SAF) files that can be held by a group. Therefore, the maximum amount of the disk space taken by SAF files is equal to AREA_SIZE * NUM_SAF_AREAS. A setting of zero (0) will disable Store and Forward Journaling. |
|
NUM_DLJ_AREAS |
This parameter determines the maximum number of Dead Letter Journal (DLJ) files that can be held by a group. Therefore, the maximum amount of the disk space taken by DLJ files is equal to AREA_SIZE * NUM_DQF_AREAS. A setting of zero (0) will disable Dead Letter Journaling. |
|
NUM_PCJ_AREAS |
This parameter determines the maximum number of Post Confirmation Journal (PCJ) files that can be held by a group. Therefore, the maximum amount of the disk space taken by PCJ files is equal to AREA_SIZE * NUM_PCJ_AREAS. A setting of zero (0) will disable Post Confirmation Journaling. |
Sizing the Journal File Chunk Size
The journal file chunk size defaults to 512 bytes. This value is controlled by the DMQ$MRS_CHUNK_SIZE logical in the DMQ$SET_SERVER_LOGICALS.COM file in the groups DMQ$USER area. This value can be set for performance tuning. A chunk size of 512 bytes can hold up to a 480 byte message including the message header. Each chunk has a 32 byte record header. It is recommended that the chunk size reflect the average message size, but cannot exceed 32767 bytes. The chunk size will be block aligned by MRS.
Sizing MRS Server In-Memory Data Structures at Startup
The NUM_QUEUES and NUM_MESSAGES fields are used to set the initial size of the key MRS data structures when the MRS Server starts up.
NUM_QUEUES sets the number of queues (both remote and local) that MRS can keep track of. NUM_MESSAGES sets the number of messages that MRS can track. Both these parameters expand dynamically as needed.
Both NUM_QUEUES and NUM_MESSAGES can be automatically sized to the largest values from previous runs based on the setting of the USE_HIGH_WATER_MARK parameter. This allows MRS to allocate space more efficiently than if it expands its data structures several times. You can control the sizing by setting these parameters in the MRS and JRN initialization table.
While the MRS Server is running, it might request additional space to hold messages until confirmed.
Note: If the value for maximum outgoing messages is too high, the MRS Server will not be able to allocate enough virtual memory without altering PGFLQUO for the MRS Server or the SYSGEN parameter VIRTUALPAGECNT. Large DQFs can also boost MRS memory allocation requests beyond PGFLQUO.
Reducing MRS Server Startup Time in Large-Scale Applications
Reading all recoverable messages and building an in-memory index are operations that do not scale well as the number of recoverable messages increases. To address this, recoverable files do not have to be scanned prior to use. This, however, means that reliable counts of the number of recoverable messages are not normally available immediately when the group starts up.
A switch setting is provided to enable accurate counts on startup, when accurate counts of messages are required prior to starting messaging.
Set LOAD_MRS_CTRS to NO to reduce startup time by not counting messages.
Set LOAD_MRS_CTRS to YES to have MRS count all messages prior to startup.
MRS Internal Operation Tracing at Startup
MRS tracing can be turned on or off and redirected dynamically. You should not enable MRS internal operation tracing unless instructed to do so by BEA Systems Customer Support. Tracing slows down the operation of the MRS Server and consumes a significant amount of disk space. Refer to Using BEA MessageQ System Management Utilities, for detailed information on process and error tracing.
Confirming Message Removal from the Recovery System
In BEA MessageQ, messages are removed from the recovery system through implicit or explicit confirmation. To explicitly confirm messages, the receiver program uses the pams_confirm_msg service. The action of the pams_confirm_msg service is determined by:
Using the CONFIRM_STYLE Attribute
The CONFIRM_STYLE attribute from the Queue Configuration Table (QCT) controls how messages can be confirmed. The QCT is described in Configuring Message Queues and Global Memory. This section describes the CONFIRM_STYLE attribute only.
Valid CONFIRM_STYLE attribute settings are listed in Table 5-3.
Table 5-3 Valid CONFIRM_STYLE Attribute Settings
|
Setting |
Description and Use |
|
II |
Implicit, In-order confirmation-If the confirm mode is set to [II], then each call to pams_get_msg or pams_get_msgw for an attached queue confirms the recoverable message from the previous call. Therefore, no special coding needs to be done in the application to correctly process recoverable messages. The messages are confirmed in the order they are dequeued. This is the most efficient way to confirm messages. |
|
EI |
Explicit, In-order confirmation-Explicit confirmation [EI] is useful when several messages need to be collected and then a single transaction applied using data from all the messages. The pams_confirm_msg service is used after all the information has been received and correctly applied. Setting the confirm mode to EI allows such a behavior. The messages must be confirmed in the order they are dequeued. |
|
EO |
Explicit, Out-of-order confirmation-This confirmation style is the same as explicit, in-order confirmation with the exception that several messages may be dequeued and then confirmed in any order by a call to pams_confirm_msg. This can be useful when handling multiple simultaneous transactions from several sources. |
Note: A period (.) specifies the default which is EO.
Controlling Recovery System Response with DMQ$BLOCKING_CONFIRM
The logical name DMQ$BLOCKING_CONFIRM is used to control whether or not pams_confirm_msg waits for a response from the recovery system.
DMQ$BLOCKING_CONFIRM can be set in the process table to YES or NO; the default is YES. The messaging rate for draining recoverable messages from a DQF journal can be increased significantly by setting the logical name DMQ$BLOCKING_CONFIRM to NO.
This setting causes pams_confirm_msg to release control to the application immediately after the confirmation request is made to the recovery system instead of blocking for a response that indicates that the message has been deleted.
Note: Set DMQ$BLOCKING_CONFIRM to NO only if the application can tolerate the occasional redelivery of a message in the case where a failure/restart occurs after a confirmation request is made, but before it is served.
DMQ$BLOCKING_CONFIRM can be set for a single process by defining the name in the process logical name table. It can also be set group-wide, by defining the name in DMQ$LNM_TBL.
Using RCVR_ONLY_CONFIRM Parameter
The RCVR_ONLY_CONFIRM parameter in the MRS initialization section of the group initialization file can be set (YES or NO) to limit message confirmations to the receiving process only.
|
|
|
Copyright © 2000 BEA Systems, Inc. All rights reserved.
|