Chapter 6 Managing Physical Destinations

This chapter explains how you use the imqcmd utility to manage physical destinations. A Message Queue^TM message is routed to its consumer clients by way of a physical destination on a broker. The broker manages the memory and persistent storage associated with the physical destinations, and sets their behaviors.

In a broker cluster, you create a physical destination on one broker and the cluster propagates that physical destination to all the others. An application client can subscribe to a topic or consume from a queue that is on any broker in the cluster, because the brokers cooperate to route messages across the cluster. However, only the broker to which a message was originally produced manages persistence and acknowledgment for that message.

This chapter covers the following topics:

• Using the Command Utility

• Creating a Physical Destination

• Listing Physical Destinations

• Displaying Information about Physical Destinations

• Updating Physical Destination Properties

• Pausing and Resuming Physical Destinations

• Purging Physical Destinations

• Destroying Physical Destinations

• Compacting Physical Destinations

• Using the Dead Message Queue

Table 13–5 provides full reference information about the imqcmd subcommands for managing physical destinations and accomplishing these tasks.

See the Message Queue Technical Overview for an introduction to physical destinations.

A client application uses a Destination object whenever it interacts with a physical destination. For provider-independence and portability, clients typically use administrator-created destination objects, which are called destination administered objects. You can configure administered objects for use by client applications, as described in Chapter 8, Managing Administered Objects

Using the Command Utility

You use the Message Queue Command utility (imqcmd) to manage physical destinations. The syntax of the imqcmd command is the same as when you use it for managing other broker services.

Full reference information about imqcmd, its subcommands, and its options, is available in Chapter 13, Command Line Reference.

Subcommands

Table 6–1 lists the imqcmd subcommands whose use is described in this chapter. For reference information about these subcommands, see Physical Destination Management.

Creating a Physical Destination

To create a physical destination, use the imqcmd create subcommand. This is the syntax for the create subcommand:

create dst -t destType -n 
destName [-o property=value
] [-o property=value1]
…

For example, to create a queue destination, enter a command like the following:

imqcmd create dst -n myQueue -t q -o “maxNumActiveConsumers=5”

To create a topic destination, enter a command like the following:

imqcmd create dst -n myTopic -t t -o “maxBytesPerMsg=5000”

When creating a physical destination, you specify the following:

• The physical destination type, t (topic) or q (queue).

• The physical destination name. The naming rules are as follows:

  • The name must contain only alphanumeric characters. It cannot contain spaces.

  • The name can begin with an alphabetic character, the underscore character (_) or the dollar sign ($). It cannot begin with the character string mq.

  Nondefault values for the physical destination’s properties.

You can also set properties when you update a physical destination.

Many physical destination properties affect broker memory resources and message flow. For example, you can specify the number of producers that can send to a physical destination, the number and size of the messages they can send, and the response that the broker should take when physical destination limits are reached. The limits are similar to brokerwide limits controlled by broker configuration properties.

The following properties are used for both queue destinations and topic destinations:

• maxNumMsgs. Specifies the maximum number of unconsumed messages allowed in the physical destination.

• maxTotalMsgBytes. Specifies the maximum total amount of memory (in bytes) allowed for unconsumed messages in the physical destination.

• limitBehavior. Specifies how the broker responds when a memory-limit threshold is reached.

• maxBytesPerMsg. Specifies the maximum size (in bytes) of any single message allowed in the physical destination.

• maxNumProducers. Specifies the maximum number of producers for the physical destination.

• consumerFlowLimit. Specifies the maximum number of messages to be delivered to a consumer in a single batch.

• isLocalOnly. Applies only to broker clusters. Specifies that a physical destination is not replicated on other brokers, and is limited to delivering messages only to local consumers (consumers connected to the broker on which the physical destination is created).

• useDMQ. Specifies whether a physical destination’s dead messages are discarded or put on the dead message queue.

The following properties are used for queue destinations only:

• maxNumActiveConsumers. Specifies the maximum number of consumers that can be active in load-balanced delivery from a queue destination.)

• maxNumBackupConsumers. Specifies the maximum number of backup consumers that can take the place of active consumers, if any fail during load-balanced delivery from a queue destination.

• localDeliveryPreferred. Applies only to load-balanced queue delivery in broker clusters. Specifies that messages be delivered to remote consumers only if there are no consumers on the local broker.

See Chapter 15, Physical Destination Property Reference for full reference information about physical destination properties.

For auto-created destinations, you set default property values in the broker’s instance configuration file. Reference information on auto-create properties is located in Table 14–3.

Listing Physical Destinations

You can get information about a physical destination’s current property values, about the number of producers or consumers associated with a physical destination, and about messaging metrics, such as the number and size of messages in the physical destination.

To find a physical destination about which you want to get information, list all physical destinations on a broker using the list dst subcommand. This is the syntax for the list dst subcommand:

list dst [-t destType] [-tmp]

The command lists physical destinations of the specified type. The value for the destination type (-t) option can be q (queue) or t (topic).

If you omit the destination type, physical destinations of all types are listed.

The list dst subcommand can optionally specify the type of destination to list or include temporary destinations (using the -tmp option). Temporary destinations are created by clients, normally for the purpose of receiving replies to messages sent to other clients.

For example, to get a list of all physical destinations on the broker running on myHost at port 4545, enter the following command:

imqcmd list dst -b myHost:4545

Information for the dead message queue, mq.sys.dmq, is always displayed, in addition to any other physical destinations, unless you specify the destination type t to include only topics.

Displaying Information about Physical Destinations

To get information about a physical destination’s current properties, use the query dst subcommand. This is the syntax of the query dst subcommand:

query dst -t destType -n 
destName

The command lists information about the destination of the specified type and name. For example, the following command displays information about the queue destination XQueue:

imqcmd query dst -t q -n XQueue -u admin

The command produces output like the following:

------------------------------------
Destination Name    Destination Type
------------------------------------
XQueue              Queue

On the broker specified by:

-------------------------
Host         Primary Port
-------------------------
localhost    7676

Destination Name                      XQueue
Destination Type                      Queue
Destination State                     RUNNING
Created Administratively              true

Current Number of Messages            0
Current Total Message Bytes           0
Current Number of Producers           0
Current Number of Active Consumers    0
Current Number of Backup Consumers    0

Max Number of Messages                unlimited (-1)
Max Total Message Bytes               unlimited (-1)
Max Bytes per Message                 unlimited (-1)
Max Number of Producers               100
Max Number of Active Consumers        1
Max Number of Backup Consumers        0

Limit Behavior                        REJECT_NEWEST
Consumer Flow Limit                   1000
Is Local Destination                  false
Local Delivery is Preferred           false
Use Dead Message Queue                true

The output also shows the number of producers and consumers associated with the destination. For queue destinations, the number includes active consumers and backup consumers.

You can use the update dst subcommand to change the value of one or more properties (see Updating Physical Destination Properties).

Updating Physical Destination Properties

You can change the properties of a physical destination by using the update dst subcommand and the -o option to specify the property to update. This is the syntax for the update dst subcommand:

update dst -t destType -n 
destName -o property=value [[-o 
property=value1]…]

The command updates the value of the specified properties at the specified destination. The property name can be any property listed in Table 15–1.

You can use the -o option multiple times to update multiple properties. For example, the following command changes the maxBytesPerMsg property to 1000 and the MaxNumMsgs property to 2000:

imqcmd update dst -t q -n myQueue -o “maxBytesPerMsg=1000”
              -o “maxNumMsgs=2000” -u admin

See Chapter 15, Physical Destination Property Reference for a list of the properties that you can update.

You cannot use the update dst subcommand to update the type of a physical destination or to update the isLocalOnly property.

The dead message queue is a specialized physical destination whose properties differ somewhat from those of other destinations. For more information, see Using the Dead Message Queue.

Pausing and Resuming Physical Destinations

You can pause a physical destination to control the delivery of messages from producers to the destination, or from the destination to consumers, or both. In particular, you can pause the flow of messages into a destination to help prevent destinations from being overwhelmed with messages when production of messages is much faster than consumption. You must pause a physical destination before compacting it.

To pause the delivery of messages to or from a physical destination, use the pause dst subcommand. This is the syntax of the pause dst subcommand:

pause dst [-t destType -n 
destName] [-pst pauseType]

The subcommand pauses the delivery of messages to consumers (-pst CONSUMERS ), or from producers (-pst PRODUCERS), or both ( -pst ALL), for the destination of the specified type and name. If no destination type and name are specified, all physical destinations are paused. The default is ALL.

Example:

imqcmd pause dst -n myQueue -t q -pst PRODUCERS -u admin
imqcmd pause dst -n myTopic -t t -pst CONSUMERS -u admin

To resume delivery to a paused destination, use the resume dst subcommand. This is the syntax of the resume dst subcommand:

resume dst [-t destType -n 
destName]

The subcommand resumes delivery of messages to the paused destination of the specified type and name. If no destination type and name are specified, all destinations are resumed.

Example:

imqcmd resume dst -n myQueue -t q

In a broker cluster, instances of the physical destination reside on each broker in the cluster. You must pause each one individually.

Purging Physical Destinations

You can purge all messages currently queued at a physical destination. Purging a physical destination means that all messages stored at the destination are deleted.

You might want to purge messages when the accumulated messages are taking up too much of the system’s resources. This might happen when a queue does not have registered consumer clients and is receiving many messages. It might also happen if inactive durable subscribers to a topic do not become active. In both cases, messages are held unnecessarily.

To purge messages at a physical destination, use the purge dst subcommand. This is the syntax of the purge dst subcommand:

purge dst -t destType -n 
destName

The subcommand purges messages at the physical destination of the specified type and name.

Examples:

imqcmd purge dst -n myQueue -t q -u admin
imqcmd purge dst -n myTopic -t t -u admin

If you have shut down the broker and do not want old messages to be delivered when you restart it, use the -reset messages option to purge stale messages; for example:

imqbrokerd -reset messages -u admin

This saves you the trouble of purging destinations after restarting the broker.

In a broker cluster, instances of the physical destination reside on each broker in the cluster. You must purge each of these destinations individually.

Destroying Physical Destinations

To destroy a physical destination, use the destroy dst subcommand. This is the syntax of the destroy dst subcommand:

destroy dst -t destType -n 
destName

The subcommand destroys the physical destination of the specified type and name.

Example:

imqcmd destroy dst -t q -n myQueue -u admin

Destroying a physical destination purges all messages at that destination and removes it from the broker; the operation is not reversible.

You cannot destroy the dead message queue.

Compacting Physical Destinations

If you are using a file-based data store as the persistent store for messages, you can monitor disk utilization and compact the disk when necessary.

The file-based message store is structured so that messages are stored in directories corresponding to the physical destinations in which they are being held. In each physical destination’s directory, most messages are stored in one file consisting of variable-sized records. (To alleviate fragmentation, messages whose size exceeds a configurable threshold are stored in their own individual files.)

As messages of varying sizes are persisted and then removed from the record file, holes may develop in the file where free records are not being re-used.

To manage unused free records, the Command utility includes subcommands for monitoring disk utilization per physical destination and for reclaiming free disk space when utilization drops.

Monitoring a Physical Destination’s Disk Utilization

To monitor a physical destination’s disk utilization, use a command like the following:

imqcmd metrics dst -t q -n myQueue -m dsk -u admin

This command produces output like the following:

--------------------------------------
Reserved   Used      Utilization Ratio
--------------------------------------
806400     804096    99
1793024    1793024   100
2544640    2518272   98

The columns in the subcommand output have the following meaning:

Reclaiming Unused Physical Destination Disk Space

The disk utilization pattern depends on the characteristics of the messaging application that uses a particular physical destination. Depending on the relative flow of messages into and out of a physical destination, and the relative size of messages, the reserved disk space might grow over time.

If the message producing rate is greater than the message consuming rate, free records should generally be reused and the utilization ratio should be on the high side. However, if the message producing rate is similar to or smaller than the message consuming rate, you can expect that the utilization ratio will be low.

In general, you want the reserved disk space to stabilize and the utilization to remain high. As a rule, if the system reaches a steady state in which the amount of reserved disk space generally stays constant and utilization rate is high (above 75%), there is no need to reclaim the unused disk space. If the system reaches a steady state and utilization rate is low (below 50%), you can compact the disk to reclaim the disk space occupied by free records.

Use the compact dst subcommand to compact the data store. This is the syntax for the compact dst subcommand:

compact dst [-t destType -n 
destName]

The subcommand compacts the file-based data store for the physical destination of the specified type and name. If no destination type and name are specified, all destinations are compacted. Physical destinations must be paused before they can be compacted.

If the reserved disk space continues to increase over time, reconfigure the destination’s memory management by setting destination memory limit properties and limit behaviors (see Table 15–1).

To Reclaim Unused Physical Destination Disk Space

1. Pause the destination.

   imqcmd pause dst -t q -n myQueue -u admin

2. Compact the disk.

   imqcmd compact dst -t q -n myQueue -u admin

3. Resume the physical destination.

   imqcmd resume dst -t q -n myQueue -u admin

   If destination type and name are not specified, these operations are performed for all physical destinations.

Using the Dead Message Queue

The dead message queue, mq.sys.dmq, is a system-created physical destination that holds the dead messages of a broker and its other physical destinations. The dead message queue is a tool for monitoring, tuning system efficiency, and troubleshooting. For a definition of the term “dead message” and a more detailed introduction to the dead message queue, see the Message Queue Technical Overview.

The broker automatically creates a dead message queue when it starts. The broker places messages on the queue if it cannot process them, or if their time-to-live has expired. In addition, other physical destinations can use the dead message queue to hold discarded messages. Use of the dead message queue provides information that is useful for troubleshooting the system.

Configuring Use of the Dead Message Queue

By default, a physical destination is configured to use the dead message queue. You can disable a physical destination from using the dead message queue, or enable it to do so, by setting the physical destination property useDMQ .

The following example creates a queue called myDist that uses the dead message queue by default:

imqcmd create dst -n myDist -t q

The following example disables use of the dead message queue for the same queue:

imqcmd update dst -n myDist -t q -o useDMQ=false

You can enable all auto-created physical destinations on a broker to use the dead message queue, or disable them from doing so, by setting the imq.autocreate.destination.useDMQ broker property.

Managing the Dead Message Queue

You can use the Message Queue Command utility (imqcmd) to manage the dead message queue as you manage other queues, with some differences. For example, because the dead message queue is system-created, you cannot create, pause, or destroy it. Also, as shown in Table 6–3, default values for the dead message queue sometimes differ from those of normal queues.

Dead Message Queue Properties

You configure the dead message queue as you configure other queues, but certain physical destination properties do not apply or have different default values. Table 6–3 lists queue properties that the dead message queue handles in a unique way.

Message Contents

A broker can place an entire message on the dead message queue, or it can discard the message body contents, retaining just the header and property data. By default, the dead message queue stores entire messages.

If you want to reduce the size of the dead message queue and if you do not plan to restore dead messages, consider setting the imq.destination.DMQ.truncateBody broker property to true:

imqcmd update bkr -o imq.destination.DMQ.truncateBody=true

This will discard the message body and retain only the headers and property data.

Enabling Dead Message Logging

Dead message logging is disabled by default. Enabling dead message logging allows the broker to log the following events:

• The broker moves a message to the dead message queue

• The broker discards a message from the dead message queue and from any physical destination that does not use the dead message queue

• A physical destination reaches its limits

The following command enables dead message logging:

imqcmd update bkr -o imq.destination.logDeadMsgs=true

Dead message logging applies to all physical destinations that use the dead message queue. You cannot enable or disable logging for an individual physical destination.