After installing Sun Java SystemTM Message QueueTM and performing some preparatory steps, you can begin starting brokers and clients. A broker’s configuration is governed by a set of configuration files, which can be overridden by command line options passed to the Broker utility (imqbrokerd); see Chapter 4, Configuring a Broker for more information.
This chapter contains the following sections:
Before starting a broker, there are two preliminary system-level tasks to perform: synchronizing system clocks and (on the Solaris or Linux platform) setting the file descriptor limit. The following sections describe these tasks.
Before starting any brokers or clients, it is important to synchronize the clocks on all hosts that will interact with the Message Queue system. Synchronization is particularly crucial if you are using message expiration (time-to-live). Time stamps from clocks that are not synchronized could prevent message expiration from working as expected and prevent the delivery of messages. Synchronization is also crucial for broker clusters.
Configure your systems to run a time synchronization protocol, such as Simple Network Time Protocol (SNTP). Time synchronization is generally supported by the xntpd daemon on Solaris and Linux, and by the W32Time service on Windows. (See your operating system documentation for information about configuring this service.) After the broker is running, avoid setting the system clock backward.
On the Solaris and Linux platforms, the shell in which a client or broker is running places a soft limit on the number of file descriptors that a process can use. In Message Queue, each connection a client makes, or a broker accepts, uses one of these file descriptors. Each physical destination that has persistent messages also uses a file descriptor.
As a result, the file descriptor limit constrains the number of connections a broker or client can have. By default, the maximum is 256 connections on Solaris or 1024 on Linux. (In practice, the connection limit is actually lower than this because of the use of file descriptors for persistence.) If you need more connections than this, you must raise the file descriptor limit in each shell in which a client or broker will be executing. For information on how to do this, see the ulimit man page.
You can start a broker either interactively, using the Message Queue command line utilities or the Windows Start menu, or by arranging for it to start automatically at system startup. The following sections describe how.
You can start a broker interactively from the command line, using the Broker utility (imqbrokerd). (Alternatively, on Windows, you can start a broker from the Start menu.) You cannot use the Administration Console (imqadmin) or the Command utility (imqcmd) to start a broker; the broker must already be running before you can use these tools.
On the Solaris and Linux platforms, a broker instance must always be started by the same user who initially started it. Each broker instance has its own set of configuration properties and file-based message store. When the broker instance first starts, Message Queue uses the user’s file creation mode mask (umask ) to set permissions on directories containing the configuration information and persistent data for that broker instance.
A broker instance has the instance name imqbroker by default. To start a broker from the command line with this name and the default configuration, simply use the command
imqbrokerd
This starts a broker instance named imqbroker on the local machine, with the Port Mapper at the default port of 7676 (see Port Mapper).
To specify an instance name other than the default, use the- name option to the imqbrokerd command. The following command starts a broker with the instance name myBroker:
imqbrokerd -name myBroker
Other options are available on the imqbrokerd command line to control various aspects of the broker’s operation. The following example uses the- tty option to send errors and warnings to the command window (standard output):
imqbrokerd -name myBroker -tty
You can also use the -D option on the command line to override the values of properties specified in the broker’s instance configuration file (config.properties). This example sets the imq.jms.max_threads property, raising the maximum number of threads available to the jms connection service to 2000:
imqbrokerd -name myBroker -Dimq.jms.max_threads=2000
See Broker Utility for complete information on the syntax, subcommands, and options of the imqbrokerd command. For a quick summary of this information, enter the command
imqbrokerd -help
If you have a Sun Java System Message Queue Platform Edition license, you can use the imqbrokerd command’s- license option to activate a trial Enterprise Edition license, allowing you to try Enterprise Edition features for 90 days. Specify try as the license name:
imqbrokerd -license try
You must use this option each time you start a broker; otherwise the broker will default to the standard Platform Edition license.
Instead of starting a broker explicitly from the command line, you can set it up to start automatically at system startup. How you do this depends on the platform (Solaris, Linux, or Windows) on which you are running the broker.
On Solaris and Linux systems, scripts that enable automatic startup are placed in the /etc/rc* directory tree during Message Queue installation. To enable the use of these scripts, you must edit the configuration file /etc/imq/imqbrokerd.conf (Solaris) or /etc/opt/sun/mq/imqbrokerd.conf (Linux) as follows:
To start the broker automatically at system startup, set the AUTOSTART property to YES.
To have the broker restart automatically after an abnormal exit, set the RESTART property to YES.
To set startup command line arguments for the broker, specify one or more values for the ARGS property.
To start a broker automatically at Windows system startup, you must define the broker as a Windows service. The broker will then start at system startup time and run in the background until system shutdown. Consequently, you do not use the imqbrokerd command to start the broker unless you want to start an additional instance.
A system can have no more than one broker running as a Windows service. Task Manager lists such a broker as two executable processes:
The native Windows service wrapper, imqbrokersvc.exe
The Java runtime that is running the broker
You can install a broker as a service when you install Message Queue on a Windows system. After installation, you can use the Service Administrator utility ( imqsvcadmin) to perform the following operations:
Add a broker as a Windows service
Determine the startup options for the broker service
Remove a broker that is running as a Windows service
To pass startup options to the broker, use the -args argument to the imqsvcadmin command. This works the same way as the imqbrokerd command’s -D option, as described under Starting Brokers. Use the Command utility (imqcmd) to control broker operations as usual.
See Service Administrator Utility for complete information on the syntax, subcommands, and options of the imqsvcadmin command.
The procedure for reconfiguring a broker installed as a Windows service is as follows:
Stop the service.
From the Settings submenu of the Windows Start menu, choose Control Panel.
Open the Administrative Tools control panel.
Run the Services tool by selecting its icon and choosing Open from the File menu or the pop-up context menu, or simply by double-clicking the icon.
Under Services (Local), select the Message Queue Broker service and choose Properties from the Action menu.
Alternatively, you can right-click on Message Queue Broker and choose Properties from the pop-up context menu, or simply double-click on Message Queue Broker. In either case, the Message Queue Broker Properties dialog box will appear.
Under the General tab in the Properties dialog, click Stop to stop the broker service.
Remove the service.
On the command line, enter the command
imqsvcadmin remove |
Reinstall the service, specifying different broker startup options with the -args option or different Java version arguments with the - vmargs option.
For example, to change the service’s host name and port number to broker1 and 7878, you could use the command
imqsvcadmin install -args "-name broker1 -port 7878" |
You can use either imqsvcadmin command’s -javahome or -jrehome option to specify the location of an alternative Java runtime. (You can also specify these options in the Start Parameters field under the General tab in the service’s Properties dialog window.)
The Start Parameters field treats the backslash character (\\ ) as an escape character, so you must type it twice when using it as a path delimiter: for example,
-javahome c:\\\\j2sdk1.4.0
To determine the startup options for the broker service, use the query option to the imqsvcadmin command, as shown in Example 3–1.
|
If you get an error when you try to start a broker as a Windows service, you can view error events that were logged:
Open the Windows Administrative Tools control panel.
Start the Event Viewer tool.
Select the Application event log.
Choose Refresh from the Action menu to display any error events.
The procedure for removing a broker again varies from one platform to another, as described in the following sections.
To remove a broker instance on the Solaris or Linux platform, use the imqbrokerd command with the -remove option. The command format is as follows:
imqbrokerd [options…] -remove instance
For example, if the name of the broker is myBroker, the command would be
imqbrokerd -name myBroker -remove instance
The command deletes the entire instance directory for the specified broker.
If the broker is set up to start automatically at system startup, edit the configuration file /etc/imq/imqbrokerd.conf (Solaris) or /etc/opt/sun/mq/imqbrokerd.conf (Linux) and set the AUTOSTART property to NO.
See Broker Utility for complete information on the syntax, subcommands, and options of the imqbrokerd command. For a quick summary of this information, enter the command
To remove a broker that is running as a Windows service, use the command
imqcmd shutdown bkr
to shut down the broker, followed by
imqsvcadmin remove
to remove the service.
Alternatively, you can use the Windows Services tool, reached via the Administrative Tools control panel, to stop and remove the broker service.
Restart your computer after removing the broker service.
Before starting a client application, obtain information from the application developer about how to set up the system. If you are starting Java client applications, you must set the CLASSPATH variable appropriately and make sure you have the correct .jar files installed. The Message Queue Developer's Guide for Java Clients contains information about generic steps for setting up the system, but your developer may have additional information to provide.
To start a Java client application, use the following command line format:
java clientAppName
To start a C client application, use the format supplied by the application developer.
The application’s documentation should provide information on attribute values that the application sets; you may want to override some of these from the command line. You may also want to specify attributes on the command line for any Java client that uses a Java Naming and Directory Interface (JNDI) lookup to find its connection factory. If the lookup returns a connection factory that is older than the application, the connection factory may lack support for more recent attributes. In such cases, Message Queue sets those attributes to default values; if necessary, you can use the command line to override these default values.
To specify attribute values from the command line for a Java application, use the following syntax:
java [[-Dattribute=value] …] clientAppName
The value for attribute must be a connection factory administered object attribute, as described in Chapter 16, Administered Object Attribute Reference If there is a space in the value, put quotation marks around the attribute=value part of the command line.
The following example starts a client application named MyMQClient, connecting to a broker on the host OtherHost at port 7677:
java -DimqAddressList=mq://OtherHost:7677/jms MyMQClient
The host name and port specified on the command line override any others set by the application itself.
In some cases, you cannot use the command line to specify attribute values. An administrator can set an administered object to allow read access only, or an application developer can code the client application to do so. Communication with the application developer is necessary to understand the best way to start the client program.
A broker’s configuration is governed by a set of configuration files and by the options passed to the imqbrokerd command at startup. This chapter describes the available configuration properties and how to use them to configure a broker.
The chapter contains the following sections:
For full reference information about broker configuration properties, see Chapter 14, Broker Properties Reference
Broker configuration properties can be divided into several categories, depending on the services or broker components they affect:
Connection services manage the physical connections between a broker and its clients that provide transport for incoming and outgoing messages.
Routing services route and deliver JMS payload messages, as well as control messages used by the message service to support reliable delivery.
Persistence services manage the writing and retrieval of data to and from persistent storage.
Security services authenticate users connecting to the broker and authorize their actions.
Monitoring services generate metric and diagnostic information about the broker’s performance.
The following sections describe each of these services and the properties you use to customize them for your particular needs.
Message brokers can offer various connection services supporting both application and administrative clients, using a variety of transport protocols. Broker configuration properties related to connection services are listed under Connection Properties.
Table 4–1 shows the available connection services which are distinguished by two characteristics:
The service type specifies whether the service provides JMS message delivery (NORMAL) or Message QueueTM administration services ( ADMIN).
The protocol type specifies the underlying transport protocol.
Service Name |
Service Type | |
---|---|---|
NORMAL | ||
NORMAL | ||
NORMAL | ||
NORMAL | ||
ADMIN |
TCP |
|
ADMIN |
TLS (SSL-based security) |
By setting a broker’s imq.service.activelist property, you can configure it to run any or all of these connection services. The value of this property is a list of connection services to be activated when the broker is started up; if the property is not specified explicitly, the jms and admin services will be activated by default.
Each connection service also supports specific authentication and authorization features; see Security Services for more information.
Each connection service is available at a particular port, specified by host name (or IP address) and port number. You can explicitly specify a static port number for a service or have the broker’s Port Mapper assign one dynamically. The Port Mapper itself resides at the broker’s primary port, which is normally located at the standard port number 7676. (If necessary, you can use the broker configuration property imq.portmapper.port to override this with a different port number.) By default, each connection service registers itself with the Port Mapper when it starts up. When a client creates a connection to the broker, the Message Queue client runtime first contacts the Port Mapper, requesting a port number for the desired connection service.
Alternatively, you can override the Port Mapper and explicitly assign a static port number to a connection service, using the imq.serviceName.protocolType. port configuration property (where serviceName and protocolType identify the specific connection service, as shown in Table 4–1). (Only the jms, ssljms, admin, and ssladmin connection services can be configured this way; the httpjms and httpsjms services use different configuration properties, described in Appendix C, HTTP/HTTPS Support) Static ports are generally used only in special situations, however, such as in making connections through a firewall (see Connecting Through a Firewall), and are not recommended for general use.
In cases where two or more hosts are available (such as when more than one network card is installed in a computer), you can use broker properties to specify which host the connection services should bind to. The imq.hostname property designates a single default host for all connection services; this can then be overridden, if necessary, with imq.serviceName. protocolType.hostname (for the jms, ssljms, admin, or ssladmin service) or imq.portmapper.hostname (for the Port Mapper itself).
When multiple Port Mapper requests are received concurrently, they are stored in an operating system backlog while awaiting action. The imq.portmapper.backlog property specifies the maximum number of such backlogged requests. When this limit is exceeded, any further requests will be rejected until the backlog is reduced.
Each connection service is multithreaded, supporting multiple connections. The threads needed for these connections are maintained by the broker in a separate thread pool for each service. As threads are needed by a connection, they are added to the thread pool for the service supporting that connection.
The threading model you choose specifies whether threads are dedicated to a single connection or shared by multiple connections:
In the dedicated model, each connection to the broker requires two threads: one for incoming and one for outgoing messages. This limits the number of connections that can be supported, but provides higher performance.
In the shared model, connections are processed by a shared thread when sending or receiving messages. Because each connection does not require dedicated threads, this model increases the number of possible connections, but at the cost of lower performance because of the additional overhead needed for thread management.
The broker’s imq.serviceName. threadpool_model property specifies which of the two models to use for a given connection service. This property takes either of two string values: dedicated or shared. If you don’t set the property explicitly, dedicated is assumed by default.
You can also set the broker properties imq.serviceName. min_threads and imq.serviceName. max_threads to specify a minimum and maximum number of threads in a service’s thread pool. When the number of available threads exceeds the specified minimum threshold, Message Queue will shut down threads as they become free until the minimum is reached again, thereby saving on memory resources. Under heavy loads, the number of threads might increase until the pool’s maximum number is reached; at this point, new connections are rejected until a thread becomes available.
The shared threading model uses distributor threads to assign threads to active connections. The broker property imq.shared.connectionMonitor_limit specifies the maximum number of connections that can be monitored by a single distributor thread. The smaller the value of this property, the faster threads can be assigned to connections. The imq.ping.interval property specifies the time interval, in seconds, at which the broker will periodically test (“ping”) a connection to verify that it is still active, allowing connection failures to be detected preemptively before an attempted message transmission fails.
Once clients are connected to the broker, the routing and delivery of messages can proceed. In this phase, the broker is responsible for creating and managing different types of physical destinations, ensuring a smooth flow of messages, and using resources efficiently. You can use the broker configuration properties described under Routing Properties to manage these tasks in a way that suits your application’s needs.
The performance and stability of a broker depend on the system resources (such as memory) available and how efficiently they are utilized. You can set configuration properties to prevent the broker from becoming overwhelmed by incoming messages or running out of memory. These properties function at three different levels to keep the message service operating as resources become scarce:
Systemwide message limits apply collectively to all physical destinations on the system. These include the maximum number of messages held by a broker (imq.system.max_count) and the maximum total number of bytes occupied by such messages (imq.system.max_size). If either of these limits is reached, the broker will reject any new messages until the pending messages fall below the limit. There is also a limit on the maximum size of an individual message (imq.message.max_size) and a time interval at which expired messages are reclaimed (imq.message.expiration.interval).
Individual destination limits regulate the flow of messages to a specific physical destination. The configuration properties controlling these limits are described in Chapter 15, Physical Destination Property Reference They include limits on the number and size of messages the destination will hold, the number of message producers and consumers that can be created for it, and the number of messages that can be batched together for delivery to the destination.
The destination can be configured to respond to memory limits by slowing down the delivery of message by message producers, by rejecting new incoming messages, or by throwing out the oldest or lowest-priority existing messages. Messages deleted from the destination in this way may optionally be moved to the dead message queue rather than discarded outright; the broker property imq.destination.DMQ.truncateBody controls whether the entire message body is saved in the dead message queue, or only the header and property data.
As a convenience during application development and testing, you can configure a message broker to create new physical destinations automatically whenever a message producer or consumer attempts to access a nonexistent destination. The broker properties summarized in Table 14–3 parallel the ones just described, but apply to such auto-created destinations instead of administratively created ones.
System memory thresholds define levels of memory usage at which the broker takes increasingly serious action to prevent memory overload. Four such usage levels are defined:
Green: Plenty of memory is available.
Yellow: Broker memory is beginning to run low.
Orange: The broker is low on memory.
Red: The broker is out of memory.
The memory utilization percentages defining these levels are specified by the broker properties imq.green.threshold, imq.yellow.threshold , imq.orange.threshold, and imq.red.threshold , respectively; the default values are 0% for green, 80% for yellow, 90% for orange, and 98% for red.
As memory usage advances from one level to the next, the broker responds progressively, first by swapping messages out of active memory into persistent storage and then by throttling back producers of nonpersistent messages, eventually stopping the flow of messages into the broker. (Both of these measures degrade broker performance.) The throttling back of message production is done by limiting the size of each batch delivered to the number of messages specified by the properties imq.resourceState .count, where resourceState is green , yellow, orange, or red , respectively.
The triggering of these system memory thresholds is a sign that systemwide and destination message limits are set too high. Because the memory thresholds cannot always catch potential memory overloads in time, you should not rely on them to control memory usage, but rather reconfigure the systemwide and destination limits to optimize memory resources.
For a broker to recover in case of failure, it needs to re-create the state of its message delivery operations. To do this, the broker must save state information to a persistent data store. When the broker restarts, it uses the saved data to re-create destinations and durable subscriptions, recover persistent messages, roll back open transactions, and rebuild its routing table for undelivered messages. It can then resume message delivery.
Message Queue supports both file-based and JDBC-based persistence modules (see Figure 4–1). File-based persistence uses individual files to store persistent data; JDBC-based persistence uses the Java Database Connectivity (JDBC™) interface to connect the broker to a JDBC-compliant data store. While file-based persistence is generally faster than JDBC-based, some users prefer the redundancy and administrative control provided by a JDBC-compliant store. The broker configuration property imq.persist.store (see Table 14–4) specifies which of the two forms of persistence to use.
By default, Message Queue uses a file-based persistent data store, in which individual files store persistent data such as messages, destinations, durable subscriptions, and transactions. Broker configuration properties related to file-based persistence are listed under File-Based Persistence.
The file-based store is located in a directory identified by the name of the broker instance (instanceName) to which the data store belongs:
…/instances/instanceName /fs350/
(See Appendix A, Platform-Specific Locations of Message QueueTM Data for the location of the instances directory.) Each destination on the broker has its own subdirectory holding messages delivered to that destination.
Because the persistent data store can contain messages of a sensitive or proprietary nature, you should secure the …/instances/ instanceName/fs350/ directory against unauthorized access; see Securing Persistent Data.
All persistent data other than messages is stored in separate files: one file for destinations, one for durable subscriptions, and one for transaction state information. Most messages are stored in a single file consisting of variable-sized records. You can compact this file to alleviate fragmentation as messages are added and removed (see Compacting Physical Destinations). In addition, messages above a certain threshold size are stored in their own individual files rather than in the variable-sized record file. You can configure this threshold size with the broker property imq.persist.file.message.max_record_size.
The broker maintains a file pool for these individual message files: instead of being deleted when it is no longer needed, a file is returned to the pool of free files in its destination directory so that it can later be reused for another message. The broker property imq.persist.file.destination.message.filepool.limit specifies the maximum number of files in the pool. When the number of individual message files for a destination exceeds this limit, files will be deleted when no longer needed instead of being returned to the pool.
When returning a file to the file pool, the broker can save time at the expense of storage space by simply tagging the file as available for reuse without deleting its previous contents. You can use the imq.persist.file.message.filepool.cleanratio broker property to specify the percentage of files in each destination’s file pool that should be maintained in a “clean” (empty) state rather than simply marked for reuse. The higher you set this value, the less space will be required for the file pool, but the more overhead will be needed to empty the contents of files when they are returned to the pool. If the broker’s imq.persist.file.message.cleanup property is true, all files in the pool will be emptied at broker shutdown, leaving them in a clean state; this conserves storage space but slows down the shutdown process.
In writing data to the persistent store, the operating system has some leeway in whether to write the data synchronously or “lazily” (asynchronously). Lazy storage can lead to data loss in the event of a system crash, if the broker believes the data to have been written to persistent storage when it has not. To ensure absolute reliability (at the expense of performance), you can require that all data be written synchronously by setting the broker property imq.persist.file.sync.enabled to true. In this case, the data is guaranteed to be available when the system comes back up after a crash, and the broker can reliably resume operation. Note, however, that although the data is not lost, it is not available to any other broker in a cluster, since clustered brokers do not currently share data.
Instead of using file-based persistence, you can set up a broker to access any data store accessible through a JDBC-compliant driver. This involves setting the appropriate JDBC-related broker configuration properties and using the Database Manager utility (imqdbmgr) to create a database with the proper schema. See Configuring a JDBC-Based Store for specifics.
The properties for configuring a broker to use a JDBC database are listed under JDBC-Based Persistence. You can specify these properties either in the instance configuration file (config.properties ) of each broker instance or by using the -D command line option to the Broker utility (imqbrokerd) or the Database Manager utility (imqdbmgr).
The imq.persist.jdbc.driver property gives the Java class name of the JDBC driver to use in connecting to the database. There are also properties specifying the URLs for connecting to an existing database (imq.persist.jdbc.opendburl), creating a new database (imq.persist.jdbc.createdburl), and closing a database connection (imq.persist.jdbc.closedburl).
The imq.persist.jdbc.user and imq.persist.jdbc.password properties give the user name and password for accessing the database; imq.persist.jdbc.needpassword is a boolean flag specifying whether a password is needed. For security reasons, the password should be specified only in a password file designated via the -passfile command line option; if no such password file is specified, the imqbrokerd and imqdbmgr commands will prompt for the password interactively. Similarly, the user name can be supplied from the command line using the -dbuser option to the imqbrokerd command or the -u option to imqdbmgr.
In a JDBC database shared by multiple broker instances, the configuration property imq.persist.jdbc.brokerid specifies a unique instance identifier for each, to be appended to the names of database tables. (This is usually unnecessary for an embedded database, which stores data for only one broker instance.) The remaining JDBC-related configuration properties are used to customize the SQL code that creates the database schema, one property for each database table. For instance, the imq.persist.jdbc.table.IMQSV35 property gives the SQL command for creating the version table, imq.persist.jdbc.table.IMQCCREC35 for the configuration change record table, imq.persist.jdbc.table.IMQDEST35 for the destination table, and so on; see Table 14–6 for the complete list.
Because database systems vary in the exact SQL syntax required, be sure to check the documentation from your database vendor for details.
Message Queue provides security services for user access control (authentication and authorization) and for encryption:
Authentication ensures that only verified users can establish a connection to a broker.
Authorization specifies which users or groups have the right to access resources and to perform specific operations.
Encryption protects messages from being tampered with during delivery over a connection.
As a Message Queue administrator, you are responsible for setting up the information the broker needs to authenticate users and authorize their actions. The broker properties pertaining to security services are listed under Security Properties. The boolean property imq.accesscontrol.enabled acts as a master switch that controls whether access control is applied on a brokerwide basis; for finer control, you can override this setting for a particular connection service by setting the imq.serviceName .accesscontrol.enabled property, where serviceName is the name of the connection service, as shown in Table 4–1: for example, imq.httpjms.accesscontrol.enabled.
Figure 4–2 shows the components needed by the broker to provide authentication and authorization services. These services depend on a user repository containing information about the users of the messaging system: their names, passwords, and group memberships. In addition, to authorize specific operations for a user or group, the broker consults an access control properties file that specifies which operations a user or group can perform. You can designate a single access control properties file for the broker as a whole, using the configuration property imq.accesscontrol.file.filename, or for a single connection service with imq.serviceName. accesscontrol.file.filename.
As Figure 4–2 shows, you can store user data in a flat-file user repository that is provided with the Message Queue service or you can plug in a preexisting Lightweight Directory Access Protocol (LDAP) repository:
If you choose a flat-file repository, you must use the Message Queue User Manager utility (imqusermgr) to manage the repository. This option is built-in and easy to use.
If you want to use an existing LDAP server, you use the tools provided by the LDAP vendor to populate and manage the user repository. You must also set properties in the broker’s instance configuration file to enable the broker to query the LDAP server for information about users and groups.
The broker’s imq.authentication.basic.user_repository property specifies which type of repository to use. In general, an LDAP repository is preferable if scalability is important or if you need the repository to be shared by different brokers (if you are using broker clusters, for instance). See User Authentication for more information on setting up a flat-file or LDAP user repository.
A client requesting a connection to a broker must supply a user name and password, which the broker compares with those stored in the user repository. Passwords transmitted from client to broker are encoded using either base-64 encoding (for flat-file repositories) or message digest (MD5) hashing (for LDAP repositories). The choice is controlled by the imq.authentication.type property for the broker as a whole, or by imq.serviceName. authentication.type for a specific connection service. The imq.authentication.client.response.timeout property sets a timeout interval for authentication requests.
As described under Password Files, you can choose to put your passwords in a password file instead of being prompted for them interactively. The boolean broker property imq.passfile.enabled controls this option. If this property is true, the imq.passfile.dirpath and imq.passfile.name properties give the directory path and file name for the password file. The imq.imqcmd.password property (which can be embedded in the password file) specifies the password for authenticating an administrative user to use the Command utility (imqcmd) for managing brokers, connection services, connections, physical destinations, durable subscriptions, and transactions.
If you are using an LDAP-based user repository, there are a whole range of broker properties available for configuring various aspects of the LDAP lookup. The address (host name and port number) of the LDAP server itself is specified by imq.user_repository.ldap.server. The imq.user_repository.ldap.principal property gives the distinguished name for binding to the LDAP repository, while imq.user_repository.ldap.password supplies the associated password. Other properties specify the directory bases and optional JNDI filters for individual user and group searches, the provider-specific attribute identifiers for user and group names, and so forth; see Security Properties for details.
Once authenticated, a user can be authorized to perform various Message Queue-related activities. As a Message Queue administrator, you can define user groups and assign individual users membership in them. The default access control properties file explicitly refers to only one group, admin (see Groups). A user in this group has connection permission for the admin connection service, which allows the user to perform administrative functions such as creating destinations and monitoring and controlling a broker. A user in any other group that you define cannot, by default, get an admin service connection.
When a user attempts to perform an operation, the broker checks the user’s name and group membership (from the user repository) against those specified for access to that operation (in the access control properties file). The access control properties file specifies permissions to users or groups for the following operations:
Connecting to a broker
Accessing destinations: creating a consumer, a producer, or a queue browser for any given destination or for all destinations
To encrypt messages sent between clients and broker, you need to use a connection service based on the Secure Socket Layer (SSL) standard. SSL provides security at the connection level by establishing an encrypted connection between an SSL-enabled broker and client.
To use an SSL-based Message Queue connection service, you generate a public/private key pair using the Key Tool utility (imqkeytool). This utility embeds the public key in a self-signed certificate and places it in a Message Queue key store. The key store is itself password-protected; to unlock it, you must provide a key store password at startup time, specified by the imq.keystore.password property. Once the key store is unlocked, a broker can pass the certificate to any client requesting a connection. The client then uses the certificate to set up an encrypted connection to the broker.
The imq.audit.enabled broker property controls the logging of audit records to the Message Queue broker log file; see Audit Logging for more information.
The broker includes components for monitoring and diagnosing application and broker performance. These include the following:
Components that generate data, a Metrics Generator and broker code that logs events
A Logger component that writes out information to a number of output channels
A Metrics Message Producer that sends JMS messages containing metric information to topic destinations for consumption by JMS monitoring clients
The general scheme is illustrated in Figure 4–3. Broker properties for configuring the monitoring services are listed under Monitoring Properties.
The Metrics Generator provides information about broker activity, such as message flow in and out of the broker, the number of messages in broker memory and the memory they consume, the number of open connections, and the number of threads being used. The boolean broker property imq.metrics.enabled controls whether such information is logged; imq.metrics.interval specifies how often.
The Logger takes information generated by broker code and the Metrics Generator and writes that information to standard output (the console), to a log file, and, on Solaris platforms, to the syslog daemon process in case of errors. The log file to use is identified by the imq.log.file.dirpath and imq.log.file.filename broker properties; imq.log.console.stream specifies whether console output is directed to stdout or stderr.
The imq.log.level property controls the categories of metric information that the Logger gathers: ERROR, WARNING, or INFO. Each level includes those above it, so if you specify, for example, WARNING as the logging level, error messages will be logged as well. The imq.log.console.output and imq.log.file.output properties control which of the specified categories will be written to the console and the log file, respectively. In this case, however, the categories do not include those above them; so if you want, for instance, both errors and warnings written to the log file and informational messages to the console, you must explicitly set imq.log.file.output to ERROR|WARNING and imq.log.console.output to INFO. On Solaris platforms another property, imq.log.syslog.output, specifies the categories of metric information to be written to the syslog daemon. There is also an imq.destination.logDeadMsgs property that specifies whether to log when dead messages are discarded or moved to the dead message queue.
In the case of a log file, you can specify the point at which the file is closed and output is rolled over to a new file. Once the log file reaches a specified size (imq.log.file.rolloverbytes) or age (imq.log.file.rolloversecs), it is saved and a new log file created.
See Monitoring Properties for additional broker properties related to logging, and Configuring and Using Broker Logging for further details about how to configure the Logger and how to use it to obtain performance information.
The Metrics Message Producer receives information from the Metrics Generator at regular intervals and writes the information into metrics messages, which it then sends to one of a number of metric topic destinations, depending on the type of metric information contained in the message (see Table 4–2). Message Queue clients subscribed to these metric topic destinations can consume the messages and process the metric data they contain. This allows developers to create custom monitoring tools to support messaging applications. For details of the metric quantities reported in each type of metrics message, see the Message Queue Developer's Guide for Java Clients.
Table 4–2 Metric Topic Destinations
Topic Name | |
---|---|
mq.metrics.broker |
Broker metrics |
mq.metrics.jvm |
Java Virtual Machine metrics |
mq.metrics.destination_list |
List of destinations and their types |
mq.metrics.destination.queue.queueName |
Destination metrics for specified queue |
mq.metrics.destination.topic.topicName |
Destination metrics for specified topic |
The broker properties imq.metrics.topic.enabled and imq.metrics.topic.interval control, respectively, whether messages are sent to metric topic destinations and how often. The imq.metrics.topic.timetolive and imq.metrics.topic.persist properties specify the lifetime of such messages and whether they are persistent.
Besides the information contained in the body of a metrics message, the header of each message includes properties that provide the following additional information:
The message type
The address (host name and port number) of the broker that sent the message
The time the metric sample was taken
These properties are useful to client applications that process metrics messages of different types or from different brokers.
You can specify a broker’s configuration properties in either of two ways:
Edit the broker’s configuration file
Supply the property values directly from the command line
The following two sections describe these two methods of configuring a broker.
Broker configuration files contain property settings for configuring a broker. They are kept in a directory whose location depends on the operating system platform you are using; see Appendix A, Platform-Specific Locations of Message QueueTM Data for details. The directory stores the following files:
A default configuration file, default.properties, that is loaded on startup. This file is not editable, but you can read it to determine default settings and find the exact names of properties you want to change.
An installation configuration file, install.properties, containing any properties specified when Message Queue was installed. This file cannot be edited after installation.
In addition, each individual broker instance has its own instance configuration file, as described below. If you connect broker instances in a cluster, you may also need to use a cluster configuration file to specify configuration information for the cluster; see Cluster Configuration Properties for more information.
At startup, the broker merges property values from the various configuration files. As shown in Figure 4–4, the files form a hierarchy in which values specified in the instance configuration file override those in the installation configuration file, which in turn override those in the default configuration file. At the top of the hierarchy, you can manually override any property values specified in the configuration files by using command line options to the imqbrokerd command.
The first time you run a broker, an instance configuration file is created containing configuration properties for that particular broker instance. The instance configuration file is named config.properties and is stored in a directory identified by the name of the broker instance to which it belongs:
…/instances/ instanceName/props/config.properties
(See Appendix A, Platform-Specific Locations of Message QueueTM Data for the location of the instances directory.) If the file does not yet exist, you must use the -name option when starting the broker (see Broker Utility), to specify an instance name that Message Queue can use to create the file.
The instances/instanceName directory and the instance configuration file are owned by the user who created the corresponding broker instance. The broker instance must always be restarted by that same user.
The instance configuration file is maintained by the broker instance and is modified when you make configuration changes using Message Queue administration utilities. You can also edit an instance configuration file by hand to customize the broker’s behavior and resource use. To do so, you must be the owner of the instances/ instanceName directory or log in as root to change the directory’s access privileges.
The broker reads its instance configuration file only at startup. To make permanent changes to the broker’s configuration, you must shut down the broker, edit the file, and then restart the broker. Property definitions in the file (or any configuration file) use the following syntax:
propertyName=value [[,value1] …]
For example, the following entry specifies that the broker will hold up to 50,000 messages in memory and persistent storage before rejecting additional messages:
imq.system.max_count=50000
The following entry specifies that a new log file will be created every day (86,400 seconds):
imq.log.file.rolloversecs=86400
See Broker Services and Chapter 14, Broker Properties Reference for information on the available broker configuration properties and their default values.
You can enter broker configuration options from the command line when you start a broker, or afterward.
At startup time, you use the Broker utility (imqbrokerd) to start a broker instance. Using the command’s -D option, you can specify any broker configuration property and its value; see Starting Brokers and Broker Utility for more information. If you start the broker as a Windows service, using the Service Administrator utility (imqsvcadmin), you use the -args option to specify startup configuration properties; see Service Administrator Utility.
You can also change certain broker properties while a broker instance is running. To modify the configuration of a running broker, you use the Command utility’s imqcmd update bkr command; see Updating Broker Properties and Broker Management.
A broker’s persistent data store holds information about physical destinations, durable subscriptions, messages, transactions, and acknowledgments. Message Queue brokers are configured by default to use a file-based persistent store, but you can reconfigure them to plug in any data store accessible through a JDBC-compliant driver. The broker configuration property imq.persist.store (see Table 14–4) specifies which of the two forms of persistence to use.
This section explains how to set up a broker to use a persistent store. It includes the following topics:
A file-based data store is automatically created when you create a broker instance. The store is located in the broker’s instance directory; see Appendix A, Platform-Specific Locations of Message QueueTM Data for the exact location.
By default, Message Queue performs asynchronous write operations to disk. The operating system can buffer these operations for efficient performance. However, if an unexpected system failure should occur between write operations, messages could be lost. To improve reliability (at the cost of reduced performance), you can set the broker property imq.persist.file.sync to write data synchronously instead. For further discussion about this property, see File-Based Persistence and Table 14–5.
When you start a broker instance, you can use the imqbrokerd command’s -reset option to clear the file system store. For more information about this option and its suboptions, see Broker Utility.
To configure a broker to use JDBC-based persistence, you set JDBC-related properties in the broker’s instance configuration file and create the appropriate database schema. The Message Queue Database Manager utility (imqdbmgr) uses your JDBC driver and the broker configuration properties to create and manage the database. You can also use the Database Manager to delete corrupted tables from the database or if you want to use a different database as a data store. See Database Manager Utility for more information.
Example configurations for Oracle and PointBase database products are available. The location of these files is platform-dependent, and is listed under “Example applications and configurations” in the relevant tables of Appendix A, Platform-Specific Locations of Message QueueTM Data. In addition, examples for PointBase embedded version, PointBase server version, and Oracle are provided as commented-out values in the instance configuration file, config.properties.
Set JDBC-related properties in the broker’s configuration file.
The relevant properties are discussed under JDBC-Based Persistence and listed in Table 14–6. In particular, you must set the broker’s imq.persist.store property to jdbc (see Persistence Properties).
Place a copy of, or a symbolic link to, your JDBC driver’s .jar file in the following location:
On Solaris:
/usr/share/lib/imq/ext/ |
On Linux:
/opt/sun/mq/share/lib/ |
On Windows:
IMQ_VARHOME\\lib\\ext |
For example, if you are using PointBase on a Solaris system, the following command copies the driver’s .jar file to the appropriate location:
% cp j2eeSDKInstallDirectory/pointbase/lib/pointbase.jar /usr/share/lib/imq/ext |
The following command creates a symbolic link instead:
% ln -s j2eeSDKID/lib/pointbase/pointbase.jar /usr/share/lib/imq/ext |
Create the database schema needed for Message Queue persistence.
Use the imqdbmgr create all command (for an embedded database) or the imqdbmgr create tbl command (for an external database); see Database Manager Utility.
Change to the directory where imqdbmgr resides:
On Solaris:
cd /usr/bin |
On Linux:
cd /opt/sun/mq/bin |
On Windows:
cd IMQ_HOME\\bin |
Enter the imqdbmgr command:
imqdbmgr create all
If you use an embedded database, it is best to create it under the following directory:
…/instances/ instanceName/dbstore/databaseName
If an embedded database is not protected by a user name and password, it is probably protected by file system permissions. To ensure that the database is readable and writable by the broker, the user who runs the broker should be the same user who created the embedded database using the imqdbmgr command.
The persistent store can contain, among other information, message files that are being temporarily stored. Since these messages may contain proprietary information, it is important to secure the data store against unauthorized access. This section describes how to secure data in a file-based or JDBC-based data store.
A broker using file-based persistence writes persistent data to a flat-file data store whose location is platform-dependent (see Appendix A, Platform-Specific Locations of Message QueueTM Data):
…/instances/ instanceName/fs350/
where instanceName is a name identifying the broker instance.
The instanceName/fs350/ directory is created when the broker instance is started for the first time. The procedure for securing this directory depends on the operating system platform on which the broker is running:
On Solaris and Linux, the directory’s permissions are determined by the file mode creation mask (umask) of the user who started the broker instance. Hence, permission to start a broker instance and to read its persistent files can be restricted by setting the mask appropriately. Alternatively, an administrator (superuser) can secure persistent data by setting the permissions on the instances directory to 700.
On Windows, the directory’s permissions can be set using the mechanisms provided by the Windows operating system. This generally involves opening a Properties dialog for the directory.
A broker using JDBC-based persistence writes persistent data to a JDBC-compliant database. For a database managed by a database server (such as Oracle), it is recommended that you create a user name and password to access the Message Queue database tables (tables whose names start with IMQ). If the database does not allow individual tables to be protected, create a dedicated database to be used only by Message Queue brokers. See the documentation provided by your database vendor for information on how to create user name/password access.
The user name and password required to open a database connection by a broker can be provided as broker configuration properties. However it is more secure to provide them as command line options when starting up the broker, using the imqbrokerd command’s -dbuserand -dbpassword options (see Broker Utility).
For an embedded database that is accessed directly by the broker via the database’s JDBC driver, security is usually provided by setting file permissions on the directory where the persistent data will be stored, as described above under Securing a File-Based Store To ensure that the database is readable and writable by both the broker and the Database Manager utility, however, both should be run by the same user.
This chapter explains how yo use the imqcmd utility to manage the broker and its services. This chapter has the following sections:
This chapter does not cover all topics related to managing a broker. Additional topics are covered in the following separate chapters:
Management of physical destinations on the broker. For information about topics such as how to create, display, update and destroy physical destinations, and how to use the dead message queue, see Chapter 6, Managing Physical Destinations
Setting up security for the broker. For information about topics such as user authentication, access control, encryption, password files, and audit logging, see Chapter 7, Managing Security
You use the imqcmd and imqusermgr command line utilities to manage the broker. Before managing the broker, you must do the following:
Start the broker using the imqbrokerd utility command. You cannot use the other command line utilities until a broker is running.
Determine whether you want to set up a Message QueueTM administrative user or use the default account. You must specify a user name and password to use management commands.
When you install Message Queue, a default flat-file user repository is installed. The repository is shipped with two default entries: an admin user and a guest user. If you are testing Message Queue, you can use the default user name and password (admin/admin) to run the imqcmd utility.
If you are setting up a production system, you must set up authentication and authorization for administrative users. See Chapter 7, Managing Security for information on setting up a file-based user repository or configuring the use of an LDAP directory server. In a production environment, it is a good security practice to use a nondefault user name and password.
Set up and enable the ssladmin service on the target broker instance, if you want to use a secure connection to the broker. For more information, see Message Encryption.
The imqcmd utility enables you to manage the broker and its services.
Reference information about the syntax, subcommands, and options of the imqcmd command is in Chapter 13, Command Line Reference. Reference information for managing physical destinations is in a separate chapter, Chapter 15, Physical Destination Property Reference.
To display help on the imqcmd utility, use the -h or -H option, and do not use a subcommand. You cannot get help about specific subcommands.
For example, the following command displays help about imqcmd:
imqcmd -H
If you enter a command line that contains the -h or -H option in addition to a subcommand or other options, the imqcmd utility processes only the -h or -H option. All other items on the command line are ignored.
To display the Message Queue product version, use the -v option. For example:
imqcmd -v
If you enter a command line that contains the -v option in addition to a subcommand or other options, the imqcmd utility processes only the -v option. All other items on the command line are ignored.
Because each imqcmd subcommand is authenticated against the user repository, it requires a user name and password. The only exceptions are commands that use the -h or -H option to display help, and commands that use the -v option to display the product version.
Use the -u option to specify an administrative user name. If you omit the user name, the command prompts you for it. For example, the following command displays information about the default broker:
imqcmd query bkr -u admin
To make the examples in this chapter easy to read, the default user name admin is shown as the argument to the -u option. In a production environment, you would use a custom user name.
Specify the password using one of the following methods:
Create a password file (passfile) and enter the password into that file. On the command line, use the -passfile option to provide the name of the password file.
Let the command prompt you for the password.
In previous versions of Message Queue, you could use the -p option to specify a password on the imqcmd command line. This option is being deprecated and will be removed in a future version.
The default broker for imqcmd is one that is running on the local host, and the default port is 7676.
If you are issuing a command to a broker running on a remote host or listening on a nondefault port, or both, you must use the -b option to specify the broker’s host and port.
The examples in this section illustrate how to use imqcmd.
The first example lists the properties of the broker running on localhost at port 7676, so the -b option is unnecessary. The command uses the default administrative user name (admin ) and omits the password, so that the command prompts for it.
imqcmd query bkr -u admin
The following example lists the properties of the broker running on the host myserver at port 1564. The user name is aladdin . (For this command to work, the user repository would need to be updated to add the user name aladdin to the admin group.)
imqcmd query bkr -b myserver:1564 -u aladdin
The following example lists the properties of the broker running on localhost at port 7676. The initial timeout for the command is set to 20 seconds and the number of retries after timeout is set to 7. The user’s password is in a password file called myPassfile, located in the current directory at the time the command is invoked.
imqcmd query bkr -u admin -passfile myPassfile -rtm 20 -rtr 7
For a secure connection to the broker, these examples could include the -secure option. The -secure option causes imqcmd to use the ssladmin service if the service has been configured and started.
To query and display information about a single broker, use the query bkr subcommand.
This is the syntax of the query bkr subcommand:
imqcmd query bkr -b hostName: portNumber
This subcommand lists the current settings of properties for the default broker or a broker at the specified host and port. It also shows the list of running brokers (in a multiple-broker cluster) that are connected to the specified broker.
imqcmd query bkr -u admin
After prompting you for the password, the command produces output like the following:
Version 3.6 Instance Name imqbroker Primary Port 7676 Current Number of Messages in System 0 Current Total Message Bytes in System 0 Current Number of Messages in Dead Message Queue 0 Current Total Message Bytes in Dead Message Queue 0 Log Dead Messages true Truncate Message Body in Dead Message Queue false Max Number of Messages in System unlimited (-1) Max Total Message Bytes in System unlimited (-1) Max Message Size 70m Auto Create Queues true Auto Create Topics true Auto Created Queue Max Number of Active Consumers 1 Auto Created Queue Max Number of Backup Consumers 0 Cluster Broker List (active) Cluster Broker List (configured) Cluster Master Broker Cluster URL Log Level INFO Log Rollover Interval (seconds) 604800 Log Rollover Size (bytes) unlimited (-1) |
You can use the update bkr subcommand to update the following broker properties:
This is the syntax of the update bkr subcommand:
imqcmd update bkr [-b hostName: portNumb er]-o attribute=value [[-o attribute=value1] …]
The subcommand changes the specified attributes for the default broker or a broker at the specified host and port. For example, the following command turns off the auto-creation of queue destinations:
imqcmd update bkr -o "imq.autocreate.queue=false" -u admin
The properties are described in Chapter 14, Broker Properties Reference
After you start the broker, you can use imqcmd subcommands to control the state of the broker.
Pausing a broker suspends the broker’s connection service threads, which causes the broker to stop listening on the connection ports. As a result, the broker will no longer be able to accept new connections, receive messages, or dispatch messages.
However, pausing a broker does not suspend the admin connection service, letting you perform administration tasks needed to regulate the flow of messages to the broker. Pausing a broker also does not suspend the cluster connection service. However message delivery within a cluster depends on the delivery functions performed by the different brokers in the cluster. Therefore, pausing a broker in a cluster might result in a slowing of some message traffic.
This is the syntax of the pause bkr subcommand:
imqcmd pause bkr [-b hostName: portNumber]
The command pauses the default broker or a broker at the specified host and port.
The following command pauses the broker running on myhost at port 1588.
imqcmd pause bkr -b myhost:1588 -u admin
You can also pause individual connection services and individual physical destinations. For more information, see Pausing and Resuming a Connection Service and Pausing and Resuming Physical Destinations.
Resuming a broker reactivates the broker’s service threads and the broker resumes listening on the ports.
This is the syntax of the resume bkr subcommand:
imqcmd resume bkr [-b hostName: portNumber]
The subcommand resumes the default broker or a broker at the specified host and port.
The following command resumes the broker running on localhost at port 7676.
imqcmd resume bkr -u admin
Shutting down the broker gracefully terminates the broker process. The broker stops accepting new connections and messages, completes delivery of existing messages, and terminates the broker process.
This is the syntax of the shutdown bkr subcommand:
imqcmd shutdown bkr [-b hostName: portNumber]
The subcommand shuts down the default broker or a broker at the specified host and port.
The following command shuts down the broker running on ctrlsrv at port 1572:
imqcmd shutdown bkr -b ctrlsrv:1572 -u admin
Use the restart bkr subcommand to shut down and restart the broker. This is the syntax of the restart bkr subcommand:
imqcmd restart bkr [-b hostName: portNumber]
The subcommand shuts down and restarts the default broker or a broker at the specified host and port, using the options specified when the broker first started. To choose different options, shut down the broker and then restart it, specifying the options you want.
To display metrics information about a broker, use the metrics bkr subcommand.
This is the syntax of the metrics bkr subcommand:
imqcmd metrics bkr [-b hostName: portNumber] [-m metricType] [-int interval] [-msp numSamples]
The subcommand displays broker metrics for the default broker or a broker at the specified host and port.
Use the -m option to specify one of the following metric types to display:
ttl Displays metrics about the messages and packets flowing into and out of the broker (default metric type).
rts Displays metrics about the rate of flow of messages and packets into and out of the broker (per second).
cxn Displays connections, virtual memory heap, and threads.
Use the -int option to specify the interval (in seconds) at which to display the metrics. The default is 5 seconds.
Use the -msp option to specify the number of samples displayed in the output. The default is an unlimited number (infinite).
For example, to get the rate of message flow into and out of the broker at ten second intervals:
imqcmd metrics bkr -m rts -int 10 -u admin
This command produces output like the following:
-------------------------------------------------------- Msgs/sec Msg Bytes/sec Pkts/sec Pkt Bytes/sec In Out In Out In Out In Out -------------------------------------------------------- 0 0 27 56 0 0 38 66 10 0 7365 56 10 10 7457 1132 0 0 27 56 0 0 38 73 0 10 27 7402 10 20 1400 8459 0 0 27 56 0 0 38 73 |
For a more detailed description about the data gathered and reported by the broker, see Brokerwide Metrics.
The imqcmd utility includes subcommands that allow you to perform the following connection service management tasks:
A broker supports connections from both application clients and administration clients. The connection services currently available from a Message Queue broker are shown in Table 5–1. As shown in the table, each service is associated with the service type it uses (NORMAL for application clients or ADMIN for administration clients) and with an underlying transport protocol.
Table 5–1 Message Queue Connection Services
Service Name |
Service Type | |
---|---|---|
NORMAL | ||
NORMAL | ||
NORMAL | ||
NORMAL | ||
ADMIN |
TCP |
|
ADMIN |
TLS (SSL-based security) |
You can use imqcmd subcommands to manage connection services as a whole or to manage a particular connection service. If the target of a subcommand is a particular service, use the -n option to specify one of the names listed in the Service Name column of Table 5–1.
To list available connection services on a broker, use the list svc subcommand.
This is the syntax of the list svc subcommand:
imqcmd list svc [-b hostName: portNumber]
The subcommand lists all connection services on the default broker or on a broker at the specified host and port.
The following command lists all services on the broker running on localhost at port 7676:
imqcmd list svc -u admin
The command will output information like the following:
------------------------------------------------ Service Name Port Number Service State ------------------------------------------------ admin 41844 (dynamic) RUNNING httpjms - UNKNOWN httpsjms - UNKNOWN jms 41843 (dynamic) RUNNING ssladmin dynamic UNKNOWN ssljms dynamic UNKNOWN |
To query and display information about a single service, use the query subcommand.
This is the syntax for the query svc subcommand:
imqcmd query svc -n serviceName [-b hostName:portNumber]
The query svc subcommand displays information about the specified service running on the default broker or on a broker at the specified host and port.
For example:
imqcmd query svc -n jms -u admin
After prompting for the password, the command produces output like the following:
Service Name jms Service State RUNNING Port Number 60920 (dynamic) Current Number of Allocated Threads 0 Current Number of Connections 0 Min Number of Threads 10 Max Number of Threads 1000 |
You can use the update subcommand to change the value of one or more of the service properties listed in Table 5–2 .
Table 5–2 Connection Service Properties Updated by imqcmd
Property |
Description |
---|---|
port |
The port assigned to the service to be updated (does not apply to httpjms or httpsjms). A value of 0 means the port is dynamically allocated by the Port Mapper. |
minThreads | |
maxThreads |
The maximum number of threads assigned to the service. |
This is the syntax of the update subcommand:
imqcmd update svc -n serviceName [-b hostName:portNumber] -o attribute=value [-o attribute=value1]…
This subcommand updates the specified attribute of the specified service running on the default broker or on a broker at the specified host and port. For a description of service attributes, see Connection Properties.
The following command changes the minimum number of threads assigned to the jms service to 20.
imqcmd update svc -n jms -o “minThreads=20” -u admin
To display metrics information about a single service, use the metrics subcommand.
This is the syntax of the metrics subcommand:
imqcmd metrics svc -n serviceName [-b hostName:portNumber] [-m metricType ] [-int interval] [-msp numSamples]
The subcommand displays metrics for the specified service on the default broker or on a broker at the specified host and port.
Use the -m option to specify the type of metric to display:
ttl Displays metrics on messages and packets flowing into and out of the broker by way of the specified connection service. (default metric type).
rts Displays metrics on rate of flow of messages and packets into and out of the broker (per second) by way of the specified connection service.
cxn Displays connections, virtual memory heap, and threads.
Use the -int option to specify the interval (in seconds) at which to display the metrics. The default is 5 seconds.
Use the -msp option to specify the number of samples displayed in the output. The default is an unlimited number (infinite).
For example, to get cumulative totals for messages and packets handled by the jms connection service:
imqcmd metrics svc -n jms -m ttl -u admin
After prompting for the password, the command produces output like the following:
------------------------------------------------- Msgs Msg Bytes Pkts Pkt Bytes In Out In Out In Out In Out ------------------------------------------------- 164 100 120704 73600 282 383 135967 102127 657 100 483552 73600 775 876 498815 149948 |
For a more detailed description of the use of imqcmd to report connection service metrics, see Connection Service Metrics.
To pause any service other than the admin service (which cannot be paused), use the pause svc and resume svc subcommands.
This is the syntax of the pause svc subcommand:
imqcmd pause svc -n serviceName [-b hostName:portNumber]
The subcommand pauses the specified service running on the default broker or on a broker at the specified host and port. For example, the following command pauses the httpjms service running on the default broker.
imqcmd pause svc -n httpjms -u admin
Pausing a service has the following effects:
The broker stops accepting new client connections on the paused service. If a Message Queue client attempts to open a new connection, it will get an exception.
All the existing connections on the paused service are kept alive, but the broker suspends all message processing on such connections until the service is resumed. (For example, if a client attempts to send a message, the send method will block until the service is resumed.)
The message delivery state of any messages already received by the broker is maintained. (For example, transactions are not disrupted and message delivery will resume when the service is resumed.)
To resume a service, use the resume svc subcommand.
This is the syntax of the resume svc subcommand:
imqcmd resume svc -n serviceName[-b hostName:portNumber]
The subcommand resumes the specified service running on the default broker or on a broker at the specified host and port.
The imqcmd utility includes subcommands that allow you to list and get information about connections.
The list cxn subcommand lists all connections of a specified service name. This is the syntax of the list cxn subcommand:
imqcmd list cxn [-svn serviceName] [-b hostName:portNumber]
The subcommand lists all connections of the specified service name on the default broker or on a broker at the specified host and port. If the service name is not specified, all connections are listed.
For example, the following command lists all connections on the default broker:
imqcmd list cxn -u admin
After prompting for the password, the command produces output like the following:
Listing all the connections on the broker specified by: ----------------------------------- Host Primary Port ------------------------------------ localhost 7676 --------------------------------------------------------------------------- Connection ID User Service Producers Consumers Host --------------------------------------------------------------------------- 1964412264455443200 guest jms 0 1 127.0.0.1 1964412264493829311 admin admin 1 1 127.0.0.1 Successfully listed connections. |
To query and display information about a single connection service, use the query subcommand.
query cxn -n connectionID [-b hostName:portNumber]
The subcommand displays information about the specified connection on the default broker or on a broker at the specified host and port.
imqcmd query cxn -n 421085509902214374 -u admin
After prompting for the password, the command produces output like the following:
Connection ID 421085509902214374 User guest Service jms Producers 0 Consumers 1 Host 111.22.333.444 Port 60953 Client ID Client Platform |
Using imqcmd subcommands you can manage a broker’s durable subscriptions by doing one or more of the following:
Listing durable subscriptions
Purging all messages for a durable subscription
Destroying a durable subscription
A durable subscription is a subscription to a topic that is registered by a client as durable; it has a unique identity and it requires the broker to retain messages for that subscription even when its consumer becomes inactive. Normally, the broker may only delete a message held for a durable subscriber when the message expires.
To list durable subscriptions for a specified physical destination, use the list dur subcommand. This is the syntax for the list dur subcommand:
imqcmd list dur -d destName
For example, the following command lists all durable subscriptions to the topic SPQuotes, using the broker at the default port on the local host:
imqcmd list dur -d SPQuotes
For each durable subscription to a topic, the list dur subcommand returns the name of the durable subscription, the client ID of the user, the number of messages queued to this topic, and the state of the durable subscription (active/inactive). For example:
Name Client ID Number of Durable Sub Messages State ---------------------------------------------------------------- myDurable myClientID 1 INACTIVE |
You can use the information returned from the list dur subcommand to identify a durable subscription you might want to destroy or for which you want to purge messages.
The purge dur subcommand purges all messages for the specified durable subscription with the specified Client Identifier. This is the syntax for the purge dur subcommand:
imqcmd purge dur -n subscrName -c clientID
The destroy dur subcommand destroys a specified durable subscription with the specified client identifier. This is the syntax for the destroy dur subcommand:
imqcmd destroy dur -n subscrName -c clientID
For example, the following command destroys the durable subscription myDurable and clientID, myClientID.
imqcmd destroy dur -n myDurable -c myClientID
All transactions initiated by client applications are tracked by the broker. These can be simple Message Queue transactions or distributed transactions managed by a distributed transaction (XA resource) manager.
Each transaction has a Message Queue transaction ID:a 64 bit number that uniquely identifies a transaction on the broker. Distributed transactions also have a distributed transaction ID (XID), up to 128 bytes long, assigned by the distributed transaction manager. Message Queue maintains the association of an Message Queue transaction ID with an XID.
For distributed transactions, in cases of failure, it is possible that transactions could be left in a PREPARED state without ever being committed. Hence, as an administrator you might need to monitor and then roll back or commit transactions left in a prepared state.
To list all transactions, being tracked by the broker, use the list txn command. This is the syntax for the list tx subcommand:
imqcmd list txn
For example, the following command lists all transactions in a broker.
imqcmd list txn
For each transaction, the list subcommand returns the transaction ID, state, user name, number of messages or acknowledgments, and creation time. For example:
--------------------------------------------------------------- Transaction ID State User name # Msgs/ Creation time # Acks --------------------------------------------------------------- 64248349708800 PREPARED guest 4/0 1/30/02 10:08:31 AM 64248371287808 PREPARED guest 0/4 1/30/02 10:09:55 AM |
The command shows all transactions in the broker, both local and distributed. You can only commit or roll back transactions in the PREPARED state. You should only do so if you know that the transaction has been left in this state by a failure and is not in the process of being committed by the distributed transaction manager.
For example, if the broker’s auto-rollback property is set to false (see Table 14–2), you must manually commit or roll back transactions found in a PREPARED state at broker startup.
The list subcommand also shows the number of messages that were produced in the transaction and the number of messages that were acknowledged in the transaction (#Msgs/#Acks). These messages will not be delivered and the acknowledgments will not be processed until the transaction is committed.
The query subcommand lets you see the same information plus a number of additional values: the Client ID, connection identification, and distributed transaction ID (XID). This is the syntax of the query txn subcommand:
imqcmd query txn -n transactionID
For example, the following example produces the output shown below:
imqcmd query txn -n 64248349708800
Client ID Connection guest@192.18.116.219:62209->jms:62195 Creation time 1/30/02 10:08:31 AM Number of acknowledgments 0 Number of messages 4 State PREPARED Transaction ID 64248349708800 User name guest XID 6469706F6C7369646577696E6465723130313234313431313030373230 |
Use the commit and rollback subcommands to commit or roll back a distributed transaction. As mentioned previously, only a transaction in the PREPARED state can be committed or rolled back.
This is the syntax of the commit subcommand:
imqcmd commit txn -n transactionID
For example:
imqcmd commit txn -n 64248349708800
This is the syntax of the rollback. subcommand:
imqcmd rollback txn -n transactionID
See the imq.transaction.autorollback property in Table 14–2 for more information.
It is also possible to configure the broker to automatically roll back transactions in the PREPARED state at broker startup.
This chapter explains how you use the imqcmd utility to manage physical destinations. A Message QueueTM message is routed to its consumer clients by way of a physical destination on a broker. The broker manages the memory and persistent storage associated with the physical destinations, and sets their behaviors.
In a broker cluster, you create a physical destination on one broker and the cluster propagates that physical destination to all the others. An application client can subscribe to a topic or consume from a queue that is on any broker in the cluster, because the brokers cooperate to route messages across the cluster. However, only the broker to which a message was originally produced manages persistence and acknowledgment for that message.
This chapter covers the following topics:
Table 13–5 provides full reference information about the imqcmd subcommands for managing physical destinations and accomplishing these tasks.
See the Message Queue Technical Overview for an introduction to physical destinations.
A client application uses a Destination object whenever it interacts with a physical destination. For provider-independence and portability, clients typically use administrator-created destination objects, which are called destination administered objects. You can configure administered objects for use by client applications, as described in Chapter 8, Managing Administered Objects
You use the Message Queue Command utility (imqcmd) to manage physical destinations. The syntax of the imqcmd command is the same as when you use it for managing other broker services.
Full reference information about imqcmd, its subcommands, and its options, is available in Chapter 13, Command Line Reference.
Table 6–1 lists the imqcmd subcommands whose use is described in this chapter. For reference information about these subcommands, see Physical Destination Management.
Table 6–1 Physical Destination Subcommands for the Command Utility
Subcommand and Argument |
Description |
---|---|
compact dst |
Compacts the file-based data store for one or more physical destinations. |
create dst |
Creates a physical destination. |
destroy dst |
Destroys a physical destination. |
list dst |
Lists physical destinations on a broker. |
metrics dst |
Displays physical destination metrics. |
pause dst |
Pauses one or more physical destinations on a broker. |
purge dst |
Purges all messages on a physical destination without destroying the physical destination. |
query dst |
Queries and displays information on a physical destination. |
resume dst |
Resumes one or more paused physical destinations on a broker. |
update dst |
Updates properties of a destination. |
To create a physical destination, use the imqcmd create subcommand. This is the syntax for the create subcommand:
create dst -t destType -n destName [-o property=value ] [-o property=value1] …
For example, to create a queue destination, enter a command like the following:
imqcmd create dst -n myQueue -t q -o “maxNumActiveConsumers=5” |
To create a topic destination, enter a command like the following:
imqcmd create dst -n myTopic -t t -o “maxBytesPerMsg=5000” |
When creating a physical destination, you specify the following:
The physical destination type, t (topic) or q (queue).
The physical destination name. The naming rules are as follows:
The name must contain only alphanumeric characters. It cannot contain spaces.
The name can begin with an alphabetic character, the underscore character (_) or the dollar sign ($). It cannot begin with the character string mq.
Nondefault values for the physical destination’s properties.
You can also set properties when you update a physical destination.
Many physical destination properties affect broker memory resources and message flow. For example, you can specify the number of producers that can send to a physical destination, the number and size of the messages they can send, and the response that the broker should take when physical destination limits are reached. The limits are similar to brokerwide limits controlled by broker configuration properties.
The following properties are used for both queue destinations and topic destinations:
maxNumMsgs. Specifies the maximum number of unconsumed messages allowed in the physical destination.
maxTotalMsgBytes. Specifies the maximum total amount of memory (in bytes) allowed for unconsumed messages in the physical destination.
limitBehavior. Specifies how the broker responds when a memory-limit threshold is reached.
maxBytesPerMsg. Specifies the maximum size (in bytes) of any single message allowed in the physical destination.
maxNumProducers. Specifies the maximum number of producers for the physical destination.
consumerFlowLimit. Specifies the maximum number of messages to be delivered to a consumer in a single batch.
isLocalOnly. Applies only to broker clusters. Specifies that a physical destination is not replicated on other brokers, and is limited to delivering messages only to local consumers (consumers connected to the broker on which the physical destination is created).
useDMQ. Specifies whether a physical destination’s dead messages are discarded or put on the dead message queue.
The following properties are used for queue destinations only:
maxNumActiveConsumers. Specifies the maximum number of consumers that can be active in load-balanced delivery from a queue destination.)
maxNumBackupConsumers. Specifies the maximum number of backup consumers that can take the place of active consumers, if any fail during load-balanced delivery from a queue destination.
localDeliveryPreferred. Applies only to load-balanced queue delivery in broker clusters. Specifies that messages be delivered to remote consumers only if there are no consumers on the local broker.
See Chapter 15, Physical Destination Property Reference for full reference information about physical destination properties.
For auto-created destinations, you set default property values in the broker’s instance configuration file. Reference information on auto-create properties is located in Table 14–3.
You can get information about a physical destination’s current property values, about the number of producers or consumers associated with a physical destination, and about messaging metrics, such as the number and size of messages in the physical destination.
To find a physical destination about which you want to get information, list all physical destinations on a broker using the list dst subcommand. This is the syntax for the list dst subcommand:
list dst [-t destType] [-tmp]
The command lists physical destinations of the specified type. The value for the destination type (-t) option can be q (queue) or t (topic).
If you omit the destination type, physical destinations of all types are listed.
The list dst subcommand can optionally specify the type of destination to list or include temporary destinations (using the -tmp option). Temporary destinations are created by clients, normally for the purpose of receiving replies to messages sent to other clients.
For example, to get a list of all physical destinations on the broker running on myHost at port 4545, enter the following command:
imqcmd list dst -b myHost:4545
Information for the dead message queue, mq.sys.dmq, is always displayed, in addition to any other physical destinations, unless you specify the destination type t to include only topics.
To get information about a physical destination’s current properties, use the query dst subcommand. This is the syntax of the query dst subcommand:
query dst -t destType -n destName
The command lists information about the destination of the specified type and name. For example, the following command displays information about the queue destination XQueue:
imqcmd query dst -t q -n XQueue -u admin
The command produces output like the following:
------------------------------------ Destination Name Destination Type ------------------------------------ XQueue Queue On the broker specified by: ------------------------- Host Primary Port ------------------------- localhost 7676 Destination Name XQueue Destination Type Queue Destination State RUNNING Created Administratively true Current Number of Messages 0 Current Total Message Bytes 0 Current Number of Producers 0 Current Number of Active Consumers 0 Current Number of Backup Consumers 0 Max Number of Messages unlimited (-1) Max Total Message Bytes unlimited (-1) Max Bytes per Message unlimited (-1) Max Number of Producers 100 Max Number of Active Consumers 1 Max Number of Backup Consumers 0 Limit Behavior REJECT_NEWEST Consumer Flow Limit 1000 Is Local Destination false Local Delivery is Preferred false Use Dead Message Queue true |
The output also shows the number of producers and consumers associated with the destination. For queue destinations, the number includes active consumers and backup consumers.
You can use the update dst subcommand to change the value of one or more properties (see Updating Physical Destination Properties).
You can change the properties of a physical destination by using the update dst subcommand and the -o option to specify the property to update. This is the syntax for the update dst subcommand:
update dst -t destType -n destName -o property=value [[-o property=value1]…]
The command updates the value of the specified properties at the specified destination. The property name can be any property listed in Table 15–1.
You can use the -o option multiple times to update multiple properties. For example, the following command changes the maxBytesPerMsg property to 1000 and the MaxNumMsgs property to 2000:
imqcmd update dst -t q -n myQueue -o “maxBytesPerMsg=1000” -o “maxNumMsgs=2000” -u admin
See Chapter 15, Physical Destination Property Reference for a list of the properties that you can update.
You cannot use the update dst subcommand to update the type of a physical destination or to update the isLocalOnly property.
The dead message queue is a specialized physical destination whose properties differ somewhat from those of other destinations. For more information, see Using the Dead Message Queue.
You can pause a physical destination to control the delivery of messages from producers to the destination, or from the destination to consumers, or both. In particular, you can pause the flow of messages into a destination to help prevent destinations from being overwhelmed with messages when production of messages is much faster than consumption. You must pause a physical destination before compacting it.
To pause the delivery of messages to or from a physical destination, use the pause dst subcommand. This is the syntax of the pause dst subcommand:
pause dst [-t destType -n destName] [-pst pauseType]
The subcommand pauses the delivery of messages to consumers (-pst CONSUMERS ), or from producers (-pst PRODUCERS), or both ( -pst ALL), for the destination of the specified type and name. If no destination type and name are specified, all physical destinations are paused. The default is ALL.
Example:
imqcmd pause dst -n myQueue -t q -pst PRODUCERS -u admin imqcmd pause dst -n myTopic -t t -pst CONSUMERS -u admin
To resume delivery to a paused destination, use the resume dst subcommand. This is the syntax of the resume dst subcommand:
resume dst [-t destType -n destName]
The subcommand resumes delivery of messages to the paused destination of the specified type and name. If no destination type and name are specified, all destinations are resumed.
Example:
imqcmd resume dst -n myQueue -t q
In a broker cluster, instances of the physical destination reside on each broker in the cluster. You must pause each one individually.
You can purge all messages currently queued at a physical destination. Purging a physical destination means that all messages stored at the destination are deleted.
You might want to purge messages when the accumulated messages are taking up too much of the system’s resources. This might happen when a queue does not have registered consumer clients and is receiving many messages. It might also happen if inactive durable subscribers to a topic do not become active. In both cases, messages are held unnecessarily.
To purge messages at a physical destination, use the purge dst subcommand. This is the syntax of the purge dst subcommand:
purge dst -t destType -n destName
The subcommand purges messages at the physical destination of the specified type and name.
Examples:
imqcmd purge dst -n myQueue -t q -u admin imqcmd purge dst -n myTopic -t t -u admin
If you have shut down the broker and do not want old messages to be delivered when you restart it, use the -reset messages option to purge stale messages; for example:
imqbrokerd -reset messages -u admin
This saves you the trouble of purging destinations after restarting the broker.
In a broker cluster, instances of the physical destination reside on each broker in the cluster. You must purge each of these destinations individually.
To destroy a physical destination, use the destroy dst subcommand. This is the syntax of the destroy dst subcommand:
destroy dst -t destType -n destName
The subcommand destroys the physical destination of the specified type and name.
Example:
imqcmd destroy dst -t q -n myQueue -u admin
Destroying a physical destination purges all messages at that destination and removes it from the broker; the operation is not reversible.
You cannot destroy the dead message queue.
If you are using a file-based data store as the persistent store for messages, you can monitor disk utilization and compact the disk when necessary.
The file-based message store is structured so that messages are stored in directories corresponding to the physical destinations in which they are being held. In each physical destination’s directory, most messages are stored in one file consisting of variable-sized records. (To alleviate fragmentation, messages whose size exceeds a configurable threshold are stored in their own individual files.)
As messages of varying sizes are persisted and then removed from the record file, holes may develop in the file where free records are not being re-used.
To manage unused free records, the Command utility includes subcommands for monitoring disk utilization per physical destination and for reclaiming free disk space when utilization drops.
To monitor a physical destination’s disk utilization, use a command like the following:
imqcmd metrics dst -t q -n myQueue -m dsk -u admin
This command produces output like the following:
-------------------------------------- Reserved Used Utilization Ratio -------------------------------------- 806400 804096 99 1793024 1793024 100 2544640 2518272 98 |
The columns in the subcommand output have the following meaning:
Table 6–2 Physical Destination Disk Utilization Metrics
The disk utilization pattern depends on the characteristics of the messaging application that uses a particular physical destination. Depending on the relative flow of messages into and out of a physical destination, and the relative size of messages, the reserved disk space might grow over time.
If the message producing rate is greater than the message consuming rate, free records should generally be reused and the utilization ratio should be on the high side. However, if the message producing rate is similar to or smaller than the message consuming rate, you can expect that the utilization ratio will be low.
In general, you want the reserved disk space to stabilize and the utilization to remain high. As a rule, if the system reaches a steady state in which the amount of reserved disk space generally stays constant and utilization rate is high (above 75%), there is no need to reclaim the unused disk space. If the system reaches a steady state and utilization rate is low (below 50%), you can compact the disk to reclaim the disk space occupied by free records.
Use the compact dst subcommand to compact the data store. This is the syntax for the compact dst subcommand:
compact dst [-t destType -n destName]
The subcommand compacts the file-based data store for the physical destination of the specified type and name. If no destination type and name are specified, all destinations are compacted. Physical destinations must be paused before they can be compacted.
If the reserved disk space continues to increase over time, reconfigure the destination’s memory management by setting destination memory limit properties and limit behaviors (see Table 15–1).
Pause the destination.
imqcmd pause dst -t q -n myQueue -u admin |
Compact the disk.
imqcmd compact dst -t q -n myQueue -u admin |
Resume the physical destination.
imqcmd resume dst -t q -n myQueue -u admin |
If destination type and name are not specified, these operations are performed for all physical destinations.
The dead message queue, mq.sys.dmq, is a system-created physical destination that holds the dead messages of a broker and its other physical destinations. The dead message queue is a tool for monitoring, tuning system efficiency, and troubleshooting. For a definition of the term “dead message” and a more detailed introduction to the dead message queue, see the Message Queue Technical Overview.
The broker automatically creates a dead message queue when it starts. The broker places messages on the queue if it cannot process them, or if their time-to-live has expired. In addition, other physical destinations can use the dead message queue to hold discarded messages. Use of the dead message queue provides information that is useful for troubleshooting the system.
By default, a physical destination is configured to use the dead message queue. You can disable a physical destination from using the dead message queue, or enable it to do so, by setting the physical destination property useDMQ .
The following example creates a queue called myDist that uses the dead message queue by default:
imqcmd create dst -n myDist -t q
The following example disables use of the dead message queue for the same queue:
imqcmd update dst -n myDist -t q -o useDMQ=false
You can enable all auto-created physical destinations on a broker to use the dead message queue, or disable them from doing so, by setting the imq.autocreate.destination.useDMQ broker property.
You can use the Message Queue Command utility (imqcmd) to manage the dead message queue as you manage other queues, with some differences. For example, because the dead message queue is system-created, you cannot create, pause, or destroy it. Also, as shown in Table 6–3, default values for the dead message queue sometimes differ from those of normal queues.
You configure the dead message queue as you configure other queues, but certain physical destination properties do not apply or have different default values. Table 6–3 lists queue properties that the dead message queue handles in a unique way.
Table 6–3 Dead Message Queue Treatment of Standard Physical Destination Properties
A broker can place an entire message on the dead message queue, or it can discard the message body contents, retaining just the header and property data. By default, the dead message queue stores entire messages.
If you want to reduce the size of the dead message queue and if you do not plan to restore dead messages, consider setting the imq.destination.DMQ.truncateBody broker property to true:
imqcmd update bkr -o imq.destination.DMQ.truncateBody=true
This will discard the message body and retain only the headers and property data.
Dead message logging is disabled by default. Enabling dead message logging allows the broker to log the following events:
The broker moves a message to the dead message queue
The broker discards a message from the dead message queue and from any physical destination that does not use the dead message queue
A physical destination reaches its limits
The following command enables dead message logging:
imqcmd update bkr -o imq.destination.logDeadMsgs=true
Dead message logging applies to all physical destinations that use the dead message queue. You cannot enable or disable logging for an individual physical destination.
You manage security by configuring a user repository to authenticate users, to define access control, to configure a Secure Socket Layer (SSL) connection service that encrypts client-broker communication, and to set up a password file for use in broker startup.
The chapter includes the following sections:
When a user attempts to connect to the broker, the broker authenticates the user by inspecting the name and password provided. The broker grants the connection if the name and password match those in a broker-specific user repository that each broker is configured to consult.
You are responsible for maintaining a list of users, their groups, and their passwords in a user repository. You can use a different user repository for each broker instance. This section explains how you create, populate, and manage that repository.
The repository can be one of the following types:
A flat-file repository that is shipped with Message QueueTM
This type of user repository is very easy to use. You can populate and manage the repository using the User Manager utility (imqusermgr). To enable authentication, you populate the user repository with each user’s name and password and the name of the user’s group.
For more information on setting up and managing the user repository, see Using a Flat-File User Repository
An LDAP server
This could be an existing or new LDAP directory server that uses the LDAP v2 or v3 protocol. It is not as easy to use as the flat-file repository, but it is more scalable, and therefore better for production environments.
If you are using an LDAP user repository, you use the tools provided by the LDAP vendor to populate and manage the user repository. For more information, see Using an LDAP Server for a User Repository.
Message Queue provides a flat-file user repository and a command line tool, the User Manager utility (imqusermgr), that you can use to populate and manage the flat-file user repository. The following sections describe the flat-file user repository and how you use the User Manager utility to populate and manage that repository.
The flat-file user repository is instance-specific. A default user repository (named passwd) is automatically created for each broker instance that you start. This user repository is placed in a directory identified by the name of the broker instance with which the repository is associated (see Appendix A, Platform-Specific Locations of Message QueueTM Data):
…/instances/instanceName/etc/passwd
The repository is created with two entries. Each row of Table 7–1 shows an entry.
Table 7–1 Initial Entries in User Repository
User Name |
Password |
Group |
State |
---|---|---|---|
admin |
admin |
admin |
active |
guest |
guest |
anonymous |
active |
These initial entries allow the Message Queue broker to be used immediately after installation without intervention by the administrator:
The initial guest user entry allows clients to connect to a broker instance using the default guest user name and password.
The initial admin user entry lets you use imqcmd commands to administer a broker instance using the default admin user name and password. You should update this initial entry to change the password (see Changing the Default Administrator Password).
The following sections explain how you populate and manage a flat-file user repository.
The Message Queue User Manager utility (imqusermgr) lets you edit or populate a flat-file user repository. This section introduces the User Manager utility. Subsequent sections explain how you use the imqusermgr subcommands to accomplish specific tasks.
For full reference information about the imqusermgr command, see Chapter 13, Command Line Reference
Before using the User Manager, keep the following things in mind:
If a broker-specific user repository does not yet exist, you must start up the corresponding broker instance to create it.
The imqusermgr command has to be run on the host where the broker is installed.
You must have appropriate permissions to write to the repository,: namely, on Solaris and Linux, you must be the root user or the user who first created the broker instance.
Examples in the following sections assume the default broker instance.
The imqusermgr command has the subcommands add, delete, list, and update.
The add subcommand adds a user and associated password to the specified (or default) broker instance repository, and optionally specifies the user’s group. The subcommand syntax is as follows:
add [-i instanceName] -u userName -p passwd [-g group] [ -s]
The delete subcommand deletes the specified user from the specified (or default) broker instance repository. The subcommand syntax is as follows:
delete [-i instanceName] -u userName [ -s] [-f]
The list subcommand displays information about the specified user or all users in the specified (or default) broker instance repository. The subcommand syntax is as follows:
list [ -i instanceName] [-u userName]
The update subcommand updates the password and/or state of the specified user in the specified (or default) broker instance repository. The subcommand syntax is as follows:
update [ -i instanceName] -u userName -p passwd [ -a state] [-s] [ -f]
update [-i instanceName] -u userName -a state [-p passwd] [-s] [-f]
Table 7–2 lists the options to the imqusermgr command.
Table 7–2 imqusermgr Options
Option |
Description |
---|---|
-a activeState |
Specifies (true/false) whether the user’s state should be active. A value of true means that the state is active. This is the default. |
-f |
Performs action without user confirmation. |
-h |
Displays usage help. Nothing else on the command line is executed. |
-i instanceName |
Specifies the broker instance name to which the command applies. If not specified, the default instance name, imqbroker, is assumed. |
-p passwd |
Specifies the user’s password. |
-g group |
Specifies the user group. Valid values are admin, user, anonymous. |
-s |
Sets silent mode. |
-u userName |
Specifies the user name. |
-v |
Displays version information. Nothing else on the command line is executed. |
When adding a user entry to the user repository for a broker instance, you can specify one of three predefined groups: admin, user, or anonymous. If no group is specified, the default group user is assigned. Groups should be assigned as follows:
admin group. For broker administrators. Users who are assigned this group can, by default, configure, administer, and manage the broker. You can assign more than one user to the admin group.
user group. For normal (non-administration) Message Queue client users. Most client users are in the user group. By default, users in this group can produce messages to all topics and queues, consume messages from all topics and queues, and browse messages in any queue.
anonymous group. For Message Queue clients that do not want a user name that is known to the broker, possibly because the client application does not know of a real user name to use. This account is analogous to the anonymous account present in most FTP servers. You can assign only one user at a time to the anonymous group. You should restrict the access privileges of this group as compared to the user group or you should remove users from the group at deployment time.
To change a user’s group, you must delete the user entry and then add another entry for the user, specifying the new group.
You cannot rename or delete these system-created groups, or create new groups. However, you can specify access rules that define the operations that the members of that group can perform. For more information, see User Authorization: The Access Control Properties File.
When you add a user to a repository, the user’s state is active by default. To make the user inactive, you must use the update command. For example, the following command makes the user JoeD inactive:
imqusermgr update -u JoeD -a false
Entries for users that have been rendered inactive are retained in the repository; however, inactive users cannot open new connections. If a user is inactive and you add another user who has the same name, the operation will fail. You must delete the inactive user entry or change the new user’s name or use a different name for the new user. This prevents you from adding duplicate user names.
User names and passwords must follow these guidelines:
A user name cannot contain an asterisk (*), comma (,), colon (:), or a new-line or carriage-return character.
A user name or password must be at least one character long.
If a user name or password contains a space, the entire name or password must be enclosed in quotation marks.
There is no limit on the length of passwords or user names, except for command shell restrictions on the maximum number of characters that can be entered on a command line.
Use the add subcommand to add a user to a repository. For example, the following command adds the user Katharine with the password sesame to the default broker instance user repository.
imqusermgr add -u Katharine -p sesame -g user
Use the delete subcommand to delete a user from a repository. For example, the following command deletes the user, Bob:
imqusermgr delete -u Bob
Use the update subcommand to change a user’s password or state. For example, the following command changes Katharine’s password to aladdin:
imqusermgr update -u Katharine -p aladdin
To list information about one user or all users, use the list command. The following command shows information about the user named isa:
imqusermgr list -u isa
% imqusermgr list -u isa User repository for broker instance: imqbroker ---------------------------------- User Name Group Active State ---------------------------------- isa admin true |
The following command lists information about all users:
imqusermgr list
% imqusermgr list User repository for broker instance: imqbroker -------------------------------------- User Name Group Active State -------------------------------------- admin admin true guest anonymous true isa admin true testuser1 user true testuser2 user true testuser3 user true testuser4 user false testuser5 user false |
For the sake of security, you should change the default password of admin to one that is known only to you. The following command changes the default administrator password for the mybroker broker instance from admin to grandpoobah.
imqusermgr update mybroker -u admin -p grandpoobah
You can quickly confirm that this change is in effect by running any of the command line tools when the broker instance is running. For example, the following command will prompt you for a password:
imqcmd list svc mybroker -u admin
Entering the new password (grandpoobah) should work; the old password should fail.
After changing the password, you should supply the new password any time you use any of the Message Queue administration tools, including the Administration Console.
To use an LDAP server for a user repository, you perform the following tasks:
Editing the instance configuration file
Setting up access control for administrators
To have a broker use a directory server, you set the values for certain properties in the broker instance configuration file, config.properties. These properties enable the broker instance to query the LDAP server for information about users and groups whenever a user attempts to connect to the broker instance or perform messaging operations.
The instance configuration file is located in a directory under the broker instance directory. The path has the following format:
…/instances/instanceName /props/config.properties
For information about the operating system-specific location of instance directories, see Appendix A, Platform-Specific Locations of Message QueueTM Data
Specify that you are using an LDAP user repository by setting the following property:
imq.authentication.basic.user_repository=ldap |
Set the imq.authentication.type property to determine whether a password should be passed from client to broker in base-64 (basic ) or MD5 (digest) encoding. When using an LDAP directory server for a user repository, you must set the authentication type to basic . For example,
imq.authentication.type=basic |
You must also set the broker properties that control LDAP access. These properties are stored in a broker’s instance configuration file. The properties are discussed under Security Services and summarized under Security Properties.
Message Queue uses JNDI APIs to communicate with the LDAP directory server. Consult JNDI documentation for more information on syntax and on terms referenced in these properties. Message Queue uses a Sun JNDI LDAP provider and uses simple authentication.
Message Queue supports LDAP authentication failover: you can specify a list of LDAP directory servers for which authentication will be attempted (see the reference information for the imq.user.repos.ldap.server property).
See the broker’s config.properties file for a sample of how to set properties related to LDAP user-repository.
If necessary, you need to edit the users/groups and rules in the access control properties file. For more information about the use of access control property files, see User Authorization: The Access Control Properties File.
If you want the broker to communicate with the LDAP directory server over SSL during connection authentication and group searches, you need to activate SSL in the LDAP server and then set the following properties in the broker configuration file:
Specify the port used by the LDAP server for SSL communications. For example:
imq.user_repository.ldap.server=myhost:7878 |
Set the broker property imq.user_repository.ldap.ssl.enabled to true.
When employing multiple LDAP directory servers, use ldap:// to specify each additional directory server. For example:
imq.user_repository.ldap.server= myHost:7878 ldap:// otherHost:7878 …
Separate each additional directory server with a space. All directory servers in the list must use the same values for other LDAP-related properties.
To create administrative users, you use the access control properties file to specify users and groups that can create ADMIN connections. These users and groups must be predefined in the LDAP directory.
Any user or group who can create an ADMIN connection can issue administrative commands.
Enable the use of the access control file by setting the broker property imq.accesscontrol.enabled to true, which is the default value.
The imq.accesscontrol.enabled property enables use of the access control file.
Open the access control file, accesscontrol.properties. The location for the file is listed in Appendix A, Platform-Specific Locations of Message QueueTM Data
The file contains an entry such as the following:
service connection access control##################################connection.NORMAL.allow.user=*connection.ADMIN.allow.group=admin
The entries listed are examples. Note that the admin group exists in the file-based user repository but does not exist by default in the LDAP directory. You must substitute the name of a group that is defined in the LDAP directory, to which you want to grant Message Queue administrator privileges.
To grant Message Queue administrator privileges to users, enter the user names as follows:
connection.ADMIN.allow.user= userName[[,userName2] …]
To grant Message Queue administrator privileges to groups, enter the group names as follows:
connection.ADMIN.allow.group= groupName[[,groupName2] …]
An access control properties file (ACL file) contains rules that specify the operations that users and groups of users can perform. You edit the ACL file to restrict operations to certain users and groups. You can use a different ACL file for each broker instance.
The ACL file is used whether user information is placed in a flat-file user repository or in an LDAP user repository. A broker checks its ACL file when a client application performs one of the following operations:
Creates a connection
Creates a producer
Creates a consumer
Browses a queue
The broker checks the ACL file to determine whether the user that generated the request, or a group to which the user belongs, is authorized to perform the operation.
If you edit an ACL file, the new settings take effect the next time the broker checks the file to verify authorization. You need not restart the broker after editing the file.
The ACL file is instance specific. Each time you start a broker instance, a default file named accesscontrol.properties is created in the instance directory. The path to the file has the following format (see Appendix A, Platform-Specific Locations of Message QueueTM Data):
…/instances/brokerInstanceName/etc/accesscontrol.properties
The ACL file is formatted like a Java properties file. It starts by defining the version of the file and then specifies access control rules in three sections:
Connection access control
Physical destination access control
Physical destination auto-create access control
The version property defines the version of the ACL properties file; you may not change this entry.
version=JMQFileAccessControlModel/100
The three sections of the ACL file that specify access control are described below, following a description of the basic syntax of access rules and an explanation of how permissions are calculated.
In the ACL properties file, access control defines what access specific users or groups have to protected resources like physical destinations and connection services. Access control is expressed by a rule or set of rules, with each rule presented as a Java property:
The basic syntax of these rules is as follows:
resourceType.resourceVariant .operation.access. principalType=principals
Table 7–3 describes the elements of syntax rules.
Table 7–3 Syntactic Elements of Access Rules
Element |
Description |
---|---|
resourceType |
One of the following: connection, queue or topic. |
resourceVariant |
An instance of the type specified by resourceType. For example, myQueue. The wild card character (*) may be used to mean all connection service types or all physical destinations. |
operation |
Value depends on the kind of access rule being formulated. |
access |
One of the following: allow or deny. |
principalType |
One of the following: user or group. For more information, see Groups. |
principals |
Who may have the access specified on the left-hand side of the rule. This may be an individual user or a list of users (comma delimited) if the principalType is user; it may be a single group or a list of groups (comma delimited list) if the principalType is group. The wild card character (*) may be used to represent all users or all groups. |
Here are some examples of access rules:
The following rule means that all users may send a message to the queue named q1.
queue.q1.produce.allow.user=*
The following rule means that any user may send messages to any queue.
queue.*.produce.allow.user=*
To specify non-ASCII user, group, or destination names, use Unicode escape (\\uXXXX) notation. If you have edited and saved the ACL file with these names in a non-ASCII encoding, you can convert the file to ASCII with the Java native2ascii tool. For more detailed information, see
http://java.sun.com/j2se/1.4/docs/guide/intl/faq.html
When there are multiple access rules in the file, permissions are computed as follows:
Specific access rules override general access rules. After applying the following two rules, all users can send to all queues, but Bob cannot send to tq1.
queue.*.produce.allow.user=* queue.tq1.produce.deny.user=Bob
Access given to an explicit principal overrides access given to a * principal. The following rules deny Bob the right to produce messages to tq1, but allow everyone else to do it.
queue.tq1.produce.allow.user=* queue.tq1.produce.deny.user=Bob
The * principal rule for users overrides the corresponding * principal for groups. For example, the following two rules allow all authenticated users to send messages to tq1.
queue.tq1.produce.allow.user=* queue.tq1.produce.deny.group=*
Access granted a user overrides access granted to the user’s group. In the following example, even if Bob is a member of User, he cannot produce messages to tq1. All other members of User will be able to do so.
queue.tq1.produce.allow.group=User queue.tq1.produce.deny.user=Bob
Any access permission not explicitly granted through an access rule is implicitly denied. For example, if the ACL file contains no access rules, all users are denied all operations.
Deny and allow permissions for the same user or group cancel themselves out. For example, the following two rules cause Bob to be unable to browse q1:
queue.q1.browse.allow.user=Bob queue.q1.browse.deny.user=Bob
The following two rules prevent the group User from consuming messages at q5.
queue.q5.consume.allow.group=User queue.q5.consume.deny.group=User
When multiple same left-hand rules exist, only the last entry takes effect.
The connection access control section in the ACL properties file contains access control rules for the broker’s connection services. The syntax of connection access control rules is as follows:
connection.resourceVariant. access.principalType= principals
Two values are defined for resourceVariant: NORMAL and ADMIN. These predefined values are the only types of connection services to which you can grant access.
The default ACL properties file gives all users access to NORMAL connection services and gives users in the group admin access to ADMIN connection services:
connection.NORMAL.allow.user=* connection.ADMIN.allow.group=admin
If you are using a file-based user repository, the default group admin is created by the User Manager utility. If you are using an LDAP user repository, you can do one of the following to use the default ACL properties file:
Define a group called admin in the LDAP directory.
Replace the name admin in the ACL properties file with the names of one or more groups that are defined in the LDAP directory.
You can restrict connection access privileges. For example, the following rules deny Bob access to NORMAL but allow everyone else:
connection.NORMAL.deny.user=Bob connection.NORMAL.allow.user=*
You can use the asterisk (*) character to specify all authenticated users or groups.
The way that you use the ACL properties file to grant access to ADMIN connections differs for file-based user repositories and LDAP user repositories, as follows:
File-based user repository
If access control is disabled, users in the group admin have ADMIN connection privileges.
If access control is enabled, edit the ACL file. Explicitly grant users or groups access to the ADMIN connection service.
LDAP user repository. If you are using an LDAP user repository, do all of the following:
Enable access control.
Edit the ACL file and provide the names of users or groups who can make ADMIN connections. Specify any users or groups that are defined in the LDAP directory server.
The destination access control section of the access control properties file contains physical destination-based access control rules. These rules determine who (users/groups) may do what (operations) where (physical destinations). The types of access that are regulated by these rules include sending messages to a queue, publishing messages to a topic, receiving messages from a queue, subscribing to a topic, and browsing messages in a queue.
By default, any user or group can have all types of access to any physical destination. You can add more specific destination access rules or edit the default rules. The rest of this section explains the syntax of physical destination access rules, which you must understand to write your own rules.
The syntax of destination rules is as follows:
resourceType.resourceVariant.operation.access.principalType=principals
Table 7–4 describes these elements:
Table 7–4 Elements of Physical Destination Access Control Rules
Component |
Description |
---|---|
resourceType |
Can be queue or topic. |
resourceVariant |
A physical destination name or all physical destinations (*), meaning all queues or all topics. |
operation |
Can be produce, consume, or browse. |
access |
Can be allow or deny. |
principalType |
Can be user or group. |
Access can be given to one or more users and/or one or more groups.
The following examples illustrate different kinds of physical destination access control rules:
Allow all users to send messages to any queue destinations.
queue.*.produce.allow.user=*
Deny any member of the group user the ability to subscribe to the topic Admissions.
topic.Admissions.consume.deny.group=user
The final section of the ACL properties file, includes access rules that specify for which users and groups the broker will auto-create a physical destination.
When a user creates a producer or consumer at a physical destination that does not already exist, the broker will create the destination if the broker’s auto-create property has been enabled.
By default, any user or group has the privilege of having a physical destination auto-created by the broker. This privilege is specified by the following rules:
queue.create.allow.user=* topic.create.allow.user=*
You can edit the ACL file to restrict this type of access.
The general syntax for physical destination auto-create access rules is as follows:
resourceType.create.access.principalType=principals
Where resourceType is either queue or topic.
For example, the following rules allow the broker to auto-create topic destinations for everyone except Snoopy.
topic.create.allow.user=* topic.create.deny.user=Snoopy
Note that the effect of physical destination auto-create rules must be congruent with that of physical destination access rules. For example, if you 1) change the destination access rule to forbid any user from sending a message to a destination but 2) enable the auto-creation of the destination, the broker will create the physical destination if it does not exist but it will not deliver a message to it.
This section explains how to set up a connection service based on the Secure Socket Layer (SSL) standard, which sends encrypted messages between clients and broker. Message Queue supports the following SSL-based connection services:
The ssljms service delivers secure, encrypted messages between a client and a broker, using the TCP/IP transport protocol.
The ssladmin service creates a secure, encrypted connection between the Message Queue Command utility (imqcmd) and a broker, using the TCP/IP transport protocol. Encrypted connections are not supported for the Administration Console (imqadmin).
The cluster service provides secure, encrypted communication between brokers in a cluster, using the TCP/IP transport protocol.
The httpsjms service delivers secure, encrypted messages between a client and a broker, using an HTTPS tunnel servlet with the HTTP transport protocol.
The remainder of this section describes how to set up secure connections over TCP/IP, using the ssljms, ssladmin, and cluster connection services.For information on setting up secure connections over HTTP with the httpsjms service, see Appendix C, HTTP/HTTPS Support.
To use an SSL-based connection service over TCP/IP, you generate a public/private key pair using the Key Tool utility (imqkeytool). This utility embeds the public key in a self-signed certificate that is passed to any client requesting a connection to the broker, and the client uses the certificate to set up an encrypted connection. This section describes how to set up an SSL-based service using such self-signed certificates.
For a stronger level of authentication, you can use signed certificates verified by a certification authority. The use of signed certificates involves some additional steps beyond those needed for self-signed certificates: you must first perform the steps described in this section and then follow them with the additional ones in Using Signed Certificates.
Message Queue's support for SSL with self-signed certificates is oriented toward securing on-the-wire data, on the assumption that the client is communicating with a known and trusted server. The following procedure shows the steps needed to set up an SSL-based connection service to use self-signed certificates. The subsections that follow describe each of these steps in greater detail.
Generate a self-signed certificate.
Enable the ssljms, ssladmin, or cluster connection service in the broker.
Start the broker.
Configure and run the client.
This step applies only to the ssljms connection service and not to ssladmin or cluster.
Run the Key Tool utility (imqkeytool) to generate a self-signed certificate for the broker. (On UNIX® systems, you may need to run the utility as the superuser (root) in order to have permission to create the key store.) The same certificate can be used for the ssljms, ssladmin, or cluster connection service.
Enter the following at the command prompt:
imqkeytool -broker
The Key Tool utility prompts you for a key store password:
Generating keystore for the broker ... Enter keystore password:
Next, the utility prompts you for information identifying the broker to which this certificate belongs. The information you supply will make up an X.500 distinguished name. Table 7–5 shows the prompts and the values to be provided for each. Values are case-insensitive and can include spaces.
Table 7–5 Distinguished Name Information Required for a Self-Signed Certificate
Prompt |
X.500 Attribute |
Description |
Example |
---|---|---|---|
What is your first and last name? |
commonName (CN) |
Fully qualified name of server running the broker |
mqserver.sun.com |
What is the name of your organizational unit? |
organizationalUnit (OU) |
Name of department or division |
purchasing |
What is the name of your organization? |
organizationName (ON) |
Name of larger organization, such as a company or government entity |
My Company, Inc. |
What is the name of your city or locality? |
localityName (L) |
Name of city or locality |
San Francisco |
What is the name of your state or province? |
stateName (ST) |
Full (unabbreviated) name of state or province |
California |
What is the two-letter country code for this unit? |
country (C) |
Standard two-letter country code |
US |
When you have entered the information, the Key Tool utility displays it for confirmation. For example:
Is CN=mqserver.sun.com, OU=purchasing, ON=My Company, Inc., L=San Francisco, ST=California, C=US correct?
To accept the current values and proceed, enter yes; to reenter values, accept the default or enter no. After you confirm, the utility pauses while it generates a key pair.
Next, the utility asks for a password to lock the key pair (key password). Press Return in response to this prompt to use the same password as the key password and key store password.
Remember the password you specify. You must provide this password when you start the broker, to allow the broker to open the key store. You can store the key store password in a password file (see Password Files).
The Key Tool utility generates a self-signed certificate and places it in Message Queue’s key store. The key store is located in a directory that depends upon the operating system, as shown in Appendix A, Platform-Specific Locations of Message QueueTM Data.
The following are the configurable properties for the Message Queue key store for SSL-based connection services:
imq.keystore.file.dirpath: The path to the directory containing the key store file. For the default value, see Appendix A, Platform-Specific Locations of Message QueueTM Data.
In some circumstances, you may need to regenerate a key pair in order to solve certain problems: for example, if you forget the key store password or if the SSL-based service fails to initialize when you start a broker and you get the exception
java.security.UnrecoverableKeyException: Cannot recover key
(This exception may result if you provided a key password different from the key store password when you generated the self-signed certificate.)
Remove the broker’s key store, located as shown in Appendix A, Platform-Specific Locations of Message QueueTM Data.
Run imqkeytool again to generate a new key pair, as described above.
To enable an SSL-based connection service in the broker, you need to add ssljms (or ssladmin) to the imq.service.activelist property.
Open the broker’s instance configuration file.
The instance configuration file is located in a directory identified by the name of the broker instance (instanceName) with which the configuration file is associated (see Appendix A, Platform-Specific Locations of Message QueueTM Data):
.../instances/instanceName/props/config.properties
Add an entry (if one does not already exist) for the imq.service.activelist property and include the desired SSL-based service(s) in the list.
By default, the property includes the jms and admin connection services. Add the SSL-based service or services you wish to activate (ssljms, ssladmin, or both):
imq.service.activelist=jms,admin,ssljms,ssladmin
The SSL-based cluster connection service is enabled using the imq.cluster.transport property rather than the imq.service.activelist property; see Connecting Brokers.
Save and close the instance configuration file.
Start the broker, providing the key store password. You can provide the password in either of two ways:
Allow the broker to prompt you for the password when it starts up:
imqbrokerd Please enter Keystore password:
Put the password in a password file, as described in Password Files. Then set the property imq.passfile.enabled = true and do one of the following:
Pass the location of the password file to the imqbrokerd command:
imqbrokerd -passfile /passfileDirectory/passfileName
Start the broker without the -passfile option, but specify the location of the password file using the following two broker configuration properties:
imq.passfile.dirpath=/passfileDirectoryimq.passfile.name=passfileName
When you start a broker or client with SSL, you may notice a sharp increase in CPU usage for a few seconds. This is because the JSSE (Java Secure Socket Extension) method java.security.SecureRandom, which Message Queue uses to generate random numbers, takes a significant amount of time to create the initial random number seed. Once the seed is created, the CPU usage level will drop to normal.
The procedure for configuring a client to use an SSL-based connection service differs depending on whether it is an application client (using the ssljms connection service) or a Message Queue administrative client such as imqcmd (using the ssladmin connection service.)
For application clients, you must make sure the client has the following .jar files specified in its CLASSPATH variable:
imq.jar
jms.jar
If you are using a version of the Java 2 Software Development Kit (J2SDK) earlier than 1.4, you must also include the following Java Secure Socket Extension (JSSE) and Java Naming and Directory Interface (JNDI) .jar files:
jsse.jar
jnet.jar
jcert.jar
jndi.jar
(It is not necessary to include these files if you are using J2SDK 1.4 or later, which has JSSE and JNDI support built in.)
Once the CLASSPATH files are properly specified, one way to start the client and connect to the broker’s ssljms connection service is by entering a command like the following:
java -DimqConnectionType=TLS clientAppName
This tells the connection to use an SSL-based connection service.
For administrative clients, you can establish a secure connection by including the -secure option when you invoke the imqcmd command: for example,
imqcmd list svc -b hostName:portNumber -u adminName -secure
where adminName is a valid entry in the Message Queue user repository. The command will prompt you for the password. (If you are using a flat-file repository, see Changing the Default Administrator Password).
Listing the connection services is a way to verify that the ssladmin service is running and that you can successfully make a secure administrative connection, as shown in the following output:
Listing all the services on the broker specified by: Host Primary Port localhost 7676 Service Name Port Number Service State admin 33984 (dynamic) RUNNING httpjms - UNKNOWN httpsjms - UNKNOWN jms 33983 (dynamic) RUNNING ssladmin 35988 (dynamic) RUNNING ssljms dynamic UNKNOWN Successfully listed services. |
Signed certificates provide a stronger level of server authentication than self-signed certificates. You can implement signed certificates only between a client and broker, and not between multiple brokers in a cluster. This requires the following extra steps in addition to the ones described above for configuring self-signed certificates. These steps are described in greater detail in the subsections that follow.
Install the certificate in the key store.
Configure the Message Queue client to require signed certificates when establishing an SSL-based connection to the broker.
The following procedures explain how to obtain and install a signed certificate.
Use the J2SE keytool command to generate a certificate signing request (CSR) for the self-signed certificate you generated in the preceding section.
Information about the keytool command can be found at
http://java.sun.com/j2se/1.5.0/docs/tooldocs/solaris/keytool.html
Here is an example:
keytool -certreq -keyalg RSA -alias imq -file certreq.csr -keystore /etc/imq/keystore -storepass myStorePassword |
This generates a CSR encapsulating the certificate in the specified file (certreq.csr in the example).
Use the CSRto generate or request a signed certificate.
You can do this by either of the following methods:
Have the certificate signed by a well known certification authority (CA), such as Thawte or Verisign. See your CA’s documentation for more information on how to do this.
Sign the certificate yourself, using an SSL signing software package.
The resulting signed certificate is a sequence of ASCII characters. If you receive the signed certificate from a CA, it may arrive as an e-mail attachment or in the text of a message.
Save the signed certificate in a file.
The instructions below use the example name broker.cer to represent the broker certificate.
Check whether J2SE supports your certification authority by default.
The following command lists the root CAs in the system key store:
keytool -v -list -keystore $JAVA_HOME/lib/security/cacerts
If your CA is listed, skip the next step.
If your certification authority is not supported in J2SE, import the CA’s root certificate into the Message Queue key store.
Here is an example:
keytool -import -alias ca -file ca.cer -noprompt -trustcacerts -keystore /etc/imq/keystore -storepass myStorePassword
where ca.cer is the file containing the root certificate obtained from the CA.
If you are using a CA test certificate, you probably need to import the test CA root certificate. Your CA should have instructions on how to obtain a copy.
Import the signed certificate into the key store to replace the original self-signed certificate.
Here is an example:
keytool -import -alias imq -file broker.cer -noprompt -trustcacerts -keystore /etc/imq/keystore -storepass myStorePassword
where broker.cer is the file containing the signed certificate that you received from the CA.
The Message Queue key store now contains a signed certificate to use for SSL connections.
You must now configure the Message Queue client runtime to require signed certificates, and ensure that it trusts the certification authority that signed the certificate.
Set the connection factory's imqSSLIsHostTrusted attribute to false.
By default, the imqSSLIsHostTrusted attribute of the connection factory object that the client will be using to establish broker connections is set to true, meaning that the client runtime will accept any certificate presented to it. You must change this value to false so that the client runtime will attempt to validate all certificates presented to it. Validation will fail if the signer of the certificate is not in the client's trust store.
Verify whether the signing authority is registered in the client's trust store.
To test whether the client will accept certificates signed by your certification authority, try to establish an SSL connection, as described above under Configuring and Running an SSL-Based Client.If the CA is in the client's trust store, the connection will succeed and you can skip the next step. If the connection fails with a certificate validation error, go on to the next step.
Install the signing CA’s root certificate in the client’s trust store.
The client searches the key store files cacerts and jssecacerts by default, so no further configuration is necessary if you install the certificate in either of those files. The following example installs a test root certificate from the Verisign certification authority from a file named testrootca.cer into the default system certificate file, cacerts.The example assumes that J2SE is installed in the directory $JAVA_HOME/usr/j2se:
keytool -import -keystore /usr/j2se/jre/lib/security/cacerts -alias VerisignTestCA -file testrootca.cer -noprompt -trustcacerts -storepass myStorePassword
An alternative (and recommended) option is to install the root certificate into the alternative system certificate file, jssecacerts :
keytool -import -keystore /usr/j2se/jre/lib/security/jssecacerts -alias VerisignTestCA -file testrootca.cer -noprompt -trustcacerts -storepass myStorePassword
A third possibility is to install the root certificate into some other key store file and configure the client to use that as its trust store.The following example installs into the file /home/smith/.keystore:
keytool -import -keystore /home/smith/.keystore -alias VerisignTestCA -file testrootca.cer -noprompt -trustcacerts -storepass myStorePassword
Since the client does not search this key store by default, you must explicitly provide its location to the client to use as a trust store. You do this by setting the Java system property javax.net.ssl.trustStore once the client is running:
javax.net.ssl.trustStore=/home/smith/.keystore
Several types of commands require passwords. In Table 7–6, the first column lists the commands that require passwords and the second column lists the reason that passwords are needed.
Table 7–6 Commands That Use Passwords
Command |
Purpose |
Purpose of Password |
---|---|---|
imqbrokerd |
Start the broker |
Access a JDBC-based persistent data store, an SSL certificate key store, or an LDAP user repository |
imqcmd |
Manage the broker |
Authenticate an administrative user who is authorized to use the command |
imqdbmgr |
Manage a JDBC-based data store |
Access the data store |
You can specify these passwords in a password file and use the -passfile option to specify the name of the file. This is the format for the -passfile option:
imqbrokerd -passfile myPassfile
In previous releases, you could use the -p, -password, -dbpassword, and -ldappassword options to specify passwords on a command line. These options are deprecated and will be removed in a future release. In the current release, a value on the command line for one of these options supersedes the associated value in a password file.
Specifying a password interactively, in response to a prompt, is the most secure method of specifying a password, unless your monitor is visible to other people. You can also specify a password file on the command line. For non-interactive use of commands, however, you must use a password file.
A password file is unencrypted, so you must set its permissions to protect it from unauthorized access. Set permissions such that they limit the users who can view the file, but provide read access to the user who starts the broker.
A password file is a simple text file that contains a set of properties and values. Each value is a password used by a command.
A password file can contain the passwords shown in Table 7–7:
Table 7–7 Passwords in a Password File
A sample password file is part of the Message Queue product. For the location of the sample file, see Appendix A, Platform-Specific Locations of Message QueueTM Data
When a client application is separated from the broker by a firewall, special measures are needed in order to establish a connection. One approach is to use the httpjms or httpsjms connection service, which can “tunnel” through the firewall; see Appendix C, HTTP/HTTPS Support for details. HTTP connections are slower than other connection services, however; a faster alternative is to bypass the Message Queue Port Mapper and explicitly assign a static port address to the desired connection service, and then open that specific port in the firewall. This approach can be used to connect through a firewall using the jms or ssljms connection service (or, in unusual cases, admin or ssladmin).
Table 7–8 Broker Configuration Properties for Static Port Addresses
Connection Service |
Configuration Property |
---|---|
jms |
imq.jms.tcp.port |
ssljms |
imq.ssljms.tls.port |
admin |
imq.admin.tcp.port |
ssladmin |
imq.ssladmin.tls.port |
Assign a static port address to the connection service you wish to use.
To bypass the Port Mapper and assign a static port number directly to a connection service, set the broker configuration property imq.serviceName.protocolType.port, where serviceName is the name of the connection service and protocolType is its protocol type (see Table 7–8). As with all broker configuration properties, you can specify this property either in the broker's instance configuration file or from the command line when starting the broker. For example, to assign port number 10234 to the jms connection service, either include the line
imq.jms.tcp.port=10234
in the configuration file or start the broker with the command
imqbrokerd -name myBroker -Dimq.jms.tcp.port=10234
Configure the firewall to allow connections to the port number you assigned to the connection service.
You must also allow connections through the firewall to Message Queue's Port Mapper port (normally 7676, unless you have reassigned the Port Mapper to some other port). In the example above, for instance, you would need to open the firewall for ports 10234 and 7676.
Message Queue supports audit logging in Enterprise Edition only. When audit logging is enabled, Message Queue generates a record for the following types of events:
Startup, shutdown, restart, and removal of a broker instance
User authentication and authorization
Reset of a persistent store
Creation, purge, and destruction of a physical destination
Administrative destruction of a durable subscriber
To log audit records to the Message Queue broker log file, set the imq.audit.enabled broker property to true . All audit records in the log contain the keyword AUDIT.
Administered objects encapsulate provider-specific configuration and naming information, enabling the development of client applications that are portable from one JMS provider to another. A Message QueueTM administrator typically creates administered objects for client applications to use in obtaining broker connections for sending and receiving messages.
This chapter tells how to use the Object Manager utility (imqobjmgr ) to create and manage administered objects. It contains the following sections:
Administered objects are placed in a readily available object store where they can be accessed by client applications via the Java Naming and Directory Interface (JNDI). There are two types of object store you can use: a standard Lightweight Directory Access Protocol (LDAP) directory server or a directory in the local file system.
An LDAP server is the recommended object store for production messaging systems. LDAP servers are designed for use in distributed systems and provide security features that are useful in production environments.
LDAP implementations are available from a number of vendors. To manage an object store on an LDAP server with Message Queue administration tools, you may first need to configure the server to store Java objects and perform JNDI lookups; see the documentation provided with your LDAP implementation for details.
To use an LDAP server as your object store, you must specify the attributes shown in Table 8–1. These attributes fall into the following categories:
Initial context. The java.naming.factory.initial attribute specifies the initial context for JNDI lookups on the server. The value of this attribute is fixed for a given LDAP object store.
Location. The java.naming.provider.url attribute specifies the URL and directory path for the LDAP server. You must verify that the specified directory path exists.
Security. The attributes java.naming.security.principal , java.naming.security.credentials, and java.naming.security.authentication govern the authentication of callers attempting to access the object store. The exact format and values of these attributes depend on the LDAP service provider; see the documentation provided with your LDAP implementation for details and to determine whether security information is required on all operations or only on those that change the stored data.
Attribute |
Description |
---|---|
Initial context for JNDI lookup Example: com.sun.jndi.ldap.LdapCtxFactory |
|
Server URL and directory path Example: ldap://myD.com:389/ou=mq1,o=App where administered objects are stored in the directory /App/mq1 . |
|
Identity of the principal for authenticating callers The format of this attribute depends on the authentication scheme: for example, uid=homerSimpson,ou=People,o=mq If this attribute is unspecified, the behavior is determined by the LDAP service provider. |
|
Credentials of the authentication principal The value of this attribute depends on the authentication scheme: for example, it might be a hashed password, a clear-text password, a key, or a certificate. If this property is unspecified, the behavior is determined by the LDAP service provider. |
|
Security level for authentication The value of this attribute is one of the keywords none, simple, or strong. For example, If you specify simple, you will be prompted for any missing principal or credential values. This will allow you a more secure way of providing identifying information. If this property is unspecified, the behavior is determined by the LDAP service provider. |
Message Queue also supports the use of a directory in the local file system as an object store for administered objects. While this approach is not recommended for production systems, it has the advantage of being very easy to use in development environments. Note, however, that for a directory to be used as a centralized object store for clients deployed across multiple computer nodes, all of those clients must have access to the directory. In addition, any user with access to the directory can use Message Queue administration tools to create and manage administered objects.
To use a file-system directory as your object store, you must specify the attributes shown in Table 8–2. These attributes have the same general meanings described above for LDAP object stores; in particular, the java.naming.provider.url attribute specifies the directory path of the directory holding the object store. This directory must exist and have the proper access permissions for the user of Message Queue administration tools as well as the users of the client applications that will access the store.
Table 8–2 File-system Object Store Attributes
Attribute |
Description |
|
---|---|---|
|
Initial context for JNDI lookup Example: com.sun.jndi.fscontext.RefFSContextFactory |
|
|
Directory path Example: file:///C:/myapp/mqobjs |
Message Queue administered objects are of two basic kinds:
Connection factories are used by client applications to create connections to brokers.
Destinations represent locations on a broker with which client applications can exchange (send and retrieve) messages.
A special SOAP endpoint administered object is used for SOAP messaging; see the Message Queue Developer's Guide for Java Clients for more information.
Each type of administered object has certain attributes that determine the object’s properties and behavior. This section describes how to use the Object Manager command line utility (imqobjmgr) to set these attributes; you can also set them with the GUI Administration Console, as described in Working With Administered Objects.
Client applications use connection factory administered objects to create connections with which to exchange messages with a broker. A connection factory’s attributes define the properties of all connections it creates. Once a connection has been created, its properties cannot be changed; thus the only way to configure a connection’s properties is by setting the attributes of the connection factory used to create it.
Message Queue defines two classes of connection factory objects:
ConnectionFactory objects support normal messaging and nondistributed transactions.
XAConnectionFactory objects support distributed transactions.
Both classes share the same configuration attributes, which you can use to optimize resources, performance, and message throughput. These attributes are listed and described in detail in Chapter 16, Administered Object Attribute Reference and are discussed in the following sections below:
Connection handling attributes specify the broker address to which to connect and, if required, how to detect connection failure and attempt reconnection. They are summarized in Table 16–1.
The most important connection handling attribute is imqAddressList , which specifies the broker or brokers to which to establish a connection. The value of this attribute is a string containing a broker address or (in the case of a broker cluster) multiple addresses separated by commas. Broker addresses can use a variety of addressing schemes, depending on the connection service to be used (see Connection Services) and the method of establishing a connection:
mq uses the broker’s Port Mapper to assign a port dynamically for either the jms or ssljms connection service.
mqtcp bypasses the Port Mapper and connects directly to a specified port, using the jms connection service.
mqssl makes a Secure Socket Layer (SSL) connection to a specified port, using the ssljms connection service.
http makes a Hypertext Transport Protocol (HTTP) connection to a Message Queue tunnel servlet at a specified URL, using the httpjms connection service.
https makes a Secure Hypertext Transport Protocol (HTTPS) connection to a Message Queue tunnel servlet at a specified URL, using the httpsjms connection service.
These addressing schemes are summarized in Table 16–2.
The general format for each broker address is
scheme://address
where scheme is one of the addressing schemes listed above and address denotes the broker address itself. The exact syntax for specifying the address varies depending on the addressing scheme, as shown in the last column of Table 16–2. Table 16–3 shows examples of the various address formats.
In a multiple-broker cluster environment, the address list can contain more than one broker address. If the first connection attempt fails, the Message Queue client runtime will attempt to connect to another address in the list, and so on until the list is exhausted. Two additional connection factory attributes control the way this is done:
imqAddressListBehavior specifies the order in which to try the specified addresses. If this attribute is set to the string PRIORITY , addresses will be tried in the order in which they appear in the address list. If the attribute value is RANDOM, the addresses will instead be tried in random order; this is useful, for instance, when many Message Queue clients are sharing the same connection factory object, to prevent them from all attempting to connect to the same broker address.
imqAddressListIterations specifies how many times to cycle through the list before giving up and reporting failure. A value of -1 denotes an unlimited number of iterations: the client runtime will keep trying until it succeeds in establishing a connection or until the end of time, whichever occurs first.
By setting a connection factory’s imqReconnectEnabled attribute to true, you can enable a client to reconnect automatically to a broker if a connection fails. The imqReconnectAttempts attribute controls the number of reconnection attempts to a given broker address; imqReconnectInterval specifies the interval, in milliseconds, to wait between attempts.
In a broker cluster, where the broker address list (imqAddressList ) specifies multiple addresses, a failed connection can be restored not only on the original broker, but also on a different one in the cluster. If reconnection to the original broker fails, the client runtime will try the other addresses in the list. The imqAddressListBehavior and imqAddressListIterations attributes control the order in which addresses are tried and the number of iterations through the list, as described in the preceding section. Each address is tried repeatedly at intervals of imqReconnectInterval milliseconds, up to the maximum number of attempts specified by imqReconnectAttempts.
Automatic reconnection supports all client acknowledgment modes for message consumption. Once a connection has been reestablished, the broker will redeliver all unacknowledged messages it had previously delivered, marking them with a Redeliver flag. Application code can use this flag to determine whether any message has already been consumed but not yet acknowledged. (In the case of nondurable subscribers, however, the broker does not hold messages once their connections have been closed. Thus any messages produced for such subscribers while the connection is down cannot be delivered after reconnection and will be lost.) Message production is blocked while automatic reconnection is in progress; message producers cannot send messages to the broker until after the connection has been reestablished.
Automatic reconnection provides connection failover, but not data failover: persistent messages and other state information held by a failed or disconnected broker can be lost when the client is reconnected to a different broker instance. While attempting to reestablish a connection, Message Queue does maintain objects (such as sessions, message consumers, and message producers) provided by the client runtime. Temporary destinations are also maintained for a time when a connection fails, because clients might reconnect and access them again; after giving clients time to reconnect and use these destinations, the broker will delete them. In circumstances where the client-side state cannot be fully restored on the broker on reconnection (for example, when using transacted sessions, which exist only for the duration of a connection), automatic reconnection will not take place and the connection’s exception handler will be called instead. It is then up to the application code to catch the exception, reconnect, and restore state.
The Message Queue client runtime can be configured to periodically test, or “ping,” a connection, allowing connection failures to be detected preemptively before an attempted message transmission fails. Such testing is particularly important for client applications that only consume messages and do not produce them, since such applications cannot otherwise detect when a connection has failed. Clients that produce messages only infrequently can also benefit from this feature.
The connection factory attribute imqPingInterval specifies the frequency, in seconds, with which to ping a connection. By default, this interval is set to 30 seconds; a value of -1 disables the ping operation.
The response to an unsuccessful ping varies from one operating-system platform to another. On some operating systems, an exception is immediately thrown to the client application’s exception listener. (If the client does not have an exception listener, its next attempt to use the connection will fail.) Other systems may continue trying to establish a connection to the broker, buffering successive pings until one succeeds or the buffer overflows.
The connection factory attributes listed in Table 16–4 support client authentication and the setting of client identifiers for durable subscribers.
All attempts to connect to a broker must be authenticated by user name and password against a user repository maintained by the message service. The connection factory attributes imqDefaultUsername and imqDefaultPassword specify a default user name and password to be used if the client does not supply them explicitly when creating a connection.
As a convenience for developers who do not wish to bother populating a user repository during application development and testing, Message Queue provides a guest user account with user name and password both equal to guest. This is also the default value for the imqDefaultUsername and imqDefaultPassword attributes, so that if they are not specified explicitly, clients can always obtain a connection under the guest account. In a production environment, access to broker connections should be restricted to users who are explicitly registered in the user repository.
The Java Message Service Specification requires that a connection provide a unique client identifier whenever the broker must maintain a persistent state on behalf of a client. Message Queue uses such client identifiers to keep track of durable subscribers to a topic destination. When a durable subscriber becomes inactive, the broker retains all incoming messages for the topic and delivers them when the subscriber becomes active again. The broker identifies the subscriber by means of its client identifier.
While it is possible for a client application to set its own client identifier programmatically using the connection object’s setClientID method, this makes it difficult to coordinate client identifiers to ensure that each is unique. It is generally better to have Message Queue automatically assign a unique identifier when creating a connection on behalf of a client. This can be done by setting the connection factory’s imqConfiguredClientID attribute to a value of the form
${u}factoryID
The characters ${u} must be the first four characters of the attribute value. (Any character other than u between the braces will cause an exception to be thrown on connection creation; in any other position, these characters have no special meaning and will be treated as plain text.) The value for factoryID is a character string uniquely associated with this connection factory object.
When creating a connection for a particular client, Message Queue will construct a client identifier by replacing the characters ${u} with u:userName, where userName is the user name authenticated for the connection. This ensures that connections created by a given connection factory, although identical in all other respects, will each have their own unique client identifier. For example, if the user name is Calvin and the string specified for the connection factory’s imqConfiguredClientID attribute is ${u}Hobbes, the client identifier assigned will be u:CalvinHobbes.
This scheme will not work if two clients both attempt to obtain connections using the default user name guest, since each would have a client identifier with the same ${u} component. In this case, only the first client to request a connection will get one; the second client’s connection attempt will fail, because Message Queue cannot create two connections with the same client identifier.
Even if you specify a client identifier with imqConfiguredClientID , client applications can override this setting with the connection method setClientID. You can prevent this by setting the connection factory’s imqDisableSetClientID attribute to true. Note that for an application that uses durable subscribers, the client identifier must be set one way or the other: either administratively with imqConfiguredClientID or programmatically with setClientID.
Because “payload” messages sent and received by clients and control messages (such as broker acknowledgments) used by Message Queue itself pass over the same client-broker connection, excessive levels of payload traffic can interfere with the delivery of control messages. To help alleviate this problem, the connection factory attributes listed in Table 16–5 allow you to manage the relative flow of the two types of message. These attributes fall into four categories:
Acknowledgment timeout specifies the maximum time (imqAckTimeout) to wait for a broker acknowledgment before throwing an exception.
Connection flow metering limits the transmission of payload messages to batches of a specified size (imqConnectionFlowCount ), ensuring periodic opportunities to deliver any accumulated control messages.
Connection flow control limits the number of payload messages (imqConnectionFlowLimit) that can be held pending on a connection, waiting to be consumed. When the limit is reached, delivery of payload messages to the connection is suspended until the number of messages awaiting consumption falls below the limit. Use of this feature is controlled by a boolean flag (imqConnectionFlowLimitEnabled).
Consumer flow control limits the number of payload messages (imqConsumerFlowLimit) that can be held pending for any single consumer, waiting to be consumed. (This limit can also be specified as a property of a specific queue destination, consumerFlowLimit.) When the limit is reached, delivery of payload messages to the consumer is suspended until the number of messages awaiting consumption, as a percentage of imqConsumerFlowLimit , falls below the limit specified by the imqConsumerFlowThreshold attribute. This helps improve load balancing among multiple consumers by preventing any one consumer from starving others on the same connection.
The use of any of these flow control techniques involves a tradeoff between reliability and throughput; see Client Runtime Message Flow Adjustments for further discussion.
Table 16–6 lists connection factory attributes affecting client queue browsing and server sessions. The imqQueueBrowserMaxMessagesPerRetrieve attribute specifies the maximum number of messages to retrieve at one time when browsing the contents of a queue destination; imqQueueBrowserRetrieveTimeout gives the maximum waiting time for retrieving them. (Note that imqQueueBrowserMaxMessagesPerRetrieve does not affect the total number of messages browsed, only the way they are chunked for delivery to the client runtime: fewer but larger chunks or more but smaller ones. The client application will always receive all messages in the queue. Changing the attribute's value may affect performance, but will not affect the total amount of data retrieved.) The boolean attribute imqLoadMaxToServerSession governs the behavior of connection consumers in an application server session: if the value of this attribute is true, the client will load up to the maximum number of messages into a server session; if false, it will load only a single message at a time.
The Java Message Service Specification defines certain standard message properties, which JMS providers (such as Message Queue) may optionally choose to support. By convention, the names of all such standard properties begin with the letters JMSX. The connection factory attributes listed in Table 16–7 control whether the Message Queue client runtime sets certain of these standard properties. For produced messages, these include the following properties:
JMSXUserID Identity of the user sending the message
JMSXAppID Identity of the application sending the message
JMSXProducerTXID Transaction identifier of the transaction within which the message was produced
For consumed messages, they include
JMSXConsumerTXID Transaction identifier of the transaction within which the message was consumed
JMSXRcvTimestamp Time the message was delivered to the consumer
You can use the connection factory attributes listed in Table 16–8 to override the values set by a client for certain JMS message header fields. The settings you specify will be used for all messages produced by connections obtained from that connection factory. Header fields that you can override in this way are
There are two attributes for each of these fields: one boolean, to control whether the field can be overridden, and another to specify its value. For instance, the attributes for setting the priority level are imqOverrideJMSPriority and imqJMSPriority. There is also an additional attribute, imqOverrideJMSHeadersToTemporaryDestinations, that controls whether override values apply to temporary destinations.
Because overriding message headers may interfere with the needs of specific applications, these attributes should only be used in consultation with an application’s designers or users.
The destination administered object that identifies a physical queue or topic destination has only two attributes, listed in Table 16–9. The important one is imqDestinationName, which gives the name of the physical destination that this administered object represents; this is the name that was specified with the -n option to the imqcmd create dst command that created the physical destination. (Note that there is not necessarily a one-to-one relationship between destination administered objects and the physical destinations they represent: a single physical destination can be referenced by more than one administered object, or by none at all.) There is also an optional descriptive string, imqDestinationDescription, which you can use to help identify the destination object and distinguish it from others you may have created.
The Message Queue Object Manager utility (imqobjmgr) allows you to create and manage administered objects. The imqobjmgr command provides the following subcommands for performing various operations on administered objects:
Add an administered object to an object store
Delete an administered object from an object store
List existing administered objects in an object store
Display information about an administered object
Modify the attributes of an administered object
See Object Manager Utility for reference information about the syntax, subcommands, and options of the imqobjmgr command.
Most Object Manager operations require you to specify the following information as options to the imqobjmgr command:
The JNDI lookup name (-l) of the administered object
This is the logical name by which client applications can look up the administered object in the object store, using the Java Naming and Directory Interface.
The attributes of the JNDI object store (-j)
See Object Stores for information on the possible attributes and their values.
The type (-t) of the administered object
Possible types include the following:
Queue destination
Topic destination
Connection factory
Queue connection factory
Topic connection factory
Connection factory for distributed transactions
Queue connection factory for distributed transactions
Topic connection factory for distributed transactions
SOAP endpoint
The attributes (-o) of the administered object
See Administered Object Attributes for information on the possible attributes and their values.
The imqobjmgr command’s add subcommand adds administered objects for connection factories and topic or queue destinations to the object store. Administered objects stored in an LDAP object store must have lookup names beginning with the prefix cn=; lookup names in a file-system object store need not begin with any particular prefix, but must not include the slash character (/).
The Object Manager lists and displays only Message Queue administered objects. If an object store should contain a non–Message Queue object with the same lookup name as an administered object that you wish to add, you will receive an error when you attempt the add operation.
To enable client applications to create broker connections, add a connection factory administered object for the type of connection to be created: a queue connection factory or a topic connection factory. Example 8–1shows a command to add a queue connection factory (administered object type qf ) to an LDAP object store. The object has lookup name cn=myQCF and connects to a broker running on host myHost at port number 7272, using the jms connection service.
imqobjmgr add -l "cn=myQCF" -j "java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory" -j "java.naming.provider.url=ldap://mydomain.com:389/o=imq" -j "java.naming.security.principal=uid=homerSimpson,ou=People,o=imq" -j "java.naming.security.credentials=doh" -j "java.naming.security.authentication=simple" -t qf -o "imqAddressList=mq://myHost:7272/jms" |
When creating an administered object representing a destination, it is good practice to create the physical destination first, before adding the administered object to the object store. Use the Command utility (imqcmd) to create the physical destination, as described in Creating a Physical Destination.
The command shown in Example 8–2 adds an administered object to an LDAP object store representing a topic destination with lookup name myTopic and physical destination name physTopic. The command for adding a queue destination would be similar, except that the administered object type (-t option) would be q (for “queue destination”) instead of t (for “topic destination”).
imqobjmgr add -l "cn=myTopic" -j "java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory" -j "java.naming.provider.url=ldap://mydomain.com:389/o=imq" -j "java.naming.security.principal=uid=homerSimpson,ou=People,o=imq" -j "java.naming.security.credentials=doh" -j "java.naming.security.authentication=simple" -t t -o "imqDestinationName=physTopic" |
Example 8–3 shows the same command, but with the administered object stored in a Solaris file system instead of an LDAP server.
imqobjmgr add -l "cn=myTopic" -j "java.naming.factory.initial= com.sun.jndi.fscontext.RefFSContextFactory" -j "java.naming.provider.url=file:///home/foo/imq_admin_objects" -t t -o "imqDestinationName=physTopic" |
To delete an administered object from the object store, you use the delete subcommand of the imqobjmgr command, specify lookup name, type, and location of the object to be deleted. The command shown in Example 8–4 deletes the object that was added in Adding a Destination above.
imqobjmgr delete -l "cn=myTopic" -j "java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory" -j "java.naming.provider.url=ldap://mydomain.com:389/o=imq" -j "java.naming.security.principal=uid=homerSimpson,ou=People,o=imq" -j "java.naming.security.credentials=doh" -j "java.naming.security.authentication=simple" -t t |
You can use the Object Manager’s list subcommand to get a list of all administered objects in an object store or those of a specific type. Example 8–5 shows how to list all administered objects on an LDAP server.
imqobjmgr list -j "java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory" -j "java.naming.provider.url=ldap://mydomain.com:389/o=imq" -j "java.naming.security.principal=uid=homerSimpson,ou=People,o=imq" -j "java.naming.security.credentials=doh" -j "java.naming.security.authentication=simple" |
Example 8–6 lists all queue destinations (type q).
imqobjmgr list -j "java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory" -j "java.naming.provider.url=ldap://mydomain.com:389/o=imq" -j "java.naming.security.principal=uid=homerSimpson,ou=People,o=imq" -j "java.naming.security.credentials=doh" -j "java.naming.security.authentication=simple" -t q |
The query subcommand displays information about a specified administered object, identified by its lookup name and the attributes of the object store containing it. Example 8–7 displays information about an object whose lookup name is cn=myTopic.
imqobjmgr query -l "cn=myTopic" -j "java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory" -j "java.naming.provider.url=ldap://mydomain.com:389/o=imq" -j "java.naming.security.principal=uid=homerSimpson,ou=People,o=imq" -j "java.naming.security.credentials=doh" -j "java.naming.security.authentication=simple" |
To modify the attributes of an administered object, use the imqobjmgr update subcommand. You supply the object’s lookup name and location, and use the -o option to specify the new attribute values.
Modifying Administered Object Attributes changes the value of the imqReconnectAttempts attribute for the queue connection factory that was added to the object store in Example 8–8.
imqobjmgr update -l "cn=myQCF" -j "java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory" -j "java.naming.provider.url=ldap://mydomain.com:389/o=imq" -j "java.naming.security.principal=uid=homerSimpson,ou=People,o=imq" -j "java.naming.security.credentials=doh" -j "java.naming.security.authentication=simple" -t qf -o "imqReconnectAttempts=3" |
The -i option to the imqobjmgr command allows you to specify the name of a command file that uses Java property file syntax to represent all or part of the subcommand clause. This feature is especially useful for specifying object store attributes, which typically require a lot of typing and are likely to be the same across multiple invocations of imqobjmgr. Using a command file can also allow you to avoid exceeding the maximum number of characters allowed for the command line.
Example 8–9 shows the general syntax for an Object Manager command file. Note that the version property is not a command line option: it refers to the version of the command file itself (not that of the Message Queue product) and must be set to the value 2.0 .
version=2.0 cmdtype=[ add | delete | list | query | update ] obj.lookupName=lookup name objstore.attrs.objStoreAttrName1=value1 objstore.attrs.objStoreAttrName2=value2 . . . objstore.attrs.objStoreAttrNameN=valueN obj.type=[ q | t | cf | qf | tf | xcf | xqf | xtf | e ] obj.attrs.objAttrName1=value1 obj.attrs.objAttrName2=value2 . . . obj.attrs.objAttrNameN=valueN |
As an example, consider the Object Manager command shown earlier in Example 8–1, which adds a queue connection factory to an LDAP object store. This command can be encapsulated in a command file as shown in Example 8–10. If the command file is named MyCmdFile, you can then execute the command with the command line
imqobjmgr -i MyCmdFile
version=2.0 cmdtype=add obj.lookupName=cn=myQCF objstore.attrs.java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory objstore.attrs.java.naming.provider.url=ldap://mydomain.com:389/o=imq objstore.attrs.java.naming.security.principal=\\ uid=homerSimpson,ou=People,o=imq objstore.attrs.java.naming.security.credentials=doh objstore.attrs.java.naming.security.authentication=simple obj.type=qf obj.attrs.imqAddressList=mq://myHost:7272/jms |
A command file can also be used to specify only part of the imqobjmgr subcommand clause, with the remainder supplied explicitly on the command line. For example, the command file shown in Example 8–11 specifies only the attribute values for an LDAP object store.
version=2.0 objstore.attrs.java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory objstore.attrs.java.naming.provider.url=ldap://mydomain.com:389/o=imq objstore.attrs.java.naming.security.principal=\\ uid=homerSimpson,ou=People,o=imq objstore.attrs.java.naming.security.credentials=doh objstore.attrs.java.naming.security.authentication=simple |
You could then use this command file to specify the object store in an imqobjmgr command while supplying the remaining options explicitly, as shown in Example 8–12.
imqobjmgr add -l "cn=myQCF" -i MyCmdFile -t qf -o "imqAddressList=mq://myHost:7272/jms" |
Additional examples of command files can be found at the following locations, depending on your platform:
Solaris: /usr/demo/imq/imqobjmgr Linux: /opt/sun/mq/examples/imqobjmgr Windows: IMQ_HOME/demo/imqobjmgr
Message QueueTM Enterprise Edition supports the use of broker clusters: groups of brokers working together to provide message delivery services to clients. Clusters enable a message service to scale its operations with the volume of message traffic by distributing client connections among multiple brokers. See the Message Queue Technical Overview for a general discussion of clusters and how they operate.
This chapter describes how to manage broker clusters, connect brokers to them, and configure them. It contains the following sections:
You define a cluster by specifying cluster configuration properties for each of its member brokers. You can set these properties individually for each broker in the cluster, but it is generally more convenient to collect them into a central cluster configuration file that all of the brokers reference. This prevents the settings from getting out of agreement and ensures that all brokers in a cluster share the same, consistent configuration information.
The cluster configuration properties are described in detail in Table 14–9. They include the following:
imq.cluster.brokerlist gives the host names and port numbers for all brokers belonging to the cluster.
imq.cluster.masterbroker designates which broker (if any) is the master broker that keeps track of state changes.
imq.cluster.url specifies the location of the cluster configuration file, if any.
imq.cluster.hostname gives the host name or IP address for the cluster connection service, used for internal communication between brokers in the cluster. This setting can be useful if more than one host is available: for example, if there is more than one network interface card in a computer.
imq.cluster.port gives the port number for the cluster connection service.
imq.cluster.transport specifies the transport protocol used by the cluster connection service, such as tcp or ssl.
The hostname and port properties can be set independently for each individual broker, but brokerlist, masterbroker, url, and transport must have the same values for all brokers in the cluster.
The following sections describe how to set a broker’s cluster configuration properties, either individually for each broker in a cluster or centrally, using a cluster configuration file.
You can set a broker’s cluster configuration properties in its instance configuration file (or on the command line when you start the broker). For example, to create a cluster consisting of brokers at port 9876 on host1, port 5000 on host2, and the default port (7676) on ctrlhost , you would include the following property in the instance configuration files for all three brokers:
imq.cluster.brokerlist=host1:9876,host2:5000,ctrlhost
Notice that if you need to change the cluster configuration, this method requires you to update the instance configuration file for every broker in the cluster.
For consistency and ease of maintenance, it is recommended that you collect all of the shared cluster configuration properties into a single cluster configuration file instead of setting them separately for each individual broker. In this method, each broker’s instance configuration file must set the imq.cluster.url property to point to the location of the cluster configuration file: for example,
imq.cluster.url=file:/home/cluster.properties
The cluster configuration file then defines the shared configuration properties for all of the brokers in the cluster, such as the list of brokers to be connected (imq.cluster.brokerlist), the transport protocol to use for the cluster connection service (imq.cluster.transport ), and optionally, the address of the master broker (imq.cluster.masterbroker). The following code defines the same cluster as in the previous example, with the broker running on ctrlhost serving as the master broker:
imq.cluster.brokerlist=host1:9876,host2:5000,ctrlhost imq.cluster.masterbroker=ctrlhost
This section describes how to connect a set of brokers to form a cluster, add new brokers to an existing cluster, and remove brokers from a cluster.
There are two general methods of connecting brokers into a cluster: from the command line (using the -cluster option) or by setting the imq.cluster.brokerlist property in the cluster configuration file. Whichever method you use, each broker that you start attempts to connect to the other brokers every five seconds; the connection will succeed once the master broker is started up (if one is configured). If a broker in the cluster starts before the master broker, it will remain in a suspended state, rejecting client connections, until the master broker starts; the suspended broker then will automatically become fully functional.
To configure a broker cluster from the command line, use the -cluster option to the imqbrokerd command to specify the complete list of brokers in the cluster when you start each one. For example, the following command starts a new broker and connects it to the brokers running at the default port (7676) on host1, at port 5000 on host2, and at port 9876 on the default host (localhost):
imqbrokerd -cluster host1,host2:5000,:9876
An alternative method, better suited for production systems, is to create a cluster configuration file that uses the imq.cluster.brokerlist property to specify the list of brokers to be connected. Each broker in the cluster must then set its own imq.cluster.url property to point to this cluster configuration file.
Whichever method you use, you must make sure that no broker in the cluster is given an address that resolves to the network loopback IP address (127.0.0.1). Any broker configured with this address will be unable to connect to other brokers in the cluster.
Some Linux installers automatically set the localhost entry to the network loopback address. On such systems, you must modify the system IP address so that all brokers in the cluster can be addressed properly.
For all Linux systems that participate in a cluster, check the /etc/hosts file as part of cluster setup. If the system uses a static IP address, edit the /etc/hosts file to specify the correct address for localhost. If the address is registered with Domain Name Service (DNS), edit the file /etc/nsswitch.conf to change the order of the entries so that DNS lookup is performed before consulting the local hosts file. The line in /etc/nsswitch.conf should read as follows:
hosts: dns files
If you want secure, encrypted message delivery between brokers in a cluster, configure the cluster connection service to use an SSL-based transport protocol. For each broker in the cluster, set up SSL-based connection services, as described in Message Encryption. Then set each broker’s imq.cluster.transport property to ssl, either in the cluster configuration file or individually for each broker.
The procedure for adding a new broker to a cluster depends on whether the cluster uses a cluster configuration file.
Add the new broker to the imq.cluster.brokerlist property in the cluster configuration file.
Issue the following command to any broker in the cluster:
imqcmd reload cls |
This forces each broker to reload the cluster configuration, ensuring that all persistent information for brokers in the cluster is up to date. Note that it is not necessary to issue this command to every broker in the cluster; executing it for any one broker will cause all of them to reload the cluster configuration.
(Optional) Set the value of the imq.cluster.url property in the broker’s config.properties file to point to the cluster configuration file.
Start the new broker.
If you did not perform Adding Brokers to a Cluster, use the -D option on the imqbrokerd command line to set the value of imq.cluster.url.
Set the value of the following properties, either by editing the config.properties file or by using the -D option on the imqbrokerd command line:
The method you use to remove a broker from a cluster depends on whether you originally created the cluster via the command line or by means of a central cluster configuration file.
If you used the imqbrokerd command from the command line to connect the brokers into a cluster, you must stop each of the brokers and then restart them, specifying the new set of cluster members on the command line. The procedure is as follows:
Stop each broker in the cluster, using the imqcmd command.
Restart the brokers that will remain in the cluster, using the imqbrokerd command’s -cluster option to specify only those remaining brokers.
For example, suppose you originally created a cluster consisting of brokers A, B, and C by starting each of the three with the command
imqbrokerd -cluster A,B, C |
To remove broker A from the cluster, restart brokers B and C with the command
imqbrokerd -cluster B,C |
If you originally created a cluster by specifying its member brokers with the imq.cluster.brokerlist property in a central cluster configuration file, it isn’t necessary to stop the brokers in order to remove one of them. Instead, you can simply edit the configuration file to exclude the broker you want to remove, force the remaining cluster members to reload the cluster configuration, and reconfigure the excluded broker so that it no longer points to the same cluster configuration file. Here is the procedure:
Edit the cluster configuration file to remove the excluded broker from the list specified for the imq.cluster.brokerlist property.
Issue the following command to each broker remaining in the cluster:
imqcmd reload cls |
This forces the broker to reload the cluster configuration.
Stop the broker you’re removing from the cluster.
Edit that broker’s config.properties file, removing or specifying a different value for its imq.cluster.url property.
A cluster can optionally have one master broker, which maintains a configuration change record to keep track of any changes in the cluster’s persistent state. The master broker is identified by the imq.cluster.masterbroker configuration property, either in the cluster configuration file or in the instance configuration files of the individual brokers.
The configuration change record contains information about changes in the persistent entities associated with the cluster, such as durable subscriptions and administrator-created physical destinations. All brokers in the cluster consult the master broker during startup in order to update their information about these persistent entities. Failure of the master broker makes such synchronization impossible; see When a Master Broker Is Unavailable for more information.
Because of the important information that the configuration change record contains, it is important to back it up regularly so that it can be restored in case of failure. Although restoring from a backup will lose any changes in the cluster’s persistent state that have occurred since the backup was made, frequent backups can minimize this potential loss of information. The backup and restore operations also have the positive effect of compressing and optimizing the change history contained in the configuration change record, which can grow significantly over time.
Use the -backup option of the imqbrokerd command, specifying the name of the backup file. For example:
imqbrokerd -backup mybackuplog
Shut down all brokers in the cluster.
Restore the master broker’s configuration change record from the backup file with the command
imqbrokerd -restore mybackuplog |
If you assign a new name or port number to the master broker, update the imq.cluster.brokerlist and imq.cluster.masterbroker properties accordingly in the cluster configuration file.
Restart all brokers in the cluster.
Because all brokers in a cluster need the master broker in order to perform persistent operations, the following imqcmd subcommands for any broker in the cluster will return an error when no master broker is available:
create dst
destroy dst
update dst
destroy dur
Auto-created physical destinations and temporary destinations are unaffected.
In the absence of a master broker, any client application attempting to create a durable subscriber or unsubscribe from a durable subscription will get an error. However, a client can successfully specify and interact with an existing durable subscription.
This chapter describes the tools you can use to monitor a broker and how you can get metrics data. The chapter has the following sections:
Reference information on specific metrics is available in Chapter 18, Metrics Reference
There are three monitoring interfaces for Message QueueTM information: log files, interactive commands, and a client API that can obtain metrics. Each has its advantages and disadvantages, as follows:
Log files provide a long-term record of metrics data, but cannot easily be parsed.
Commands enable you to quickly sample information tailored to your needs, but do not enable you to look at historical information or manipulate the data programmatically.
The client API lets you extract information, process it, manipulate the data, present graphs or send alerts. However, to use it, you must write a custom application to capture and analyze the data.
Table 10–1 compares the different tools.
Table 10–1 Benefits and Limitations of Metrics Monitoring Tools
Metrics Monitoring Tool |
Benefits |
Limitations |
---|---|---|
imqcmd metrics |
Remote monitoring Convenient for spot checking Reporting interval set in command option; can be changed on the fly Easy to select specific data of interest Data presented in easy tabular format |
No single command gets all data Difficult to analyze data programmatically Doesn’t create historical record Difficult to see historical trends |
Log files |
Regular sampling Creates a historical record |
Need to configure broker properties; must shut down and restart broker to take effect Local monitoring only Data format very difficult to read or parse; no parsing tools Reporting interval cannot be changed on the fly; the same for all metrics data Does not provide flexibility in selection of data Broker metrics only; destination and connection service metrics not included Possible performance hit if interval set too short |
Client API |
Remote monitoring Easy to select specific data of interest Data can be analyzed programmatically and presented in any format |
Need to configure broker properties; must shut down and restart broker to take effect You need to write your own metrics monitoring client Reporting interval cannot be changed on the fly; the same for all metrics data |
In addition to the differences shown in the table, each tool gathers a somewhat different subset of the metrics information generated by the broker. For information on which metrics data is gathered by each monitoring tool, see Chapter 18, Metrics Reference.
The Message Queue logger takes information generated by broker code, a debugger, and a metrics generator and writes that information to a number of output channels: to standard output (the console), to a log file, and, on Solaris™ operating systems, to the syslog daemon process.
You can specify the type of information gathered by the logger as well as the type written to each of the output channels. In particular, you can specify that you want metrics information written out to a log file.
This section describes the default logging configuration for the broker and explains how to redirect log information to alternative output channels, how to change log file rollover criteria, and how to send metrics data to a log file.
A broker is automatically configured to save log output to a set of rolling log files. The log files are located in a directory identified by the instance name of the associated broker (see Appendix A, Platform-Specific Locations of Message QueueTM Data):
…/instances/instanceName/log
For a broker whose life cycle is controlled by the Application Server, the log files are located in a subdirectory of the domain directory for the domain for which the broker was started:
…/appServer_domainName_dir/imq/instances/imqbroker/log
The log files are simple text files. They are named as follows, from earliest to latest:
log.txt log_1.txt log_2.txt …log_9.txt
By default, log files are rolled over once a week; the system maintains nine backup files.
To change the directory in which the log files are kept, set the property imq.log.file.dirpath to the desired path.
To change the root name of the log files from log to something else, set the imq.log.file.filename property.
The broker supports three log levels: ERROR, WARNING , INFO. Table 10–2 explains each level.
Table 10–2 Logging Levels
Level |
Description |
---|---|
ERROR |
Messages indicating problems that could cause system failure. |
WARNING |
Alerts that should be heeded but will not cause system failure. |
INFO |
Reporting of metrics and other informational messages. |
Setting a logging level gathers messages for that level and all higher levels. The default log level is INFO, so ERROR, WARNING, and INFO messages are all logged by default.
A logged message consists of a time stamp, message code, and the message itself. The volume of information varies with the log level you have set. The following is an example of an INFO message.
[13/Sep/2000:16:13:36 PDT] [B1004]: Starting the broker service using tcp [25374,100] with min threads 50 and max threads of 500 |
To change the time stamp time zone, see information about the imq.log.timezone property, which is described in Table 14–8.
Log-related properties are described in Table 14–8.
Set the log level.
Set the output channel (file, console, or both) for one or more logging categories.
If you log output to a file, configure the rollover criteria for the file.
You complete these steps by setting logger properties. You can do this in one of two ways:
Change or add logger properties in the config.properties file for a broker before you start the broker.
Specify logger command line options in the imqbrokerd command that starts the broker. You can also use the broker option -D to change logger properties (or any broker property).
Options passed on the command line override properties specified in the broker instance configuration files. The following imqbrokerd options affect logging:
Logging interval for broker metrics, in seconds
Logging level (ERROR, WARNING, INFO, or NONE)
Silent mode (no logging to console)
Log all messages to console
The following sections describe how you can change the default configuration in order to do the following:
Change the output channel (the destination of log messages)
Change rollover criteria
By default, error and warning messages are displayed on the terminal as well as being logged to a log file. (On Solaris, error messages are also written to the system’s syslog daemon.)
You can change the output channel for log messages in the following ways:
To have all log categories (for a given level) output displayed on the screen, use the -tty option to the imqbrokerd command.
To prevent log output from being displayed on the screen, use the -silent option to the imqbrokerd command.
Use the imq.log.file.output property to specify which categories of logging information should be written to the log file. For example,
imq.log.file.output=ERROR
Use the imq.log.console.output property to specify which categories of logging information should be written to the console. For example,
imq.log.console.output=INFO
On Solaris, use the imq.log.syslog.output property to specify which categories of logging information should be written to Solaris syslog. For example,
imq.log.syslog.output=NONE
Before changing logger output channels, you must make sure that logging is set at a level that supports the information you are mapping to the output channel. For example, if you set the log level to ERROR and then set the imq.log.console.output property to WARNING, no messages will be logged because you have not enabled the logging of WARNING messages.
There are two criteria for rolling over log files: time and size. The default is to use a time criteria and roll over files every seven days.
To change the time interval, you need to change the property imq.log.file.rolloversecs. For example, the following property definition changes the time interval to ten days:
imq.log.file.rolloversecs=864000
To change the rollover criteria to depend on file size, you need to set the imq.log.file.rolloverbytes property. For example, the following definition directs the broker to rollover files after they reach a limit of 500,000 bytes
imq.log.file.rolloverbytes=500000
If you set both the time-related and the size-related rollover properties, the first limit reached will trigger the rollover. As noted before, the broker maintains up to nine rollover files.
You can set or change the log file rollover properties when a broker is running. To set these properties, use the imqcmd update bkr command.
This section describes the procedure for using broker log files to report metrics information. For general information on configuring the logger, see Configuring and Using Broker Logging.
Configure the broker’s metrics generation capability:
Confirm imq.metrics.enabled=true
Generation of metrics for logging is turned on by default.
Set the metrics generation interval to a convenient number of seconds.
imq.metrics.interval=interval
This value can be set in the config.properties file or using the -metrics interval command line option when starting up the broker.
Confirm that the logger gathers metrics information:
imq.log.level=INFO |
This is the default value. This value can be set in the config.properties file or using the -loglevel level command line option when starting up the broker.
Confirm that the logger is set to write metrics information to the log file:
imq.log.file.output=INFO |
This is the default value. It can be set in the config.properties file.
Start up the broker.
The following shows sample broker metrics output to the log file:
[21/Jul/2004:11:21:18 PDT] Connections: 0 JVM Heap: 8323072 bytes (7226576 free) Threads: 0 (14-1010) In: 0 msgs (0bytes) 0 pkts (0 bytes) Out: 0 msgs (0bytes) 0 pkts (0 bytes) Rate In: 0 msgs/sec (0 bytes/sec) 0 pkts/sec (0 bytes/sec) Rate Out: 0 msgs/sec (0 bytes/sec) 0 pkts/sec (0 bytes/sec) |
For reference information about metrics data, see Chapter 18, Metrics Reference
You can monitor physical destinations by enabling dead message logging for a broker. You can log dead messages whether or not you are using a dead message queue.
If you enable dead message logging, the broker logs the following types of events:
A physical destination exceeded its maximum size.
The broker removed a message from a physical destination, for a reason such as the following:
The destination size limit has been reached.
The message time to live expired.
The message is too large.
An error occurred when the broker attempted to process the message.
If a dead message queue is in use, logging also includes the following types of events:
The broker moved a message to the dead message queue.
The broker removed a message from the dead message queue and discarded it.
The following is an example of the log format for dead messages:
[29/Mar/2006:15:35:39 PST] [B1147]: Message 8-129.145.180.87(e7:6b:dd:5d:98:aa)- 35251-1143675279400 from destination Q:q0 has been placed on the DMQ because [B0053]: Message on destination Q:q0 Expired: expiration time 1143675279402, arrival time 1143675279401, JMSTimestamp 1143675279400 |
Dead message logging is disabled by default. To enable it, set the broker attribute imq.destination.logDeadMsgs.
A Message Queue broker can report the following types of metrics:
Java Virtual Machine (JVM) metrics. Information about the JVM heap size.
Brokerwide metrics. Information about messages stored in a broker, message flows into and out of a broker, and memory use. Messages are tracked in terms of numbers of messages and numbers of bytes.
Connection Service metrics. Information about connections and connection thread resources, and information about message flows for a particular connection service.
Destination metrics. Information about message flows into and out of a particular physical destination, information about a physical destination’s consumers, and information about memory and disk space usage.
The imqcmd command can obtain metrics information for the broker as a whole, for individual connection services, and for individual physical destinations. To obtain metrics data, you generally use the metrics subcommand of imqcmd. Metrics data is written at an interval you specify, or the number of times you specify, to the console screen.
You can also use the query subcommand to view similar data that also includes configuration information. See imqcmd query for more information.
The syntax and options of imqcmd metrics are shown in Table 10–3 and Table 10–4, respectively.
Table 10–3 imqcmd metrics Subcommand Syntax
Subcommand Syntax |
Metrics Data Provided |
---|---|
metrics bkr [-b hostName:portNumber] [-m metricType] [-int interval] [-msp numSamples] |
Displays broker metrics for the default broker or a broker at the specified host and port. |
metrics svc -n serviceName [-b hostName:portNumber] [-m metricType] [-int interval] [-msp numSamples] |
Displays metrics for the specified service on the default broker or on a broker at the specified host and port. |
metrics dst -t destType -n destName [-b hostName:portNumber] [-m metricType] [-int interval] [-msp numSamples] |
Displays metrics information for the physical destination of the specified type and name. |
Table 10–4 imqcmd metrics Subcommand Options
Subcommand Options |
Description |
---|---|
-b hostName: portNumber |
Specifies the hostname and port of the broker for which metrics data is reported. The default is localhost:7676. |
-int interval |
Specifies the interval (in seconds) at which to display the metrics. The default is 5 seconds. |
-m metricType |
Specifies the type of metric to display: ttl Displays metrics on messages and packets flowing into and out of the broker, service, or destination (default metric type). rts Displays metrics on rate of flow of messages and packets into and out of the broker, connection service, or destination (per second). cxn Displays connections, virtual memory heap, and threads (brokers and connection services only). con Displays consumer-related metrics (destinations only). dsk Displays disk usage metrics (destinations only). |
-msp numSamples |
Specifies the number of samples displayed in the output. The default is an unlimited number (infinite). |
-n destName |
Specifies the name of the physical destination (if any) for which metrics data is reported. There is no default. |
-n serviceName |
Specifies the connection service (if any) for which metrics data is reported. There is no default. |
-t destType |
Specifies the type (queue or topic) of the physical destination (if any) for which metrics data is reported. There is no default. |
This section describes the procedure for using the metrics subcommand to report metrics information.
Start the broker for which metrics information is desired.
See Starting Brokers.
Issue the appropriate imqcmd metrics subcommand and options as shown in Table 10–3 and Table 10–4.
This section contains examples of output for the imqcmd metrics subcommand. The examples show brokerwide, connection service, and physical destination metrics.
To get the rate of message and packet flow into and out of the broker at 10 second intervals, use the metrics bkr subcommand:
imqcmd metrics bkr -m rts -int 10 -u admin
This command produces output similar to the following (see data descriptions in Table 18–2):
-------------------------------------------------------- Msgs/sec Msg Bytes/sec Pkts/sec Pkt Bytes/sec In Out In Out In Out In Out -------------------------------------------------------- 0 0 27 56 0 0 38 66 10 0 7365 56 10 10 7457 1132 0 0 27 56 0 0 38 73 0 10 27 7402 10 20 1400 8459 0 0 27 56 0 0 38 73 |
To get cumulative totals for messages and packets handled by the jms connection service, use the metrics svc subcommand:
imqcmd metrics svc -n jms -m ttl -u admin
This command produces output similar to the following (see data descriptions in Table 18–3):
------------------------------------------------- Msgs Msg Bytes Pkts Pkt Bytes In Out In Out In Out In Out ------------------------------------------------- 164 100 120704 73600 282 383 135967 102127 657 100 483552 73600 775 876 498815 149948 |
To get metrics information about a physical destination, use the metrics dst subcommand:
imqcmd metrics dst -t q -n XQueue -m ttl -u admin
This command produces output similar to the following (see data descriptions in Table 18–4):
----------------------------------------------------------------------------- Msgs Msg Bytes Msg Count Total Msg Bytes (k) Largest In Out In Out Current Peak Avg Current Peak Avg Msg (k) ----------------------------------------------------------------------------- 200 200 147200 147200 0 200 0 0 143 71 0 300 200 220800 147200 100 200 10 71 143 64 0 300 300 220800 220800 0 200 0 0 143 59 0 |
To get information about a physical destination’s consumers, use the following metrics dst subcommand:
imqcmd metrics dst -t q -n SimpleQueue -m con -u admin
This command produces output similar to the following (see data descriptions in Table 18–4):
------------------------------------------------------------------ Active Consumers Backup Consumers Msg Count Current Peak Avg Current Peak Avg Current Peak Avg ------------------------------------------------------------------ 1 1 0 0 0 0 944 1000 525 |
The syntax and options of imqcmd query are shown in Table 10–5 along with a description of the metrics data provided by the command.
Table 10–5 imqcmd query Subcommand Syntax
Subcommand Syntax |
Metrics Data Provided |
|
---|---|---|
|
Information on the current number of messages and message bytes stored in broker memory and persistent store (see Displaying Broker Information). |
|
or | ||
|
Information on the current number of allocated threads and number of connections for a specified connection service (see Displaying Connection Service Information). |
|
or | ||
|
Information on the current number of producers, active and backup consumers, and messages and message bytes stored in memory and persistent store for a specified destination (see Displaying Information about Physical Destinations). |
Because of the limited metrics data provided by imqcmd query , this tool is not represented in the tables presented in Chapter 18, Metrics Reference.
Message Queue provides a metrics monitoring capability by which the broker can write metrics data into JMS messages, which it then sends to one of a number of metrics topic destinations, depending on the type of metrics information contained in the message.
You can access this metrics information by writing a client application that subscribes to the metrics topic destinations, consumes the messages in these destinations, and processes the metrics information contained in the messages.
There are five metrics topic destinations, whose names are shown in Table 10–6, along with the type of metrics messages delivered to each destination.
Table 10–6 Metrics Topic Destinations
Topic Name | |
---|---|
mq.metrics.broker | |
mq.metrics.jvm | |
mq.metrics.destination_list | |
mq.metrics.destination.queue.monitoredDestinationName |
Destination metrics for queue of specified name |
mq.metrics.destination.topic.monitoredDestinationName |
Destination metrics for topic of specified name |
This section describes the procedure for using the message-based monitoring capability to gather metrics information. The procedure includes both client development and administration tasks.
Write a metrics monitoring client.
See the Message Queue Developer's Guide for Java Clients for instructions on programming clients that subscribe to metrics topic destinations, consume metrics messages, and extract the metrics data from these messages.
Configure the broker’s Metrics Message Producer by setting broker property values in the config.properties file:
Enable metrics message production.
Set imq.metrics.topic.enabled=true
The default value is true.
Set the interval (in seconds) at which metrics messages are generated.
Set imq.metrics.topic.interval=interval .
The default is 60 seconds.
Specify whether you want metrics messages to be persistent (that is, whether they will survive a broker failure).
Set imq.metrics.topic.persist .
The default is false.
Specify how long you want metrics messages to remain in their respective destinations before being deleted.
Set imq.metrics.topic.timetolive .
The default value is 300 seconds.
Set any access control you desire on metrics topic destinations.
See the discussion in Security and Access Considerations below.
Start up your metrics monitoring client.
When consumers subscribe to a metrics topic, the metrics topic destination will automatically be created. Once a metrics topic has been created, the broker’s metrics message producer will begin sending metrics messages to the metrics topic.
There are two reasons to restrict access to metrics topic destinations:
Metrics data might include sensitive information about a broker and its resources.
Excessive numbers of subscriptions to metrics topic destinations might increase broker overhead and negatively affect performance.
Because of these considerations, it is advisable to restrict access to metrics topic destinations.
Monitoring clients are subject to the same authentication and authorization control as any other client. Only users maintained in the Message Queue user repository are allowed to connect to the broker.
You can provide additional protections by restricting access to specific metrics topic destinations through an access control properties file, as described in User Authorization: The Access Control Properties File.
For example, the following entries in an accesscontrol.properties file will deny access to the mq.metrics.broker metrics topic to everyone except user1 and user 2.
topic.mq.metrics.broker.consume.deny.user=* topic.mq.metrics.broker.consume.allow.user=user1,user2 |
The following entries will only allow users user3 to monitor topic t1.
topic.mq.metrics.destination.topic.t1.consume.deny.user=* topic.mq.metrics.destination.topic.t1.consume.allow.user=user3 |
Depending on the sensitivity of metrics data, you can also connect your metrics monitoring client to a broker using an encrypted connection. For information on using encrypted connections, see Message Encryption.
The metrics data outputs you get using the message-based monitoring API is a function of the metrics monitoring client you write. You are limited only by the data provided by the metrics generator in the broker. For a complete list of this data, see Chapter 18, Metrics Reference.
This chapter covers a number of topics about how to analyze and tune a Message QueueTM service to optimize the performance of your messaging applications. It includes the following topics:
This section provides some background information on performance tuning.
The performance you get out of a messaging application depends on the interaction between the application and the Message Queue service. Hence, maximizing performance requires the combined efforts of both the application developer and the administrator.
The process of optimizing performance begins with application design and continues through to tuning the message service after the application has been deployed. The performance tuning process includes the following stages:
Defining performance requirements for the application
Designing the application taking into account factors that affect performance (especially tradeoffs between reliability and performance)
Establishing baseline performance measures
Tuning or reconfiguring the message service to optimize performance
The process outlined above is often iterative. During deployment of the application, a Message Queue administrator evaluates the suitability of the message service for the application’s general performance requirements. If the benchmark testing meets these requirements, the administrator can tune the system as described in this chapter. However, if benchmark testing does not meet performance requirements, a redesign of the application might be necessary or the deployment architecture might need to be modified.
In general, performance is a measure of the speed and efficiency with which a message service delivers messages from producer to consumer. However, there are several different aspects of performance that might be important to you, depending on your needs.
The number of message producers, or message consumers, or the number of concurrent connections a system can support.
The number of messages or message bytes that can be pumped through a messaging system per second.
The time it takes a particular message to be delivered from message producer to message consumer.
The overall availability of the message service or how gracefully it degrades in cases of heavy load or failure.
The efficiency of message delivery; a measure of message throughput in relation to the computing resources employed.
These different aspects of performance are generally interrelated. If message throughput is high, that means messages are less likely to be backlogged in the broker, and as a result, latency should be low (a single message can be delivered very quickly). However, latency can depend on many factors: the speed of communication links, broker processing speed, and client processing speed, to name a few.
In any case, there are several different aspects of performance. Which of them are most important to you generally depends on the requirements of a particular application.
Benchmarking is the process of creating a test suite for your messaging application and of measuring message throughput or other aspects of performance for this test suite.
For example, you could create a test suite by which some number of producing clients, using some number of connections, sessions, and message producers, send persistent or nonpersistent messages of a standard size to some number of queues or topics (all depending on your messaging application design) at some specified rate. Similarly, the test suite includes some number of consuming clients, using some number of connections, sessions, and message consumers (of a particular type) that consume the messages in the test suite’s physical destinations using a particular acknowledgment mode.
Using your standard test suite you can measure the time it takes between production and consumption of messages or the average message throughput rate, and you can monitor the system to observe connection thread usage, message storage data, message flow data, and other relevant metrics. You can then ramp up the rate of message production, or the number of message producers, or other variables, until performance is negatively affected. The maximum throughput you can achieve is a benchmark for your message service configuration.
Using this benchmark, you can modify some of the characteristics of your test suite. By carefully controlling all the factors that might have an effect on performance (see Application Design Factors Affecting Performance), you can note how changing some of these factors affects the benchmark. For example, you can increase the number of connections or the size of messages five-fold or ten-fold, and note the effect on performance.
Conversely, you can keep application-based factors constant and change your broker configuration in some controlled way (for example, change connection properties, thread pool properties, JVM memory limits, limit behaviors, file-based versus JDBC-based persistence, and so forth) and note how these changes affect performance.
This benchmarking of your application provides information that can be valuable when you want to increase the performance of a deployed application by tuning your message service. A benchmark allows the effect of a change or a set of changes to be more accurately predicted.
As a general rule, benchmarks should be run in a controlled test environment and for a long enough period of time for your message service to stabilize. (Performance is negatively affected at startup by the just-in-time compilation that turns Java code into machine code.)
Once a messaging application is deployed and running, it is important to establish baseline use patterns. You want to know when peak demand occurs and you want to be able to quantify that demand. For example, demand normally fluctuates by number of end users, activity levels, time of day, or all of these.
To establish baseline use patterns you need to monitor your message service over an extended period of time, looking at data such as the following:
Number of connections
Number of messages stored in the broker (or in particular physical destinations)
Message flows into and out of a broker (or particular physical destinations)
Numbers of active consumers
You can also use average and peak values provided in metrics data.
It is important to check these baseline metrics against design expectations. By doing so, you are checking that client code is behaving properly: for example, that connections are not being left open or that consumed messages are not being left unacknowledged. These coding errors consume broker resources and could significantly affect performance.
The base-line use patterns help you determine how to tune your system for optimal performance. For example:
If one physical destination is used significantly more than others, you might want to set higher message memory limits on that physical destination than on others, or to adjust limit behaviors accordingly.
If the number of connections needed is significantly greater than allowed by the maximum thread pool size, you might want to increase the thread pool size or adopt a shared thread model.
If peak message flows are substantially greater than average flows, that might influence the limit behaviors you employ when memory runs low.
In general, the more you know about use patterns, the better you are able to tune your system to those patterns and to plan for future needs.
Message latency and message throughput, two of the main performance indicators, generally depend on the time it takes a typical message to complete various steps in the message delivery process. These steps are shown below for the case of a persistent, reliably delivered message. The steps are described following the illustration.
The message is delivered from producing client to broker.
The broker reads in the message.
The message is placed in persistent storage (for reliability).
The broker confirms receipt of the message (for reliability).
The broker determines the routing for the message.
The broker writes out the message.
The message is delivered from broker to consuming client.
The consuming client acknowledges receipt of the message (for reliability).
The broker processes client acknowledgment (for reliability).
The broker confirms that client acknowledgment has been processed.
Since these steps are sequential, any one of them can be a potential bottleneck in the delivery of messages from producing clients to consuming clients. Most of the steps depend on physical characteristics of the messaging system: network bandwidth, computer processing speeds, message service architecture, and so forth. Some, however, also depend on characteristics of the messaging application and the level of reliability it requires.
The following subsections discuss the effect of both application design factors and messaging system factors on performance. While application design and messaging system factors closely interact in the delivery of messages, each category is considered separately.
Application design decisions can have a significant effect on overall messaging performance.
The most important factors affecting performance are those that affect the reliability of message delivery. Among these are the following:
Other application design factors affecting performance are the following:
The sections that follow describe the effect of each of these factors on messaging performance. As a general rule, there is a tradeoff between performance and reliability: factors that increase reliability tend to decrease performance.
Table 11–1 shows how the various application design factors generally affect messaging performance. The table shows two scenarios—one high-reliability, low-performance, and one high-performance, low-reliability—and the choices of application design factors that characterize each. Between these extremes, there are many choices and tradeoffs that affect both reliability and performance.
Table 11–1 Comparison of High-Reliability and High-Performance Scenarios
Application DesignFactor |
High-Reliability, Low-Performance Scenario |
High-Performance, Low-Reliability Scenario |
---|---|---|
Delivery mode |
Persistent messages |
Nonpersistent messages |
Use of transactions |
Transacted sessions |
No transactions |
Acknowledgment mode |
AUTO_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE |
DUPS_OK_ACKNOWLEDGE |
Durable/nondurable subscriptions |
Durable subscriptions |
Nondurable subscriptions |
Use of selectors |
Message filtering |
No message filtering |
Message size |
Large number of small messages |
Small number of large messages |
Message body type |
Complex body types |
Simple body types |
Persistent messages guarantee message delivery in case of broker failure. The broker stores the message in a persistent store until all intended consumers acknowledge they have consumed the message.
Broker processing of persistent messages is slower than for nonpersistent messages for the following reasons:
A broker must reliably store a persistent message so that it will not be lost should the broker fail.
The broker must confirm receipt of each persistent message it receives. Delivery to the broker is guaranteed once the method producing the message returns without an exception.
Depending on the client acknowledgment mode, the broker might need to confirm a consuming client’s acknowledgment of a persistent message.
For both queues and topics with durable subscribers, performance was approximately 40% faster for nonpersistent messages. We obtained these results using 10k-sized messages and AUTO_ACKNOWLEDGE mode
A transaction is a guarantee that all messages produced in a transacted session and all messages consumed in a transacted session will be either processed or not processed (rolled back) as a unit.
Message Queue supports both local and distributed transactions.
A message produced or acknowledged in a transacted session is slower than in a nontransacted session for the following reasons:
Additional information must be stored with each produced message.
In some situations, messages in a transaction are stored when normally they would not be (for example, a persistent message delivered to a topic destination with no subscriptions would normally be deleted, however, at the time the transaction is begun, information about subscriptions is not available).
Information on the consumption and acknowledgment of messages within a transaction must be stored and processed when the transaction is committed.
One mechanism for ensuring the reliability of JMS message delivery is for a client to acknowledge consumption of messages delivered to it by the Message Queue broker.
If a session is closed without the client acknowledging the message or if the broker fails before the acknowledgment is processed, the broker redelivers that message, setting a JMSRedelivered flag.
For a nontransacted session, the client can choose one of three acknowledgment modes, each of which has its own performance characteristics:
AUTO_ACKNOWLEDGE. The system automatically acknowledges a message once the consumer has processed it. This mode guarantees at most one redelivered message after a provider failure.
CLIENT_ACKNOWLEDGE. The application controls the point at which messages are acknowledged. All messages processed in that session since the previous acknowledgment are acknowledged. If the broker fails while processing a set of acknowledgments, one or more messages in that group might be redelivered.
DUPS_OK_ACKNOWLEDGE. This mode instructs the system to acknowledge messages in a lazy manner. Multiple messages can be redelivered after a provider failure.
(Using CLIENT_ACKNOWLEDGE mode is similar to using transactions, except there is no guarantee that all acknowledgments will be processed together if a provider fails during processing.)
Acknowledgment mode affects performance for the following reasons:
Extra control messages between broker and client are required in AUTO_ACKNOWLEDGE and CLIENT_ACKNOWLEDGE modes. The additional control messages add additional processing overhead and can interfere with JMS payload messages, causing processing delays.
In AUTO_ACKNOWLEDGE and CLIENT_ACKNOWLEDGE modes, the client must wait until the broker confirms that it has processed the client’s acknowledgment before the client can consume additional messages. (This broker confirmation guarantees that the broker will not inadvertently redeliver these messages.)
The Message Queue persistent store must be updated with the acknowledgment information for all persistent messages received by consumers, thereby decreasing performance.
Subscribers to a topic destination fall into two categories, those with durable and nondurable subscriptions.
Durable subscriptions provide increased reliability but slower throughput, for the following reasons:
The Message Queue message service must persistently store the list of messages assigned to each durable subscription so that should a broker fail, the list is available after recovery.
Persistent messages for durable subscriptions are stored persistently, so that should a broker fail, the messages can still be delivered after recovery, when the corresponding consumer becomes active. By contrast, persistent messages for nondurable subscriptions are not stored persistently (should a broker fail, the corresponding consumer connection is lost and the message would never be delivered).
We compared performance for durable and nondurable subscribers in two cases: persistent and nonpersistent 10k-sized messages. Both cases use AUTO_ACKNOWLEDGE acknowledgment mode. We found an effect on performance only in the case of persistent messages which slowed durables by about 30%
Application developers often want to target sets of messages to particular consumers. They can do so either by targeting each set of messages to a unique physical destination or by using a single physical destination and registering one or more selectors for each consumer.
A selector is a string requesting that only messages with property values that match the string are delivered to a particular consumer. For example, the selector NumberOfOrders >1 delivers only the messages with a NumberOfOrders property value of 2 or more.
Creating consumers with selectors lowers performance (as compared to using multiple physical destinations) because additional processing is required to handle each message. When a selector is used, it must be parsed so that it can be matched against future messages. Additionally, the message properties of each message must be retrieved and compared against the selector as each message is routed. However, using selectors provides more flexibility in a messaging application.
Message size affects performance because more data must be passed from producing client to broker and from broker to consuming client, and because for persistent messages a larger message must be stored.
However, by batching smaller messages into a single message, the routing and processing of individual messages can be minimized, providing an overall performance gain. In this case, information about the state of individual messages is lost.
In our tests, which compared throughput in kilobytes per second for 1k, 10k, and 100k-sized messages to a queue destination and AUTO_ACKNOWLEDGE acknowledgment mode, we found that nonpersistent messaging was about 50% faster for 1k messages, about 20% faster for 10k messages, and about 5% faster for 100k messages. The size of the message affected performance significantly for both persistent and nonpersistent messages. 100k messages are about 10 times faster than 10k, and 10k are about 5 times faster than 1k.
JMS supports five message body types, shown below roughly in the order of complexity:
BytesMessage contains a set of bytes in a format determined by the application.
TextMessage is a simple Java string.
StreamMessage contains a stream of Java primitive values.
MapMessage contains a set of name-value pairs.
ObjectMessage contains a Java serialized object.
While, in general, the message type is dictated by the needs of an application, the more complicated types (MapMessage and ObjectMessage) carry a performance cost: the expense of serializing and deserializing the data. The performance cost depends on how simple or how complicated the data is.
The performance of a messaging application is affected not only by application design, but also by the message service performing the routing and delivery of messages.
The following sections discuss various message service factors that can affect performance. Understanding the effect of these factors is key to sizing a message service and diagnosing and resolving performance bottlenecks that might arise in a deployed application.
The most important factors affecting performance in a Message Queue service are the following:
The sections below describe the effect of each of these factors on messaging performance.
For both the Message Queue broker and client applications, CPU processing speed and available memory are primary determinants of message service performance. Many software limitations can be eliminated by increasing processing power, while adding memory can increase both processing speed and capacity. However, it is generally expensive to overcome bottlenecks simply by upgrading your hardware.
Because of the efficiencies of different operating systems, performance can vary, even assuming the same hardware platform. For example, the thread model employed by the operating system can have an important effect on the number of concurrent connections a broker can support. In general, all hardware being equal, Solaris is generally faster than Linux, which is generally faster than Windows.
The broker is a Java process that runs in and is supported by the host JVM. As a result, JVM processing is an important determinant of how fast and efficiently a broker can route and deliver messages.
In particular, the JVM’s management of memory resources can be critical. Sufficient memory has to be allocated to the JVM to accommodate increasing memory loads. In addition, the JVM periodically reclaims unused memory, and this memory reclamation can delay message processing. The larger the JVM memory heap, the longer the potential delay that might be experienced during memory reclamation.
The number and speed of connections between client and broker can affect the number of messages that a message service can handle as well as the speed of message delivery.
All access to the broker is by way of connections. Any limit on the number of concurrent connections can affect the number of producing or consuming clients that can concurrently use the broker.
The number of connections to a broker is generally limited by the number of threads available. Message Queue can be configured to support either a dedicated thread model or a shared thread model (see Thread Pool Management).
The dedicated thread model is very fast because each connection has dedicated threads, however the number of connections is limited by the number of threads available (one input thread and one output thread for each connection). The shared thread model places no limit on the number of connections, however there is significant overhead and throughput delays in sharing threads among a number of connections, especially when those connections are busy.
Message Queue software allows clients to communicate with the broker using various low-level transport protocols. Message Queue supports the connection services (and corresponding protocols) described in Connection Services.
The choice of protocols is based on application requirements (encrypted, accessible through a firewall), but the choice affects overall performance.
Our tests compared throughput for TCP and SSL for two cases: a high-reliability scenario (1k persistent messages sent to topic destinations with durable subscriptions and using AUTO_ACKNOWLEDGE acknowledgment mode) and a high-performance scenario (1k nonpersistent messages sent to topic destinations without durable subscriptions and using DUPS_OK_ACKNOWLEDGE acknowledgment mode).
In general we found that protocol has less effect in the high-reliability case. This is probably because the persistence overhead required in the high-reliability case is a more important factor in limiting throughput than the protocol speed. Additionally:
TCP provides the fastest method to communicate with the broker.
SSL is 50 to 70 percent slower than TCP when it comes to sending and receiving messages (50 percent for persistent messages, closer to 70 percent for nonpersistent messages). Additionally, establishing the initial connection is slower with SSL (it might take several seconds) because the client and broker (or Web Server in the case of HTTPS) need to establish a private key to be used when encrypting the data for transmission. The performance drop is caused by the additional processing required to encrypt and decrypt each low-level TCP packet.
HTTP is slower than either the TCP or SSL. It uses a servlet that runs on a Web server as a proxy between the client and the broker. Performance overhead is involved in encapsulating packets in HTTP requests and in the requirement that messages go through two hops--client to servlet, servlet to broker--to reach the broker.
HTTPS is slower than HTTP because of the additional overhead required to encrypt the packet between client and servlet and between servlet and broker.
A Message Queue message service can be implemented as a single broker or as a cluster consisting of multiple interconnected broker instances.
As the number of clients connected to a broker increases, and as the number of messages being delivered increases, a broker will eventually exceed resource limitations such as file descriptor, thread, and memory limits. One way to accommodate increasing loads is to add more broker instances to a Message Queue message service, distributing client connections and message routing and delivery across multiple brokers.
In general, this scaling works best if clients are evenly distributed across the cluster, especially message producing clients. Because of the overhead involved in delivering messages between the brokers in a cluster, clusters with limited numbers of connections or limited message delivery rates, might exhibit lower performance than a single broker.
You might also use a broker cluster to optimize network bandwidth. For example, you might want to use slower, long distance network links between a set of remote brokers within a cluster, while using higher speed links for connecting clients to their respective broker instances.
For more information on clusters, see Chapter 9, Working With Broker Clusters
The message throughput that a broker might be required to handle is a function of the use patterns of the messaging applications the broker supports. However, the broker is limited in resources: memory, CPU cycles, and so forth. As a result, it would be possible for a broker to become overwhelmed to the point where it becomes unresponsive or unstable.
The Message Queue message broker has mechanisms built in for managing memory resources and preventing the broker from running out of memory. These mechanisms include configurable limits on the number of messages or message bytes that can be held by a broker or its individual physical destinations, and a set of behaviors that can be instituted when physical destination limits are reached.
With careful monitoring and tuning, these configurable mechanisms can be used to balance the inflow and outflow of messages so that system overload cannot occur. While these mechanisms consume overhead and can limit message throughput, they nevertheless maintain operational integrity.
Message Queue supports both file-based and JDBC-based persistence modules. File-based persistence uses individual files to store persistent data. JDBC-based persistence uses a Java Database Connectivity (JDBC™) interface and requires a JDBC-compliant data store. File-based persistence is generally faster than JDBC-based; however, some users prefer the redundancy and administrative control provided by a JDBC-compliant store.
In the case of file-based persistence, you can maximize reliability by specifying that persistence operations synchronize the in-memory state with the data store. This helps eliminate data loss due to system crashes, but at the expense of performance.
The Message Queue client runtime provides client applications with an interface to the Message Queue message service. It supports all the operations needed for clients to send messages to physical destinations and to receive messages from such destinations. The client runtime is configurable (by setting connection factory attribute values), allowing you to control aspects of its behavior, such as connection flow metering, consumer flow limits, and connection flow limits, that can improve performance and message throughput. See Client Runtime Message Flow Adjustments for more information on these features and the attributes used to configure them.
The following sections explain how configuration adjustments can affect performance.
The following sections describe adjustments you can make to the operating system, JVM, and communication protocols.
See your system documentation for tuning your operating system.
By default, the broker uses a JVM heap size of 192MB. This is often too small for significant message loads and should be increased.
When the broker gets close to exhausting the JVM heap space used by Java objects, it uses various techniques such as flow control and message swapping to free memory. Under extreme circumstances it even closes client connections in order to free the memory and reduce the message inflow. Hence it is desirable to set the maximum JVM heap space high enough to avoid such circumstances.
However, if the maximum Java heap space is set too high, in relation to system physical memory, the broker can continue to grow the Java heap space until the entire system runs out of memory. This can result in diminished performance, unpredictable broker crashes, and/or affect the behavior of other applications and services running on the system. In general, you need to allow enough physical memory for the operating system and other applications to run on the machine.
In general it is a good idea to evaluate the normal and peak system memory footprints, and configure the Java heap size so that it is large enough to provide good performance, but not so large as to risk system memory problems.
To change the minimum and maximum heap size for the broker, use the -vmargs command line option when starting the broker. For example:
/usr/bin/imqbrokerd -vmargs "-Xms256m -Xmx1024m"
This command will set the starting Java heap size to 256MB and the maximum Java heap size to 1GB.
On Solaris or Linux, if starting the broker via /etc/rc* (that is, /etc/init.d/imq), specify broker command line arguments in the file /etc/imq/imqbrokerd.conf (Solaris) or /etc/opt/sun/mq/imqbrokerd.conf (Linux). See the comments in that file for more information.
On Windows, if starting the broker as a Window’s service, specify JVM arguments using the -vmargs option to the imqsvcadmin install command. See Service Administrator Utility in Chapter 13, Command Line Reference
In any case, verify settings by checking the broker’s log file or using the imqcmd metrics bkr -m cxn command.
Once a protocol that meets application needs has been chosen, additional tuning (based on the selected protocol) might improve performance.
A protocol’s performance can be modified using the following three broker properties:
For TCP and SSL protocols, these properties affect the speed of message delivery between client and broker. For HTTP and HTTPS protocols, these properties affect the speed of message delivery between the Message Queue tunnel servlet (running on a Web server) and the broker. For HTTP/HTTPS protocols there are additional properties that can affect performance (see HTTP/HTTPS Tuning).
The protocol tuning properties are described in the following sections.
The nodelay property affects Nagle’s algorithm (the value of the TCP_NODELAY socket-level option on TCP/IP) for the given protocol. Nagle’s algorithm is used to improve TCP performance on systems using slow connections such as wide-area networks (WANs).
When the algorithm is used, TCP tries to prevent several small chunks of data from being sent to the remote system (by bundling the data in larger packets). If the data written to the socket does not fill the required buffer size, the protocol delays sending the packet until either the buffer is filled or a specific delay time has elapsed. Once the buffer is full or the timeout has occurred, the packet is sent.
For most messaging applications, performance is best if there is no delay in the sending of packets (Nagle’s algorithm is not enabled). This is because most interactions between client and broker are request/response interactions: the client sends a packet of data to the broker and waits for a response. For example, typical interactions include:
Creating a connection
Creating a producer or consumer
Sending a persistent message (the broker confirms receipt of the message)
Sending a client acknowledgment in an AUTO_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE session (the broker confirms processing of the acknowledgment)
For these interactions, most packets are smaller than the buffer size. This means that if Nagle’s algorithm is used, the broker delays several milliseconds before sending a response to the consumer.
However, Nagle’s algorithm may improve performance in situations where connections are slow and broker responses are not required. This would be the case where a client sends a nonpersistent message or where a client acknowledgment is not confirmed by the broker (DUPS_OK_ACKNOWLEDGE session).
The inbufsz property sets the size of the buffer on the input stream reading data coming in from a socket. Similarly, outbufsz sets the buffer size of the output stream used by the broker to write data to the socket.
In general, both parameters should be set to values that are slightly larger than the average packet being received or sent. A good rule of thumb is to set these property values to the size of the average packet plus 1 kilobyte (rounded to the nearest kilobyte). For example, if the broker is receiving packets with a body size of 1 kilobyte, the overall size of the packet (message body plus header plus properties) is about 1200 bytes; an inbufsz of 2 kilobytes (2048 bytes) gives reasonable performance. Increasing inbufsz or outbufsz greater than that size may improve performance slightly, but increases the memory needed for each connection.
In addition to the general properties discussed in the previous two sections, HTTP/HTTPS performance is limited by how fast a client can make HTTP requests to the Web server hosting the Message Queue tunnel servlet.
A Web server might need to be optimized to handle multiple requests on a single socket. With JDK version 1.4 and later, HTTP connections to a Web server are kept alive (the socket to the Web server remains open) to minimize resources used by the Web server when it processes multiple HTTP requests. If the performance of a client application using JDK version 1.4 is slower than the same application running with an earlier JDK release, you might need to tune the Web server keep-alive configuration parameters to improve performance.
In addition to such Web server tuning, you can also adjust how often a client polls the Web server. HTTP is a request-based protocol. This means that clients using an HTTP-based protocol periodically need to check the Web server to see if messages are waiting. The imq.httpjms.http.pullPeriod broker property (and the corresponding imq.httpsjms.https.pullPeriod property) specifies how often the Message Queue client runtime polls the Web server.
If the pullPeriod value is -1 (the default value), the client runtime polls the server as soon as the previous request returns, maximizing the performance of the individual client. As a result, each client connection monopolizes a request thread in the Web server, possibly straining Web server resources.
If the pullPeriod value is a positive number, the client runtime periodically sends requests to the Web server to see if there is pending data. In this case, the client does not monopolize a request thread in the Web server. Hence, if large numbers of clients are using the Web server, you might conserve Web server resources by setting the pullPeriod to a positive value.
For information on tuning the file-based persistent store, see Persistence Services.
The following sections describe adjustments you can make to broker properties to improve performance.
Memory management can be configured on a destination-by-destination basis or on a systemwide level (for all destinations, collectively).
For information on physical destination limits, see Chapter 6, Managing Physical Destinations
If message producers tend to overrun message consumers, messages can accumulate in the broker. The broker contains a mechanism for throttling back producers and swapping messages out of active memory in low memory conditions, but it is wise to set a hard limit on the total number of messages (and message bytes) that the broker can hold.
Control these limits by setting the imq.system.max_count and the imq.system.max_size broker properties.
For example:
imq.system.max_count=5000
The defined value above means that the broker will only hold up to 5000 undelivered/unacknowledged messages. If additional messages are sent, they are rejected by the broker. If a message is persistent then the producer will get an exception when it tries to send the message. If the message is nonpersistent, the broker silently drops the message.
When an exception is returned in sending a message, the client should pause for a moment and retry the send again. (Note that the exception will never be due to the broker’s failure to receive a message; the only exceptions raised are those detected by the client on the sending side.)
The efficiency with which multiple queue consumers process messages in a queue destination depends on the following configurable queue destination attributes:
The number of active consumers (maxNumActiveConsumers)
The maximum number of messages that can be delivered to a consumer in a single batch (consumerFlowLimit)
To achieve optimal message throughput there must be a sufficient number of active consumers to keep up with the rate of message production for the queue, and the messages in the queue must be routed and then delivered to the active consumers in such a way as to maximize their rate of consumption. The general mechanism for balancing message delivery among multiple consumers is described in the Sun Java SystemTM Message Queue Technical Overview.
If messages are accumulating in the queue, it is possible that there is an insufficient number of active consumers to handle the message load. It is also possible that messages are being delivered to the consumers in batch sizes that cause messages to be backing up on the consumers. For example, if the batch size (consumerFlowLimit) is too large, one consumer might receive all the messages in a queue while other active consumers receive none. If consumers are very fast, this might not be a problem.
However, if consumers are relatively slow, you want messages to be distributed to them evenly, and therefore you want the batch size to be small. The smaller the batch size, the more overhead is required to deliver messages to consumers. Nevertheless, for slow consumers, there is generally a net performance gain to using small batch sizes.
This section discusses flow control behaviors that affect performance (see Client Runtime Configuration). These behaviors are configured as attributes of connection factory administered objects. For information on setting connection factory attributes, see Chapter 8, Managing Administered Objects
Messages sent and received by clients (payload messages), as well as Message Queue control messages, pass over the same client-broker connection. Delays in the delivery of control messages, such as broker acknowledgments, can result if control messages are held up by the delivery of payload messages. To prevent this type of congestion, Message Queue meters the flow of payload messages across a connection.
Payload messages are batched (as specified with the connection factory attribute imqConnectionFlowCount) so that only a set number are delivered. After the batch has been delivered, delivery of payload messages is suspended and only pending control messages are delivered. This cycle repeats, as additional batches of payload messages are delivered followed by pending control messages.
The value of imqConnectionFlowCount should be kept low if the client is doing operations that require many responses from the broker: for example, if the client is using CLIENT_ACKNOWLEDGE or AUTO_ACKNOWLEDGE mode, persistent messages, transactions, or queue browsers, or is adding or removing consumers. If, on the other hand, the client has only simple consumers on a connection using DUPS_OK_ACKNOWLEDGE mode, you can increase imqConnectionFlowCount without compromising performance.
There is a limit to the number of payload messages that the Message Queue client runtime can handle before encountering local resource limitations, such as memory. When this limit is approached, performance suffers. Hence, Message Queue lets you limit the number of messages per consumer (or messages per connection) that can be delivered over a connection and buffered in the client runtime, waiting to be consumed.
When the number of payload messages delivered to the client runtime exceeds the value of imqConsumerFlowLimit for any consumer, message delivery for that consumer stops. It is resumed only when the number of unconsumed messages for that consumer drops below the value set with imqConsumerFlowThreshold .
The following example illustrates the use of these limits: consider the default settings for topic consumers:
imqConsumerFlowLimit=1000 imqConsumerFlowThreshold=50
When the consumer is created, the broker delivers an initial batch of 1000 messages (providing they exist) to this consumer without pausing. After sending 1000 messages, the broker stops delivery until the client runtime asks for more messages. The client runtime holds these messages until the application processes them. The client runtime then allows the application to consume at least 50% (imqConsumerFlowThreshold ) of the message buffer capacity (i.e. 500 messages) before asking the broker to send the next batch.
In the same situation, if the threshold were 10%, the client runtime would wait for the application to consume at least 900 messages before asking for the next batch.
The next batch size is calculated as follows:
imqConsumerFlowLimit - (current number of pending msgs in buffer )
So if imqConsumerFlowThreshold is 50%, the next batch size can fluctuate between 500 and 1000, depending on how fast the application can process the messages.
If the imqConsumerFlowThreshold is set too high (close to 100%), the broker will tend to send smaller batches, which can lower message throughput. If the value is set too low (close to 0%), the client may be able to finish processing the remaining buffered messages before the broker delivers the next set, again degrading message throughput. Generally speaking, unless you have specific performance or reliability concerns, you will not need to change the default value of imqConsumerFlowThreshold attribute.
The consumer-based flow controls (in particular, imqConsumerFlowLimit ) are the best way to manage memory in the client runtime. Generally, depending on the client application, you know the number of consumers you need to support on any connection, the size of the messages, and the total amount of memory that is available to the client runtime.
In the case of some client applications, however, the number of consumers may be indeterminate, depending on choices made by end users. In those cases, you can still manage memory using connection-level flow limits.
Connection-level flow controls limit the total number of messages buffered for all consumers on a connection. If this number exceeds the value of imqConnectionFlowLimit, delivery of messages through the connection stops until that total drops below the connection limit. (The imqConnectionFlowLimit attribute is enabled only if you set imqConnectionFlowLimitEnabled to true.)
The number of messages queued up in a session is a function of the number of message consumers using the session and the message load for each consumer. If a client is exhibiting delays in producing or consuming messages, you can normally improve performance by redesigning the application to distribute message producers and consumers among a larger number of sessions or to distribute sessions among a larger number of connections.
This chapter explains how to understand and resolve the following problems:
When problems occur, it is useful to check the version number of the installed Message QueueTM software. Use the version number to ensure that you are using documentation whose version matches the software version. You also need the version number to report a problem to Sun. To check the version number, issue the following command:
imqcmd -v
Symptoms:
Client cannot make a new connection.
Client cannot auto-reconnect on failed connection.
Possible causes:
Broker is not running or there is a network connectivity problem.
Too few threads available for the number of connections required.
TCP backlog limits the number of simultaneous new connection requests that can be established.
Operating system limits the number of concurrent connections.
Possible cause: Client applications are not closing connections, causing the number of connections to exceed resource limitations.
To confirm this cause of the problem: List all connections to a broker:
imqcmd list cxn
The output will list all connections and the host from which each connection has been made, revealing an unusual number of open connections for specific clients.
To resolve the problem: Rewrite the offending clients to close unused connections.
Possible cause: Broker is not running or there is a network connectivity problem.
To confirm this cause of the problem:
Telnet to the broker’s primary port (for example, the default of 7676) and verify that the broker responds with Port Mapper output.
Verify that the broker process is running on the host.
To resolve the problem:
Start up the broker.
Fix the network connectivity problem.
Possible cause: Connection service is inactive or paused.
To confirm this cause of the problem: Check the status of all connection services:
imqcmd list svc
If the status of a connection service is shown as unknown or paused, clients will not be able to establish a connection using that service.
To resolve the problem:
If the status of a connection service is shown as unknown , it is missing from the active service list (imq.service.active ). In the case of SSL-based services, the service might also be improperly configured, causing the broker to make the following entry in the broker log:
ERROR [B3009]: Unable to start service ssljms:[B4001]: Unable to open protocol tls for ssljms service...
followed by an explanation of the underlying cause of the exception.
To properly configure SSL services, see Message Encryption.
If the status of a connection service is shown as paused, resume the service (see Pausing and Resuming a Connection Service).
Possible cause: Too few threads available for the number of connections required.
To confirm this cause of the problem: Check for the following entry in the broker log:
WARNING [B3004]: No threads are available to process a new connection on service ...Closing the new connection.
Also check the number of connections on the connection service and the number of threads currently in use, using one of the following formats:
imqcmd query svc -n serviceNameimqcmd metrics svc -n serviceName -m cxn
Each connection requires two threads: one for incoming messages and one for outgoing messages (see Thread Pool Management).
To resolve the problem:
If you are using a dedicated thread pool model (imq.serviceName.threadpool_model=dedicated), the maximum number of connections is half the maximum number of threads in the thread pool. Therefore, to increase the number of connections, increase the size of the thread pool (imq.serviceName.max_threads) or switch to the shared thread pool model.
If you are using a shared thread pool model (imq.serviceName.threadpool_model=shared), the maximum number of connections is half the product of the connection monitor limit (imq.serviceName.connectionMonitor_limit) and the maximum number of threads (imq.serviceName.max_threads). Therefore, to increase the number of connections, increase the size of the thread pool or increase the connection monitor limit.
Ultimately, the number of supportable connections (or the throughput on connections) will reach input/output limits. In such cases, use a multiple-broker cluster to distribute connections among the broker instances within the cluster.
Possible cause: Too few file descriptors for the number of connections required on the Solaris or Linux platform.
For more information about this issue, see Setting the File Descriptor Limit.
To confirm this cause of the problem: Check for an entry in the broker log similar to the following:
Too many open files
To resolve the problem: Increase the file descriptor limit, as described in the ulimit man page.
Possible cause: TCP backlog limits the number of simultaneous new connection requests that can be established.
The TCP backlog places a limit on the number of simultaneous connection requests that can be stored in the system backlog (imq.portmapper.backlog) before the Port Mapper rejects additional requests. (On the Windows platform there is a hard-coded backlog limit of 5 for Windows desktops and 200 for Windows servers.)
The rejection of requests because of backlog limits is usually a transient phenomenon, due to an unusually high number of simultaneous connection requests.
To confirm this cause of the problem: Examine the broker log. First, check to see whether the broker is accepting some connections during the same time period that it is rejecting others. Next, check for messages that explain rejected connections. If you find such messages, the TCP backlog is probably not the problem, because the broker does not log connection rejections due to the TCP backlog. If some successful connections are logged, and no connection rejections are logged, the TCP backlog is probably the problem.
To resolve the problem:
Program the client to retry the attempted connection after a short interval of time (this normally works because of the transient nature of this problem).
Increase the value of imq.portmapper.backlog.
Check that clients are not closing and then opening connections too often.
Possible cause: Operating system limits the number of concurrent connections.
The Windows operating system license places limits on the number of concurrent remote connections that are supported.
To confirm this cause of the problem: Check that there are plenty of threads available for connections (using imqcmd query svc) and check the terms of your Windows license agreement. If you can make connections from a local client, but not from a remote client, operating system limitations might be the cause of the problem.
To resolve the problem:
Upgrade the Windows license to allow more connections.
Distribute connections among a number of broker instances by setting up a multiple-broker cluster.
Possible cause: Authentication or authorization of the user is failing.
The authentication may be failing for any of the following reasons:
Incorrect password
No entry for user in user repository
User does not have access permission for connection service
To confirm this cause of the problem: Check entries in the broker log for the Forbidden error message. This will indicate an authentication error, but will not indicate the reason for it.
If you are using a file-based user repository, enter the following command:
imqusermgr list -i instanceName -u userName
If the output shows a user, the wrong password was probably submitted. If the output shows the following error, there is no entry for the user in the user repository:
Error [B3048]: User does not exist in the password file
If you are using an LDAP server user repository, use the appropriate tools to check whether there is an entry for the user.
Check the access control properties file to see whether there are restrictions on access to the connection service.
To resolve the problem:
If the wrong password was used, provide the correct password.
If there is no entry for the user in the user repository, add one (see Populating and Managing a User Repository).
If the user does not have access permission for the connection service, edit the access control properties file to grant such permission (see Access Control for Connection Services).
Symptoms:
Message throughput does not meet expectations.
The number of supported connections to a broker is not limited as described in A Client Cannot Establish a Connection, but rather by message input/output rates.
Possible causes:
Possible cause: Network connection or WAN is too slow.
To confirm this cause of the problem:
Ping the network, to see how long it takes for the ping to return, and consult a network administrator.
Send and receive messages using local clients and compare the delivery time with that of remote clients (which use a network link).
To resolve the problem: Upgrade the network link.
Possible cause: Connection service protocol is inherently slow compared to TCP.
For example, SSL-based or HTTP-based protocols are slower than TCP (see Transport Protocols).
To confirm this cause of the problem: If you are using SSL-based or HTTP-based protocols, try using TCP and compare the delivery times.
To resolve the problem: Application requirements usually dictate the protocols being used, so there is little you can do other than attempt to tune the protocol as described in Tuning Transport Protocols.
Possible cause: Connection service protocol is not optimally tuned.
To confirm this cause of the problem: Try tuning the protocol to see whether it makes a difference.
To resolve the problem: Try tuning the protocol, as described in Tuning Transport Protocols.
Possible cause: Messages are so large that they consume too much bandwidth.
To confirm this cause of the problem: Try running your benchmark with smaller-sized messages.
To resolve the problem:
Have application developers modify the application to use the message compression feature, which is described in the Message Queue Developer's Guide for Java Clients.
Use messages as notifications of data to be sent, but move the data using another protocol.
Possible cause: What appears to be slow connection throughput is actually a bottleneck in some other step of the message delivery process.
To confirm this cause of the problem: If what appears to be slow connection throughput cannot be explained by any of the causes above, see Factors Affecting Performance for other possible bottlenecks and check for symptoms associated with the following problems:
To resolve the problem: Follow the problem resolution guidelines provided in the troubleshooting sections listed above.
Symptom:
A message producer cannot be created for a physical destination; the client receives an exception.
Possible causes:
Possible cause: A physical destination has been configured to allow only a limited number of producers.
One of the ways of avoiding the accumulation of messages on a physical destination is to limit the number of producers (maxNumProducers) that it supports.
To confirm this cause of the problem: Check the physical destination:
imqcmd query dst
(see Displaying Information about Physical Destinations). The output will show the current number of producers and the value of maxNumProducers. If the two values are the same, the number of producers has reached its configured limit. When a new producer is rejected by the broker, the broker returns the exception
ResourceAllocationException [C4088]: A JMS destination limit was reached
and makes the following entry in the broker log:
[B4183]: Producer can not be added to destination
To resolve the problem: Increase the value of the maxNumProducers attribute (see Updating Physical Destination Properties).
Possible cause: The user is not authorized to create a message producer due to settings in the access control properties file.
To confirm this cause of the problem: When a new producer is rejected by the broker, the broker returns the exception
JMSSecurityException [C4076]: Client does not have permission to create producer on destination
and makes the following entries in the broker log:
[B2041]: Producer on destination denied[B4051]: Forbidden guest.
To resolve the problem: Change the access control properties to allow the user to produce messages (see Access Control for Physical Destinations).
Symptoms:
When sending persistent messages, the send method does not return and the client blocks.
When sending a persistent message, the client receives an exception.
A producing client slows down.
Possible causes:
The broker is backlogged and has responded by slowing message producers.
The broker cannot save a persistent message to the data store.
Possible cause: The broker is backlogged and has responded by slowing message producers.
A backlogged broker accumulates messages in broker memory. When the number of messages or message bytes in physical destination memory reaches configured limits, the broker attempts to conserve memory resources in accordance with the specified limit behavior. The following limit behaviors slow down message producers:
FLOW_CONTROL: The broker does not immediately acknowledge receipt of persistent messages (thereby blocking a producing client).
REJECT_NEWEST: The broker rejects new persistent messages.
Similarly, when the number of messages or message bytes in brokerwide memory (for all physical destinations) reaches configured limits, the broker will attempt to conserve memory resources by rejecting the newest messages. Also, when system memory limits are reached because physical destination or brokerwide limits have not been set properly, the broker takes increasingly serious action to prevent memory overload. These actions include throttling back message producers.
To confirm this cause of the problem: When a message is rejected by the broker because of configured message limits, the broker returns the exception
JMSException [C4036]: A server error occurred
and makes the following entry in the broker log:
[B2011]: Storing of JMS message from IMQconn failed
This message is followed by another indicating the limit that has been reached:
[B4120]: Cannot store message on destination destName because capacity of maxNumMsgs would be exceeded.
if the exceeded message limit is on a physical destination, or
[B4024]: The maximum number of messages currrently in the system has been exceeded, rejecting message.
if the limit is brokerwide.
More generally, you can check for message limit conditions before the rejections occur as follows:
Query physical destinations and the broker and inspect their configured message limit settings.
Monitor the number of messages or message bytes currently in a physical destination or in the broker as a whole, using the appropriate imqcmd commands. See Chapter 18, Metrics Reference for information about metrics you can monitor and the commands you use to obtain them.
To resolve the problem:
Modify the message limits on a physical destination (or brokerwide), being careful not to exceed memory resources.
In general, you should manage memory at the individual destination level, so that brokerwide message limits are never reached. For more information, see Broker Adjustments.
Change the limit behaviors on a destination so as not to slow message production when message limits are reached, but rather to discard messages in memory.
For example, you can specify the REMOVE_OLDEST and REMOVE_LOW_PRIORITY limit behaviors, which delete messages that accumulate in memory (see Table 15–1).
Possible cause: The broker cannot save a persistent message to the data store.
If the broker cannot access a data store or write a persistent message to it, the producing client is blocked. This condition can also occur if destination or brokerwide message limits are reached, as described above.
To confirm this cause of the problem: If the broker is unable to write to the data store, it makes one of the following entries in the broker log:
[B2011]: Storing of JMS message from connectionID failed[B4004]: Failed to persist message messageID
To resolve the problem:
In the case of file-based persistence, try increasing the disk space of the file-based data store.
In the case of a JDBC-compliant data store, check that JDBC-based persistence is properly configured (see Configuring a Persistent Data Store). If so, consult your database administrator to troubleshoot other database problems.
Possible cause: Broker acknowledgment timeout is too short.
Because of slow connections or a lethargic broker (caused by high CPU utilization or scarce memory resources), a broker may require more time to acknowledge receipt of a persistent message than allowed by the value of the connection factory’s imqAckTimeout attribute.
To confirm this cause of the problem: If the imqAckTimeout value is exceeded, the broker returns the exception
JMSException [C4000]: Packet acknowledge failed
To resolve the problem: Change the value of the imqAckTimeout connection factory attribute (see Reliability And Flow Control).
Possible cause: A producing client is encountering JVM limitations.
To confirm this cause of the problem:
Find out whether the client application receives an out-of-memory error.
Check the free memory available in the JVM heap, using runtime methods such as freeMemory, maxMemory, and totalMemory.
To resolve the problem: Adjust the JVM (see Java Virtual Machine Adjustments).
Symptoms:
Message production is delayed or produced messages are rejected by the broker.
Messages take an unusually long time to reach consumers.
The number of messages or message bytes in the broker (or in specific destinations) increases steadily over time.
To see whether messages are accumulating, check how the number of messages or message bytes in the broker changes over time and compare to configured limits. First check the configured limits:
imqcmd query bkr
The imqcmd metrics bkr subcommand does not display this information.
Then check for message accumulation in each destination:
imqcmd list dst
To see whether messages have exceeded configured destination or brokerwide limits, check the broker log for the entry
[B2011]: Storing of JMS message from … failed.
This entry will be followed by another identifying the limit that has been exceeded.
Possible causes:
There are inactive durable subscriptions on a topic destination.
Too few consumers are available to consume messages in a queue.
Message consumers are processing too slowly to keep up with message producers.
Client acknowledgment processing is slowing down message consumption.
Client code defects; consumers are not acknowledging messages.
Possible cause: There are inactive durable subscriptions on a topic destination.
If a durable subscription is inactive, messages are stored in a destination until the corresponding consumer becomes active and can consume the messages.
To confirm this cause of the problem: Check the state of durable subscriptions on each topic destination:
imqcmd list dur -d destName
To resolve the problem:
Purge all messages for the offending durable subscriptions (see Managing Durable Subscriptions).
Specify message limit and limit behavior attributes for the topic (see Table 15–1). For example, you can specify the REMOVE_OLDEST and REMOVE_LOW_PRIORITY limit behaviors, which delete messages that accumulate in memory.
Purge all messages from the corresponding destinations (see Purging Physical Destinations).
Limit the time messages can remain in memory by rewriting the producing client to set a time-to-live value on each message. You can override any such settings for all producers sharing a connection by setting the imqOverrideJMSExpiration and imqJMSExpiration connection factory attributes (see Message Header Overrides).
Possible cause: Too few consumers are available to consume messages in a queue.
If there are too few active consumers to which messages can be delivered, a queue destination can become backlogged as messages accumulate. This condition can occur for any of the following reasons:
Too few active consumers exist for the destination.
Consuming clients have failed to establish connections.
No active consumers use a selector that matches messages in the queue.
To confirm this cause of the problem: To help determine the reason for unavailable consumers, check the number of active consumers on a destination:
imqcmd metrics dst -n destName -t q -m con
To resolve the problem: Depending on the reason for unavailable consumers,
Create more active consumers for the queue by starting up additional consuming clients.
Adjust the imq.consumerFlowLimit broker property to optimize queue delivery to multiple consumers (see Multiple Consumer Queue Performance).
Specify message limit and limit behavior attributes for the queue (see Table 15–1). For example, you can specify the REMOVE_OLDEST and REMOVE_LOW_PRIOROTY limit behaviors, which delete messages that accumulate in memory.
Purge all messages from the corresponding destinations (see Purging Physical Destinations).
Limit the time messages can remain in memory by rewriting the producing client to set a time-to-live value on each message. You can override any such setting for all producers sharing a connection by setting the imqOverrideJMSExpiration and imqJMSExpiration connection factory attributes (see Message Header Overrides).
Possible cause: Message consumers are processing too slowly to keep up with message producers.
In this case, topic subscribers or queue receivers are consuming messages more slowly than the producers are sending messages. One or more destinations are getting backlogged with messages because of this imbalance.
To confirm this cause of the problem: Check for the rate of flow of messages into and out of the broker:
imqcmd metrics bkr -m rts
Then check flow rates for each of the individual destinations:
imqcmd metrics bkr -t destType -n destName -m rts
To resolve the problem:
Optimize consuming client code.
For queue destinations, increase the number of active consumers (see Multiple Consumer Queue Performance).
Possible cause: Client acknowledgment processing is slowing down message consumption.
Two factors affect the processing of client acknowledgments:
Significant broker resources can be consumed in processing client acknowledgments. As a result, message consumption may be slowed in those acknowledgment modes in which consuming clients block until the broker confirms client acknowledgments.
JMS payload messages and Message Queue control messages (such as client acknowledgments) share the same connection. As a result, control messages can be held up by JMS payload messages, slowing message consumption.
To confirm this cause of the problem:
Check the flow of messages relative to the flow of packets. If the number of packets per second is out of proportion to the number of messages, client acknowledgments may be a problem.
Check to see whether the client has received the following exception:
JMSException [C4000]: Packet acknowledge failed
To resolve the problem:
Modify the acknowledgment mode used by clients: for example, switch to DUPS_OK_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE.
If using CLIENT_ACKNOWLEDGE or transacted sessions, group a larger number of messages into a single acknowledgment.
Adjust consumer and connection flow control parameters (see Client Runtime Message Flow Adjustments ).
Possible cause: The broker cannot keep up with produced messages.
In this case, messages are flowing into the broker faster than the broker can route and dispatch them to consumers. The sluggishness of the broker can be due to limitations in any or all of the following:
CPU
Network socket read/write operations
Disk read/write operations
Memory paging
Persistent store
JVM memory limits
To confirm this cause of the problem: Check that none of the other possible causes of this problem are responsible.
To resolve the problem:
Upgrade the speed of your computer or data store.
Use a broker cluster to distribute the load among multiple broker instances.
Possible cause: Client code defects; consumers are not acknowledging messages.
Messages are held in a destination until they have been acknowledged by all consumers to which they have been sent. If a client is not acknowledging consumed messages, the messages accumulate in the destination without being deleted.
For example, client code might have the following defects:
Consumers using the CLIENT_ACKNOWLEDGE acknowledgment mode or transacted session may not be calling Session.acknowledge or Session.commit regularly.
Consumers using the AUTO_ACKNOWLEDGE acknowledgment mode may be hanging for some reason.
To confirm this cause of the problem: First check all other possible causes listed in this section. Next, list the destination with the following command:
imqcmd list dst
Notice whether the number of messages listed under the UnAcked header is the same as the number of messages in the destination. Messages under this header were sent to consumers but not acknowledged. If this number is the same as the total number of messages, then the broker has sent all the messages and is waiting for acknowledgment.
To resolve the problem: Request the help of application developers in debugging this problem.
Symptom:
Message throughput sporadically drops and then resumes normal performance.
Possible causes:
JVM memory reclamation (garbage collection) is taking place.
The JVM is using the just-in-time compiler to speed up performance.
Possible cause: The broker is very low on memory resources.
Because destination and broker limits were not properly set, the broker takes increasingly serious action to prevent memory overload; this can cause the broker to become sluggish until the message backlog is cleared.
To confirm this cause of the problem: Check the broker log for a low memory condition
[B1089]: In low memory condition, broker is attempting to free up resources
followed by an entry describing the new memory state and the amount of total memory being used. Also check the free memory available in the JVM heap:
imqcmd metrics bkr -m cxn
Free memory is low when the value of total JVM memory is close to the maximum JVM memory value.
To resolve the problem:
Adjust the JVM (see Java Virtual Machine Adjustments).
Increase system swap space.
Possible cause: JVM memory reclamation (garbage collection) is taking place.
Memory reclamation periodically sweeps through the system to free up memory. When this occurs, all threads are blocked. The larger the amount of memory to be freed up and the larger the JVM heap size, the longer the delay due to memory reclamation.
To confirm this cause of the problem: Monitor CPU usage on your computer. CPU usage drops when memory reclamation is taking place.
Also start your broker using the following command line options:
-vmargs -verbose:gc
Standard output indicates the time when memory reclamation takes place.
To resolve the problem: In computers with multiple CPUs, set the memory reclamation to take place in parallel:
-XX:+UseParallelGC=true
Possible cause: The JVM is using the just-in-time compiler to speed up performance.
To confirm this cause of the problem: Check that none of the other possible causes of this problem are responsible.
To resolve the problem: Let the system run for awhile; performance should improve.
Symptom:
Messages sent by producers are not received by consumers.
Possible causes:
Limit behaviors are causing messages to be deleted on the broker.
Consuming client failed to start message delivery on a connection.
Possible cause: Limit behaviors are causing messages to be deleted on the broker.
When the number of messages or message bytes in destination memory reach configured limits, the broker attempts to conserve memory resources. Three of the configurable behaviors adopted by the broker when these limits are reached will cause messages to be lost:
REMOVE_OLDEST: Delete the oldest messages.
REMOVE_LOW_PRIORITY: Delete the lowest-priority messages according to age.
REJECT_NEWEST: Reject new persistent messages.
To confirm this cause of the problem: Check the dead message queue, as described under Dead Message Queue Contains Messages. Specifically, use the instructions under “The number of messages, or their sizes, exceed destination limits.” Look for the REMOVE_OLDEST or REMOVE_LOW_PRIORITY reason.
To resolve the problem: Increase the destination limits. For example:
imqcmd update dst -n MyDest -o maxNumMsgs=1000
Possible cause: Message timeout value is expiring.
The broker deletes messages whose timeout value has expired. If a destination gets sufficiently backlogged with messages, messages whose time-to-live value is too short may be deleted.
To confirm this cause of the problem: Use the QBrowser demo application to look at the contents of the dead message queue and see whether messages are timing out. For the QBrowser demo’s platform-specific location, see Appendix A, Platform-Specific Locations of Message QueueTM Data and look in the tables for “Example Applications and Locations.”
Here is an example invocation on the Windows platform:
cd \MessageQueue3\demo\applications\qbrowser java QBrowser
When the QBrowser main window appears, select the queue name mq.sys.dmq and then click Browse. A list like the following appears:
Double-click any message to display details about that message:
Note whether the JMS_SUN_DMQ_UNDELIVERED_REASON property for messages has the value EXPIRED.
To resolve the problem: Contact the application developers and have them increase the time-to-live value.
Possible cause: Clocks are not synchronized.
If clocks are not synchronized, broker calculations of message lifetimes can be wrong, causing messages to exceed their expiration times and be deleted.
To confirm this cause of the problem: In the broker log file, look for any of the following messages: B2102, B2103, B2104. These messages all report that possible clock skew was detected.
To resolve the problem: Check that you are running a time synchronization program, as described in Preparing System Resources.
Possible cause: Consuming client failed to start message delivery on a connection.
Messages cannot be delivered until client code establishes a connection and starts message delivery on the connection.
To confirm this cause of the problem: Check that client code establishes a connection and starts message delivery.
To resolve the problem: Rewrite the client code to establish a connection and start message delivery.
Symptom:
When you list destinations, you see that the dead message queue contains messages. For example, issue a command like the following:
imqcmd list dst
After you supply a user name and password, output like the following appears:
Listing all the destinations on the broker specified by: --------------------------------- Host Primary Port --------------------------------- localhost 7676 ---------------------------------------------------------------------- Name Type State Producers Consumers Msgs Total Count UnAck Avg Size ------------------------------------------------- ---------------------- MyDest Queue RUNNING 0 0 5 0 1177.0 mq.sys.dmq Queue RUNNING 0 0 35 0 1422.0 Successfully listed destinations. |
In this example, the dead message queue, mq.sys.dmq, contains 35 messages.
Possible causes:
Possible cause: The number of messages, or their sizes, exceed destination limits.
To confirm this cause of the problem: Use the QBrowser demo application to look at the contents of the dead message queue. For the QBrowser demo’s platform-specific location, see Appendix A, Platform-Specific Locations of Message QueueTM Data and look in the tables for “Example Applications and Locations.”
Here is an example invocation on the Windows platform:
cd \MessageQueue3\demo\applications\qbrowser java QBrowser
When the QBrowser main window appears, select the queue name mq.sys.dmq and then click Browse. A list like the one shown earlier under “Message timeout value is expiring” appears. Double-click any message to display details about that message, as shown under “Message timeout value is expiring.”
Note the values for the following message properties:
JMS_SUN_DMQ_UNDELIVERED_REASON
JMS_SUN_DMQ_UNDELIVERED_COMMENT
JMS_SUN_DMQ_UNDELIVERED_TIMESTAMP
Under JMS Headers, note the value for JMSDestination to determine the destination whose messages are becoming dead.
To resolve the problem: Increase the destination limits. For example:
imqcmd update dst -n MyDest -o maxNumMsgs=1000
Possible cause: The broker clock and producer clock are not synchronized.
To confirm this cause of the problem: Using the QBrowser application, view the message details for messages in the dead message queue. Check the value for JMS_SUN_DMQ_UNDELIVERED_REASON, looking for messages with the reason EXPIRED.
In the broker log file, look for any of the following messages: B2102, B2103, B2104. These messages all report that possible clock skew was detected.
To resolve the problem: Check that you are running a time synchronization program, as described in Preparing System Resources.
Possible cause: Consumers are not receiving messages before they time out.
To verify this cause of the problem: Using the QBrowser application, view the message details for messages in the dead message queue. Check the value for JMS_SUN_DMQ_UNDELIVERED_REASON, looking for messages with the reason EXPIRED.
Check to see whether there any consumers on the destination. For example:
imqcmd query dst -t q -n MyDest
Check the value listed for Current Number of Active Consumers. If there are active consumers, one of the following is true:
A consumer’s connection is paused.
The message timeout is too short for the speed at which the consumer executes.
To resolve the problem: Request that application developers increase message time-to-live values.
Possible cause: There are too many producers for the number of consumers.
To confirm this cause of the problem: Using the QBrowser application, view the message details for messages in the dead message queue. Check the value for JMS_SUN_DMQ_UNDELIVERED_REASON. If the reason is REMOVE_OLDEST or REMOVE_LOW_PRIORITY, use the imqcmd query dst command to check the number of producers and consumers on the destination. If the number of producers exceeds the number of consumers, production rate may be overwhelming consumption rate.
To resolve the problem: Add more consumer clients or set the destination’s limit behavior to FLOW_CONTROL (which uses consumption rate to control production rate), using a command such as the following:
imqcmd update dst -n myDst -t q -o consumerFlowLimit=FLOW_CONTROL
Possible cause: Producers are faster than consumers.
To confirm this cause of the problem: To determine whether slow consumers are causing producers to slow down, set the destination’s limit behavior to FLOW_CONTROL (which uses consumption rate to control production rate), using a command such as the following:
imqcmd update dst -n myDst -t q -o consumerFlowLimit=FLOW_CONTROL
Use metrics to examine the destination’s input and output, using a command such as the following:
imqcmd metrics dst -n myDst -t q -m rts
In the metrics output, examine the following values:
Msgs/sec Out: Shows how many messages per second the broker is removing. The broker removes messages when all consumers acknowledge receiving them, so the metric reflects consumption rate.
Msgs/sec In: Shows how many messages per second the broker is receiving from producers. The metric reflects production rate.
Because flow control aligns production to consumption, note whether production slows or stops. If so, there is a discrepancy between the processing speeds of producers and consumers. You can also check the number of unacknowledged (UnAcked) messages sent, by using the imqcmd list dst command. If the number of unacknowledged messages is less than the size of the destination, the destination has additional capacity and is being held back by client flow control.
To resolve the problem: If production rate is consistently faster than consumption rate, consider using flow control regularly, to keep the system aligned. In addition, using the subsequent sections, consider and attempt to resolve each of the following possible factors:
Possible cause: A consumer is too slow.
To confirm this cause of the problem: Use metrics to determine the rate of production and consumption, as described above under “Producers are faster than consumers.”
To resolve the problem:
Set the destinations’ limit behavior to FLOW_CONTROL, using a command such as the following:
imqcmd update dst -n myDst -t q -o consumerFlowLimit=FLOW_CONTROL
Use of flow control slows production to the rate of consumption and prevents the accumulation of messages on the broker. Producer applications hold messages until the destination can process them, with less risk of expiration.
Find out from application developers whether producers send messages at a steady rate or in periodic bursts. If an application sends bursts of messages, increase destination limits as described in the next item.
Increase destination limits based on number of messages or bytes, or both. To change the number of messages on a destination, enter a command with the following format:
imqcmd update dst -n destName -t {q|t} -o maxNumMsgs=number
To change the size of a destination, enter a command with the following format:
imqcmd update dst -n destName -t {q|t} -o maxTotalMsgBytes=number
Be aware that raising limits increases the amount of memory that the broker uses. If limits are too high, the broker could run out of memory and become unable to process messages.
Consider whether you can accept loss of messages during periods of high production load.
Possible cause: Clients are not committing messages.
To confirm this cause of the problem: Check with application developers to find out whether the application uses transactions. If so, list the active transactions as follows:
imqcmd list txn
Here is an example of the command output:
---------------------------------------------------------------------- Transaction ID State User name # Msgs/# Acks Creation time ---------------------------------------------------------------------- 6800151593984248832 STARTED guest 3/2 7/19/04 11:03:08 AM |
Note the numbers of messages and number of acknowledgments. If the number of messages is high, producers may be sending individual messages but failing to commit transactions. Until the broker receives a commit, it cannot route and deliver the messages for that transaction. If the number of acknowledgments is high, consumers may be sending acknowledgments for individual messages but failing to commit transactions. Until the broker receives a commit, it cannot remove the acknowledgments for that transaction.
To resolve the problem: Contact application developers to fix the coding error.
Possible cause: Consumers are failing to acknowledge messages.
To confirm this cause of the problem: Contact application developers to determine whether the application uses system-based acknowledgment or client-based acknowledgment. If the application uses system-based acknowledgment, skip this section; if it uses client-based acknowledgment (CLIENT_ACKNOWLEDGE), first decrease the number of messages stored on the client, using a command like the following:
imqcmd update dst -n myDst -t q -o consumerFlowLimit=1
Next, you will determine whether the broker is buffering messages because a consumer is slow, or whether the consumer processes messages quickly but does not acknowledge them. List the destination, using the following command:
imqcmd list dst
After you supply a user name and password, output like the following appears:
Listing all the destinations on the broker specified by: --------------------------------- Host Primary Port --------------------------------- localhost 7676 ---------------------------------------------------------------------- Name Type State Producers Consumers Msgs Total Count UnAck Avg Size ------------------------------------------------ ----------------------- MyDest Queue RUNNING 0 0 5 200 1177.0 mq.sys.dmq Queue RUNNING 0 0 35 0 1422.0 Successfully listed destinations. |
The UnAck number represents messages that the broker has sent and for which it is waiting for acknowledgment. If this number is high or increasing, you know that the broker is sending messages, so it is not waiting for a slow consumer. You also know that the consumer is not acknowledging the messages.
To resolve the problem: Contact application developers to fix the coding error.
Possible cause: Durable consumers are inactive.
To confirm this cause of the problem: Look at the topic’s durable subscribers, using the following command format:
imqcmd list dur -d topicName
To resolve the problem:
Purge the durable consumers using the imqcmd purge dur command.
Restart the consumer applications.
Possible cause: An unexpected broker error has occurred.
To confirm this cause of the problem: Use QBrowser to examine a message, as described earlier under “Producers are faster than consumers.” If the value for JMS_SUN_DMQ_UNDELIVERED_REASON is ERROR, a broker error occurred.
To resolve the problem:
Examine the broker log file to find the associated error.
Contact Sun Technical Support to report the broker problem.