Skip Navigation Links | |
Exit Print View | |
Oracle GlassFish Server Message Queue 4.5 Administration Guide |
Part I Introduction to Message Queue Administration
1. Administrative Tasks and Tools
3. Starting Brokers and Clients
6. Configuring and Managing Connection Services
8. Configuring Persistence Services
9. Configuring and Managing Security Services
10. Configuring and Managing Broker Clusters
The Cluster Configuration File
Cluster Configuration Properties
Cluster Connection Service Properties
Conventional Broker Cluster Properties
Enhanced Broker Cluster Properties
Displaying a Cluster Configuration
Managing Conventional Clusters
Connecting Brokers into a Conventional Cluster
Adding Brokers to a Conventional Cluster
Removing Brokers From a Conventional Cluster
Changing the Master Broker in a Conventional Cluster with Master Broker
Managing a Conventional Cluster's Configuration Change Record
Converting Between Types of Conventional Clusters
Connecting Brokers into an Enhanced Cluster
Adding and Removing Brokers in an Enhanced Cluster
Preventing or Forcing Broker Failover
Backing up a Shared Data Store
Converting a Conventional Cluster to an Enhanced Cluster
11. Managing Administered Objects
12. Configuring and Managing Bridge Services
13. Monitoring Broker Operations
14. Analyzing and Tuning a Message Service
17. Broker Properties Reference
18. Physical Destination Property Reference
19. Administered Object Attribute Reference
20. JMS Resource Adapter Property Reference
21. Metrics Information Reference
22. JES Monitoring Framework Reference
A. Distribution-Specific Locations of Message Queue Data
B. Stability of Message Queue Interfaces
The following sections describe how to perform various administrative management tasks for conventional and enhanced clusters, respectively.
The procedures in this section show how to perform the following tasks for a conventional cluster:
Changing the Master Broker in a Conventional Cluster with Master Broker
Managing a Conventional Cluster's Configuration Change Record
There are two general methods of connecting brokers into a conventional cluster: from the command line (using the -cluster option) or by setting the imq.cluster.brokerlist property in the cluster configuration file.
Whichever method you use, each broker that you start attempts to connect to the other brokers in the cluster every five seconds until the connection succeeds.
For a cluster configured with master broker, the connection will succeed once the master broker is started up (if one is configured). If a broker in the cluster starts before the master broker, it will remain in a suspended state, rejecting client connections, until the master broker starts; the suspended broker then will automatically become fully functional. It is therefore a good idea to start the master broker first and then the others, after the master broker has completed its startup.
When connecting brokers into a conventional cluster, you should be aware of the following issues:
Mixed broker versions. A conventional cluster can contain brokers of different versions if all brokers have a version at least as great as that of the master broker. If the cluster is not configured to use a master broker, then all brokers must be of the same version.
Matching broker property values. In addition to cluster configuration properties, the following broker properties also must have the same value for all brokers in a cluster:
imq.service.activelist
imq.autocreate.queue
imq.autocreate.topic
imq.autocreate.queue.maxNumActiveConsumers
imq.autocreate.queue.maxNumBackupConsumers
This restriction is particularly important when a cluster contains mixed broker versions that might contain properties with different default values. For example, If you are clustering a Message Queue version 4.1 or later broker together with those from earlier versions than Message Queue 4.1, you must set the value of the imq.autocreate.queue.maxNumActiveConsumers property, which has different default values before and after version 4.1 (1 and -1, respectively), to be the same. Otherwise the brokers will not be able to establish a cluster connection.
Multiple interface cards. On a multi-homed computer, in which there is more than one network interface card, be sure to explicitly set the network interface to be used by the broker for client connection services (imq.hostname) and for the cluster connection service (imq.cluster.hostname). Setting the imq.hostname value also effectively sets the value for other properties that use imq.hostname as their default value, such as imq.portmapper.hostname, imq.cluster.hostname, and so on. If imq.cluster.hostname is not set, then connections between brokers might not succeed and as a result, the cluster will not be established.
Network loopback IP address. You must make sure that no broker in the cluster is given an address that resolves to a loopback network (127.*.*.*) IP address. Any broker configured with such an address will be unable to connect to other brokers in the cluster.
In particular, some Linux installers automatically set the local host to a loopback network address, most commonly 127.0.0.1. On such systems, you must do the following: For each Linux system participating in the cluster, check the /etc/hosts file as part of cluster setup. If the system uses a static IP address, edit the /etc/hosts file to specify the correct address for the local host. If the address is registered with Domain Name Service (DNS), edit the file /etc/nsswitch.conf so that DNS lookup is performed before consulting the local hosts file.
The method best suited for production systems is to use a cluster configuration file to specify the configuration of the cluster:
Use the imqdbmgr create sharecc_tbl command to create the database table for the configuration change record.
Place a copy of, or a symbolic link to, your JDBC driver’s .jar file in the Message Queue external resource files directory, IMQ_HOME/lib/ext, on each host where a broker will run.
If you are using a master broker, identify it with the imq.cluster.masterbroker property in the configuration file.
If you are using a cluster of peer brokers, set the imq.cluster.nomasterbroker property to true, and set imq.cluster.sharecc.persist.jdbc.* properties as appropriate in the configuration file.
If there is a master broker, start it first, then the others after it has completed its startup.
Connecting brokers to a cluster from the command line involves starting each broker with the imqbrokerd command using the -cluster option to specify the complete list of brokers to be included in the cluster.
For example, the following command starts a broker as part of a cluster consisting of the brokers running at the default port (7676) on host1, at port 5000 on host2, and at port 9876 on the default host (localhost):
imqbrokerd -cluster host1,host2:5000,:9876
The value specified for the -cluster option must be the same for all brokers in the cluster.
Before You Begin
Set any necessary broker properties, except imq.cluster.brokerlist, in each broker's configuration file before performing the following procedure.
Use the imqdbmgr create sharecc_tbl command to create the database table for the configuration change record.
Place a copy of, or a symbolic link to, your JDBC driver’s .jar file in the Message Queue external resource files directory, IMQ_HOME/lib/ext, on each host where a broker will run.
If you want secure, encrypted message delivery between brokers in a cluster, configure the cluster connection service to use an SSL-based transport protocol:
The procedure for adding a new broker to a conventional cluster depends on whether the cluster uses a cluster configuration file.
This forces each broker to reload the imq.cluster.brokerlist property. It is not necessary to issue this command to every broker in the cluster; executing it for any one broker will cause all of them to reload the cluster configuration.
If you did not perform step 3, use the -D option on the imqbrokerd command line to set the value of imq.cluster.url to the location of the cluster configuration file.
When the newly added broker starts, it connects and exchanges data with all the other brokers in the imq.cluster.brokerlist value.
This step is not strictly necessary to add a broker to a functioning cluster. However, should any broker need to be restarted, its imq.cluster.brokerlist value must include all other brokers in the cluster, including the newly added broker.
If you did not perform step 1, use the -D option on the imqbrokerd command line to set the property values listed there.
The method you use to remove a broker from a conventional cluster depends on whether you originally created the cluster using a cluster configuration file or by means of command line options.
Note - Before you remove from a conventional cluster the broker instance serving as the cluster's master broker, first change the master broker to another broker instance in the cluster, as described in Changing the Master Broker in a Conventional Cluster with Master Broker
If you originally created a cluster by specifying its member brokers with the imq.cluster.brokerlist property in a central cluster configuration file, it isn’t necessary to stop the brokers in order to remove one of them. Instead, you can simply edit the configuration file to exclude the broker you want to remove, force the remaining cluster members to reload the cluster configuration, and reconfigure the excluded broker so that it no longer points to the same cluster configuration file:
Use the imqcmd query bkr command periodically to check the number of messages in the broker.
Use the imqcmd list txn command to view prepared open transactions, and use the imqcmd rollback txn and imqcmd commit txn to roll back and commit transactions.
This forces the brokers to reload the cluster configuration.
If you used the imqbrokerd command from the command line to connect the brokers into a cluster, you must stop each of the brokers and then restart them, specifying the new set of cluster members on the command line:
Use the imqcmd query bkr command periodically to check the number of messages in the broker.
Use the imqcmd list txn command to view prepared open transactions, and use the imqcmd rollback txn and imqcmd commit txn to roll back and commit transactions.
For example, suppose you originally created a cluster consisting of brokers A, B, and C by starting each of the three with the command
imqbrokerd -cluster A,B,CTo remove broker A from the cluster, restart brokers B and C with the command
imqbrokerd -cluster B,CMessage Queue provides two ways to change the broker instance serving as the master broker to a different broker instance in the cluster:
Dynamically while the cluster is running
Manually by stopping the cluster and migrating the configuration change record from one broker to another
To change the master broker dynamically, you must first configure the brokers in the cluster to support dynamic changing of the master broker.
If using a cluster configuration file, you can instead set the imq.cluster.dynamicChangeMasterBrokerEnabled property to true in the cluster configuration file.
When the imq.cluster.dynamicChangeMasterBrokerEnabled property is set to true, the imq.cluster.masterbroker property cannot be specified on the command line to start a broker. Therefore, it must be set in the brokers' properties files, or in the cluster configuration file if one is being used.
To dynamically change the broker instance serving as the master broker to a different broker instance in the cluster, use the imqcmd changemaster cls command.
Follow this procedure, for example, before you remove from a cluster the broker instance serving as the master broker.
Caution - Do not use the imqcmd changemaster cls command to dynamically change the master broker in a Message Queue cluster managed by GlassFish Server as an Embedded or Local JMS host. Instead, use the asadmin change-master-broker command as described in To Change the Master Broker in an Embedded or Local Broker Cluster in Oracle GlassFish Server 3.1-3.1.1 High Availability Administration Guide. |
Before You Begin
To ensure a successful dynamic changing of the master broker, verify that all brokers in the cluster are running before issuing the imqcmd changemaster cls command.
imqcmd changemaster cls -o imq.cluster.masterbroker=newMaster
The value newMaster has the form hostName:portNumber, where hostName and portNumber are the Port Mapper host name and port number, respectively, of the new master broker's host.
The broker returns one of the following status values for the operation:
The operation failed, and the cluster's configuration was unchanged. The old master broker is still the master broker for the cluster.
The operation failed. Use the imqcmd query bkr command on the old master broker to discover which broker is the master broker:
If the master broker listed is the old master broker, the failure occurred before the cluster's configuration change record was transferred to the new master broker. In this case, retry the command.
If the master broker listed is the new master broker, the cluster's configuration change record was transferred successfully to the new master broker, but some other activity failed later in the operation. In this case, stop all brokers in the cluster, manually update their configurations to refer to the new master broker, and then restart them all.
imqbrokerd -backup backupFile
Additionally, if necessary, update the imq.cluster.brokerlist property in the configurations for all brokers in the cluster.
imqbrokerd -restore backupFile
When using this command, specify as backupFile the file you saved in Step 2.
As noted earlier, a conventional cluster maintains a configuration change record to keep track of any changes in the cluster’s persistent state. This configuration change record is maintained either by the master broker or in a shared JDBC data store, depending on the type of the conventional cluster.
Because of the important information that the configuration change record contains, it is important to back it up regularly so that it can be restored in case of failure. Although restoring from a backup will lose any changes in the cluster’s persistent state that have occurred since the backup was made, frequent backups can minimize this potential loss of information. The backup and restore operations also have the positive effect of compressing and optimizing the change history contained in the configuration change record, which can grow significantly over time.
For example:
imqbrokerd -backup mybackuplogimqdbmgr backup sharecc_tbl -file fileName -Dimq.cluster.url=clusterConfigUrl
imqdbmgr recreate sharecc_tbl -Dimq.cluster.url=clusterConfigUrl
imqdbmgr restore sharecc_tbl -file fileName -Dimq.cluster.url=clusterConfigUrl
To convert between types of conventional clusters, you change where the configuration change record is maintained: in a master broker or in a shared JDBC data store. The following topics provide instructions to convert between types:
To Convert from Using a Master Broker to Using a Shared JDBC Data Store
To Convert from Using a Shared JDBC Data Store to Using a Master Broker
Set the imq.cluster.nomasterbroker property to true.
Set additional properties as described in Additional Properties for Conventional Clusters of Peer Brokers.
Set the imq.cluster.nomasterbroker property to false.
Set additional properties as described in Additional Properties for Conventional Clusters with Master Broker.
This section presents step-by-step procedures for performing a variety of administrative tasks for an enhanced cluster:
Because enhanced clusters are self-configuring, there is no need to explicitly specify the list of brokers to be included in the cluster. Instead, all that is needed is to set each broker’s configuration properties appropriately and then start the broker; as long as its properties are set properly, it will automatically be incorporated into the cluster. Enhanced Broker Cluster Properties describes the required properties, which include vendor-specific JDBC database properties.
Note - In addition to creating an enhanced cluster as described in this section, you must also configure clients to successfully reconnect to a failover broker in the event of broker or connection failure. You do this by setting the imqReconnectAttempts connection factory attribute to a value of -1.
The property values needed for brokers in an enhanced cluster can be set separately in each broker’s instance configuration file, or they can be specified in a cluster configuration file that all the brokers reference. The procedures are as follows:
The method best suited for production systems is to use a cluster configuration file to specify the configuration of the cluster.
Enhanced Broker Cluster Properties shows the required property values. However, do not include the imq.brokerid property in the cluster configuration file; this must be specified separately for each individual broker in the cluster.
The vendor-specific properties required for MySQL are shown in Example 8-1.
The first time a broker instance is run, an instance configuration file (config.properties) is automatically created.
Use the imqcmd shutdown bkr command.
In the broker’s instance configuration file, set the imq.cluster.url property to point to the location of the cluster configuration file you created in step 1.
Set the imq.brokerid property in the instance configuration file to the broker’s unique broker identifier. This value must be different for each broker.
Use the imqdbmgr create tbl command; see Database Manager Utility.
The brokers will automatically register themselves into the cluster on startup.
The first time a broker instance is run, an instance configuration file (config.properties) is automatically created.
Use the imqcmd shutdown bkr command.
Enhanced Broker Cluster Properties shows the required property values. Be sure to set the brokerid property uniquely for each broker.
The vendor-specific properties required for MySQL are shown in Example 8-1.
Use the imqdbmgr create tbl command; see Database Manager Utility.
The brokers will automatically register themselves into the cluster on startup.
Because enhanced clusters are self-configuring, the procedures for adding and removing brokers are simpler than for a conventional cluster.
You can do this either by specifying the individual properties in the broker’s instance configuration file (config.properties) or, if there is a cluster configuration file, by setting the broker’s imq.cluster.url property to point to it.
The broker will automatically register itself into the cluster on startup.
If necessary, use the command
imqcmd shutdown bkrto shut down the broker.
This command deletes all database tables for the corresponding broker.
After a broker has failed, you can restart it using the imqbrokerd command. Normally, the broker will automatically be re-registered into the cluster on startup.
However, if the broker slated to take over the failed broker's persistent data failed as it was taking over the persistent data, the running brokers in the cluster will not permit the failed broker to rejoin the cluster for 60 seconds or twice the value of imq.cluster.monitor.interval in seconds, whichever is greater.
Although the takeover of a failed broker’s persistent data by a failover broker in an enhanced cluster is normally automatic, there may be times when you want to prevent such failover from occurring. To suppress automatic failover when shutting down a broker, use the -nofailover option to the imqcmd shutdown bkr subcommand:
imqcmd shutdown bkr -nofailover -b hostName:portNumberwhere hostName and portNumber are the host name and port number of the broker to be shut down.
Conversely, you may sometimes need to force a broker failover to occur manually. (This might be necessary, for instance, if a failover broker were to itself fail before completing the takeover process.) In such cases, you can initiate a failover manually from the command line: first shut down the broker to be taken over with the -nofailover option, as shown above, then issue the command
imqcmd takeover bkr -n brokerIDwhere brokerID is the broker identifier of the broker to be taken over. If the specified broker appears to be running, the Command utility will display a confirmation message:
The broker associated with brokerID last accessed the database # seconds ago. Do you want to take over for this broker?You can suppress this message, and force the takeover to occur unconditionally, by using the -f option to the imqcmd takeover bkr command:
imqcmd takeover bkr -f -n brokerIDNote - The imqcmd takeover bkr subcommand is intended only for use in failed-takeover situations. You should use it only as a last resort, and not as a general way of forcibly taking over a running broker.
For durability and reliability, it is a good idea to back up an enhanced cluster’s shared data store periodically to backup files. This creates a snapshot of the data store that you can then use to restore the data in case of catastrophic failure. The command for backing up the data store is
imqdbmgr backup -dir backupDirwhere backupDir is the path to the directory in which to place the backup files. To restore the data store from these files, use the command
imqdbmgr restore -dir backupDirBefore restoring the data store, you should shut down all brokers in the enhanced cluster.
The best approach to converting a conventional broker cluster to an enhanced broker cluster is to drain your messaging system of all persistent data before attempting the conversion. This lets you create a new shared data store without worrying about loss of data. However, if you are using individual JDBC-based data stores for your brokers, a utility is available for converting a standalone datastore to a shared data store.
If the brokers in your conventional cluster are using file-based data stores, use the following procedure to convert to an enhanced cluster.
Stop all producer clients from producing messages, and wait for all messages in the system to be consumed.
See Enhanced Broker Cluster Properties. It is recommended that you use a cluster configuration file to specify cluster configuration property values, such as the imq.cluster.clusterid, imq.persist.store, and additional shared JDBC database properties.
Client re-connection behavior is specified by connection handling attributes of the connection factory administered objects (see the Connection Handling). In the case of enhanced broker clusters, the imqAddressList, imqAddressListBehavior, and imqAddressListIterations attributes are ignored, however the imqReconnectAttempts attribute should be set to a value of -1 (unlimited).
If the brokers in your conventional cluster are using JDBC-based data stores, use the following procedure to convert to an enhanced cluster. The procedure assumes that individual standalone broker data stores reside on the same JDBC database server.
Use proprietary JDBC database tools.
Use the Message Queue Database Manager utility (imqdbmgr) subcommand
imqdbmgr upgrade hastoreto convert an existing standalone JDBC database to a shared JDBC database.
See Enhanced Broker Cluster Properties. It is recommended that you use a cluster configuration file to specify cluster configuration property values, such as the imq.cluster.clusterid, imq.persist.store, and additional shared JDBC database properties.
Client re-connection behavior is specified by connection handling attributes of the connection factory administered objects (see the Connection Handling). In the case of enhanced broker clusters, the imqAddressList, imqAddressListBehavior, and imqAddressListIterations attributes are ignored, however the imqReconnectAttempts attribute should be set to a value of -1 (unlimited).