test

Sun Java System Message Queue 4.1 Administration Guide

Persistence Services

For a broker to recover in case of failure, it needs to re-create the state of its message delivery operations. To do this, the broker must save state information to a persistent data store. When the broker restarts, it uses the saved data to re-create destinations and durable subscriptions, recover persistent messages, roll back open transactions, and rebuild its routing table for undelivered messages. It can then resume message delivery.

An important use for persistent data stores is for providing high availability message service. In this model, two or more brokers are joined together in a broker cluster sharing the same persistent store. If one of the brokers should fail, another broker within the cluster can then take over ownership of its messages and see that they are delivered to their destinations without interruption of service. (See Chapter 8, Broker Clusters and the Message Queue Technical Overviewfor more information on clusters and how they operate.)

Message Queue supports both file-based and JDBC-based persistence modules (see Figure 4–1). File-based persistence uses individual files to store persistent data; JDBC-based persistence uses the Java Database Connectivity (JDBC) interface to connect the broker to a JDBC-compliant data store. While file-based persistence is generally faster than JDBC-based, some users prefer the redundancy and administrative control provided by a JDBC-compliant store. The broker configuration property imq.persist.store (see Table 14–4) specifies which of the two forms of persistence to use.

Figure 4–1 Persistent Data Storage

Diagram showing that persistence services use either a flat file-based or a JDBC-based data store.

Note –

Changes in the file formats for both file-based and JDBC-based persistent data stores were introduced in Message Queue 3.7, with further JDBC changes in version 4.0. As a result of these changes, the persistent store version numbers were updated to 370 for file-based and 400 for JDBC-based stores. You can use the imqdbmgr query command (see Displaying Information About the Persistent Store) to determine the version number of your existing data store.

On first startup, the Message Queue Broker utility (imqbrokerd) will check for the presence of an older persistent store and automatically migrate it to the latest format:

This upgrade leaves the older copy of the persistent store intact, allowing you to roll back the upgrade if necessary. To do so, you can uninstall Message Queue 4.0 or 4.1 and reinstall the earlier version you were previously running. The older version’s message brokers will locate and use the older copy of the persistent store.

File-Based Persistence

By default, Message Queue uses a file-based persistent data store, in which individual files store persistent data such as messages, destinations, durable subscriptions, and transactions. Broker configuration properties related to file-based persistence are listed under File-Based Persistence Properties.

The file-based store is located in a directory identified by the name of the broker instance (instanceName) to which the data store belongs: 

   …/instances/instanceName/fs370

(See Appendix A, Platform-Specific Locations of Message Queue Data for the location of the instances directory.) Each destination on the broker has its own subdirectory holding messages delivered to that destination.

Note –

Because the persistent data store can contain messages of a sensitive or proprietary nature, you should secure the …/instances/instanceName/fs370 directory against unauthorized access; see Securing Persistent Data.

All persistent data other than messages is stored in separate files: one file each for destinations, durable subscriptions, and transaction state information. Most messages are stored in a single file consisting of variable-size records. You can compact this file to alleviate fragmentation as messages are added and removed (see Managing Physical Destination Disk Utilization). In addition, messages above a certain threshold size are stored in their own individual files rather than in the variable-sized record file. You can configure this threshold size with the broker property imq.persist.file.message.max_record_size.

The broker maintains a file pool for these individual message files: instead of being deleted when it is no longer needed, a file is returned to the pool of free files in its destination directory so that it can later be reused for another message. The broker property imq.persist.file.destination.message.filepool.limit specifies the maximum number of files in the pool. When the number of individual message files for a destination exceeds this limit, files will be deleted when no longer needed instead of being returned to the pool.

When returning a file to the file pool, the broker can save time at the expense of storage space by simply tagging the file as available for reuse without deleting its previous contents. You can use the imq.persist.file.message.filepool.cleanratio broker property to specify the percentage of files in each destination’s file pool that should be maintained in a “clean” (empty) state rather than simply marked for reuse. The higher you set this value, the less space will be required for the file pool, but the more overhead will be needed to empty the contents of files when they are returned to the pool. If the broker’s imq.persist.file.message.cleanup property is true, all files in the pool will be emptied at broker shutdown, leaving them in a clean state; this conserves storage space but slows down the shutdown process.

In writing data to the persistent store, the operating system has some leeway in whether to write the data synchronously or “lazily” (asynchronously). Lazy storage can lead to data loss in the event of a system crash, if the broker believes the data to have been written to persistent storage when it has not. To ensure absolute reliability (at the expense of performance), you can require that all data be written synchronously by setting the broker property imq.persist.file.sync.enabled to true. In this case, the data is guaranteed to be available when the system comes back up after a crash, and the broker can reliably resume operation. Note, however, that although the data is not lost, it is not available to any other broker in a cluster, since clustered brokers do not currently share data.

JDBC-Based Persistence

Instead of using file-based persistence, you can set up a broker to access any data store accessible through a JDBC-compliant driver. This involves setting the appropriate JDBC-related broker configuration properties and using the Database Manager utility (imqdbmgr) to create the proper database schema. See Configuring a JDBC-Based Store for specifics.

The properties for configuring a broker to use a JDBC database are listed in Table 14–6. You can specify these properties either in the instance configuration file (config.properties) of each broker instance or by using the -D command line option to the Broker utility (imqbrokerd) or the Database Manager utility (imqdbmgr).

The imq.persist.jdbc.dbVendor property identifies the database vendor being used for the persistent data store; all of the remaining properties are qualified by this vendor name. The imq.persist.jdbc.vendorName.driver property gives the Java class name of the JDBC driver to use in connecting to the database. There are also properties specifying the URLs for connecting to an existing database (imq.persist.jdbc.vendorName.opendburl), creating a new database (imq.persist.jdbc.vendorName.createdburl), and closing a database connection (imq.persist.jdbc.vendorName.closedburl).

The imq.persist.jdbc.vendorName.user and imq.persist.jdbc.vendorName.password properties give the user name and password for accessing the database, if required; imq.persist.jdbc.vendorName.needpassword is a boolean flag specifying whether a password is needed. For security reasons, the password should be specified only in a password file designated with the -passfile command line option; if no such password file is specified, the imqbrokerd and imqdbmgr commands will prompt for the password interactively. Similarly, the user name can be supplied from the command line using the -dbuser option to the imqbrokerd command or the -u option to imqdbmgr. If any additional, vendor-specific properties are required, you can set them by using broker properties of the form imq.persist.jdbc.vendorName.property.propName.

In a JDBC database shared by multiple broker instances, the configuration property imq.brokerid specifies a unique instance identifier for each, to be appended to the names of database tables. (This is usually unnecessary for an embedded database, which stores data for only one broker instance.)

Besides setting all of the broker’s needed JDBC configuration properties, you must also install your JDBC driver’s .jar file in the appropriate directory location, depending on your operating-system platform (as listed in Appendix A, Platform-Specific Locations of Message Queue Data) and then execute the Database Manager command 

   imqdbmgr create tbl

to create the database schema for the persistent data store.