Verify that the broker has a backlog of undelivered messages.
Examine the broker's persistent message store in the appropriate directory for your operating system:
On Solaris: /var/imq/instances/psw-broker/filestore/message/
On Linux: /var/opt/sun/mq/instances/isw-broker/filestore/message/
On Windows: installation_root\isw-machine_name\imq\var\instances\isw-broker\filestore\message\
Each file in this directory contains a single undelivered message. If there are more than 10,000 files in this directory, then the broker has a backlog of messages. Otherwise, an undelivered message backlog is not causing the problem with the broker.
The message backlog usually contains log files related to an idsync resync operation and you can safely remove them.
Stop the Message Queue broker.
For more information, see Starting and Stopping Services in Sun Java System Directory Server Enterprise Edition 6.3 Installation Guide.
Remove all files in the persistent message store.
The easiest way to remove these files is by recursively removing the message/ directory and then recreating it.
Restart the Message Queue broker.
To avoid running out of memory in the future, take the steps described earlier in this section.