Monitoring Data Exporter

After the Exporter has been configured and is operational, you may choose to monitor it to ensure its continuous operation. The Exporter has several JMX beans that are useful for determining how the Exporter is behaving. The JMX beans include statistics on the average read/write rates for the Exporter, the current/maximum size of the internal memory queue, and the size of the persistent queue. The Exporter also produces audit records during export, one record for each cycle of each data type. The audit record includes how many records of the type were exported, and how long the export took.

Data Exporter provides the following JMX management beans that monitor the Exporter.

Data Exporter can be configured to queue export records to a queuing table during normal Identity Manager operation. Because the queue needs to potentially scale to a large number of records and survive a server restart, the queue is backed by a table in the Identity Manager repository. Since writes to the repository would typically slow down normal Identity Manager operations, the queue uses a small memory cache to buffer records in memory until they can be persisted in the repository.

The DataQueue MBean attributes can be plotted to show the largest number of records queued in memory (on a single Identity Manager server). On a balanced system, the number of records in the memory cache should be small and trend quickly to zero. If you observe this number get large (in the thousands) or not return to zero within a few seconds, you should investigate the write performance of the repository.

The ExportTask MBean contains two error counts, one for read and one for write. These counts should be zero, but there are a number of reasons that errors might occur, especially during write. The most common write error will result from the exported data not fitting within the warehouse table columns - typically a string overflow. Some exported String data is unbounded, where the export table columns must have some upper limit.