Maintaining Data Exporter

This section describes how to track the status of Data Exporter. This information is organized into the following topics:

• Monitoring Data Exporter

• Monitoring Logging

Monitoring Data Exporter

After the Exporter has been configured and is operational, you may choose to monitor it to ensure its continuous operation. The Exporter has several JMX beans that are useful for determining how the Exporter is behaving. The JMX beans include statistics on the average read/write rates for the Exporter, the current/maximum size of the internal memory queue, and the size of the persistent queue. The Exporter also produces audit records during export, one record for each cycle of each data type. The audit record includes how many records of the type were exported, and how long the export took.

Data Exporter provides the following JMX management beans that monitor the Exporter.

Data Exporter can be configured to queue export records to a queuing table during normal Waveset operation. Because the queue needs to potentially scale to a large number of records and survive a server restart, the queue is backed by a table in the Waveset repository. Since writes to the repository would typically slow down normal Waveset operations, the queue uses a small memory cache to buffer records in memory until they can be persisted in the repository.

The DataQueue MBean attributes can be plotted to show the largest number of records queued in memory (on a single Waveset server). On a balanced system, the number of records in the memory cache should be small and trend quickly to zero. If you observe this number get large (in the thousands) or not return to zero within a few seconds, you should investigate the write performance of the repository.

The ExportTask MBean contains two error counts, one for read and one for write. These counts should be zero, but there are a number of reasons that errors might occur, especially during write. The most common write error will result from the exported data not fitting within the warehouse table columns - typically a string overflow. Some exported String data is unbounded, where the export table columns must have some upper limit.

Monitoring Logging

Waveset has two sets of objects that grow without bounds: the audit log and the system log. Data Exporter addresses some of the maintenance problems associated with the log tables.

Audit Logs

Waveset writes immutable audit records to the audit log to serve as a historical audit trail of the operations it performs. Waveset uses these records in certain reports, and the data from the records may be displayed in the administrator interface. However, because the audit log grows without bounds and it grows at a modest rate, the deployer must determine when to truncate the audit log. Before Data Exporter, if you wanted to preserve the records prior to truncation, you were forced to dump the tables from the repository. If Data Exporter is enabled and configured to export log records, then the old records are preserved in the warehouse, and Waveset may truncate the audit tables as needed.

System Logs

System logs have the same immutable property that the audit logs have, but system logs are not typically generated as frequently. Data Exporter does not export system logs. To truncate the system log and preserve old records, you must dump the tables in the repository.