1. Administrator's Guide
  2. Administrative Procedures
  3. Monitoring the Store

Monitoring the Store

You can obtain Information about the performance and availability of your store from both server side and client side perspectives:

  • Your Oracle NoSQL Database applications can obtain performance statistics using the oracle.kv.KVStore.getStats() class. This provides a client side view of the complete round trip performance for Oracle NoSQL Database operations.

  • Oracle NoSQL Database automatically captures Replication Node performance statistics into a log file that you can into into spreadsheet software for analysis. The store tracks, logs, and writes statistics at a user specified interval to a CSV file. The file is je.stat.csv, located in the Environment directory. Logging occurs per-Environment when the Environment is opened in read/write mode.

    Configuration parameters control the size and number of rotating log files to use (similar to java logging, see java.util.logging.FileHandler). For a rotating set of files, as each file reaches a given size limit, it is closed, rotated out, and a new file is opened. Successively older files are named with an incrementing numeric suffix to the file name. The name format is je.stat[version].csv.

  • The Oracle NoSQL Database administrative service collects and aggregates status information, alerts, and performance statistics components that the store generates. This provides a detailed view of the behavior and performance of the Oracle NoSQL Database server.

  • Each Oracle NoSQL Database Storage Node maintains detailed logs of trace information from the services that the node supports. The administrative service presents an aggregated, store-wide view of these component logs. Logs are available on each Storage Node in the event that the administrative service is not available, or if it is more convenient to examine individual Storage Node logs.

  • Oracle NoSQL Database supports the optional Java Management Extensions (JMX) agents for monitoring. The JMX interfaces allow you to poll the Storage Nodes for information about the storage node and any replication nodes that it hosts. For more information on JMX monitoring, see Standardized Monitoring Interfaces. For information on using JMX securely, see Guidelines for using JMX securely in the Security Guide.

You can monitor the status of the store by verifying it from within the CLI. See Verifying the Store. You can also use the CLI to examine events.

Events

Events are special messages that inform you of the state of your system. As events are generated, they are routed through the monitoring system so that you can see them. There are four types of events that the store reports:

  1. State Change events are issued when a service starts up or shuts down.

  2. Performance events report statistics about the performance of various services.

  3. Log events are records produced by the various system components to provide trace information about debugging. These records are produced by the standard java.util.logging package.

  4. Plan Change events record the progress of plans as they execute, are interrupted, fail or are canceled.

Note:

Some events are considered critical. These events are recorded in the administration service's database, and can be retrieved and viewed using the CLI.

You cannot view Plan Change events directly through Oracle NoSQL Database's administrative interfaces. However, State Change events, Performance events, and Log events are recorded using the EventRecorder facility, which is internal to the Admin. Only events considered critical are recorded, and the criteria for being designated as such varies with the event type. These are the events considered critical:

  • All state changes.
  • Log events classified as SEVERE.
  • Any performance events reported as below a certain threshold.

You can view all of these critical events using the Admin CLI show events and show event commands.

Use the CLI show events command with no arguments to see all of the unexpired events in the database. Use the -from and -to arguments to limit the range of events that display. Use the -type or -id arguments to filter events by type or id, respectively.

For example, this is part of the output from a show events command: 

kv-> show events
idarpdfbS STAT 2015-08-13 22:18:39.287 UTC sn1 RUNNING sev1
idarpeg0S STAT 2015-08-13 22:18:40.608 UTC sn2 RUNNING sev1
idarphmuS STAT 2015-08-13 22:18:44.742 UTC rg1-rn1 RUNNING sev1
idarpjLLS STAT 2015-08-13 22:18:47.289 UTC rg1-rn2 RUNNING sev1
idartfcuS STAT 2015-08-13 22:21:48.414 UTC rg1-rn2 UNREACHABLE sev2
                                                   (reported by admin1)

This result shows four service state change events (sev1) and one log event (UNREACHABLE), classified as sev2. Tags at the beginning of each line are individual event record identifiers. To see detailed information for a particular event, use the show event command, which takes an event record identifier, such as idartfcuS as its argument: 

kv-> show event -id idartfcuS
idartfcuS STAT 2015-08-13 22:21:48.414 UTC rg1-rn2 UNREACHABLE sev2
  (reported by admin1)

Using this method of event identifiers, you can see a complete stack trace.

Events are removed from the system if the total number of events is greater than a set maximum number, or if the Event is older than a set period. The default maximum number of events is 10,000, while the default time period is 30 days.

Both Sev1 and Sev2 flags are associated with specific service state change events. Sev1 flags report the current state. Sev2 flags report errors during attempted state changes, as follows:

Sev1 Flags Sev2 Flags
STARTING ERROR_RESTARTING
WAITING_FOR_DEPLOY ERROR_NO_RESTART
RUNNING UNREACHABLE
STOPPING  
STOPPED