Critical Event Logging

When critical events occur, TimesTen collects the daemon log entries at the moment of the critical event to assist when diagnosing critical failures.

An example of critical events include database failure or a log-based catch up failure. Database invalidation occurs when TimesTen detects that the data in the database is corrupt and therefore unusable.

  • TimesTen generates critical event files containing a snippet of the daemon log file that was collected at the moment of the critical event. Critical event log files are created and stored in the same directory as the daemon log files (specified by supportlog configuration attribute in the timesten.conf file). Critical event log files are named with the format of ttmesg.log.ts_timestamp.gz where timestamp is the current time on the host.

  • When a critical event occurs, TimesTen records in the daemon log file that critical event information was collected and the name of the critical event log file.

  • If a database invalidation occurs, then TimesTen records in the .inval file that critical event information was collected at the time of the database invalidation and the name of the critical event log file. The .inval file is located in the same directory configured as the DataStore directory.

If more than one critical event occurs in quick succession, TimesTen collects only new information for each subsequent event in the next critical event log file.

You can configure the duration of the daemon log collection with the daemon_log_snippet_interval_in_mins configuration attribute in the timesten.conf file. As with all changes to the timesten.conf file, you must restart the main daemon for any change to take effect. See the TimesTen Instance Configuration File in the Oracle TimesTen In-Memory Database Reference.

The occurrence of a critical event and the name of the critical event log file are noted in the daemon log file. The following example shows the message you would see in the daemon log file if two critical events occur in quick succession.

17:08:00.224 Err : : 12532: A critical event has happened. Saving last 600 seconds 
snippet of daemon log at /timesten/instance/diag/ttmesg.log.ts_1568160480.gz.
17:08:00.643 Err : : 12532: Read 543878 bytes from daemon log file and wrote into
 the daemon log snippet file located at /timesten/instance/diag/ttmesg.log.ts_1568160480.gz
 
17:11:12.657 Err : : 12532: A critical event has happened. Saving last 600 seconds
 snippet of daemon log at /timesten/instance/diag/ttmesg.log.ts_1568160672.gz.
17:11:12.657 Err : : 12532: Part of the daemon log snippet has already been
 recorded in a prior snippet file ending with timestamp 1568160480.gz
17:11:13.077 Err : : 12532: Read 2678582 bytes from daemon log file and wrote
 into the daemon log snippet file located at
 /timesten/instance/diag/ttmesg.log.ts_1568160672.gz

After which, you can evaluate the collected log records in the specified ttmesg.log.ts_timestamp.gz files.

Any database invalidation event and the name of the critical event log file are noted in the .inval file. The following example shows the messages you could see in the .inval file if two critical events occur in quick succession. 

2019-09-10 17:08:00.752
Hostname: myhost
Invalidated data store: /timesten/instance/datastores/mydb
Data store created: 2019/09/10 17:04:25
TimesTen Release 18.1.3.1.0 (Linux x86-64, 64-bit dbg) (myhost)
Data store created by Release 18.1.3.1.0 (Linux x86-64, 64-bit dbg) (myhost)
Source: Data store marked invalid by master daemon: grid LBCU pre-condition check failed
A critical event has happened. Saving last 600 seconds snippet of daemon log at 
/timesten/instance/diag/ttmesg.log.ts_1568160480.gz.
Read 953765 bytes from daemon log file and wrote into the daemon log snippet file 
located at /timesten/instance/diag/ttmesg.log.ts_1568160480.gz
 
2019-09-10 17:11:13.905
Hostname: myhost
Invalidated data store: /timesten/instance/datastores/mydb
Data store created: 2019/09/10 17:04:25
TimesTen Release 18.1.3.1.0 (Linux x86-64, 64-bit dbg) 
Data store created by Release 18.1.3.1.0 (Linux x86-64, 64-bit dbg) 
Source: Data store marked invalid by master daemon: subdaemon managing database 
exited or died
A critical event has happened. Saving last 600 seconds snippet of daemon log at 
/timesten/instance/diag/ttmesg.log.ts_1568160673.gz.
Part of the daemon log snippet has already been recorded in a prior snippet file 
ending with timestamp ts_1568160480.gz
Read 781026 bytes from daemon log file and wrote into the daemon log snippet file 
located at /timesten/instance/diag/ttmesg.log.ts_1568160673.gz

After which, you can evaluate the collected log records in the specified ttmesg.log.ts_timestamp.gz files.