Transaction Logging
TimesTen maintains a transaction log for each database to track all updates made within each transaction, so that those updates can be undone if the transaction is rolled back.
TimesTen recovers transactions using both the most recent checkpoint file and the most recent transaction log, which was written from the time of the last checkpoint operation after a system failure.
A transaction log record is created for each database modification, commit, and rollback. However, transaction log records are not generated for read-only transactions. Log records are first written to the transaction log buffer, which resides in the same shared memory segment as the database. The contents of the log buffer are then subsequently flushed to the latest transaction log file.
The transaction log is a logically ordered sequence of transaction log records, but physically consists of a set of one or more transaction log files and the in-memory log buffer within the TimesTen database. This log is shared by all concurrent connections.
The following sections describe how to manage and monitor the transaction log buffers and file:
Managing Transaction Log Buffers and Files
There are configuration options for transaction log buffers and files.
-
Transaction log buffers: There is one transaction log buffer for each database and the size of the transaction log buffer can be configured using the
LogBufMB
DSN connection attribute.Strands divide the transaction log buffer available memory into a number of different regions, which can be accessed concurrently by different connections. The number of transaction log buffer strands is configured with the
LogBufParallelism
connection attribute. After which, each connection can run data independent DML statements in parallel using those strands as if each has its own transaction log buffer. See LogBufParallelism in the Oracle TimesTen In-Memory Database Reference. -
Transaction log files: The transaction log files are created in the location that the
LogDir
connection attribute specifies.Note:
The
LogDir
connection attribute is optional and if it is not specified, the transaction log files are created in the same directory as the checkpoint files. However, for best performance, TimesTen recommends that you use theLogDir
connection attribute to place the transaction log files in a different physical device from the checkpoint files. If separated, I/O operations for checkpoints do not block I/O operations to the transaction log and vice versa.If the database is already loaded into RAM and the transaction log files and checkpoint files for your database are on the same physical device, TimesTen writes a message to the daemon log file.
See LogDir in the Oracle TimesTen In-Memory Database Reference and Check Transaction Log File Use of File System Space in the Oracle TimesTen In-Memory Database Monitoring and Troubleshooting Guide.
You can configure the maximum size for the transaction log files with the
LogFileSize
DSN connection attribute. See LogFileSize in the Oracle TimesTen In-Memory Database Reference.The transaction log file names have the form
dsname
.log
n
. Thedsname
is the database path name that is specified by theDataStore
DSN connection attribute and is provided within the database's DSN. The suffixn
is the transaction log file number, starting at zero.When the database is created, TimesTen creates reserve files named
dsname
.res0
,dsname
.res1
, anddsname
.res2
. These files contain pre-allocated space that serves as reserved transaction log space. Reserved transaction log space allows for a limited continuation of transaction logging if the file system that holds the transaction log files becomes full. If the file system that contains the transaction logs becomes full, the database does not allow any new transactions to begin. Transactions that are in progress are allowed to continue by using the reserve files until they either commit or rollback. If there are transactions that are still in progress and the reserve files are all consumed, all database operations that generate log files are blocked.
Using NFS-Mounted Systems for Checkpoint and Transaction Log Files
To enable TimesTen checkpoint and transaction log files to be located on NFS-mounted
file systems (on Linux platforms platforms only), ensure that
allow_network_files=1
in the timesten.conf
file. The
allow_network_files=1
configuration is the default setting.
See TimesTen Instance Configuration File in Oracle TimesTen In-Memory Database Reference.
Monitoring Accumulation of Transaction Log Files
It is important to verify at frequent intervals that there are no transaction log holds that could result in an excessive accumulation of transaction log files.
If too many transaction log files accumulate and fill up available file system space, new transactions in the TimesTen database cannot begin until the transaction log hold is advanced and transaction log files are purged by the next checkpoint operation.
The following sections describe transaction log operations, log holds, and accumulation of log files:
Log Holds by TimesTen Components or Operations
Several TimesTen components or operations can cause transaction log holds. A transaction log hold prevents log files, beyond a certain point, from being purged until they are no longer needed.
In standard circumstances, the log hold position is regularly advanced and log files are purged appropriately. However, if operations are not functioning properly and the hold position does not advance, there can be an excessive accumulation of log files beyond the hold position that can no longer be purged, which eventually fills available file system space.
These components and operations include the following:
-
Transactions writing log records to the transaction log file have been committed or rolled back. These can be either local database transactions or XA transactions.
-
Changes recorded in the transaction log file have been written to both checkpoint files.
-
Replication: There is a transaction log hold until the transmitting replication agent confirms that the log files have been fully processed by the receiving host.
Possible failure modes include the following:
-
The network is down or there is a standby crash and replication is unable to deliver data to one or more subscribers. If necessary, the application can direct that logs no longer be held, then duplicate the master database to the standby when standard operations resume. Criteria for when to do this includes the amount of time required to duplicate, the amount of available file system space on the master for log files, and the transaction log growth rate.
-
The overall database transaction rate exceeds the ability of replication to keep the active and standby databases synchronized. An application can reduce the application transaction rate or the number of replicated tables.
See Improving Replication Performance in Oracle TimesTen In-Memory Database Replication Guide and Troubleshooting Replication in Oracle TimesTen In-Memory Database Monitoring and Troubleshooting Guide.
-
-
XLA: There is a transaction log hold until the XLA bookmark advances.
A possible failure mode occurs when the bookmark becomes stuck, which can occur if an XLA application terminates unexpectedly or if it disconnects without first deleting its bookmark or disabling change-tracking. If a bookmark gets too far behind, the application can delete it. If the XLA reader process is still active, it must first be terminated, so that another XLA process can connect and delete the bookmark.
-
Active standby pairs that replicate AWT cache groups: There is a transaction log hold until the replication agent confirms that the transaction corresponding to the log hold has been committed on the Oracle Database. With an active standby pair, the active database typically receives the confirmation from the standby database. If the standby database is down, the replication agent receives confirmation from Oracle Database directly.
Possible failure modes include the following:
-
Oracle Database is down or there is a lock or resource contention.
-
The network is down, slow, or saturated.
-
With an active standby pair, replication to the standby database falls behind. Check log holds on the standby database.
-
The transaction rate to TimesTen exceeds the maximum sustainable rate that TimesTen can propagate to Oracle Database.
See Monitoring AWT Cache Groups in Oracle TimesTen In-Memory Database Cache Guide and Troubleshooting AWT Cache Groups in Oracle TimesTen In-Memory Database Monitoring and Troubleshooting Guide.
-
-
Cache groups configured with
AUTOREFRESH
: There is a transaction log hold until the replication agent on the active database confirms the log files have been fully processed by the standby database.Possible failure modes include the following:
-
Replication from the active database to the standby database is impacted because the standby database falls behind due to large workloads resulting from
AUTOREFRESH
mode. -
The standby database is down or recovering, but has not been marked as
FAILED
through a call, initiated by either the user application or Oracle Clusterware, to thettRepStateSave
built-in procedure. The active database does not take over propagation to the Oracle Database until the state of the standby database is marked asFAILED
. While the standby database is down or recovering, transaction log files are held for the Oracle Database.
See Monitoring Cache Groups with Autorefresh in Oracle TimesTen In-Memory Database Monitoring and Troubleshooting Guide.
-
-
Incremental TimesTen backup: There is a log hold until you manually run an incremental backup and until that backup completes.
The most likely failure is if you configure incremental backups (which starts TimesTen holding log files), but you do not actually run an incremental backup, which would free the log file hold.
Another possible failure mode can occur if the incremental backup falls too far behind the most recent entries in the transaction log. For example, ensure that an unexpected burst of transaction activity cannot fill up available transaction log file system space due to the backup holding a log file that is too old. An application can perform another incremental backup to work around this situation.
-
Long-running transaction or XA transaction: There is a transaction log hold until the transaction commits or rolls back.
A possible failure mode can occur if an application transaction does not commit or rollback for a long time, so that it becomes necessary for the application to terminate the long-running transaction.
Alternatively, you can rollback a transaction using the
ttXactAdmin
utility with the-xactIdRollback
option. See ttXactAdmin in Oracle TimesTen In-Memory Database Reference. -
Oldest transaction undo: Each transaction that has not yet committed or rolled back establishes a log hold. This hold starts with the first (oldest) log record of the transaction and marks a point for the transaction that the checkpoint log purge operation cannot pass. You can retrieve this log hold information from the
SYS.GV$CKPT_HISTORY
orSYS.V$CKPT_HISTORY
system views or from the output of thettCkptHistory
built-in procedure.
Purging Transaction Log Files
Any transaction log file is kept until TimesTen determines it can be purged. Under standard TimesTen operating conditions, unneeded transaction log files are purged each time a checkpoint is initiated.
A checkpoint can be initiated either through a configurable time interval with the
CkptFrequency
connection attribute or a configurable log volume
with the CkptLogVolume
connection attribute. In TimesTen Classic, you
can also initiate a checkpoint (either manually or in a background checkpointing
application thread) with the ttCkpt
built-in function.
If you are running out of file system space because of log files accumulating, use the CkptLogVolume
connection attribute instead of the CkptFrequency
connection attribute. In addition, you can tell if reclamation is blocked when frequently selecting from the SYS.GV$LOG_HOLDS
or SYS.V$LOG_HOLDS
system views or calling the ttLogHolds
built-in procedure.
Note:
To improve performance, locate your log files on a separate physical device from the one on which the checkpoint files are located. See Managing Transaction Log Buffers and Files.
See Checkpointing in the Oracle TimesTen In-Memory Database
Introduction for general information on
checkpointing. See Configuring or Turning Off Background Checkpointing for more details on CkptFrequency
and
CkptLogVolume
connection attributes.
For more information on log holds, see SYS.GV$LOG_HOLDS or SYS.V$LOG_HOLDS in the Oracle TimesTen In-Memory Database SQL Reference or ttLogHolds in the Oracle TimesTen In-Memory Database Reference.
See CkptFrequency, CkptLogVolume, and ttCkpt in the Oracle TimesTen In-Memory Database Reference.
Monitoring Log Holds and Log File Accumulation
There are options for periodic monitoring of excessive transaction log accumulation.
-
For details of all log holds, select from the
SYS.GV$LOG_HOLDS
orSYS.V$LOG_HOLDS
system views or call thettLogHolds
built-in procedure. The information includes the following, as applicable:-
Log file number, the offset of the hold position, and the type of hold, which can be checkpoint, replication, backup, XLA, long-running transaction, or long-running XA transaction
-
Name of the checkpoint file for a checkpoint hold
-
Name of the subscriber and the parallel track ID it uses for replication
-
Backup path for a backup hold
-
Name of the persistent subscription and process ID of the last process to open it for XLA
-
Transaction ID for a long-running transaction
-
XA XID for a long-running XA transaction
See SYS.GV$LOG_HOLDS or SYS.V$LOG_HOLDS in the Oracle TimesTen In-Memory Database SQL Reference or ttLogHolds in the Oracle TimesTen In-Memory Database Reference.
-
-
Select from the
SYS.GV$CKPT_HISTORY
orSYS.V$CKPT_HISTORY
system views or call thettCkptHistory
built-in procedure to check the last several checkpoints to confirm none of the returned rows has a status ofFAILED
.See SYS.GV$CKPT_HISTORY or SYS.V$CKPT_HISTORY in the Oracle TimesTen In-Memory Database SQL Reference or ttCkptHistory in the Oracle TimesTen In-Memory Database Reference.
-
Check the
SYS.SYSTEMSTATS
table for operational metrics. Each transaction log file has a unique sequence number, which starts at 0 for the first log file and increments by 1 for each subsequent log file. The number of the current log file is available inSYS.SYSTEMSTATS.log.file.latest
. The number of the oldest log file not yet purged is available inSYS.SYSTEMSTATS.log.file.earliest
. You should raise an error or warning if the difference in the sequence numbers exceeds an inappropriate threshold.See SYS.SYSTEMSTATS in the Oracle TimesTen In-Memory Database System Tables and Views Reference.
-
For XLA, check the
SYS.TRANSACTION_LOG_API
table that provides bookmark information, such as the process ID of the connected application, which could help diagnose the reason why a bookmark may be stuck or lagging.See SYS.TRANSACTION_LOG_API in the Oracle TimesTen In-Memory Database System Tables and Views Reference.