Six Things Everyone Should Know about JE Log Files

JE log files are not like the log files of other database systems. Nor are they like the log files or database files created by Berkeley DB C Edition. In this guide you will learn more about log files as you go along, but it is good to keep the following points in mind as you begin using JE.

  1. JE log files are "append only". Record insertions, deletions, and updates are always added at the end of the current file. The first file is named 00000000.jdb. When that file grows to a certain size (10 MB by default) a new file named 00000001.jdb becomes the current file, and so on.

  2. There are no separate database files. Unlike Berkeley DB C Edition, databases are not stored in files that are separate from the transaction log. The transaction log and the database records are stored together in a single sequential log consisting of multiple log files.

  3. The JE cleaner is responsible for reclaiming unused disk space. When the records in a log file are superseded by deletions or updates recorded in a later log file, the older log file is no longer fully utilized. The cleaner, which operates by default as a separate thread, determines the least utilized log files, copies any still utilized records in those files to the end of the current log file, and finally deletes the now completely un-utilized log file.

    See The Cleaner Thread for more information on the cleaner.

  4. Cleaning does not start immediately and never produces 100% utilization. Until you have written enough data to create several log files, and some of that data is obsoleted through deletions and updates, you will not notice any log files being deleted by the cleaner. By default cleaning occurs in the background and maintains the log files at 50% utilization. You can configure a higher utilization value, but configuring too high a utilization value will reduce overall performance.

  5. Cleaning is not automatically performed when closing the environment. If you wish to reduce unused disk space to a minimum at a particular point in time, you must explicitly call a method to perform log cleaning. See the Closing Database Environments for more information.

  6. Log file deletion only occurs after a checkpoint. The cleaner prepares log files to be deleted, but file deletion must be performed after a checkpoint to ensure that the files are no longer referenced. Checkpoints occur on their own schedule, which is every 20 MB of log written, by default. This is part of the reason that you will not see log files being deleted until after several files have been created.

    When using JE's replication (high availability) feature, a checkpoint does not delete the cleaned log file but instead instead changes the status of the file to reserved. Reserved files are only deleted when disk utilization exceeds the values set for either EnvironmentConfig.MAX_DISK or EnvironmentConfig.FREE_DISK.

  7. Log files can be spread across multiple directories, and therefore across multiple disks through the use of links or mount points. See Multiple Environment Subdirectories for more information.