Library Version 22.214.171.124, Release 6.0.11
The open source license for JE has been changed. The open source license for JE 6.0 is the GNU AFFERO GENERAL PUBLIC LICENSE. See the LICENSE file for details. Commercial licensees are not impacted by this change.
The Java compatibility requirement for JE has changed from Java SE 6 to 7. JE is compatible with Java SE 7 and later, and has been tested and certified against Oracle JDK 7u51 and IBM J9 7.0. We encourage you to upgrade to the latest Java releases to take advantage of the latest bug fixes and performance improvements.
The change is forward compatible in that JE files created with release 5.0 and earlier can be read when opened with JE 6.0 or later. The change is not backward compatible in that files created with JE 6.0 or later cannot be read by earlier releases. Note that if an existing environment is opened read/write, a new log file is written by JE 6.0 or later, and the environment can no longer be read by earlier releases.
One of two utility programs must be used, which are available in the release package for JE 4.1.20, or a later release of JE 4.1. If you are currently running a release earlier than JE 4.1.20, then you must download the latest JE 4.1 release package in order to run these utilities.
The steps for upgrading are as follows.
java -jar je-4.1.20.jar DbPreUpgrade_4_1 -h <dir>If you are using a JE
java -jar je-4.1.20.jar DbRepPreUpgrade_4_1 -h <dir> -groupName <group name> -nodeName <node name> -nodeHostPort <host:port>
The second step -- running the utility program -- does not perform data conversion. This step simply performs a special checkpoint to prepare the environment for upgrade. It should take no longer than an ordinary startup and shutdown.
During the last step -- when the application opens the JE environment using the
current release (JE 5 or later) -- all databases configured for duplicates will
automatically be converted before the
ReplicatedEnvironment constructor returns. Note that a database
might be explicitly configured for duplicates using
DatabaseConfig.setSortedDuplicates(true), or implicitly configured
for duplicates by using a DPL MANY_TO_XXX relationship
The duplicate database conversion only rewrites internal nodes in the Btree, not leaf nodes. In a test with a 500 MB cache, conversion of a 10 million record data set (8 byte key and data) took between 1.5 and 6.5 minutes, depending on number of duplicates per key. The high end of this range is when 10 duplicates per key were used; the low end is with 1 million duplicates per key.
To make the duplicate database conversion predictable during deployment, users
should measure the conversion time on a non-production system before upgrading
a deployed system. When duplicates are converted, the Btree internal nodes are
preloaded into the JE cache. A new configuration option,
EnvironmentConfig.ENV_DUP_CONVERT_PRELOAD_ALL, can be set to false
to optimize this process if the cache is not large enough to hold the internal
nodes for all databases. For more information, see the javadoc for this
If an application has no databases configured for duplicates, then the last step simply opens the JE environment normally, and no data conversion is performed.
If the user fails to run the DbPreUpgrade_4_1 or DbRepPreUpgrade_4_1 utility
program before opening an environment with JE 5 or later for the first time, an
exception such as the following will normally be thrown by the
com.sleepycat.je.EnvironmentFailureException: (JE 6.0.1) JE 4.1 duplicate DB entries were found in the recovery interval. Before upgrading to JE 5.0, the following utility must be run using JE 4.1 (4.1.20 or later): DbPreUpgrade_4_1. See the change log. UNEXPECTED_STATE: Unexpected internal state, may have side effects. at com.sleepycat.je.EnvironmentFailureException.unexpectedState(EnvironmentFailureException.java:376) at com.sleepycat.je.recovery.RecoveryManager.checkLogVersion8UpgradeViolations(RecoveryManager.java:2694) at com.sleepycat.je.recovery.RecoveryManager.buildTree(RecoveryManager.java:549) at com.sleepycat.je.recovery.RecoveryManager.recover(RecoveryManager.java:198) at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:610) ...
If the user fails to run the DbPreUpgrade_4_1 or DbRepPreUpgrade_4_1 utility
program, but no exception is thrown when the environment is opened with JE 5
or later, this is probably because the application performed an
Environment.sync before last closing the environment with JE 4.1
or earlier, and nothing else happened to be written (by the application or JE
background threads) after the sync operation. In this case, running the
upgrade utility is not necessary.
Changes include adding the
enumeration constant, and the
ReplicationGroup.getDataNodes methods. [#22482] (6.0.1)
As part of the performance improvement work, the following statistics were added.
nCachedBINDeltas: EnvironmentStats.getNCachedBINDeltas-- Number of BIN-deltas (partial BINs) in cache.
nBINDeltasFetchMiss: EnvironmentStats.getNBINDeltasFetchMiss-- Number of BIN-deltas fetched to satisfy btree operations.
nBINsMutated: EnvironmentStats.getNBINsMutated-- The number of BINs mutated to BIN-deltas by eviction.
lastCheckpointInterval: EnvironmentStats.getLastCheckpointInterval-- Byte length from last checkpoint start to the previous checkpoint start.
In addition, the EnvironmentConfig.TREE_MAX_DELTA param has been deprecated. As of JE 5.0, the benefit from logging BIN-deltas is unrelated to the number of deltas that have been logged since the last full BIN. To configure BIN-delta logging, use EnvironmentConfig.TREE_BIN_DELTA.
As described under 'Upgrading from JE 5.0 or earlier' at the top of this document, to support this cleaner optimization a change was made involving partial Btree and duplicate comparators. Partial comparators are an advanced feature that few applications use. As of JE 6.0, using partial comparators is not recommended. Applications that do use partial comparators must now change their comparator classes to implement the new PartialComparator tag interface, before running the application with JE 6. Failure to do so may cause incorrect behavior during transaction aborts. See the PartialComparator javadoc for more information.
ReplicationConfig.REP_STREAM_TIMEOUTparameter. The system does not store information about replication progress for secondary replicas, though, so a different approach has been added.
The modified algorithm estimates the costs of replication replay and network restore, and protects log files from deletion that could be used for replay if there is sufficient disk space and replay would be less expensive than network restore. These computations apply to all replicas, but are particularly useful for secondary replicas, for which log files will not otherwise be retained if the replicas become temporarily unreachable. Note that disk space calculations are only performed when running with Java 7 or later.
ReplicationConfig parameters were added:
REPLAY_COST_PERCENT- The cost of replaying the replication stream as compared to the cost of performing a network restore.
REPLAY_FREE_DISK_PERCENT- The target amount of free disk space to maintain when selecting log files to retain for use in replay.
To prevent these problems, the size of each logged record is now stored in the Btree BINs (bottom internal nodes), so that utilization can be calculated correctly during record updates and deletions, while still avoiding a fetch of the old version of the record. With this change, the utilization adjustment facility in the log cleaner, which attempted to compensate for this problem by estimating utilization, is no longer needed by most applications.
Therefore the EnvironmentConfig.CLEANER_ADJUST_UTILIZATION parameter is now false by default rather than true, and will be disabled completely in a future version of JE. For more information, see the javadoc for this parameter.
In addition, the new eviction approach implements a more accurate LRU which ensures that dirty nodes are evicted last and thereby reduces unnecessary logging.
As part of this change, the following configuration parameters were deprecated and are ignored by JE:
EnvironmentConfig.EVICTOR_NODES_PER_SCAN EnvironmentConfig.EVICTOR_LRU_ONLYAnd the following configuration parameter was added:
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 5.0.97) node2(2):foo\node2 Read invisible log entry at 0x0/0xcb776 hdr type="INS_LN_TX/8" vlsn v="19,373" isReplicated="1" isInvisible="1" prev="0xcb74c" size="17" cksum="2626620732" LOG_INTEGRITY: Log information is incorrect, problem is likely persistent. fetchTarget of 0x0/0xcb776 parent IN=29 IN class=com.sleepycat.je.tree.BIN lastFullVersion=0x0/0xf154c lastLoggedVersion=0x0/0xf588e parent.getDirty()=true state=3 at com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:1054) at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:906) at com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:867) at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1427) at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1250) at com.sleepycat.je.recovery.RecoveryManager.undo(RecoveryManager.java:2415) at com.sleepycat.je.recovery.RecoveryManager.rollbackUndo(RecoveryManager.java:2268) ...[#22848] (6.0.10)