Berkeley DB 4.8.28 Change Log

Changes between 4.8.26 and 4.8.28:
Known bugs in 4.8
Changes between 4.8.24 and 4.8.26:
Changes between 4.8.21 and 4.8.24:
Changes between 4.7 and 4.8.21:
Database or Log File On-Disk Format Changes:
New Features:
Database Environment Changes:
Concurrent Data Store Changes:
General Access Method Changes:
Btree Access Method Changes:
Hash Access Method Changes:
Queue Access Method Changes:
Recno Access Method Changes:
C-specific API Changes:
C++-specific API Changes:
Java-specific API Changes:
Direct Persistence Layer (DPL), Bindings and Collections API:
Tcl-specific API Changes:
RPC-specific Client/Server Changes:
Replication Changes:
XA Resource Manager Changes:
Locking Subsystem Changes:
Logging Subsystem Changes:
Memory Pool Subsystem Changes:
Mutex Subsystem Changes:
Test Suite Changes
Transaction Subsystem Changes:
Utility Changes:
Configuration, Documentation, Sample Application, Portability and Build Changes:

Changes between 4.8.26 and 4.8.28:

  1. Limit the size of a log record generated by freeing pages from a database so it fits in the log file size. [#17313]

  2. Fix a bug that could cause a file to be removed if it was both the source and target of two renames within a transaction. [#18069]

  3. Modified how we go about selecting a usable buffer in the cache. Place more emphasis on single version and obsolete buffers. [#18114]

Known bugs in 4.8

  1. Sharing logs across mixed-endian systems does not work.[#18032]

Changes between 4.8.24 and 4.8.26:

  1. Fixed a bug where the truncate log record could be too large when freeing too many pages during a compact. [#17313]

  2. Fixed a bug where the deadlock detector might not run properly. [#17555]

  3. Fixed three bugs related to properly detecting thread local storage for DbStl. [#17609] [#18001] [#18038]

  4. Fixed a bug that prevented some of our example code from running correctly in a Windows environment. [#17627]

  5. Fixed a bug where a "unable to allocate space from buffer cache" error was improperly generated. [#17630]

  6. Fixed a bug where DB->exists() did not accept the DB_AUTO_COMMIT flag. [#17687]

  7. Fixed a bug where DB_TXN_SNAPSHOT was not getting ignored when DB_MULTIVERSION not set. [#17706]

  8. Fixed a bug that prevented callback based partitioning through the Java API. [#17735]

  9. Fixed a replication bug where log files were not automatically removed from the client side. [#17899]

  10. Fixed a bug where code generated from db_sql stored the key in both the data and key DBTs. [#17925]

  11. Fixed a bug that prevented a sequence from closing properly after the EntityStore closed. [#17951]

  12. Fixed a bug where gets fail if the DB_GET_BOTH_FLAG is specified in a hash, sorted duplicates database.[#17997]

Changes between 4.8.21 and 4.8.24:

  1. Fixed a bug in the C# API where applications in a 64-bit environment could hang. [#17461]

  2. Fixed a bug in MVCC where an exclusive latch was not removed when we couldn't obtain a buffer. [#17479]

  3. Fixed a bug where a lock wasn't removed on a non-transactional locker. [#17509]

  4. Fixed a bug which could trigger an assertion when performing a B-tree page split and running out of log space or with MVCC enabled. [#17531]

  5. Fixed a bug in the repquote example that could cause the application to crash. [#17547]

  6. Fixed a couple of bugs when using the GCC 4.4 compiler to build the examples and the dbstl API. [#17504] [#17476]

  7. Fixed an incorrect representation of log system configuration info. [#17532]

Changes between 4.7 and 4.8.21:

Database or Log File On-Disk Format Changes:

  1. The log file format changed in 4.8.

New Features:

  1. Improved scalability and throughput when using BTree databases especially when running with multiple threads that equal or exceed the number of available CPUs.

  2. Berkeley DB has added support for C#. In addition to the new C# api, C# specific tests and sample applications were also added. [#16137]

  3. Berkeley DB has added an STL API, which is compatible with and very similar to C++ Standard Template Library (STL). Tests and sample applications and documentation were also added. [#16217]

  4. Berkeley DB has added database partitioning. BTree or Hash databases may now be partitioned across multiple directories. Partitioned databases can be used to increase concurrency and to improve performance by spreading access across disk subsystems. [#15862]

  5. Berkeley DB now supports bulk insertion and deletion of data. Similar to the bulk get interface, the bulk put and bulk delete allow the developer to populate a buffer of key-value pairs and then pass it to the BDB library with a single API call.

  6. Berkeley DB now supports compression when using BTree.

  7. Berkeley DB introduces a new utility named db_sql which replaces db_codegen. Similar to db_codegen, db_sql accepts an input file with DDL statements and generates a Berkeley DB application using the C API that creates and performs CRUD operations on the defined tables. The developer can then use that code as a basis for further application development.

  8. The Replication Manager now supports shared access to the Master database environment from multiple processes. In earlier versions, multiple process support on the Master required use of the Base Replication API. [#15982]

  9. Foreign Key Support has been added to Berkeley DB.

  10. Several enhancements were made to DB_REGISTER & DB_ENV->failchk().

  11. Berkeley now supports 100% in-memory replication.

  12. Berkeley DB now has the ability to compare two cursors for equality. [#16811]

Database Environment Changes:

  1. Fixed a bug that could cause an allocation error while trying to allocate thread tracking information for the DB_ENV->failcheck system. [#16300]

  2. Fixed a bug that could cause a trap if an environment open failed and failchk thread tracking was enabled.[#16770]

Concurrent Data Store Changes:

None.

General Access Method Changes:

  1. Fixed a bug where doing an insert with secondary indices and the NOOVERWRITE flag could corrupt the secondary index. [#15912]

  2. Fixed a possible file handle leak that occurred while aborting the create of a database whose metadata page was not initialized. [#16359]

  3. Fixed a bug so that we now realloc the filename buffer only if we need it to grow. [#16385] [#16219]

  4. Fixed a race freeing a transaction object when using MVCC. [#16381]

  5. Added missing get methods for the DB and DB_ENV classes where there already was a corresponding set method. [#16505]

  6. Fixed a bug to now ensure that DB_STAT_SUBSYSTEM is distinct from other stat flags. [#16798]

  7. Fixed a bug related to updating multiple secondary keys (using DB_MULTIPLE). [#16885]

  8. Fixed a bug so that verify (db->verify, db_verify) will now report when it cannot read a page rather than just saying the database is bad. [#16916]

  9. Fixed a bug that could cause memory corruption if a transaction allocating a page aborted while DB->compact was running on that database. [#16862]

  10. Fixed a bug where logging was occurring during remove of an in-memory database when the DB_TXN_NOT_DURABLE flag was set. [#16571]

  11. Fixed a bug to remove a race condition during database/file create. [#17020]

  12. Fixed a bug where a call to DB->verify and specifying DB_SALVAGE could leak memory when the call returned. [#17161]

  13. Fixed a bug to avoid accessing freed memory during puts on primaries with custom comparators. [#17189]

  14. Fixed a bug that could cause old versions of pages to be written over new versions if an existing database is opened with the DB_TRUNCATE flag. [#17191]

Btree Access Method Changes:

  1. Fixed a bug which could cause DB->compact to fail with DB_NOTFOUND or DB_PAGE_NOTFOUND if the height of the tree was reduced by another thread while compact was active. The bug could also cause a page split to trigger splitting of internal nodes which did not need to be split. [#16192]

  2. Fixed a bug that caused Db->compact to loop if run on an empty RECNO database when there were pages in the free list. [#16778]

  3. Added a new flag, DB_OVERWRITE_DUP, to DB->put and DBC->put. This flag is equivalent to DB_KEYLAST in almost all cases: the exception is that with sorted duplicates, if a matching key/data pair exists, we overwrite it rather than returning DB_KEYEXIST. [#16803]

Hash Access Method Changes:

  1. Fixed a bug to now force a group allocation that rolls forward to reinit all the pages. Otherwise a previous aborted allocation may change the header. [#15414]

  2. Fixed a bug to now return the expected buffer size on a DB_BUFFER_SMALL condition. [#16881]

Queue Access Method Changes:

  1. Fixed a bug that would cause the LSN reset functionality to not process queue extents. [#16213]

  2. Fixed a bug that prevented a partial put on a queue database with secondaries configured. [#16460]

  3. Fixed a bug to now prevent an unpinned page to be returned if a delete from a HASH database deadlocked. [#16371]

  4. Fixed a bug that could cause a queue extent to be recreated if an application deleted a record that was already deleted in that extent. [#17004]

  5. Added the DB_CONSUME flag to DB->del and DBC->del to force adjustment of the head of the queue. [#17004]

Recno Access Method Changes:

  1. Fixed a bug which could cause DB->compact of a RECNO database to loop if the number of pages on the free list was reduced by another thread while compact was active. [#16199]

  2. Fixed a bug that occurs when deleting from a Recno database and using DB_READ_UNCOMMITTED where we could try to downgrade a lock twice. [#16347]

  3. Fixed a bug to now disallow passing DB_DUP and DB_RECNUM together to __db_set_flags. [#16585]

C-specific API Changes:

  1. Add get functions for each set functions of DB and DB_ENV structures which didn't have one.[#16505]

C++-specific API Changes:

  1. The get and set_lk_partitions methods are now available.

  2. Add get functions for each set functions of Db and DbEnv classes which didn't have one.[#16505]

  3. Fixed a memory leak when using nested transactions.[#16956]

Java-specific API Changes:

  1. Fixed a bug where the replication finer-grained verbose flags were not available in the Java API. [#15419]

  2. Fixed a bug in the BTree prefix compression API when called from the Java API. DBTs were not properly initialized. [#16417]

  3. Fixed a bug so that LogCursor will work correctly from the Java API. [#16827]

  4. Fixed a bug so that position(), limit() and capacity() of ByteBuffers are obeyed by DatabaseEntry objects. [#16982]

Direct Persistence Layer (DPL), Bindings and Collections API:

  1. The StoredMap class now implements the standard java.util.concurrent.ConcurrentMap interface. [#15382]

  2. Report a meaningful IllegalArgumentException when @Persistent is incorrectly declared on an enum class. Before, the confusing message Persistent class has non-persistent superclass: java.lang.Enum was reported. [#15623]

  3. Report a meaningful IllegalArgumentException when @Persistent is incorrectly declared on an interface. Before, a NullPointerException was reported. [#15841]

  4. Several validation checks have been added or corrected having to do with entity subclasses, which are @Persistent classes that extend an @Entity class. [#16077]

  5. Optimized marshaling for large numbers of embedded objects improving performance. [#16198]

  6. The StoredMap class now implements the Java 1.5 ConcurrentMap interface. [#16218]

  7. Fix a DPL bug that caused exceptions when using a class Converter for an instance containing non-simple fields. [#16233]

  8. Add EntityCursor.setCacheMode and getCacheMode. See the com.sleepycat.je.CacheMode class for more information. [#16239]

  9. Fix a bug that prevents evolution of @SecondaryKey information in an entity subclass (a class that extends an @Entity class). [#16253]

  10. Report a meaningful IllegalArgumentException when @Persistent or @Entity is incorrectly used on an inner class (a non-static nested class). Before, the confusing message No default constructor was reported. [#16279]

  11. Improved the reliability of Entity subclasses that define secondary keys by requiring that they be registered prior to storing an instance of the class. [#16399]

  12. Fix a bug that under certain circumstances causes "IllegalArgumentException: Not a key class" when calling EntityStore.getSubclassIndex, EntityStore.getPrimaryConfig, EntityStore.getSecondaryConfig, or PrimaryIndex.put, and a composite key class is used. [#16407]

  13. Fixed a bug so that one can now compile DPL in the Java API on Windows. [#16570]

  14. The com.sleepycat.collections.TransactionRunner.handleException method has been added to allow overriding the default transaction retry policy. See the javadoc for this method for more information. [#16574]

  15. Fix a bug that causes an assertion to fire or a NullPointerException (when assertions are disabled) from the EntityStore constructor. The problem occurs only when the previously created EntityStore contains an entity with a secondary key definition in which the key name has been overridden and is different than the field name. [#16819]

  16. Key cursors have been optimized to significantly reduce I/O when the READ_UNCOMMITTED isolation mode is used. See EntityIndex.keys for more information. [#16859]

  17. Report a meaningful IllegalArgumentException when NULLIFY is used with a @SecondaryKey and the field is a primitive type. Before, the confusing message Key field object may not be null was reported. [#17011]

  18. Enum fields may now be used as DPL keys, including primary keys, secondary keys, and fields of composite key classes. Comparators are supported for composite key classes containing enum fields. [#17140]

  19. Fix a bug that prevented the use of custom key comparisons (composite key classes that implement Comparable) for secondary keys defined as ONE_TO_MANY or MANY_TO_MANY.[#17207]

  20. The db.jar file now contains a Premain class which enables bytecode enhancement using the JVM instrumentation commands. The built-in proxy classes are also now enhanced in the db.jar file, which enables off-line bytecode enhancement. For more information on DPL bytecode enhancement and how to use both instrumentation and off-line enhancement, please see the com.sleepycat.persist.model.ClassEnhancer javadoc. [#17233]

Tcl-specific API Changes:

  1. The mutex API is now available when using Tcl. [#16342]

RPC-specific Client/Server Changes:

  • RPC support has been removed from Berkeley DB. [#16785]

Replication Changes:

  1. Improved testing of initial conditions for rep and repmgr APIs and added heartbeat timeouts to rep_get_timeout.[#14977]

  2. Added DB_REP_CONF_INMEM replication configuration flag to store replication information exclusively in-memory without creating any files on-disk. [#15257]

  3. Added repmgr support for multi-process shared env [#15982]

  4. Fixed a bug where opening a cursor from a database handle failed to check whether the database handle was still fresh. If the database handle had been invalidated by a replication client synchronizing with a new master, it could point to invalid information. [#15990]

  5. Fixed a bug so that if LOG_REQ gets an archived LSN, replication sends VERIFY_FAIL. [#16004]

  6. Added timestamp and process/thread id to replication verbose messages. [#16098]

  7. Fixed a bug where, in very rare circumstances, two repmgr sites could connect to each other at the exact same time, the connection attempts "collide" and fail, and the same collision repeats in time synchronization indefinitely. [#16114]

  8. Fixed a bug where a missing database file (FILE_FAIL error condition) can interrupt a client synchronization without restarting it. [#16130]

  9. Fixed a bug by adding REP_F_INREPSTART flag to prevent racing threads in rep_start. [#16247]

  10. Fixed a bug to not return HOLDELECTION if we are already in the middle of an election. Updated the egen so the election thread will notice. [#16270]

  11. Fixed a bug in buffer space computation, which could have led to memory corruption in rare circumstances, when using bulk transfer. [#16357]

  12. Fixed a bug that prevented replication clients from opening a sequence. The sequence is opened for read operations only. [#16406]

  13. Fixed a bug by removing an assertion about priority in elections. It is not correct because it could have changed by then. Remove unused recover_gen field. [#16412]

  14. Fixed a bug to now ignore a message from client if it is an LSN not recognized in a LOG_REQ. [#16444]

  15. Fixed a bug so that on POSIX systems, repmgr no longer restores default SIGPIPE action upon env close, if it was necessary to change it during start-up. This allows remaining repmgr environments within the same process, if any, to continue operating after one of them is closed. [#16454]

  16. After a replication client restarts with recovery, any named in-memory databases are now re-materialized from the rest of the replication group upon synchronization with the master. [#16495]

  17. Fixed a bug by adding missing rep_get_config flags. [#16527]

  18. Instead of sleeping if the bulk buffer is in transmission, return so that we can send as a singleton. [#16537]

  19. Fixed a bug by changing __env_refresh to not hit assert on -private -rep env with an in-memory database. [#16546]

  20. Fixed a bug in the Windows implementation of repmgr where a large number of commit threads concurrently awaiting acknowledgments could result in memory corruption, and leaking Win32 Event Objects. [#16548]

  21. Fixed a bug by changing repmgr to count a dropped connection when noticing a lacking heartbeat; fixed hearbeat test to check for election, rather than connection drop count, and more reasonable time limit; fixed test to poll until desired result, rather than always sleeping max possible time. [#16550]

  22. Fixed "master changes" stat to count when local site becomes master too. [#16562]

  23. Fixed a bug where a c2c client would send UPDATE_REQ to another client [#16592]

  24. Removed code to proactively expire leases when we don't get acks. Leases maintain their own LSNs to know. [#16494]

  25. Fixed a bug where a client may not sync pages during internal init. [#16671]

  26. Fixed a bug where a client that received and skipped a log record from the master during an election, then won the election, could then try to request a copy of the skipped log record. The result was an attempt to send a request to the local site, which is invalid: this could confuse a replication Base API application, or cause the Replication Manager to crash. [#16700]

  27. Fixed a bug which could have caused data loss or corruption (at the client only) if a replication client rolled back existing transactions in order to synchronize with a new master, and then crashed/recovered before a subsequent checkpoint operation had been replicated from the master. [#16732]

  28. Fixed a bug so that replication now retries on DB_LOCK_NOTGRANTED. [#16741]

  29. Fixed a potential deadlock in rep_verify_fail. [#16779]

  30. Fixed a bug so that an application will no longer segv if nsites given was smaller than number of sites that actually exists. [#16825]

XA Resource Manager Changes:

  1. The XA Resource Manager has been removed from Berkeley DB. [#6459]

Locking Subsystem Changes:

  1. Fixed a bug to prevent unlocking a mutex twice if we ran out of transactional locks. [#16285]

  2. Fixed a bug to prevent a segmentation trap in __lock_open if there were an error during the opening of an environment. [#16307]

  3. Fixed a bug to now avoid a deadlock if user defined locks are used only one lock partition is defined.[#16415]

  4. Fixed concurrency problems in __dd_build, __dd_abort by adding LOCK_SYSTEM_LOCK() calls to __dd_build and __dd_abort. [16489]

  5. Fixed a bug that could cause a panic if a transaction which updated a database that was supporting READ_UNCOMMITED readers aborted and it hit a race with a thread running the deadlock detector. [#16490]

  6. Fixed a race condition in deadlock detection that could overwrite heap. [#16541]

  7. Fixed a bug so that DB_STAT_CLEAR now restores the value of st_partitions. [#16701]

Logging Subsystem Changes:

  1. Fixed a bug so that the header checksum is only ignored when the log is from a previous version [#16281]

  2. Fixed a bug by removing a possible race condition with logc_get(DB_FIRST) and log archiving. [#16387]

  3. Fixed a bug that could cause a recovery failure of a create of a database that was aborted. [#16824]

  4. An in-memory database creation has an intermediate phase where we have a semi-open DBP. If we crash in that state, then recovery was failing because it tried to use a partically open database handle. This fix checks for that case, and avoids trying to undo page writes for databases in that interim step. [#17203]

Memory Pool Subsystem Changes:

  1. Fixed a bug that occurred after all open handles on a file are closed. Needed to clear the TXN_NOT_DURABLE flag (if set) and mark the file as DURABLE_UNKNOWN in the memory pool. [#16091]

  2. Fixed a possible race condition between dirtying and freeing a buffer that could result in a panic or corruption. [#16530]

  3. Fixed a memory leak where allocated space for temporary file names are not released. [#16956]

Mutex Subsystem Changes:

  1. Fixed a bug when using mutexes for SMP MIPS/Linux systems. [#15914]

  2. POSIX mutexes are now the default on Solaris. [#16066]

  3. Fixed a bug in mutex allocation with multiple cache regions. [#16178]

  4. Fixed MIPS/Linux mutexes in 4.7. [#16209]

  5. Fixed a bug that would cause a mutex to be unlocked a second time if we ran out of space while tracking pinned pages. [#16228]

  6. Fixed a bug Sparc/GCC when using test-and-set mutexes. They are now aligned on an 8-byte boundary. [#16243]

  7. Fixed a bug to now prevent a thread calling DB_ENV->failcheck to hang on a mutex held by a dead thread. [#16446]

  8. Fixed a bug so that __db_pthread_mutex_unlock() now handles the failchk case of finding a busy mutex which was owned by a now-dead process. [#16557]

  9. Removed support for the mutex implementation based on the "fcntl" system call. Anyone configuring Berkeley DB to use this type of mutex in an earlier release will need to either switch to a different mutex type or contact Oracle for support. [#17470]

Test Suite Changes

  1. Fixed a bug when using failchk(), where a mutex was not released. [#15982]

  2. Added a set of basic repmgr tests to run_std and run_all. [#16092]

  3. Added control wrapper for db_reptest to test suite. [#16161]

  4. Fixed a bug to now skip tests if db_reptest is not configured. [#16161]

  5. Changed name of run_db_in_mem to run_inmem_db, and run_inmem to run_inmem_log and made the arg orders consistent. [#16358]

  6. Fixed a bug to now clean up stray handles when rep_verify doesn't work. [#16390]

  7. Fixed a bug to avoid db_reptest passing the wrong flag to repmgr_start when there is already a master. [#16475]

  8. Added new tests for abbreviated internal init. Fixed test not to expect in-memory database to survive recovery. [#16495]

  9. Fix a bug, to add page size for txn014 if the default page size is too small. Move files instead of renaming directory for env015 on QNX. [#16627]

  10. Added new rep088 test for log truncation integrity. [#16732]

  11. Fixed a bug by adding a checkpoint in rep061 to make sure we have messages to process. Otherwise we could hang with client stuck in internal init, and no incoming messages to trigger rerequest. [#16781]

Transaction Subsystem Changes:

  1. Fixed a bug to no longer generate an error if DB_ENV->set_flags (DB_TXN_NOSYNC) was called after the environment was opened. [#16492]

  2. Fixed a bug to remove a potential hang condition in replication os_yield loops when DB_REGISTER used with replication by adding PANIC_CHECKS. [#16502]

  3. Fix a bug to now release mutex obtained before special condition returns in __db_cursor_int and __txn_record_fname. [#16665]

  4. Fixed a leak in the transaction region when a snapshot update transaction accesses more than 4 databases. [#16734]

  5. Enabled setting of set_thread_count via the DB_CONFIG file. [#16878]

  6. Fixed a mutex leak in some corner cases. [#16665]

Utility Changes:

  1. The db_stat utility with the -RA flags will now print a list of known remote replication flags when using repmgr. [#15484]

  2. Restructured DB salvage to walk known leaf pages prior to looping over all db pages. [#16219]

  3. Fixed a problem with upgrades to 4.7 on big endian machines. [#16411]

  4. Fixed a bug so that now db_load consistently returns >1 on failure. [#16765]

  5. The db_dump utility now accepts a "-m" flag to dump information from a named in-memory database. [#16896]

  6. Fixed a bug that would cause db_hotbackup to fail if a database file was removed while it was running. [#17234]

Configuration, Documentation, Sample Application, Portability and Build Changes:

  1. Fixed a bug to now use the correct Perl include path. [#16058]

  2. Updated the version of the Microsoft runtime libraries shipped. [#16058]

  3. Upgraded the Visual Studio build files to be based on Visual Studio 8 (2005+). The build is now simplified. Users can still upgrade the Visual Studio 6.0 project files, if they want to use Visual Studio .NET (7.1) [#16108]

  4. Expanded the ex_rep example with checkpoint and log archive threads, deadlock detection, new options for acknowledgment policy and bulk transfer, and use of additional replication features and events. [#16109]

  5. Fixed a bug so that optimizations on AIX are re-enabled, avoiding incorrect code generation. [#16141]

  6. Removed a few compiler warnings and three type redefinitons when using vxworks and the GNU compiler. [#16341]

  7. Fixed a bug on Sparc v9 so that MUTEX_MEMBAR() now uses membar_enter() to get a #storeload barrier rather than just stbar's #storestor. [#16468]

  8. Berkeley DB no longer supports Win9X and Windows Me (Millenium edition).

  9. Fixed lock_get and lock_vec examples from the Java (and C#) API. Updated the Java lock example. [#16506]

  10. Fixed a bug to correctly handle the TPC-B history record on 64-bit systems. [#16709]

  11. Add STL API to Linux build. Can be enabled via the --enable-stl flag. [#16786]

  12. Add STL API to Windows build, by building the db_stl project in the solution. There are also stl's test and examples projects in this solution. [#16786]

  13. Add support to build dll projects for WinCE, in order to enable users to build DB into a dll in addition to a static library.[#16625]

  14. Fixed a weakness where several malloc/realloc return values are not checked before use.[#16664]

  15. Enabled DB->compact for WinCE.[#15897]

  16. HP-UX 10 is no longer supported.