Chapter 17.  The Memory Pool Subsystem

Table of Contents

Introduction to the memory pool subsystem
Configuring the memory pool

Introduction to the memory pool subsystem

The Memory Pool subsystem is the general-purpose shared memory buffer pool used by Berkeley DB. This module is useful outside of the Berkeley DB package for processes that require page-oriented, shared and cached file access. (However, such "use outside of Berkeley DB" is not supported in replicated environments.)

A memory pool is a memory cache shared among any number of threads of control. The DB_INIT_MPOOL flag to the DB_ENV->open() method opens and optionally creates a memory pool. When that pool is no longer in use, it should be closed using the DB_ENV->close() method.

The DB_ENV->memp_fcreate() method returns a DB_MPOOLFILE handle on an underlying file within the memory pool. The file may be opened using the DB_MPOOLFILE->open() method. The DB_MPOOLFILE->get() method is used to retrieve pages from files in the pool. All retrieved pages must be subsequently returned using the DB_MPOOLFILE->put() method. At the time pages are returned, they may be marked dirty, which causes them to be written to the underlying file before being discarded from the pool. If there is insufficient room to bring a new page in the pool, a page is selected to be discarded from the pool using a least-recently-used algorithm. All dirty pages in the pool from the file may be flushed using the DB_MPOOLFILE->sync() method. When the file handle is no longer in use, it should be closed using the DB_MPOOLFILE->close() method.

There are additional configuration interfaces that apply when opening a new file in the memory pool:

There are additional interfaces for the memory pool as a whole:

  • It is possible to gradually flush buffers from the pool in order to maintain a consistent percentage of clean buffers in the pool using the DB_ENV->memp_trickle() method.
  • Because special-purpose processing may be necessary when pages are read or written (for example, endian conversion, or page checksums), the DB_ENV->memp_register() function allows applications to specify automatic input and output processing in these cases.
  • The db_stat utility uses the DB_ENV->memp_stat() method to display statistics about the efficiency of the pool.
  • All dirty pages in the pool may be flushed using the DB_ENV->memp_sync() method. In addition, DB_ENV->memp_sync() takes an argument that is specific to database systems, and which allows the memory pool to be flushed up to a specified log sequence number (DB_LSN).
  • The entire pool may be discarded using the DB_ENV->remove() method.
Memory Pools and Related Methods Description
DB->get_mpf() handle Return DB's underlying DB_MPOOLFILE handle
DB_ENV->memp_stat() Return memory pool statistics
DB_ENV->memp_sync() Flush pages from a memory pool
DB_ENV->memp_trickle() Trickle flush pages from a memory pool
Memory Pool Configuration  
DB_ENV->memp_register() Register input/output functions for a file in a memory pool
DB_ENV->set_cachesize() Set the environment cache size
DB_ENV->set_cache_max() Set the maximum cache size
DB_ENV->set_mp_max_openfd() Set the maximum number of open file descriptors
DB_ENV->set_mp_max_write() Set the maximum number of sequential disk writes
DB_ENV->set_mp_mmapsize() Set maximum mapped-in database file size
Memory Pool Files  
DB_ENV->memp_fcreate() Create a memory pool file handle
DB_MPOOLFILE->close() Close a file in a memory pool
DB_MPOOLFILE->get() Get page from a file in a memory pool
DB_MPOOLFILE->open() Open a file in a memory pool
DB_MPOOLFILE->put() Return a page to a memory pool
DB_MPOOLFILE->sync() Flush pages from a file in a memory pool
Memory Pool File Configuration  
DB_MPOOLFILE->set_clear_len() Set file page bytes to be cleared
DB_MPOOLFILE->set_fileid() Set file unique identifier
DB_MPOOLFILE->set_flags() General memory pool file configuration
DB_MPOOLFILE->set_ftype() Set file type
DB_MPOOLFILE->set_lsn_offset() Set file log-sequence-number offset
DB_MPOOLFILE->set_maxsize() Set maximum file size
DB_MPOOLFILE->set_pgcookie() Set file cookie for pgin/pgout
mempset_priority() Set memory pool file priority