Package com.tangosol.io.journal

Class RamJournalRM

java.lang.Object
com.tangosol.io.journal.AbstractJournalRM
com.tangosol.io.journal.RamJournalRM
All Implemented Interfaces:
Disposable, BinaryStoreManager, ClassLoaderAware, JournalMBean, Controllable, AutoCloseable
public class RamJournalRM extends AbstractJournalRM
A RamJournalRM manages memory buffers for journal-based storage in memory, and acts as a shared resource for any number of journals that share a common configuration. While the RamJournalRM can be used by itself, it is intended to be used with a FlashJournalRM instance: So that large objects can be stored using flash; to allow for spill-over when the amount of total memory allocated to the RAM journal is used; and to handle the case when the journal "garbage collection" is temporarily not able to keep up with demand.

To use the Resource Manager, configure it using Dependency Injection by passing in the populated dependency object at instantiation.. Once configured, start the Resource Manager via the AbstractJournalRM.start() method, and then obtain BinaryStore instances via the AbstractJournalRM.createBinaryStore() method. When a BinaryStore instance is no longer required, dispose of it via the Disposable interface, and similarly stop() or AbstractJournalRM.dispose() of the Resource Manager when it is no longer needed to ensure that any resources it allocated are cleaned up and released.

The limits on the journal are as follows:

  • Values are limited by default to 16KB (and maximum 4MB);
  • An individual buffer (i.e. a journal "file") is limited by default to 2MB (and maximum 2GB);
  • A journal is composed of up to 512 files;
  • The total memory used by the journal is limited to 1GB by default (and maximum 64GB).

Note that with a flash journal backing up the RAM journal, the limits have a different meaning:

  • Values that exceed the size limit will automatically be delegated to the flash journal;
  • All data that is attempted to be written when the total memory allocated to the RAM journal has been used will automatically be delegated to the flash journal.

Note that, outside of the initial configuration, there is no difference in the use of the RAM journal with or without the flash journal; the combination of the RAM journal with the flash journal is transparent to the clients, with the only obvious difference being the significantly reduced possibility of an exception due to the RAM being exhausted.

Since:
Coherence 3.7
Author:
cp/cf 2010-06-24

  • Field Details

    • SHIFT_RAM_FLAG

      protected static final int SHIFT_RAM_FLAG
      The location of the RAM flag within the ticket.
      See Also:

    • MASK_RAM_FLAG

      protected static final long MASK_RAM_FLAG
      The 64-bit bitmask for the RAM flag.

      The RAM flag is the bit that signifies that the ticket was produced by the RAM Journal Resource Manager, as opposed to either a "real" compact ticket or a delegated flash ticket.

      See Also:

    • SHIFT_FILE_ID

      protected static final int SHIFT_FILE_ID
      The location of the file id within the ticket.
      See Also:

    • MASK_FILE_ID

      protected static final long MASK_FILE_ID
      The 64-bit bitmask for the file index. The 9 bits from 61-53.
      See Also:

    • SHIFT_OFFSET

      protected static final int SHIFT_OFFSET
      The location of the value-offset within the ticket.
      See Also:

    • MASK_OFFSET

      protected static final long MASK_OFFSET
      The 64-bit bitmask for the value offset. The 31 bits 52-22.
      See Also:

    • SHIFT_LENGTH

      protected static final int SHIFT_LENGTH
      The location of the value-length within the ticket.
      See Also:

    • MASK_LENGTH

      protected static final long MASK_LENGTH
      The 64-bit bitmask for the value length. The 22 bits 21-0.
      See Also:

    • TYPE_NAME

      protected static final String TYPE_NAME
      A simple type name to be used as part of MBean Objectnames.
      See Also:

    • m_pool

      protected RamJournalRM.BufferPool m_pool
      The buffer pool.

    • m_jrnlrm

      protected FlashJournalRM m_jrnlrm
      A reference to a flash journal resource manager to use to read/write to/from flash for data that doesn't fit in memory.

    • m_flash

      protected RamJournalRM.FlashConsumer m_flash
      The conduit between this RAM journal resource manager and the flash journal resource manager.

  • Constructor Details

  • Method Details

    • stop

      public void stop()
      Hard-stop the controllable service. Use Controllable.shutdown() for normal service termination. Calling this method for a service that has already stopped has no effect.
      Specified by:
      stop in interface Controllable
      Overrides:
      stop in class AbstractJournalRM

    • getBacklogCount

      public int getBacklogCount()
      Determine the number of serialized values that have not yet been persisted to disk.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the number of serialized values queued to be written

    • getBacklogSize

      public int getBacklogSize()
      Determine the total size in bytes of the serialized values that have not yet been persisted to disk. This value is also referred to as the "backlog".

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the number of bytes queued to be written

    • getMaxBacklogSize

      public int getMaxBacklogSize()
      Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the maximum allowable size of the backlog

    • getPoolSize

      public int getPoolSize()
      Determine the size (in bytes) of the buffers that are currently available in the pool. This is not a measurement of how many buffers are currently in use or how many have been allocated.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the total size in bytes of all of the buffers that are currently in the buffer pool

    • getMaxTotalRam

      public long getMaxTotalRam()
      Determine the total amount, in bytes, of RAM that will be used for the Journal.

      This property is specific to a RamJournalRM implementation.

      Returns:
      the maximum number of bytes that will be allocated for Journal storage

    • isNioRam

      public boolean isNioRam()
      Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).

      This property is specific to a RamJournalRM implementation.

      Returns:
      true iff the journal is configured to use NIO "direct buffer" RAM

    • getMaxPoolSize

      public long getMaxPoolSize()
      Determine the maximum size (in bytes) of the buffers that can be held by the pool. This is not a limit of how many buffers can be allocated, since some may be in use at any time, but rather how many will be held by the pool (i.e. recycled) as they are released.
      Returns:
      the maximum size in bytes of all of the buffers that the buffer pool can hold onto

    • getBufferSize

      public int getBufferSize()
      Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the size, in bytes, of each buffer

    • getHighFileCount

      public int getHighFileCount()
      Determine the high file count for this Journal. Compulsory compaction (GC) occurs when the journal file count reaches the high file count.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the high file count for this journal

    • getFlashJournalRM

      public FlashJournalRM getFlashJournalRM()
      Obtain the FlashJournalRM used to store large values or whatever values don't fit in the configured amount of RAM.
      Returns:
      the FlashJournalRM used by this RamJournalRM, or null if none

    • getName

      protected String getName()
      Returns the name value to be used in MBean Objectname key "name".
      Specified by:
      getName in class AbstractJournalRM
      Returns:
      the name

    • getDependencies

      public RamJournalRM.Dependencies getDependencies()
      Return the Dependencies object.
      Overrides:
      getDependencies in class AbstractJournalRM
      Returns:
      the journal dependencies object

    • cloneDependencies

      public RamJournalRM.DefaultDependencies cloneDependencies(AbstractJournalRM.Dependencies deps)
      Clone the Dependencies object.
      Specified by:
      cloneDependencies in class AbstractJournalRM
      Parameters:
      deps - the populated Dependencies object
      Returns:
      the cloned Dependencies object

    • onDependencies

      protected void onDependencies(AbstractJournalRM.Dependencies deps)
      Process the Dependencies after they have been injected into this object.
      Overrides:
      onDependencies in class AbstractJournalRM
      Parameters:
      deps - the populated Dependencies object

    • startThreads

      protected void startThreads()
      Create and start the various threads used by this Journal Resource Manager.
      Overrides:
      startThreads in class AbstractJournalRM

    • getDescription

      protected String getDescription()
      Format the object attributes into a String for inclusion in the String returned from the AbstractJournalRM.toString() method.
      Overrides:
      getDescription in class AbstractJournalRM
      Returns:
      a comma-delimited String listing the attributes of this object in the form "attribute=value"

    • getCollectorLoadFactor

      public double getCollectorLoadFactor()
      Determine the load factor threshold at which files become eligible for garbage collection (compaction). The load factor threshold is the ratio between the portion of the file used for live data and the total size of the file, and is expressed as a double value in the interval (0.0, 1.0).

      When the portion of released data exceeds (1 - threshold), the file becomes eligible for compaction (garbage collection).

      Specified by:
      getCollectorLoadFactor in interface JournalMBean
      Overrides:
      getCollectorLoadFactor in class AbstractJournalRM
      Returns:
      the compaction factor threshold between 0.0 and 1.0

    • shouldNotifyCollector

      protected boolean shouldNotifyCollector()
      Check if the Collector daemon needs to be notified to come out of sleep
      Overrides:
      shouldNotifyCollector in class AbstractJournalRM
      Returns:
      true if collector daemon should be notified

    • getMaxJournalFiles

      protected int getMaxJournalFiles()
      Determine the maximum number of journal files.
      Overrides:
      getMaxJournalFiles in class AbstractJournalRM
      Returns:
      the maximum number of simultaneous journal files that will be used to store the contents of this journal

    • getMinCollectorSleepMillis

      protected long getMinCollectorSleepMillis()
      Determine the minimum sleep cycle between collections.
      Overrides:
      getMinCollectorSleepMillis in class AbstractJournalRM
      Returns:
      the minimum number of milliseconds to sleep between collections

    • isDedupEnabled

      protected boolean isDedupEnabled()
      Determine if this journal resource manager should de-dup the keys that its JournalBinaryStore instances are managing.
      Overrides:
      isDedupEnabled in class AbstractJournalRM
      Returns:
      true iff this journal resource manager should routinely cause its JournalBinaryStore instances to de-dup their keys

    • isSingleEvacuation

      protected boolean isSingleEvacuation()
      Determine if this journal resource manager should only evacuate one single JournalFile (at the most) per collection cycle.
      Overrides:
      isSingleEvacuation in class AbstractJournalRM
      Returns:
      true iff this journal resource manager should only evacuate the emptiest JournalFile instead of all the JournalFile instances that qualify for evacuation

    • getFlashJournal

      protected Journal getFlashJournal()
      Obtain the Journal used to read/write flash.
      Returns:
      the flash Journal or null if there is none

    • getSurvivorFile

      protected AbstractJournalRM.JournalFile getSurvivorFile()
      Return a AbstractJournalRM.JournalFile based on the context of the calling thread or null if it is determined that an exhaustive evacuation will not reclaim sufficient memory. This method should only be invoked in a "depleted" state when:
      • the maximum number of files is used
      • all those files are non-appending
      When called on a client thread, any operation against this file will block until the full compaction is performed. There is no guarantee that compaction results in available memory thus an out of memory condition may still occur.

      When called on the CollectorDaemon thread this method returns a normal JournalFile, with the only exception that it will be in excess to the maximum number of permitted journal files and used exclusively by the CollectorDaemon to perform evacuation in a depleted state.

      Overrides:
      getSurvivorFile in class AbstractJournalRM
      Returns:
      either a JournalFile whose usage triggers a full collection or a JournalFile used exclusively by the CollectorDaemon. null may be returned if it is determined that an exhaustive evacuation will not reclaim sufficient memory

    • instantiateJournal

      protected RamJournalRM.JournalImpl instantiateJournal()
      Factory: Instantiate a JournalImpl object.
      Overrides:
      instantiateJournal in class AbstractJournalRM
      Returns:
      a JournalImpl instance or subclass thereof

    • instantiateConsumer

      protected RamJournalRM.FlashConsumer instantiateConsumer()
      Factory: Instantiate a FlashConsumer object.
      Returns:
      an instance of FlashConsumer or a subclass thereof.

    • getFlashConsumer

      protected RamJournalRM.FlashConsumer getFlashConsumer()
      Obtain the FlashConsumer object used to read/write flash.
      Returns:
      the FlashConsumer instance, or null if there is no flash configured to use

    • getJournalFile

      protected RamJournalRM.JournalFile getJournalFile(int nFileId)
      Obtain a JournalFile by its ID.
      Overrides:
      getJournalFile in class AbstractJournalRM
      Parameters:
      nFileId - the file id (0-511)
      Returns:
      the JournalFile for that file id, or null if there is currently no JournalFile for that file id

    • instantiateJournalFile

      protected RamJournalRM.JournalFile instantiateJournalFile(int nFile)
      Factory: Instantiate a JournalFile or subclass thereof.
      Specified by:
      instantiateJournalFile in class AbstractJournalRM
      Parameters:
      nFile - the file number in the range 0-511 (inclusive)
      Returns:
      a new JournalFile instance

    • isCompact

      protected boolean isCompact(long lTicket)
      Determine if the ticket is in the compact form.
      Overrides:
      isCompact in class AbstractJournalRM
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      true iff the ticket is a compact ticket

    • isRam

      protected boolean isRam(long lTicket)
      Determine if the ticket is in the RAM form.
      Parameters:
      lTicket - a "ticket" that was returned previously by the Journal
      Returns:
      true iff the ticket is a RAM ticket

    • isFlash

      protected boolean isFlash(long lTicket)
      Determine if the ticket is neither in compact or RAM form.
      Parameters:
      lTicket - a "ticket" that was returned previously by the Journal
      Returns:
      true iff the ticket is from an underlying FlashJournalRM

    • getEvacuationMask

      protected long getEvacuationMask()
      Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
      Specified by:
      getEvacuationMask in class AbstractJournalRM
      Returns:
      the bit mask for the file index within a ticket and whatever other bits are necessary to ensure that a ticket refers to that file

    • extractFileId

      protected int extractFileId(long lTicket)
      Extract a file index (a file ID) from the passed ticket.
      Specified by:
      extractFileId in class AbstractJournalRM
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file index for the value represented by that ticket

    • extractOffset

      protected long extractOffset(long lTicket)
      Extract a file offset from the passed ticket.
      Specified by:
      extractOffset in class AbstractJournalRM
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file offset for the value represented by that ticket

    • extractLength

      protected int extractLength(long lTicket)
      Extract a file offset from the passed ticket.
      Specified by:
      extractLength in class AbstractJournalRM
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file offset for the value represented by that ticket

    • encodeTicket

      protected long encodeTicket(int nFile, long of, int cb)
      Encode a file number, offset and length into a "ticket".
      Specified by:
      encodeTicket in class AbstractJournalRM
      Parameters:
      nFile - a file number (aka a file index)
      of - a value offset
      cb - a value length
      Returns:
      a "ticket" containing the file number, offset and length

    • getTicketDescription

      protected String getTicketDescription(long lTicket)
      Format the information encoded in a ticket into a debug string.
      Overrides:
      getTicketDescription in class AbstractJournalRM
      Parameters:
      lTicket - the ticket value
      Returns:
      a description of the contents of the ticket value

    • instantiateBufferPool

      protected RamJournalRM.BufferPool instantiateBufferPool()
      Factory: Instantiate a pool of buffers.
      Returns:
      a BufferPool or subclass thereof

    • getBufferPool

      protected RamJournalRM.BufferPool getBufferPool()
      Obtain the BufferPool instance.
      Returns:
      the BufferPool for this FlashJournalRM