Class FlashJournalRM

  • All Implemented Interfaces:
    Disposable, BinaryStoreManager, ClassLoaderAware, JournalMBean, Controllable, AutoCloseable

    public class FlashJournalRM
    extends AbstractJournalRM
    A FlashJournalRM manages disk I/O for temporary journal-based files, and acts as a shared resource for any number of journals that share a common configuration.

    To use the Resource Manager, configure it using Dependency Injection by passing in the populated dependency object at instantiation. Once configured, start the Resource Manager via the AbstractJournalRM.start() method, and then obtain BinaryStore instances via the AbstractJournalRM.createBinaryStore() method. When a BinaryStore instance is no longer required, dispose of it via the Disposable interface, and similarly stop() or AbstractJournalRM.dispose() of the Resource Manager when it is no longer needed to ensure that any resources it allocated are cleaned up and released.

    While the Resource Manager is running, there will be one or more files created in the specified temporary directory (or the default temporary directory if none is specified). The number of files is not related to the number of BinaryStore instances, but rather to the total amount of data that the Resource Manager is managing at any given time. Additionally, the Resource Manager uses three daemon threads to help manage the data: "Journal-Preparer", "Journal-Writer" and "Journal-Collector". These threads respectively lay out the data to be written, write the data, and perform asynchronous GC of the storage files.

    The limits on the journal are as follows:

    • Values are limited by default to 64MB;
    • An individual file is limited by default to 2GB (and maximum 4GB);
    • A journal is composed of up to 512 files;
    • A journal is thus limited by default to 1TB, with a theoretical maximum of 2TB

    Since:
    Coherence 3.7
    Author:
    cp/cf 2010-06-10
    • Field Detail

      • SHIFT_FILE_ID

        protected static final int SHIFT_FILE_ID
        The location of the file id within the ticket.
        See Also:
        Constant Field Values
      • MASK_FILE_ID

        protected static final long MASK_FILE_ID
        The 64-bit bitmask for the file index. The 9 bits from 62-54.
        See Also:
        Constant Field Values
      • SHIFT_OFFSET

        protected static final int SHIFT_OFFSET
        The location of the value-offset within the ticket.
        See Also:
        Constant Field Values
      • SHIFT_OFFSET_RAW

        protected static final int SHIFT_OFFSET_RAW
        From an encoding point of view, the raw offset is shifted four bits less than necessary, because the four least significant bits of the offset are always zero (due to paragraph alignment).
        See Also:
        Constant Field Values
      • MASK_OFFSET

        protected static final long MASK_OFFSET
        The 64-bit bitmask for the value offset. The 28 bits 53-26.

        Note that the least significant four bits of the offset MUST ALWAYS be zero, because they are NOT encoded into the ticket -- hence the 16-byte paragraph boundary for entries.

        See Also:
        Constant Field Values
      • SHIFT_LENGTH

        protected static final int SHIFT_LENGTH
        The location of the value-length within the ticket.
        See Also:
        Constant Field Values
      • MASK_LENGTH

        protected static final long MASK_LENGTH
        The 64-bit bitmask for the value length. The 26 bits 25-0.
        See Also:
        Constant Field Values
      • TYPE_NAME

        protected static final String TYPE_NAME
        A simple name to be used as part of MBean ObjectNames.
        See Also:
        Constant Field Values
      • m_dirTemp

        protected File m_dirTemp
        Directory to store the temporary files.
      • m_fLoggedDirectory

        protected boolean m_fLoggedDirectory
        Set to true after the directory location has been logged.
      • m_cbBacklog

        protected AtomicInteger m_cbBacklog
        Total amount of data enqueued to be written.
      • m_mapLocators

        protected ConcurrentMap<Long,​Binary> m_mapLocators
        Map of locators to Binary values that have been queued to be written. A "locator" is a ticket without an encoded length.
      • m_pool

        protected FlashJournalRM.BufferPool m_pool
        The BufferPool of Buffer objects that are used to format data to append to the journal.
      • m_daemonPreparer

        protected FlashJournalRM.PreparerDaemon m_daemonPreparer
        The "rendering" thread that renders (copies and encrypts) the queued binaries into buffers to be written.
      • m_daemonWriter

        protected FlashJournalRM.WriterDaemon m_daemonWriter
        The "writing" thread that just does appends of prepared buffers to journal files.
    • Method Detail

      • getMaxTotalRam

        public long getMaxTotalRam()
        Determine the total amount, in bytes, of RAM that will be used for the Journal.

        This property is specific to a RamJournalRM implementation.

        Returns:
        the maximum number of bytes that will be allocated for Journal storage
      • isNioRam

        public boolean isNioRam()
        Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).

        This property is specific to a RamJournalRM implementation.

        Returns:
        true iff the journal is configured to use NIO "direct buffer" RAM
      • getMaxBacklogSize

        public int getMaxBacklogSize()
        Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.

        This property is specific to a FlashJournalRM implementation.

        Returns:
        the maximum allowable size of the backlog
      • getBacklogSize

        public int getBacklogSize()
        Determine the total size in bytes of the serialized values that have not yet been persisted to disk. This value is also referred to as the "backlog".

        This property is specific to a FlashJournalRM implementation.

        Returns:
        the number of bytes queued to be written
      • getBacklogCount

        public int getBacklogCount()
        Determine the number of serialized values that have not yet been persisted to disk.

        This property is specific to a FlashJournalRM implementation.

        Returns:
        the number of serialized values queued to be written
      • getBufferSize

        public int getBufferSize()
        Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.

        This property is specific to a FlashJournalRM implementation.

        Returns:
        the size, in bytes, of each buffer
      • getMaxPoolSize

        public long getMaxPoolSize()
        Determine the maximum size (in bytes) of the buffers that can be held by the pool. This is not a limit of how many buffers can be allocated, since some may be in use at any time, but rather how many will be held by the pool (i.e. recycled) as they are released.
        Returns:
        the maximum size in bytes of all of the buffers that the buffer pool can hold onto
      • getPoolSize

        public int getPoolSize()
        Determine the size (in bytes) of the buffers that are currently available in the pool. This is not a measurement of how many buffers are currently in use or how many have been allocated.

        This property is specific to a FlashJournalRM implementation.

        Returns:
        the total size in bytes of all of the buffers that are currently in the buffer pool
      • getHighFileCount

        public int getHighFileCount()
        Determine the high file count for this Journal. Compulsory compaction (GC) occurs when the journal file count reaches the high file count.

        This property is specific to a FlashJournalRM implementation.

        Returns:
        the high file count for this journal
      • getDirectoryFile

        public File getDirectoryFile()
        Determine the directory used to store the underlying disk files for the journal.
        Returns:
        the directory that is used to hold the journal files, or null if the default temporary directory is used
      • getPurgeDelayMillis

        public long getPurgeDelayMillis()
        Return the delay since the last touch of a tmp file until it is eligible for purging. Tmp files are purged at startup of the FlashJournalRM to clean up tmp files from a previous instance of the FlashJournalRM.
        Returns:
        the delay in milliseconds
      • getName

        protected String getName()
        Returns the name value to be used in MBean Objectname key "name".
        Specified by:
        getName in class AbstractJournalRM
        Returns:
        the name
      • getWriterTimeout

        protected long getWriterTimeout()
        Return the number of milliseconds the Writer daemon can be unresponsive prior to considering it timed out.
        Returns:
        the number of milliseconds the Writer can be unresponsive
      • startThreads

        protected void startThreads()
        Create and start the various threads used by this Journal Resource Manager.
        Overrides:
        startThreads in class AbstractJournalRM
      • purgeOldTempFiles

        protected void purgeOldTempFiles()
        Delete any old temp files that weren't cleaned up properly by a previous are configured to be "deleted on exit", it is possible that the temp files would not have been deleted if the JVM crashed or if there was a fault at the physical machine level.
      • stopThreads

        protected void stopThreads()
        Stop the various threads used by this Journal Resource Manager.
        Overrides:
        stopThreads in class AbstractJournalRM
      • calculateCurrentLoadFactor

        protected double calculateCurrentLoadFactor()
        Calculate current load factor for compaction (GC).
        Overrides:
        calculateCurrentLoadFactor in class AbstractJournalRM
        Returns:
        calculated load factor
      • shouldNotifyCollector

        protected boolean shouldNotifyCollector()
        Check if the Collector daemon needs to be notified to come out of sleep
        Overrides:
        shouldNotifyCollector in class AbstractJournalRM
        Returns:
        true if collector daemon should be notified
      • getDescription

        protected String getDescription()
        Format the object attributes into a String for inclusion in the String returned from the AbstractJournalRM.toString() method.
        Overrides:
        getDescription in class AbstractJournalRM
        Returns:
        a comma-delimited String listing the attributes of this object in the form "attribute=value"
      • getJournalFile

        protected FlashJournalRM.JournalFile getJournalFile​(int nFileId)
        Obtain a JournalFile by its ID.
        Overrides:
        getJournalFile in class AbstractJournalRM
        Parameters:
        nFileId - the file id (0-511)
        Returns:
        the JournalFile for that file id, or null if there is currently no JournalFile for that file id
      • instantiateJournalFile

        protected FlashJournalRM.JournalFile instantiateJournalFile​(int nFile)
        Factory: Instantiate a JournalFile or subclass thereof.
        Specified by:
        instantiateJournalFile in class AbstractJournalRM
        Parameters:
        nFile - the file number in the range 0-511 (inclusive)
        Returns:
        a new JournalFile instance
      • instantiateBufferPool

        protected FlashJournalRM.BufferPool instantiateBufferPool()
        Factory: Instantiate a pool of buffers.
        Returns:
        a BufferPool or subclass thereof
      • getBufferPool

        protected FlashJournalRM.BufferPool getBufferPool()
        Obtain the BufferPool instance.
        Returns:
        the BufferPool for this FlashJournalRM
      • instantiatePreparerDaemon

        protected FlashJournalRM.PreparerDaemon instantiatePreparerDaemon()
        Factory: Instantiate a PreparerDaemon.
        Returns:
        a PreparerDaemon or a sub-class thereof
      • getPreparerDaemon

        protected FlashJournalRM.PreparerDaemon getPreparerDaemon()
        Obtain the PreparerDaemon instance.
        Returns:
        the PreparerDaemon for this FlashJournalRM
      • instantiateWriterDaemon

        protected FlashJournalRM.WriterDaemon instantiateWriterDaemon()
        Factory: Instantiate a WriterDaemon.
        Returns:
        a WriterDaemon or a sub-class thereof
      • getWriterDaemon

        protected FlashJournalRM.WriterDaemon getWriterDaemon()
        Obtain the WriterDaemon instance.
        Returns:
        the WriterDaemon for this FlashJournalRM
      • getEvacuationMask

        protected long getEvacuationMask()
        Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
        Specified by:
        getEvacuationMask in class AbstractJournalRM
        Returns:
        the the bit mask for the file index within a ticket and whatever other bits are necessary to ensure that a ticket refers to that file
      • extractFileId

        protected int extractFileId​(long lTicket)
        Extract a file index (a file ID) from the passed ticket.
        Specified by:
        extractFileId in class AbstractJournalRM
        Parameters:
        lTicket - a "ticket" that was created by the Journal
        Returns:
        the file index for the value represented by that ticket
      • extractOffset

        protected long extractOffset​(long lTicket)
        Extract a file offset from the passed ticket.
        Specified by:
        extractOffset in class AbstractJournalRM
        Parameters:
        lTicket - a "ticket" that was created by the Journal
        Returns:
        the file offset for the value represented by that ticket
      • extractLength

        protected int extractLength​(long lTicket)
        Extract a file offset from the passed ticket.
        Specified by:
        extractLength in class AbstractJournalRM
        Parameters:
        lTicket - a "ticket" that was created by the Journal
        Returns:
        the file offset for the value represented by that ticket
      • encodeTicket

        protected long encodeTicket​(int nFile,
                                    long of,
                                    int cb)
        Encode a file number, offset and length into a "ticket".
        Specified by:
        encodeTicket in class AbstractJournalRM
        Parameters:
        nFile - a file number (aka a file index)
        of - a value offset
        cb - a value length
        Returns:
        a "ticket" containing the file number, offset and length