Package com.tangosol.io.journal

Class AbstractJournalRM

  • All Implemented Interfaces:
    Disposable, BinaryStoreManager, ClassLoaderAware, JournalMBean, Controllable, AutoCloseable
    Direct Known Subclasses:
    FlashJournalRM, RamJournalRM
    public abstract class AbstractJournalRM
extends Object
implements Controllable, Disposable, BinaryStoreManager, JournalMBean
    The AbstractJournalRM represents a Journal Resource Manager that manages storage for journaled data, and acts as a shared resource for any number of journals that share a common configuration. This implementation is intended to support both file-based and memory-based implementations.

    The limits on the journal are as follows:

    • Values are limited by default to 64MB;
    • An individual file is limited by default to 2GB (and maximum 4GB);
    • A journal is composed of up to 512 files;
    • A journal is thus limited by default to 1TB, with a theoretical maximum of 2TB.

    Since:
    Coherence 3.7
    Author:
    cp/cf 2010-06-23

    • Field Detail

      • STATE_INITIAL

        protected static final int STATE_INITIAL
        State: Initial (unconfigured).
        See Also:
        Constant Field Values

      • STATE_CONFIGURED

        protected static final int STATE_CONFIGURED
        State: Configured (ready to be started).
        See Also:
        Constant Field Values

      • STATE_RUNNING

        protected static final int STATE_RUNNING
        State: Running (able to be shut down or stopped).
        See Also:
        Constant Field Values

      • STATE_STOPPING

        protected static final int STATE_STOPPING
        State: Stopping.
        See Also:
        Constant Field Values

      • STATE_STOPPED

        protected static final int STATE_STOPPED
        State: Stopped.
        See Also:
        Constant Field Values

      • SHIFT_COMPACT_FLAG

        protected static final int SHIFT_COMPACT_FLAG
        The location of the compact flag within the ticket.
        See Also:
        Constant Field Values

      • MASK_COMPACT_FLAG

        protected static final long MASK_COMPACT_FLAG
        The 64-bit bitmask for the compact flag.

        The compact flag is the bit that signifies whether the ticket contains the actual value encoded into the long, versus a "proper" journal file ticket.

        See Also:
        Constant Field Values

      • SHIFT_COMPACT_LENGTH

        protected static final int SHIFT_COMPACT_LENGTH
        The location of the compact length within the ticket.
        See Also:
        Constant Field Values

      • MASK_COMPACT_LENGTH

        protected static final long MASK_COMPACT_LENGTH
        The 64-bit bitmask for the compact length.
        See Also:
        Constant Field Values

      • DEDUPE_ENABLED

        protected static final boolean DEDUPE_ENABLED
        Whether de-duping (interning) the byte arrays held by the BinaryRadixTree should be requested.

      • m_nState

        protected int m_nState
        The current state of the Resource Manager; one of the STATE_* constants.

      • m_loader

        protected ClassLoader m_loader
        The ClassLoader supplied to this Controllable instance. (Controllable extends ClassLoaderAware.)

      • m_dflHighestLoadFactor

        protected double m_dflHighestLoadFactor
        The highest percentage (as a double in the range 0.00 to 1.00) of the configured capacity that this journal has reached. In other words, this is a high water mark of how full the journal has ever gotten.

      • m_dblFileSizeFactor

        protected double m_dblFileSizeFactor
        The size of the file as a ratio to the largest possible file size. Used to tune the aggressiveness of the collection.

      • m_cJournalFiles

        protected volatile int m_cJournalFiles
        The number of JournalFiles that currently exist.

      • m_daemonCollector

        protected AbstractJournalRM.CollectorDaemon m_daemonCollector
        The "garbage collector" thread that determines when journal files are ripe to be evacuated and deleted.

    • Method Detail

      • configure

        public void configure​(XmlElement xml)
        Configure the dependencies object given the xml configuration.
        Specified by:
        configure in interface Controllable
        Parameters:
        xml - the XML configuration for the object

      • start

        public void start()
        Start the controllable service.

        This method should only be called once per the life cycle of the Controllable service. This method has no affect if the service is already running.

        Specified by:
        start in interface Controllable

      • isRunning

        public boolean isRunning()
        Determine whether or not the controllable service is running. This method returns false before a service is started, while the service is starting, while a service is shutting down and after the service has stopped. It only returns true after completing its start processing and before beginning its shutdown processing.
        Specified by:
        isRunning in interface Controllable
        Returns:
        true if the service is running; false otherwise

      • shutdown

        public void shutdown()
        Stop the controllable service. This is a controlled shut-down, and is preferred to the Controllable.stop() method.

        This method should only be called once per the life cycle of the controllable service. Calling this method for a service that has already stopped has no effect.

        Specified by:
        shutdown in interface Controllable

      • stop

        public void stop()
        Hard-stop the controllable service. Use Controllable.shutdown() for normal service termination. Calling this method for a service that has already stopped has no effect.
        Specified by:
        stop in interface Controllable

      • getContextClassLoader

        public ClassLoader getContextClassLoader()
        Retrieve the context ClassLoader for this object. The context ClassLoader is provided by the creator of the object for use by the object when loading classes and resources.
        Specified by:
        getContextClassLoader in interface ClassLoaderAware
        Returns:
        the context ClassLoader for this object
        See Also:
        Thread.getContextClassLoader()

      • setContextClassLoader

        public void setContextClassLoader​(ClassLoader loader)
        Specify the context ClassLoader for this object. The context ClassLoader can be set when the object is created, and allows the creator to provide the appropriate class loader to be used by the object when when loading classes and resources.
        Specified by:
        setContextClassLoader in interface ClassLoaderAware
        Parameters:
        loader - the context ClassLoader for this object

      • dispose

        public void dispose()
        Invoked when all resources owned by the implementer can safely be released.

        Once disposed of the object should no longer be considered to be usable.

        Note the Disposable interface is compatible with try-with-resources which will automatically invoke this method.

        Specified by:
        dispose in interface Disposable

      • close

        public void close()
        Deprecated.
        use the Disposable interface
        Previous to the Disposable interface, clean-up was performed by invoking a close method via reflection.
        Specified by:
        close in interface AutoCloseable
        Specified by:
        close in interface Disposable

      • getMaxJournalFilesNumber

        public int getMaxJournalFilesNumber()
        Determine the maximum number of Journal files.
        Specified by:
        getMaxJournalFilesNumber in interface JournalMBean
        Returns:
        the maximum number of simultaneous Journal files that will be used to store the contents of this Journal

      • getBinaryStoreCount

        public int getBinaryStoreCount()
        Determine the number of active BinaryStore objects that are using this Journal.
        Specified by:
        getBinaryStoreCount in interface JournalMBean
        Returns:
        the number of BinaryStore objects that are currently active

      • getFileCount

        public int getFileCount()
        Determine the number of Journal files used by this Journal.
        Specified by:
        getFileCount in interface JournalMBean
        Returns:
        the number of files for this journal

      • getTotalDataSize

        public long getTotalDataSize()
        Determine the amount of data currently stored by this Journal. This value does not include data that have been released.
        Specified by:
        getTotalDataSize in interface JournalMBean
        Returns:
        the number of bytes for this Journal used to store Binary values that have not been released

      • getTotalFileSize

        public long getTotalFileSize()
        Determine the number of bytes in the Journal files for this Journal. This number could include released data that has not yet been compacted.
        Specified by:
        getTotalFileSize in interface JournalMBean
        Returns:
        the total size of all Journal files for this Journal

      • getMaxValueSize

        public int getMaxValueSize()
        Determine the maximum allowable size, in bytes, of a Binary value.
        Specified by:
        getMaxValueSize in interface JournalMBean
        Returns:
        the maximum allowable size for a value to write

      • getMaxFileSize

        public long getMaxFileSize()
        Determine the maximum allowable size, in bytes, of each individual file used to hold Journal data. This is not the total size limit of the Journal, since the Journal may use a number of files to store its data.
        Specified by:
        getMaxFileSize in interface JournalMBean
        Returns:
        the maximum allowable size for an underlying file

      • getCollectorLoadFactor

        public double getCollectorLoadFactor()
        Determine the load factor threshold at which files become eligible for garbage collection (compaction). The load factor threshold is the ratio between the portion of the file used for live data and the total size of the file, and is expressed as a double value in the interval (0.0, 1.0).

        When the portion of released data exceeds (1 - threshold), the file becomes eligible for compaction (garbage collection).

        Specified by:
        getCollectorLoadFactor in interface JournalMBean
        Returns:
        the compaction factor threshold between 0.0 and 1.0

      • getCurrentCollectorLoadFactor

        public double getCurrentCollectorLoadFactor()
        Determine the current load factor threshold for the journal.
        Specified by:
        getCurrentCollectorLoadFactor in interface JournalMBean
        Returns:
        the current load factor threshold between 0.0 and 1.0

      • getHighestLoadFactor

        public double getHighestLoadFactor()
        Determine the high-water mark load factor for the entire journal. The load factor is the ratio of the highest number of journal files that have concurrently existed to the maximum number of journal files that can concurrently exist.
        Specified by:
        getHighestLoadFactor in interface JournalMBean
        Returns:
        the high-water journal load factor in the range 0.00 to 1.00

      • getCompactionCount

        public int getCompactionCount()
        Determine the number of compactions (garbage collections) performed by this journal.
        Specified by:
        getCompactionCount in interface JournalMBean
        Returns:
        the number of compactions

      • getExhaustiveCompactionCount

        public int getExhaustiveCompactionCount()
        Determine the total number of exhaustive compactions performed by the collector. An exhaustive compaction is performed when there appears to be no memory available, however the garbage collector may have the ability reclaim memory via compaction.
        Specified by:
        getExhaustiveCompactionCount in interface JournalMBean
        Returns:
        number of exhaustive compactions executed

      • getExhaustiveCompactionTime

        public long getExhaustiveCompactionTime()
        Return the total amount time spent performing exhaustive compaction.
        Specified by:
        getExhaustiveCompactionTime in interface JournalMBean
        Returns:
        the total amount time spent performing exhaustive compaction

      • resetStatistics

        public void resetStatistics()
        Reset the journal statistics.
        Specified by:
        resetStatistics in interface JournalMBean

      • compact

        public void compact​(boolean fRegular)
        Execute either a regular or exhaustive compaction on the journal based on the provided parameter. The compaction is performed asynchronous to the invocation of this operation.

        An exhaustive compaction is only executed if sufficient memory has been released or sufficient time has elapsed since the previous exhaustive compaction.

        Specified by:
        compact in interface JournalMBean
        Parameters:
        fRegular - whether the compaction should be regular; false for an exhaustive compaction

      • createBinaryStore

        public JournalBinaryStore createBinaryStore()
        Create a new BinaryStore that will journal its information using this Journal Resource Manager.
        Specified by:
        createBinaryStore in interface BinaryStoreManager
        Returns:
        a new instance of JournalBinaryStore that uses this Journal Resource Manager for storage

      • destroyBinaryStore

        public void destroyBinaryStore​(BinaryStore store)
        Lifecycle method: Destroy a BinaryStore previously created by this manager.
        Specified by:
        destroyBinaryStore in interface BinaryStoreManager
        Parameters:
        store - a BinaryStore object previously created by this manager

      • createJournal

        public Journal createJournal​(Journal.JournalConsumer consumer)
        Create a new Journal that exposes the storage capabilities of this Journal Resource Manager.
        Parameters:
        consumer - the JournalConsumer that will be using the Journal
        Returns:
        a new instance of Journal that uses this Journal Resource Manager for storage

      • onDependencies

        protected void onDependencies​(AbstractJournalRM.Dependencies deps)
        Process the Dependencies after they have been injected into this object.
        Parameters:
        deps - the populated Dependencies object

      • getState

        protected int getState()
        Determine the current state of the AbstractJournalRM.
        Returns:
        one of the STATE_* constants

      • setState

        protected void setState​(int nState)
        Modify the current state of the AbstractJournalRM.
        Parameters:
        nState - one of the STATE_* constants

      • startThreads

        protected void startThreads()
        Create and start the various threads used by this Journal Resource Manager.

      • stopThreads

        protected void stopThreads()
        Stop the various threads used by this Journal Resource Manager.

      • calculateCurrentLoadFactor

        protected double calculateCurrentLoadFactor()
        Calculate current load factor for compaction (GC).
        Returns:
        calculated load factor

      • getStateDescription

        protected static String getStateDescription​(int nState)
        Format the passed state identifier into a human-readable string.
        Parameters:
        nState - a state identifier
        Returns:
        a human-readable string for the state

      • getDescription

        protected String getDescription()
        Format the object attributes into a String for inclusion in the String returned from the toString() method.
        Returns:
        a comma-delimited String listing the attributes of this object in the form "attribute=value"

      • getMaxJournalFiles

        protected int getMaxJournalFiles()
        Determine the maximum number of journal files.
        Returns:
        the maximum number of simultaneous journal files that will be used to store the contents of this journal

      • getMaxJournalSize

        protected long getMaxJournalSize()
        Return the total amount of memory, in bytes, that can be allocated for Journal storage.
        Returns:
        the maximum number of bytes that can be allocated for Journal storage

      • getMinCollectorSleepMillis

        protected long getMinCollectorSleepMillis()
        Determine the minimum sleep cycle between collections.
        Returns:
        the minimum number of milliseconds to sleep between collections

      • getMaxCollectorSleepMillis

        protected long getMaxCollectorSleepMillis()
        Determine the maximum sleep cycle between collections.
        Returns:
        the maximum number of milliseconds to sleep between collections

      • isDedupEnabled

        protected boolean isDedupEnabled()
        Determine if this journal resource manager should de-dup the keys that its JournalBinaryStore instances are managing.
        Returns:
        true iff this journal resource manager should routinely cause its JournalBinaryStore instances to de-dup their keys

      • isSingleEvacuation

        protected boolean isSingleEvacuation()
        Determine if this journal resource manager should only evacuate one single JournalFile (at the most) per collection cycle.
        Returns:
        true iff this journal resource manager should only evacuate the emptiest JournalFile instead of all the JournalFile instances that qualify for evacuation

      • getSurvivorFile

        protected AbstractJournalRM.JournalFile getSurvivorFile()
        Return a AbstractJournalRM.JournalFile based on the context of the calling thread or null if it is determined that an exhaustive evacuation will not reclaim sufficient memory. This method should only be invoked in a "depleted" state when:
        • the maximum number of files is used
        • all those files are non-appending
        When called on a client thread, any operation against this file will block until the full compaction is performed. There is no guarantee that compaction results in available memory thus an out of memory condition may still occur.

        When called on the CollectorDaemon thread this method returns a normal JournalFile, with the only exception that it will be in excess to the maximum number of permitted journal files and used exclusively by the CollectorDaemon to perform evacuation in a depleted state.

        Returns:
        either a JournalFile whose usage triggers a full collection or a JournalFile used exclusively by the CollectorDaemon. null may be returned if it is determined that an exhaustive evacuation will not reclaim sufficient memory

      • registerMBean

        protected void registerMBean()
        Register an MBean for this JournalRM.

      • unregisterMBean

        protected void unregisterMBean()
        Unregister an MBean for this JournalRM.

      • instantiateJournal

        protected AbstractJournalRM.JournalImpl instantiateJournal()
        Factory: Instantiate a JournalImpl object.
        Returns:
        a JournalImpl instance or subclass thereof

      • registerJournal

        protected void registerJournal​(AbstractJournalRM.JournalImpl journal)
        Register a Journal that uses this Journal Resource Manager.
        Parameters:
        journal - an instance of JournalImpl created by this Journal Resource Manager

      • unregisterJournal

        protected void unregisterJournal​(AbstractJournalRM.JournalImpl journal)
        Register a Journal that uses this Journal Resource Manager.
        Parameters:
        journal - an instance of JournalImpl created by this Journal Resource Manager

      • iterateJournals

        protected Iterator<AbstractJournalRM.JournalImpl> iterateJournals()
        Iterate through the Journal instances that have been created by this Journal Resource Manager but have not been disposed of yet.
        Returns:
        an Iterator of the current JournalImpl instances

      • getJournalFile

        protected AbstractJournalRM.JournalFile getJournalFile​(int nFileId)
        Obtain a JournalFile by its ID.
        Parameters:
        nFileId - the file id (0-511)
        Returns:
        the JournalFile for that file id, or null if there is currently no JournalFile for that file id

      • ensureCurrentJournalFile

        protected AbstractJournalRM.JournalFile ensureCurrentJournalFile()
        Obtain the current JournalFile that is being appended to, creating a new one if necessary.
        Returns:
        the current JournalFile, or null if the storage is full and no new JournalFile can be created as a result

      • iterateJournalFiles

        protected Iterator<AbstractJournalRM.JournalFile> iterateJournalFiles()
        Iterate all of the JournalFile objects.
        Returns:
        an Iterator of the existing JournalFile instances

      • instantiateJournalFile

        protected abstract AbstractJournalRM.JournalFile instantiateJournalFile​(int nFile)
        Factory: Instantiate a JournalFile or subclass thereof.
        Parameters:
        nFile - the file number in the range 0-511 (inclusive)
        Returns:
        a new JournalFile instance

      • instantiateCollectorDaemon

        protected AbstractJournalRM.CollectorDaemon instantiateCollectorDaemon()
        Factory: Instantiate a CollectorDaemon.
        Returns:
        a CollectorDaemon or a sub-class thereof

      • getCollectorDaemon

        protected AbstractJournalRM.CollectorDaemon getCollectorDaemon()
        Obtain the CollectorDaemon instance.
        Returns:
        the CollectorDaemon for this Journal Resource Manager

      • isCompact

        protected boolean isCompact​(long lTicket)
        Determine if the ticket is in the compact form.
        Parameters:
        lTicket - a "ticket" that was created by the Journal
        Returns:
        true iff the ticket is a compact ticket

      • getEvacuationMask

        protected abstract long getEvacuationMask()
        Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
        Returns:
        the the bit mask for the file index within a ticket and whatever other bits are necessary to ensure that a ticket refers to that file

      • extractFileId

        protected abstract int extractFileId​(long lTicket)
        Extract a file index (a file ID) from the passed ticket.
        Parameters:
        lTicket - a "ticket" that was created by the Journal
        Returns:
        the file index for the value represented by that ticket

      • extractOffset

        protected abstract long extractOffset​(long lTicket)
        Extract a file offset from the passed ticket.
        Parameters:
        lTicket - a "ticket" that was created by the Journal
        Returns:
        the file offset for the value represented by that ticket

      • extractLength

        protected abstract int extractLength​(long lTicket)
        Extract a file offset from the passed ticket.
        Parameters:
        lTicket - a "ticket" that was created by the Journal
        Returns:
        the file offset for the value represented by that ticket

      • encodeTicket

        protected abstract long encodeTicket​(int nFile,
                                     long of,
                                     int cb)
        Encode a file number, offset and length into a "ticket".
        Parameters:
        nFile - a file number (aka a file index)
        of - a value offset
        cb - a value length
        Returns:
        a "ticket" containing the file number, offset and length

      • encodeTicket

        protected long encodeTicket​(ByteSequence bin)
        Encode a Binary into a "ticket".
        Parameters:
        bin - a ByteSequence with (length() <= 7)
        Returns:
        a compact "ticket" containing the passed Binary value

      • extractBinary

        protected Binary extractBinary​(long lTicket)
        Extract a Binary from the passed ticket.
        Parameters:
        lTicket - a compact "ticket" that was created by the Journal
        Returns:
        the Binary encoded into the ticket

      • getTicketDescription

        protected String getTicketDescription​(long lTicket)
        Format the information encoded in a ticket into a debug string.
        Parameters:
        lTicket - the ticket value
        Returns:
        a description of the contents of the ticket value

      • shouldNotifyCollector

        protected boolean shouldNotifyCollector()
        Check if the Collector daemon needs to be notified to come out of sleep
        Returns:
        true if collector daemon should be notified

      • getGuardian

        protected Guardian getGuardian()
        Return the Guardian associated with the Cluster or null if absent.
        Returns:
        the Guardian associated with the Cluster or null

      • getCollectorTimeout

        protected long getCollectorTimeout()
        Return the number of milliseconds the Collector can be unresponsive prior to considering the Collector timed out.
        Returns:
        the number of milliseconds the Collector can be unresponsive

      • getName

        protected abstract String getName()
        Returns the name value to be used in MBean Objectname key "name".
        Returns:
        the name