Skip navigation links

com.tangosol.io.journal
Class RamJournalRM

java.lang.Object
com.tangosol.io.journal.AbstractJournalRM
com.tangosol.io.journal.RamJournalRM

All Implemented Interfaces:

com.oracle.common.base.Disposable, BinaryStoreManager, ClassLoaderAware, JournalMBean, Controllable

public class RamJournalRM

extends AbstractJournalRM

A RamJournalRM manages memory buffers for journal-based storage in memory, and acts as a shared resource for any number of journals that share a common configuration. While the RamJournalRM can be used by itself, it is intended to be used with a FlashJournalRM instance: So that large objects can be stored using flash; to allow for spill-over when the amount of total memory allocated to the RAM journal is used; and to handle the case when the journal "garbage collection" is temporarily not able to keep up with demand.

To use the Resource Manager, configure it using Dependency Injection by passing in the populated dependency object at instantiation.. Once configured, start the Resource Manager via the AbstractJournalRM.start() method, and then obtain BinaryStore instances via the AbstractJournalRM.createBinaryStore() method. When a BinaryStore instance is no longer required, dispose of it via the Disposable interface, and similarly stop() or AbstractJournalRM.dispose() of the Resource Manager when it is no longer needed to ensure that any resources it allocated are cleaned up and released.

The limits on the journal are as follows:

• Values are limited by default to 16KB (and maximum 4MB);
• An individual buffer (i.e. a journal "file") is limited by default to 2MB (and maximum 2GB);
• A journal is composed of up to 512 files;
• The total memory used by the journal is limited to 1GB by default (and maximum 64GB).

Note that with a flash journal backing up the RAM journal, the limits have a different meaning:

• Values that exceed the size limit will automatically be delegated to the flash journal;
• All data that is attempted to be written when the total memory allocated to the RAM journal has been used will automatically be delegated to the flash journal.

Note that, outside of the initial configuration, there is no difference in the use of the RAM journal with or without the flash journal; the combination of the RAM journal with the flash journal is transparent to the clients, with the only obvious difference being the significantly reduced possibility of an exception due to the RAM being exhausted.

Since:

Coherence 3.7

Author:

cp/cf 2010-06-24

Nested Class Summary

protected class	RamJournalRM.BufferPool A pooling implementation for WriteBuffer objects.
static class	RamJournalRM.DefaultDependencies The DefaultDependencies class provides a default implementation of Dependencies.
static interface	RamJournalRM.Dependencies The Dependencies interface provides a RamJournalRM object with its external dependencies.
protected class	RamJournalRM.FlashConsumer The FlashConsumer is a conduit between this RamJournalRM and the FlashJournalRM, allowing this journal to read/write to/from the other flash journal, and for the flash journal to notify this journal of lifecycle and other events.
protected class	RamJournalRM.JournalFile A JournalFile is created for each RAM buffer used to store the journal contents.
protected class	RamJournalRM.JournalImpl A Journal implementation managed by this Journal Resource Manager.

Nested classes/interfaces inherited from class com.tangosol.io.journal.AbstractJournalRM

AbstractJournalRM.CollectorDaemon

Field Summary

protected RamJournalRM.FlashConsumer	m_flash The conduit between this RAM journal resource manager and the flash journal resource manager.
protected FlashJournalRM	m_jrnlrm A reference to a flash journal resource manager to use to read/write to/from flash for data that doesn't fit in memory.
protected RamJournalRM.BufferPool	m_pool The buffer pool.
protected static long	MASK_FILE_ID The 64-bit bitmask for the file index.
protected static long	MASK_LENGTH The 64-bit bitmask for the value length.
protected static long	MASK_OFFSET The 64-bit bitmask for the value offset.
protected static long	MASK_RAM_FLAG The 64-bit bitmask for the RAM flag.
protected static int	SHIFT_FILE_ID The location of the file id within the ticket.
protected static int	SHIFT_LENGTH The location of the value-length within the ticket.
protected static int	SHIFT_OFFSET The location of the value-offset within the ticket.
protected static int	SHIFT_RAM_FLAG The location of the RAM flag within the ticket.
protected static java.lang.String	TYPE_NAME A simple type name to be used as part of MBean Objectnames.

Fields inherited from class com.tangosol.io.journal.AbstractJournalRM

DEDUPE_ENABLED, m_ajournalfile, m_cJournalFiles, m_daemonCollector, m_dblFileSizeFactor, m_dependencies, m_dflHighestLoadFactor, m_jrnlfile, m_loader, m_nState, m_setJournals, MASK_COMPACT_FLAG, MASK_COMPACT_LENGTH, SHIFT_COMPACT_FLAG, SHIFT_COMPACT_LENGTH, STATE_CONFIGURED, STATE_INITIAL, STATE_RUNNING, STATE_STOPPED, STATE_STOPPING

Constructor Summary

RamJournalRM(Cluster cluster, RamJournalRM.Dependencies deps)
Construct a RAMJournalRM.

RamJournalRM(Cluster cluster, RamJournalRM.Dependencies deps, FlashJournalRM jrnlmr)
Construct a RAMJournalRM.

Method Summary

RamJournalRM.DefaultDependencies	cloneDependencies(AbstractJournalRM.Dependencies deps) Clone the Dependencies object.
protected long	encodeTicket(int nFile, long of, int cb) Encode a file number, offset and length into a "ticket".
protected int	extractFileId(long lTicket) Extract a file index (a file ID) from the passed ticket.
protected int	extractLength(long lTicket) Extract a file offset from the passed ticket.
protected long	extractOffset(long lTicket) Extract a file offset from the passed ticket.
int	getBacklogCount() Determine the number of serialized values that have not yet been persisted to disk.
int	getBacklogSize() Determine the total size in bytes of the serialized values that have not yet been persisted to disk.
protected RamJournalRM.BufferPool	getBufferPool() Obtain the BufferPool instance.
int	getBufferSize() Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.
double	getCollectorLoadFactor() Determine the load factor threshold at which files become eligible for garbage collection (compaction).
RamJournalRM.Dependencies	getDependencies() Return the Dependencies object.
protected java.lang.String	getDescription() Format the object attributes into a String for inclusion in the String returned from the AbstractJournalRM.toString() method.
protected long	getEvacuationMask() Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
protected RamJournalRM.FlashConsumer	getFlashConsumer() Obtain the FlashConsumer object used to read/write flash.
protected Journal	getFlashJournal() Obtain the Journal used to read/write flash.
FlashJournalRM	getFlashJournalRM() Obtain the FlashJournalRM used to store large values or whatever values don't fit in the configured amount of RAM.
int	getHighFileCount() Determine the high file count for this Journal.
protected RamJournalRM.JournalFile	getJournalFile(int nFileId) Obtain a JournalFile by its ID.
int	getMaxBacklogSize() Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.
protected int	getMaxJournalFiles() Determine the maximum number of journal files.
long	getMaxPoolSize() Determine the maximum size (in bytes) of the buffers that can be held by the pool.
long	getMaxTotalRam() Determine the total amount, in bytes, of RAM that will be used for the Journal.
protected long	getMinCollectorSleepMillis() Determine the minimum sleep cycle between collections.
protected java.lang.String	getName() Returns the name value to be used in MBean Objectname key "name".
int	getPoolSize() Determine the size (in bytes) of the buffers that are currently available in the pool.
protected AbstractJournalRM.JournalFile	getSurvivorFile() Return a AbstractJournalRM.JournalFile based on the context of the calling thread or null if it is determined that an exhaustive evacuation will not reclaim sufficient memory.
protected java.lang.String	getTicketDescription(long lTicket) Format the information encoded in a ticket into a debug string.
protected RamJournalRM.BufferPool	instantiateBufferPool() Factory: Instantiate a pool of buffers.
protected RamJournalRM.FlashConsumer	instantiateConsumer() Factory: Instantiate a FlashConsumer object.
protected RamJournalRM.JournalImpl	instantiateJournal() Factory: Instantiate a JournalImpl object.
protected RamJournalRM.JournalFile	instantiateJournalFile(int nFile) Factory: Instantiate a JournalFile or subclass thereof.
protected boolean	isCompact(long lTicket) Determine if the ticket is in the compact form.
protected boolean	isDedupEnabled() Determine if this journal resource manager should de-dup the keys that its JournalBinaryStore instances are managing.
protected boolean	isFlash(long lTicket) Determine if the ticket is neither in compact or RAM form.
boolean	isNioRam() Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).
protected boolean	isRam(long lTicket) Determine if the ticket is in the RAM form.
protected boolean	isSingleEvacuation() Determine if this journal resource manager should only evacuate one single JournalFile (at the most) per collection cycle.
protected void	onDependencies(AbstractJournalRM.Dependencies deps) Process the Dependencies after they have been injected into this object.
protected boolean	shouldNotifyCollector() Check if the Collector daemon needs to be notified to come out of sleep
protected void	startThreads() Create and start the various threads used by this Journal Resource Manager.
void	stop() Hard-stop the controllable service.

Methods inherited from class com.tangosol.io.journal.AbstractJournalRM

calculateCurrentLoadFactor, close, compact, configure, createBinaryStore, createJournal, destroyBinaryStore, dispose, encodeTicket, ensureCurrentJournalFile, extractBinary, getBinaryStoreCount, getCluster, getCollectorDaemon, getCollectorTimeout, getCompactionCount, getContextClassLoader, getCurrentCollectorLoadFactor, getExhaustiveCompactionCount, getExhaustiveCompactionTime, getFileCount, getGuardian, getHighestLoadFactor, getMaxCollectorSleepMillis, getMaxFileSize, getMaxJournalFilesNumber, getMaxJournalSize, getMaxValueSize, getState, getStateDescription, getTotalDataSize, getTotalFileSize, instantiateCollectorDaemon, isRunning, iterateJournalFiles, iterateJournals, registerJournal, registerMBean, resetStatistics, setContextClassLoader, setState, shutdown, start, stopThreads, toString, unregisterJournal, unregisterMBean

Field Detail

SHIFT_RAM_FLAG

protected static final int SHIFT_RAM_FLAG

The location of the RAM flag within the ticket.

See Also:

Constant Field Values

MASK_RAM_FLAG

protected static final long MASK_RAM_FLAG

The 64-bit bitmask for the RAM flag.

The RAM flag is the bit that signifies that the ticket was produced by the RAM Journal Resource Manager, as opposed to either a "real" compact ticket or a delegated flash ticket.

See Also:

Constant Field Values

SHIFT_FILE_ID

protected static final int SHIFT_FILE_ID

The location of the file id within the ticket.

See Also:

Constant Field Values

MASK_FILE_ID

protected static final long MASK_FILE_ID

The 64-bit bitmask for the file index. The 9 bits from 61-53.

See Also:

Constant Field Values

SHIFT_OFFSET

protected static final int SHIFT_OFFSET

The location of the value-offset within the ticket.

See Also:

Constant Field Values

MASK_OFFSET

protected static final long MASK_OFFSET

The 64-bit bitmask for the value offset. The 31 bits 52-22.

See Also:

Constant Field Values

SHIFT_LENGTH

protected static final int SHIFT_LENGTH

The location of the value-length within the ticket.

See Also:

Constant Field Values

MASK_LENGTH

protected static final long MASK_LENGTH

The 64-bit bitmask for the value length. The 22 bits 21-0.

See Also:

Constant Field Values

TYPE_NAME

protected static final java.lang.String TYPE_NAME

A simple type name to be used as part of MBean Objectnames.

See Also:

Constant Field Values

m_pool

protected RamJournalRM.BufferPool m_pool

The buffer pool.

m_jrnlrm

protected FlashJournalRM m_jrnlrm

A reference to a flash journal resource manager to use to read/write to/from flash for data that doesn't fit in memory.

m_flash

protected RamJournalRM.FlashConsumer m_flash

The conduit between this RAM journal resource manager and the flash journal resource manager.

Constructor Detail

RamJournalRM

public RamJournalRM(Cluster cluster,
                    RamJournalRM.Dependencies deps)

Construct a RAMJournalRM.

Parameters:

cluster - the Cluster for which the RamJournalRM is storing data

deps - the Dependencies object

RamJournalRM

public RamJournalRM(Cluster cluster,
                    RamJournalRM.Dependencies deps,
                    FlashJournalRM jrnlmr)

Construct a RAMJournalRM.

Parameters:

cluster - the Cluster for which the RamJournalRM is storing data

deps - the Dependencies object

jrnlmr - the Flash Journal object

Method Detail

stop

public void stop()

Hard-stop the controllable service. Use Controllable.shutdown() for normal service termination. Calling this method for a service that has already stopped has no effect.

Specified by:

stop in interface Controllable

Overrides:

stop in class AbstractJournalRM

getBacklogCount

public int getBacklogCount()

Determine the number of serialized values that have not yet been persisted to disk.

This property is specific to a FlashJournalRM implementation.

Returns:

the number of serialized values queued to be written

getBacklogSize

public int getBacklogSize()

Determine the total size in bytes of the serialized values that have not yet been persisted to disk. This value is also referred to as the "backlog".

This property is specific to a FlashJournalRM implementation.

Returns:

the number of bytes queued to be written

getMaxBacklogSize

public int getMaxBacklogSize()

Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.

This property is specific to a FlashJournalRM implementation.

Returns:

the maximum allowable size of the backlog

getPoolSize

public int getPoolSize()

Determine the size (in bytes) of the buffers that are currently available in the pool. This is not a measurement of how many buffers are currently in use or how many have been allocated.

This property is specific to a FlashJournalRM implementation.

Returns:

the total size in bytes of all of the buffers that are currently in the buffer pool

getMaxTotalRam

public long getMaxTotalRam()

Determine the total amount, in bytes, of RAM that will be used for the Journal.

This property is specific to a RamJournalRM implementation.

Returns:

the maximum number of bytes that will be allocated for Journal storage

isNioRam

public boolean isNioRam()

Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).

This property is specific to a RamJournalRM implementation.

Returns:

true iff the journal is configured to use NIO "direct buffer" RAM

getMaxPoolSize

public long getMaxPoolSize()

Determine the maximum size (in bytes) of the buffers that can be held by the pool. This is not a limit of how many buffers can be allocated, since some may be in use at any time, but rather how many will be held by the pool (i.e. recycled) as they are released.

Returns:

the maximum size in bytes of all of the buffers that the buffer pool can hold onto

getBufferSize

public int getBufferSize()

Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.

This property is specific to a FlashJournalRM implementation.

Returns:

the size, in bytes, of each buffer

getHighFileCount

public int getHighFileCount()

Determine the high file count for this Journal. Compulsory compaction (GC) occurs when the journal file count reaches the high file count.

This property is specific to a FlashJournalRM implementation.

Returns:

the high file count for this journal

getFlashJournalRM

public FlashJournalRM getFlashJournalRM()

Obtain the FlashJournalRM used to store large values or whatever values don't fit in the configured amount of RAM.

Returns:

the FlashJournalRM used by this RamJournalRM, or null if none

getName

protected java.lang.String getName()

Returns the name value to be used in MBean Objectname key "name".

Specified by:

getName in class AbstractJournalRM

Returns:

the name

getDependencies

public RamJournalRM.Dependencies getDependencies()

Return the Dependencies object.

Overrides:

getDependencies in class AbstractJournalRM

Returns:

the journal dependencies object

cloneDependencies

public RamJournalRM.DefaultDependencies cloneDependencies(AbstractJournalRM.Dependencies deps)

Clone the Dependencies object.

Specified by:

cloneDependencies in class AbstractJournalRM

Parameters:

deps - the populated Dependencies object

Returns:

the cloned Dependencies object

onDependencies

protected void onDependencies(AbstractJournalRM.Dependencies deps)

Process the Dependencies after they have been injected into this object.

Overrides:

onDependencies in class AbstractJournalRM

Parameters:

deps - the populated Dependencies object

startThreads

protected void startThreads()

Create and start the various threads used by this Journal Resource Manager.

Overrides:

startThreads in class AbstractJournalRM

getDescription

protected java.lang.String getDescription()

Format the object attributes into a String for inclusion in the String returned from the AbstractJournalRM.toString() method.

Overrides:

getDescription in class AbstractJournalRM

Returns:

a comma-delimited String listing the attributes of this object in the form "attribute=value"

getCollectorLoadFactor

public double getCollectorLoadFactor()

Determine the load factor threshold at which files become eligible for garbage collection (compaction). The load factor threshold is the ratio between the portion of the file used for live data and the total size of the file, and is expressed as a double value in the interval (0.0, 1.0).

When the portion of released data exceeds (1 - threshold), the file becomes eligible for compaction (garbage collection).

Specified by:

getCollectorLoadFactor in interface JournalMBean

Overrides:

getCollectorLoadFactor in class AbstractJournalRM

Returns:

the compaction factor threshold between 0.0 and 1.0

shouldNotifyCollector

protected boolean shouldNotifyCollector()

Check if the Collector daemon needs to be notified to come out of sleep

Overrides:

shouldNotifyCollector in class AbstractJournalRM

Returns:

true if collector daemon should be notified

getMaxJournalFiles

protected int getMaxJournalFiles()

Determine the maximum number of journal files.

Overrides:

getMaxJournalFiles in class AbstractJournalRM

Returns:

the maximum number of simultaneous journal files that will be used to store the contents of this journal

getMinCollectorSleepMillis

protected long getMinCollectorSleepMillis()

Determine the minimum sleep cycle between collections.

Overrides:

getMinCollectorSleepMillis in class AbstractJournalRM

Returns:

the minimum number of milliseconds to sleep between collections

isDedupEnabled

protected boolean isDedupEnabled()

Determine if this journal resource manager should de-dup the keys that its JournalBinaryStore instances are managing.

Overrides:

isDedupEnabled in class AbstractJournalRM

Returns:

true iff this journal resource manager should routinely cause its JournalBinaryStore instances to de-dup their keys

isSingleEvacuation

protected boolean isSingleEvacuation()

Determine if this journal resource manager should only evacuate one single JournalFile (at the most) per collection cycle.

Overrides:

isSingleEvacuation in class AbstractJournalRM

Returns:

true iff this journal resource manager should only evacuate the emptiest JournalFile instead of all the JournalFile instances that qualify for evacuation

getFlashJournal

protected Journal getFlashJournal()

Obtain the Journal used to read/write flash.

Returns:

the flash Journal or null if there is none

getSurvivorFile

protected AbstractJournalRM.JournalFile getSurvivorFile()

Return a AbstractJournalRM.JournalFile based on the context of the calling thread or null if it is determined that an exhaustive evacuation will not reclaim sufficient memory. This method should only be invoked in a "depleted" state when:

• the maximum number of files is used
• all those files are non-appending

When called on a client thread, any operation against this file will block until the full compaction is performed. There is no guarantee that compaction results in available memory thus an out of memory condition may still occur.

When called on the CollectorDaemon thread this method returns a normal JournalFile, with the only exception that it will be in excess to the maximum number of permitted journal files and used exclusively by the CollectorDaemon to perform evacuation in a depleted state.

Overrides:

getSurvivorFile in class AbstractJournalRM

Returns:

either a JournalFile whose usage triggers a full collection or a JournalFile used exclusively by the CollectorDaemon. null may be returned if it is determined that an exhaustive evacuation will not reclaim sufficient memory

instantiateJournal

protected RamJournalRM.JournalImpl instantiateJournal()

Factory: Instantiate a JournalImpl object.

Overrides:

instantiateJournal in class AbstractJournalRM

Returns:

a JournalImpl instance or subclass thereof

instantiateConsumer

protected RamJournalRM.FlashConsumer instantiateConsumer()

Factory: Instantiate a FlashConsumer object.

Returns:

an instance of FlashConsumer or a subclass thereof.

getFlashConsumer

protected RamJournalRM.FlashConsumer getFlashConsumer()

Obtain the FlashConsumer object used to read/write flash.

Returns:

the FlashConsumer instance, or null if there is no flash configured to use

getJournalFile

protected RamJournalRM.JournalFile getJournalFile(int nFileId)

Obtain a JournalFile by its ID.

Overrides:

getJournalFile in class AbstractJournalRM

Parameters:

nFileId - the file id (0-511)

Returns:

the JournalFile for that file id, or null if there is currently no JournalFile for that file id

instantiateJournalFile

protected RamJournalRM.JournalFile instantiateJournalFile(int nFile)

Factory: Instantiate a JournalFile or subclass thereof.

Specified by:

instantiateJournalFile in class AbstractJournalRM

Parameters:

nFile - the file number in the range 0-511 (inclusive)

Returns:

a new JournalFile instance

isCompact

protected boolean isCompact(long lTicket)

Determine if the ticket is in the compact form.

Overrides:

isCompact in class AbstractJournalRM

Parameters:

lTicket - a "ticket" that was created by the Journal

Returns:

true iff the ticket is a compact ticket

isRam

protected boolean isRam(long lTicket)

Determine if the ticket is in the RAM form.

Parameters:

lTicket - a "ticket" that was returned previously by the Journal

Returns:

true iff the ticket is a RAM ticket

isFlash

protected boolean isFlash(long lTicket)

Determine if the ticket is neither in compact or RAM form.

Parameters:

lTicket - a "ticket" that was returned previously by the Journal

Returns:

true iff the ticket is from an underlying FlashJournalRM

getEvacuationMask

protected long getEvacuationMask()

Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.

Specified by:

getEvacuationMask in class AbstractJournalRM

Returns:

the the bit mask for the file index within a ticket and whatever other bits are necessary to ensure that a ticket refers to that file

extractFileId

protected int extractFileId(long lTicket)

Extract a file index (a file ID) from the passed ticket.

Specified by:

extractFileId in class AbstractJournalRM

Parameters:

lTicket - a "ticket" that was created by the Journal

Returns:

the file index for the value represented by that ticket

extractOffset

protected long extractOffset(long lTicket)

Extract a file offset from the passed ticket.

Specified by:

extractOffset in class AbstractJournalRM

Parameters:

lTicket - a "ticket" that was created by the Journal

Returns:

the file offset for the value represented by that ticket

extractLength

protected int extractLength(long lTicket)

Extract a file offset from the passed ticket.

Specified by:

extractLength in class AbstractJournalRM

Parameters:

lTicket - a "ticket" that was created by the Journal

Returns:

the file offset for the value represented by that ticket

encodeTicket

protected long encodeTicket(int nFile,
                            long of,
                            int cb)

Encode a file number, offset and length into a "ticket".

Specified by:

encodeTicket in class AbstractJournalRM

Parameters:

nFile - a file number (aka a file index)

of - a value offset

cb - a value length

Returns:

a "ticket" containing the file number, offset and length

getTicketDescription

protected java.lang.String getTicketDescription(long lTicket)

Format the information encoded in a ticket into a debug string.

Overrides:

getTicketDescription in class AbstractJournalRM

Parameters:

lTicket - the ticket value

Returns:

a description of the contents of the ticket value

instantiateBufferPool

protected RamJournalRM.BufferPool instantiateBufferPool()

Factory: Instantiate a pool of buffers.

Returns:

a BufferPool or subclass thereof

getBufferPool

protected RamJournalRM.BufferPool getBufferPool()

Obtain the BufferPool instance.

Returns:

the BufferPool for this FlashJournalRM

Skip navigation links