Skip navigation links

com.tangosol.io.journal
Class FlashJournalRM

java.lang.Object
com.tangosol.io.journal.AbstractJournalRM
com.tangosol.io.journal.FlashJournalRM

All Implemented Interfaces:

com.oracle.common.base.Disposable, BinaryStoreManager, ClassLoaderAware, JournalMBean, Controllable

public class FlashJournalRM

extends AbstractJournalRM

A FlashJournalRM manages disk I/O for temporary journal-based files, and acts as a shared resource for any number of journals that share a common configuration.

To use the Resource Manager, configure it using Dependency Injection by passing in the populated dependency object at instantiation. Once configured, start the Resource Manager via the AbstractJournalRM.start() method, and then obtain BinaryStore instances via the AbstractJournalRM.createBinaryStore() method. When a BinaryStore instance is no longer required, dispose of it via the Disposable interface, and similarly stop() or AbstractJournalRM.dispose() of the Resource Manager when it is no longer needed to ensure that any resources it allocated are cleaned up and released.

While the Resource Manager is running, there will be one or more files created in the specified temporary directory (or the default temporary directory if none is specified). The number of files is not related to the number of BinaryStore instances, but rather to the total amount of data that the Resource Manager is managing at any given time. Additionally, the Resource Manager uses three daemon threads to help manage the data: "Journal-Preparer", "Journal-Writer" and "Journal-Collector". These threads respectively lay out the data to be written, write the data, and perform asynchronous GC of the storage files.

The limits on the journal are as follows:

• Values are limited by default to 64MB;
• An individual file is limited by default to 2GB (and maximum 4GB);
• A journal is composed of up to 512 files;
• A journal is thus limited by default to 1TB, with a theoretical maximum of 2TB

Since:

Coherence 3.7

Author:

cp/cf 2010-06-10

Nested Class Summary

protected static class	FlashJournalRM.Buffer A Buffer is a handy shell with some helper methods around a byte[] and a ByteBuffer.
protected static class	FlashJournalRM.BufferPool The BufferPool class recycles up to a maximum number of Buffer objects.
protected static class	FlashJournalRM.CongestionPolicy A CongestionPolicy defines how to behave when the currently active FlashJournalRM.JournalFile becomes congested.
static class	FlashJournalRM.DefaultDependencies The DefaultDependencies class provides a default implementation of Dependencies.
static interface	FlashJournalRM.Dependencies The Dependencies interface provides a FlashJournalRM object with its external dependencies.
protected static class	FlashJournalRM.FailCongestionPolicy A FlashJournalRM.CongestionPolicy that fails, by throwing an unchecked exception, when onCongestion is invoked.
protected class	FlashJournalRM.JournalFile A JournalFile is created for each OS file used to store the journal contents.
protected class	FlashJournalRM.PreparerDaemon The PreparerDaemon takes the enqueued Binary values for the JournalFile objects, and lays them out in Buffer objects from the BufferPool, enqueuing them for the WriterDaemon to write.
protected static class	FlashJournalRM.WaitCongestionPolicy A FlashJournalRM.CongestionPolicy which blocks the current thread on the FlashJournalRM.JournalFile's monitor until awoken.
protected class	FlashJournalRM.WriterDaemon The WriterDaemon takes the enqueued Buffer objects from the PreparerDaemon, and appends them to a JournalFile.

Nested classes/interfaces inherited from class com.tangosol.io.journal.AbstractJournalRM

AbstractJournalRM.CollectorDaemon, AbstractJournalRM.JournalImpl

Field Summary

protected static FlashJournalRM.FailCongestionPolicy	FAIL_CONGESTION_ACTION A FlashJournalRM.CongestionPolicy which fails, by throwing an unchecked exception, when the file is deemed congested.
protected java.util.concurrent.atomic.AtomicInteger	m_cbBacklog Total amount of data enqueued to be written.
protected FlashJournalRM.CongestionPolicy	m_congestionPolicy The FlashJournalRM.CongestionPolicy to be used when in a congested state.
protected FlashJournalRM.PreparerDaemon	m_daemonPreparer The "rendering" thread that renders (copies and encrypts) the queued binaries into buffers to be written.
protected FlashJournalRM.WriterDaemon	m_daemonWriter The "writing" thread that just does appends of prepared buffers to journal files.
protected java.io.File	m_dirTemp Directory to store the temporary files.
protected boolean	m_fLoggedDirectory Set to true after the directory location has been logged.
protected java.util.concurrent.ConcurrentMap	m_mapLocators Map of locators to Binary values that have been queued to be written.
protected FlashJournalRM.BufferPool	m_pool The BufferPool of Buffer objects that are used to format data to append to the journal.
protected static long	MASK_FILE_ID The 64-bit bitmask for the file index.
protected static long	MASK_LENGTH The 64-bit bitmask for the value length.
protected static long	MASK_OFFSET The 64-bit bitmask for the value offset.
protected static int	SHIFT_FILE_ID The location of the file id within the ticket.
protected static int	SHIFT_LENGTH The location of the value-length within the ticket.
protected static int	SHIFT_OFFSET The location of the value-offset within the ticket.
protected static int	SHIFT_OFFSET_RAW From an encoding point of view, the raw offset is shifted four bits less than necessary, because the four least significant bits of the offset are always zero (due to paragraph alignment).
protected static java.lang.String	TYPE_NAME A simple name to be used as part of MBean ObjectNames.
protected static FlashJournalRM.WaitCongestionPolicy	WAIT_CONGESTION_ACTION The default congestion policy which blocks the client thread until awoken by the FlashJournalRM.WriterDaemon.

Fields inherited from class com.tangosol.io.journal.AbstractJournalRM

DEDUPE_ENABLED, m_ajournalfile, m_cJournalFiles, m_daemonCollector, m_dblFileSizeFactor, m_dependencies, m_dflHighestLoadFactor, m_jrnlfile, m_loader, m_nState, m_setJournals, MASK_COMPACT_FLAG, MASK_COMPACT_LENGTH, SHIFT_COMPACT_FLAG, SHIFT_COMPACT_LENGTH, STATE_CONFIGURED, STATE_INITIAL, STATE_RUNNING, STATE_STOPPED, STATE_STOPPING

Constructor Summary

FlashJournalRM(Cluster cluster, FlashJournalRM.Dependencies deps)
Construct a FlashJournalRM.

Method Summary

protected double	calculateCurrentLoadFactor() Calculate current load factor for compaction (GC).
FlashJournalRM.DefaultDependencies	cloneDependencies(AbstractJournalRM.Dependencies deps) Clone the Dependencies object.
protected long	encodeTicket(int nFile, long of, int cb) Encode a file number, offset and length into a "ticket".
protected int	extractFileId(long lTicket) Extract a file index (a file ID) from the passed ticket.
protected int	extractLength(long lTicket) Extract a file offset from the passed ticket.
protected long	extractOffset(long lTicket) Extract a file offset from the passed ticket.
int	getBacklogCount() Determine the number of serialized values that have not yet been persisted to disk.
int	getBacklogSize() Determine the total size in bytes of the serialized values that have not yet been persisted to disk.
protected FlashJournalRM.BufferPool	getBufferPool() Obtain the BufferPool instance.
int	getBufferSize() Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.
FlashJournalRM.Dependencies	getDependencies() Return the Dependencies object.
protected java.lang.String	getDescription() Format the object attributes into a String for inclusion in the String returned from the AbstractJournalRM.toString() method.
java.io.File	getDirectoryFile() Determine the directory used to store the underlying disk files for the journal.
protected long	getEvacuationMask() Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
int	getHighFileCount() Determine the high file count for this Journal.
protected FlashJournalRM.JournalFile	getJournalFile(int nFileId) Obtain a JournalFile by its ID.
int	getMaxBacklogSize() Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.
long	getMaxPoolSize() Determine the maximum size (in bytes) of the buffers that can be held by the pool.
long	getMaxTotalRam() Determine the total amount, in bytes, of RAM that will be used for the Journal.
protected java.lang.String	getName() Returns the name value to be used in MBean Objectname key "name".
int	getPoolSize() Determine the size (in bytes) of the buffers that are currently available in the pool.
protected FlashJournalRM.PreparerDaemon	getPreparerDaemon() Obtain the PreparerDaemon instance.
long	getPurgeDelayMillis() Return the delay since the last touch of a tmp file until it is eligible for purging.
protected FlashJournalRM.WriterDaemon	getWriterDaemon() Obtain the WriterDaemon instance.
protected long	getWriterTimeout() Return the number of milliseconds the Writer daemon can be unresponsive prior to considering it timed out.
protected FlashJournalRM.BufferPool	instantiateBufferPool() Factory: Instantiate a pool of buffers.
protected FlashJournalRM.JournalFile	instantiateJournalFile(int nFile) Factory: Instantiate a JournalFile or subclass thereof.
protected FlashJournalRM.PreparerDaemon	instantiatePreparerDaemon() Factory: Instantiate a PreparerDaemon.
protected FlashJournalRM.WriterDaemon	instantiateWriterDaemon() Factory: Instantiate a WriterDaemon.
boolean	isNioRam() Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).
protected void	onDependencies(AbstractJournalRM.Dependencies deps) Process the Dependencies after they have been injected into this object.
protected void	purgeOldTempFiles() Delete any old temp files that weren't cleaned up properly by a previous are configured to be "deleted on exit", it is possible that the temp files would not have been deleted if the JVM crashed or if there was a fault at the physical machine level.
protected boolean	shouldNotifyCollector() Check if the Collector daemon needs to be notified to come out of sleep
protected void	startThreads() Create and start the various threads used by this Journal Resource Manager.
void	stop() Hard-stop the controllable service.
protected void	stopThreads() Stop the various threads used by this Journal Resource Manager.

Methods inherited from class com.tangosol.io.journal.AbstractJournalRM

close, compact, configure, createBinaryStore, createJournal, destroyBinaryStore, dispose, encodeTicket, ensureCurrentJournalFile, extractBinary, getBinaryStoreCount, getCluster, getCollectorDaemon, getCollectorLoadFactor, getCollectorTimeout, getCompactionCount, getContextClassLoader, getCurrentCollectorLoadFactor, getExhaustiveCompactionCount, getExhaustiveCompactionTime, getFileCount, getGuardian, getHighestLoadFactor, getMaxCollectorSleepMillis, getMaxFileSize, getMaxJournalFiles, getMaxJournalFilesNumber, getMaxJournalSize, getMaxValueSize, getMinCollectorSleepMillis, getState, getStateDescription, getSurvivorFile, getTicketDescription, getTotalDataSize, getTotalFileSize, instantiateCollectorDaemon, instantiateJournal, isCompact, isDedupEnabled, isRunning, isSingleEvacuation, iterateJournalFiles, iterateJournals, registerJournal, registerMBean, resetStatistics, setContextClassLoader, setState, shutdown, start, toString, unregisterJournal, unregisterMBean

Field Detail

SHIFT_FILE_ID

protected static final int SHIFT_FILE_ID

The location of the file id within the ticket.

See Also:

Constant Field Values

MASK_FILE_ID

protected static final long MASK_FILE_ID

The 64-bit bitmask for the file index. The 9 bits from 62-54.

See Also:

Constant Field Values

SHIFT_OFFSET

protected static final int SHIFT_OFFSET

The location of the value-offset within the ticket.

See Also:

Constant Field Values

SHIFT_OFFSET_RAW

protected static final int SHIFT_OFFSET_RAW

From an encoding point of view, the raw offset is shifted four bits less than necessary, because the four least significant bits of the offset are always zero (due to paragraph alignment).

See Also:

Constant Field Values

MASK_OFFSET

protected static final long MASK_OFFSET

The 64-bit bitmask for the value offset. The 28 bits 53-26.

Note that the least significant four bits of the offset MUST ALWAYS be zero, because they are NOT encoded into the ticket -- hence the 16-byte paragraph boundary for entries.

See Also:

Constant Field Values

SHIFT_LENGTH

protected static final int SHIFT_LENGTH

The location of the value-length within the ticket.

See Also:

Constant Field Values

MASK_LENGTH

protected static final long MASK_LENGTH

The 64-bit bitmask for the value length. The 26 bits 25-0.

See Also:

Constant Field Values

TYPE_NAME

protected static final java.lang.String TYPE_NAME

A simple name to be used as part of MBean ObjectNames.

See Also:

Constant Field Values

WAIT_CONGESTION_ACTION

protected static final FlashJournalRM.WaitCongestionPolicy WAIT_CONGESTION_ACTION

The default congestion policy which blocks the client thread until awoken by the FlashJournalRM.WriterDaemon.

FAIL_CONGESTION_ACTION

protected static final FlashJournalRM.FailCongestionPolicy FAIL_CONGESTION_ACTION

A FlashJournalRM.CongestionPolicy which fails, by throwing an unchecked exception, when the file is deemed congested.

m_congestionPolicy

protected volatile FlashJournalRM.CongestionPolicy m_congestionPolicy

The FlashJournalRM.CongestionPolicy to be used when in a congested state.

m_dirTemp

protected java.io.File m_dirTemp

Directory to store the temporary files.

m_fLoggedDirectory

protected boolean m_fLoggedDirectory

Set to true after the directory location has been logged.

m_cbBacklog

protected java.util.concurrent.atomic.AtomicInteger m_cbBacklog

Total amount of data enqueued to be written.

m_mapLocators

protected java.util.concurrent.ConcurrentMap m_mapLocators

Map of locators to Binary values that have been queued to be written. A "locator" is a ticket without an encoded length.

m_pool

protected FlashJournalRM.BufferPool m_pool

The BufferPool of Buffer objects that are used to format data to append to the journal.

m_daemonPreparer

protected FlashJournalRM.PreparerDaemon m_daemonPreparer

The "rendering" thread that renders (copies and encrypts) the queued binaries into buffers to be written.

m_daemonWriter

protected FlashJournalRM.WriterDaemon m_daemonWriter

The "writing" thread that just does appends of prepared buffers to journal files.

Constructor Detail

FlashJournalRM

public FlashJournalRM(Cluster cluster,
                      FlashJournalRM.Dependencies deps)

Construct a FlashJournalRM.

Parameters:

cluster - the Cluster for which the FlashJournalRM is storing data

deps - the Dependencies object

Method Detail

stop

public void stop()

Hard-stop the controllable service. Use Controllable.shutdown() for normal service termination. Calling this method for a service that has already stopped has no effect.

Specified by:

stop in interface Controllable

Overrides:

stop in class AbstractJournalRM

getMaxTotalRam

public long getMaxTotalRam()

Determine the total amount, in bytes, of RAM that will be used for the Journal.

This property is specific to a RamJournalRM implementation.

Returns:

the maximum number of bytes that will be allocated for Journal storage

isNioRam

public boolean isNioRam()

Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).

This property is specific to a RamJournalRM implementation.

Returns:

true iff the journal is configured to use NIO "direct buffer" RAM

getMaxBacklogSize

public int getMaxBacklogSize()

Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.

This property is specific to a FlashJournalRM implementation.

Returns:

the maximum allowable size of the backlog

getBacklogSize

public int getBacklogSize()

Determine the total size in bytes of the serialized values that have not yet been persisted to disk. This value is also referred to as the "backlog".

This property is specific to a FlashJournalRM implementation.

Returns:

the number of bytes queued to be written

getBacklogCount

public int getBacklogCount()

Determine the number of serialized values that have not yet been persisted to disk.

This property is specific to a FlashJournalRM implementation.

Returns:

the number of serialized values queued to be written

getBufferSize

public int getBufferSize()

Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.

This property is specific to a FlashJournalRM implementation.

Returns:

the size, in bytes, of each buffer

getMaxPoolSize

public long getMaxPoolSize()

Determine the maximum size (in bytes) of the buffers that can be held by the pool. This is not a limit of how many buffers can be allocated, since some may be in use at any time, but rather how many will be held by the pool (i.e. recycled) as they are released.

Returns:

the maximum size in bytes of all of the buffers that the buffer pool can hold onto

getPoolSize

public int getPoolSize()

Determine the size (in bytes) of the buffers that are currently available in the pool. This is not a measurement of how many buffers are currently in use or how many have been allocated.

This property is specific to a FlashJournalRM implementation.

Returns:

the total size in bytes of all of the buffers that are currently in the buffer pool

getHighFileCount

public int getHighFileCount()

Determine the high file count for this Journal. Compulsory compaction (GC) occurs when the journal file count reaches the high file count.

This property is specific to a FlashJournalRM implementation.

Returns:

the high file count for this journal

getDirectoryFile

public java.io.File getDirectoryFile()

Determine the directory used to store the underlying disk files for the journal.

Returns:

the directory that is used to hold the journal files, or null if the default temporary directory is used

getPurgeDelayMillis

public long getPurgeDelayMillis()

Return the delay since the last touch of a tmp file until it is eligible for purging. Tmp files are purged at startup of the FlashJournalRM to clean up tmp files from a previous instance of the FlashJournalRM.

Returns:

the delay in milliseconds

getName

protected java.lang.String getName()

Returns the name value to be used in MBean Objectname key "name".

Specified by:

getName in class AbstractJournalRM

Returns:

the name

getWriterTimeout

protected long getWriterTimeout()

Return the number of milliseconds the Writer daemon can be unresponsive prior to considering it timed out.

Returns:

the number of milliseconds the Writer can be unresponsive

getDependencies

public FlashJournalRM.Dependencies getDependencies()

Return the Dependencies object.

Overrides:

getDependencies in class AbstractJournalRM

Returns:

the journal dependencies object

cloneDependencies

public FlashJournalRM.DefaultDependencies cloneDependencies(AbstractJournalRM.Dependencies deps)

Clone the Dependencies object.

Specified by:

cloneDependencies in class AbstractJournalRM

Parameters:

deps - the populated Dependencies object

Returns:

the cloned Dependencies object

onDependencies

protected void onDependencies(AbstractJournalRM.Dependencies deps)

Process the Dependencies after they have been injected into this object.

Overrides:

onDependencies in class AbstractJournalRM

Parameters:

deps - the populated Dependencies object

startThreads

protected void startThreads()

Create and start the various threads used by this Journal Resource Manager.

Overrides:

startThreads in class AbstractJournalRM

purgeOldTempFiles

protected void purgeOldTempFiles()

Delete any old temp files that weren't cleaned up properly by a previous are configured to be "deleted on exit", it is possible that the temp files would not have been deleted if the JVM crashed or if there was a fault at the physical machine level.

stopThreads

protected void stopThreads()

Stop the various threads used by this Journal Resource Manager.

Overrides:

stopThreads in class AbstractJournalRM

calculateCurrentLoadFactor

protected double calculateCurrentLoadFactor()

Calculate current load factor for compaction (GC).

Overrides:

calculateCurrentLoadFactor in class AbstractJournalRM

Returns:

calculated load factor

shouldNotifyCollector

protected boolean shouldNotifyCollector()

Check if the Collector daemon needs to be notified to come out of sleep

Overrides:

shouldNotifyCollector in class AbstractJournalRM

Returns:

true if collector daemon should be notified

getDescription

protected java.lang.String getDescription()

Format the object attributes into a String for inclusion in the String returned from the AbstractJournalRM.toString() method.

Overrides:

getDescription in class AbstractJournalRM

Returns:

a comma-delimited String listing the attributes of this object in the form "attribute=value"

getJournalFile

protected FlashJournalRM.JournalFile getJournalFile(int nFileId)

Obtain a JournalFile by its ID.

Overrides:

getJournalFile in class AbstractJournalRM

Parameters:

nFileId - the file id (0-511)

Returns:

the JournalFile for that file id, or null if there is currently no JournalFile for that file id

instantiateJournalFile

protected FlashJournalRM.JournalFile instantiateJournalFile(int nFile)

Factory: Instantiate a JournalFile or subclass thereof.

Specified by:

instantiateJournalFile in class AbstractJournalRM

Parameters:

nFile - the file number in the range 0-511 (inclusive)

Returns:

a new JournalFile instance

instantiateBufferPool

protected FlashJournalRM.BufferPool instantiateBufferPool()

Factory: Instantiate a pool of buffers.

Returns:

a BufferPool or subclass thereof

getBufferPool

protected FlashJournalRM.BufferPool getBufferPool()

Obtain the BufferPool instance.

Returns:

the BufferPool for this FlashJournalRM

instantiatePreparerDaemon

protected FlashJournalRM.PreparerDaemon instantiatePreparerDaemon()

Factory: Instantiate a PreparerDaemon.

Returns:

a PreparerDaemon or a sub-class thereof

getPreparerDaemon

protected FlashJournalRM.PreparerDaemon getPreparerDaemon()

Obtain the PreparerDaemon instance.

Returns:

the PreparerDaemon for this FlashJournalRM

instantiateWriterDaemon

protected FlashJournalRM.WriterDaemon instantiateWriterDaemon()

Factory: Instantiate a WriterDaemon.

Returns:

a WriterDaemon or a sub-class thereof

getWriterDaemon

protected FlashJournalRM.WriterDaemon getWriterDaemon()

Obtain the WriterDaemon instance.

Returns:

the WriterDaemon for this FlashJournalRM

getEvacuationMask

protected long getEvacuationMask()

Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.

Specified by:

getEvacuationMask in class AbstractJournalRM

Returns:

the the bit mask for the file index within a ticket and whatever other bits are necessary to ensure that a ticket refers to that file

extractFileId

protected int extractFileId(long lTicket)

Extract a file index (a file ID) from the passed ticket.

Specified by:

extractFileId in class AbstractJournalRM

Parameters:

lTicket - a "ticket" that was created by the Journal

Returns:

the file index for the value represented by that ticket

extractOffset

protected long extractOffset(long lTicket)

Extract a file offset from the passed ticket.

Specified by:

extractOffset in class AbstractJournalRM

Parameters:

lTicket - a "ticket" that was created by the Journal

Returns:

the file offset for the value represented by that ticket

extractLength

protected int extractLength(long lTicket)

Extract a file offset from the passed ticket.

Specified by:

extractLength in class AbstractJournalRM

Parameters:

lTicket - a "ticket" that was created by the Journal

Returns:

the file offset for the value represented by that ticket

encodeTicket

protected long encodeTicket(int nFile,
                            long of,
                            int cb)

Encode a file number, offset and length into a "ticket".

Specified by:

encodeTicket in class AbstractJournalRM

Parameters:

nFile - a file number (aka a file index)

of - a value offset

cb - a value length

Returns:

a "ticket" containing the file number, offset and length

Skip navigation links