public class FlashJournalRM extends AbstractJournalRM
To use the Resource Manager, configure it using Dependency Injection by passing in the populated dependency object at instantiation. Once configured, start the Resource Manager via the AbstractJournalRM.start()
method, and then obtain BinaryStore
instances via the AbstractJournalRM.createBinaryStore()
method. When a BinaryStore instance is no longer required, dispose of it via the Disposable
interface, and similarly stop()
or AbstractJournalRM.dispose()
of the Resource Manager when it is no longer needed to ensure that any resources it allocated are cleaned up and released.
While the Resource Manager is running, there will be one or more files created in the specified temporary directory (or the default temporary directory if none is specified). The number of files is not related to the number of BinaryStore instances, but rather to the total amount of data that the Resource Manager is managing at any given time. Additionally, the Resource Manager uses three daemon threads to help manage the data: "Journal-Preparer", "Journal-Writer" and "Journal-Collector". These threads respectively lay out the data to be written, write the data, and perform asynchronous GC of the storage files.
The limits on the journal are as follows:
Modifier and Type | Class and Description |
---|---|
protected static class |
FlashJournalRM.Buffer
A Buffer is a handy shell with some helper methods around a byte[] and a ByteBuffer.
|
protected static class |
FlashJournalRM.BufferPool
The BufferPool class recycles up to a maximum number of Buffer objects.
|
protected static class |
FlashJournalRM.CongestionPolicy
A CongestionPolicy defines how to behave when the currently active
FlashJournalRM.JournalFile becomes congested. |
static class |
FlashJournalRM.DefaultDependencies
The DefaultDependencies class provides a default implementation of Dependencies.
|
static interface |
FlashJournalRM.Dependencies
The Dependencies interface provides a FlashJournalRM object with its external dependencies.
|
protected static class |
FlashJournalRM.FailCongestionPolicy
A
FlashJournalRM.CongestionPolicy that fails, by throwing an unchecked exception, when FlashJournalRM.FailCongestionPolicy.onCongestion(com.tangosol.io.journal.FlashJournalRM.JournalFile) is invoked. |
protected class |
FlashJournalRM.JournalFile
A JournalFile is created for each OS file used to store the journal contents.
|
protected class |
FlashJournalRM.PreparerDaemon
The PreparerDaemon takes the enqueued Binary values for the JournalFile objects, and lays them out in Buffer objects from the BufferPool, enqueuing them for the WriterDaemon to write.
|
protected static class |
FlashJournalRM.WaitCongestionPolicy
A
FlashJournalRM.CongestionPolicy which blocks the current thread on the FlashJournalRM.JournalFile 's monitor until awoken. |
protected class |
FlashJournalRM.WriterDaemon
The WriterDaemon takes the enqueued Buffer objects from the PreparerDaemon, and appends them to a JournalFile.
|
AbstractJournalRM.CollectorDaemon, AbstractJournalRM.JournalImpl
Modifier and Type | Field and Description |
---|---|
protected static FlashJournalRM.FailCongestionPolicy |
FAIL_CONGESTION_ACTION
A
FlashJournalRM.CongestionPolicy which fails, by throwing an unchecked exception, when the file is deemed congested. |
protected AtomicInteger |
m_cbBacklog
Total amount of data enqueued to be written.
|
protected FlashJournalRM.CongestionPolicy |
m_congestionPolicy
The
FlashJournalRM.CongestionPolicy to be used when in a congested state. |
protected FlashJournalRM.PreparerDaemon |
m_daemonPreparer
The "rendering" thread that renders (copies and encrypts) the queued binaries into buffers to be written.
|
protected FlashJournalRM.WriterDaemon |
m_daemonWriter
The "writing" thread that just does appends of prepared buffers to journal files.
|
protected File |
m_dirTemp
Directory to store the temporary files.
|
protected boolean |
m_fLoggedDirectory
Set to true after the directory location has been logged.
|
protected ConcurrentMap<Long,Binary> |
m_mapLocators
Map of locators to Binary values that have been queued to be written.
|
protected FlashJournalRM.BufferPool |
m_pool
The BufferPool of Buffer objects that are used to format data to append to the journal.
|
protected static long |
MASK_FILE_ID
The 64-bit bitmask for the file index.
|
protected static long |
MASK_LENGTH
The 64-bit bitmask for the value length.
|
protected static long |
MASK_OFFSET
The 64-bit bitmask for the value offset.
|
protected static int |
SHIFT_FILE_ID
The location of the file id within the ticket.
|
protected static int |
SHIFT_LENGTH
The location of the value-length within the ticket.
|
protected static int |
SHIFT_OFFSET
The location of the value-offset within the ticket.
|
protected static int |
SHIFT_OFFSET_RAW
From an encoding point of view, the raw offset is shifted four bits less than necessary, because the four least significant bits of the offset are always zero (due to paragraph alignment).
|
protected static String |
TYPE_NAME
A simple name to be used as part of MBean ObjectNames.
|
protected static FlashJournalRM.WaitCongestionPolicy |
WAIT_CONGESTION_ACTION
The default congestion policy which blocks the client thread until awoken by the
FlashJournalRM.WriterDaemon . |
DEDUPE_ENABLED, m_ajournalfile, m_cJournalFiles, m_daemonCollector, m_dblFileSizeFactor, m_dependencies, m_dflHighestLoadFactor, m_jrnlfile, m_loader, m_nState, m_setJournals, MASK_COMPACT_FLAG, MASK_COMPACT_LENGTH, SHIFT_COMPACT_FLAG, SHIFT_COMPACT_LENGTH, STATE_CONFIGURED, STATE_INITIAL, STATE_RUNNING, STATE_STOPPED, STATE_STOPPING
Constructor and Description |
---|
FlashJournalRM(Cluster cluster, FlashJournalRM.Dependencies deps)
Construct a FlashJournalRM.
|
Modifier and Type | Method and Description |
---|---|
protected double |
calculateCurrentLoadFactor()
Calculate current load factor for compaction (GC).
|
FlashJournalRM.DefaultDependencies |
cloneDependencies(AbstractJournalRM.Dependencies deps)
Clone the Dependencies object.
|
protected long |
encodeTicket(int nFile, long of, int cb)
Encode a file number, offset and length into a "ticket".
|
protected int |
extractFileId(long lTicket)
Extract a file index (a file ID) from the passed ticket.
|
protected int |
extractLength(long lTicket)
Extract a file offset from the passed ticket.
|
protected long |
extractOffset(long lTicket)
Extract a file offset from the passed ticket.
|
int |
getBacklogCount()
Determine the number of serialized values that have not yet been persisted to disk.
|
int |
getBacklogSize()
Determine the total size in bytes of the serialized values that have not yet been persisted to disk.
|
protected FlashJournalRM.BufferPool |
getBufferPool()
Obtain the BufferPool instance.
|
int |
getBufferSize()
Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.
|
FlashJournalRM.Dependencies |
getDependencies()
Return the Dependencies object.
|
protected String |
getDescription()
Format the object attributes into a String for inclusion in the String returned from the
AbstractJournalRM.toString() method. |
File |
getDirectoryFile()
Determine the directory used to store the underlying disk files for the journal.
|
protected long |
getEvacuationMask()
Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
|
int |
getHighFileCount()
Determine the high file count for this Journal.
|
protected FlashJournalRM.JournalFile |
getJournalFile(int nFileId)
Obtain a JournalFile by its ID.
|
int |
getMaxBacklogSize()
Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.
|
long |
getMaxPoolSize()
Determine the maximum size (in bytes) of the buffers that can be held by the pool.
|
long |
getMaxTotalRam()
Determine the total amount, in bytes, of RAM that will be used for the Journal.
|
protected String |
getName()
Returns the name value to be used in MBean Objectname key "name".
|
int |
getPoolSize()
Determine the size (in bytes) of the buffers that are currently available in the pool.
|
protected FlashJournalRM.PreparerDaemon |
getPreparerDaemon()
Obtain the PreparerDaemon instance.
|
long |
getPurgeDelayMillis()
Return the delay since the last touch of a tmp file until it is eligible for purging.
|
protected FlashJournalRM.WriterDaemon |
getWriterDaemon()
Obtain the WriterDaemon instance.
|
protected long |
getWriterTimeout()
Return the number of milliseconds the Writer daemon can be unresponsive prior to considering it timed out.
|
protected FlashJournalRM.BufferPool |
instantiateBufferPool()
Factory: Instantiate a pool of buffers.
|
protected FlashJournalRM.JournalFile |
instantiateJournalFile(int nFile)
Factory: Instantiate a JournalFile or subclass thereof.
|
protected FlashJournalRM.PreparerDaemon |
instantiatePreparerDaemon()
Factory: Instantiate a PreparerDaemon.
|
protected FlashJournalRM.WriterDaemon |
instantiateWriterDaemon()
Factory: Instantiate a WriterDaemon.
|
boolean |
isNioRam()
Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).
|
protected void |
onDependencies(AbstractJournalRM.Dependencies deps)
Process the Dependencies after they have been injected into this object.
|
protected void |
purgeOldTempFiles()
Delete any old temp files that weren't cleaned up properly by a previous are configured to be "deleted on exit", it is possible that the temp files would not have been deleted if the JVM crashed or if there was a fault at the physical machine level.
|
protected boolean |
shouldNotifyCollector()
Check if the Collector daemon needs to be notified to come out of sleep
|
protected void |
startThreads()
Create and start the various threads used by this Journal Resource Manager.
|
void |
stop()
Hard-stop the controllable service.
|
protected void |
stopThreads()
Stop the various threads used by this Journal Resource Manager.
|
close, compact, configure, createBinaryStore, createJournal, destroyBinaryStore, dispose, encodeTicket, ensureCurrentJournalFile, extractBinary, getBinaryStoreCount, getCluster, getCollectorDaemon, getCollectorLoadFactor, getCollectorTimeout, getCompactionCount, getContextClassLoader, getCurrentCollectorLoadFactor, getExhaustiveCompactionCount, getExhaustiveCompactionTime, getFileCount, getGuardian, getHighestLoadFactor, getMaxCollectorSleepMillis, getMaxFileSize, getMaxJournalFiles, getMaxJournalFilesNumber, getMaxJournalSize, getMaxValueSize, getMinCollectorSleepMillis, getState, getStateDescription, getSurvivorFile, getTicketDescription, getTotalDataSize, getTotalFileSize, instantiateCollectorDaemon, instantiateJournal, isCompact, isDedupEnabled, isRunning, isSingleEvacuation, iterateJournalFiles, iterateJournals, registerJournal, registerMBean, resetStatistics, setContextClassLoader, setState, shutdown, start, toString, unregisterJournal, unregisterMBean
protected static final int SHIFT_FILE_ID
protected static final long MASK_FILE_ID
protected static final int SHIFT_OFFSET
protected static final int SHIFT_OFFSET_RAW
protected static final long MASK_OFFSET
Note that the least significant four bits of the offset MUST ALWAYS be zero, because they are NOT encoded into the ticket -- hence the 16-byte paragraph boundary for entries.
protected static final int SHIFT_LENGTH
protected static final long MASK_LENGTH
protected static final String TYPE_NAME
protected static final FlashJournalRM.WaitCongestionPolicy WAIT_CONGESTION_ACTION
FlashJournalRM.WriterDaemon
.protected static final FlashJournalRM.FailCongestionPolicy FAIL_CONGESTION_ACTION
FlashJournalRM.CongestionPolicy
which fails, by throwing an unchecked exception, when the file is deemed congested.protected volatile FlashJournalRM.CongestionPolicy m_congestionPolicy
FlashJournalRM.CongestionPolicy
to be used when in a congested state.protected File m_dirTemp
protected boolean m_fLoggedDirectory
protected AtomicInteger m_cbBacklog
protected ConcurrentMap<Long,Binary> m_mapLocators
protected FlashJournalRM.BufferPool m_pool
protected FlashJournalRM.PreparerDaemon m_daemonPreparer
protected FlashJournalRM.WriterDaemon m_daemonWriter
public FlashJournalRM(Cluster cluster, FlashJournalRM.Dependencies deps)
cluster
- the Cluster
for which the FlashJournalRM
is storing datadeps
- the Dependencies objectpublic void stop()
Controllable.shutdown()
for normal service termination. Calling this method for a service that has already stopped has no effect.stop
in interface Controllable
stop
in class AbstractJournalRM
public long getMaxTotalRam()
This property is specific to a RamJournalRM
implementation.
public boolean isNioRam()
This property is specific to a RamJournalRM
implementation.
public int getMaxBacklogSize()
This property is specific to a FlashJournalRM
implementation.
public int getBacklogSize()
This property is specific to a FlashJournalRM
implementation.
public int getBacklogCount()
This property is specific to a FlashJournalRM
implementation.
public int getBufferSize()
This property is specific to a FlashJournalRM
implementation.
public long getMaxPoolSize()
public int getPoolSize()
This property is specific to a FlashJournalRM
implementation.
public int getHighFileCount()
This property is specific to a FlashJournalRM
implementation.
public File getDirectoryFile()
public long getPurgeDelayMillis()
protected String getName()
getName
in class AbstractJournalRM
protected long getWriterTimeout()
public FlashJournalRM.Dependencies getDependencies()
getDependencies
in class AbstractJournalRM
public FlashJournalRM.DefaultDependencies cloneDependencies(AbstractJournalRM.Dependencies deps)
cloneDependencies
in class AbstractJournalRM
deps
- the populated Dependencies objectprotected void onDependencies(AbstractJournalRM.Dependencies deps)
onDependencies
in class AbstractJournalRM
deps
- the populated Dependencies objectprotected void startThreads()
startThreads
in class AbstractJournalRM
protected void purgeOldTempFiles()
protected void stopThreads()
stopThreads
in class AbstractJournalRM
protected double calculateCurrentLoadFactor()
calculateCurrentLoadFactor
in class AbstractJournalRM
protected boolean shouldNotifyCollector()
shouldNotifyCollector
in class AbstractJournalRM
protected String getDescription()
AbstractJournalRM.toString()
method.getDescription
in class AbstractJournalRM
protected FlashJournalRM.JournalFile getJournalFile(int nFileId)
getJournalFile
in class AbstractJournalRM
nFileId
- the file id (0-511)protected FlashJournalRM.JournalFile instantiateJournalFile(int nFile)
instantiateJournalFile
in class AbstractJournalRM
nFile
- the file number in the range 0-511 (inclusive)protected FlashJournalRM.BufferPool instantiateBufferPool()
protected FlashJournalRM.BufferPool getBufferPool()
protected FlashJournalRM.PreparerDaemon instantiatePreparerDaemon()
protected FlashJournalRM.PreparerDaemon getPreparerDaemon()
protected FlashJournalRM.WriterDaemon instantiateWriterDaemon()
protected FlashJournalRM.WriterDaemon getWriterDaemon()
protected long getEvacuationMask()
getEvacuationMask
in class AbstractJournalRM
protected int extractFileId(long lTicket)
extractFileId
in class AbstractJournalRM
lTicket
- a "ticket" that was created by the Journalprotected long extractOffset(long lTicket)
extractOffset
in class AbstractJournalRM
lTicket
- a "ticket" that was created by the Journalprotected int extractLength(long lTicket)
extractLength
in class AbstractJournalRM
lTicket
- a "ticket" that was created by the Journalprotected long encodeTicket(int nFile, long of, int cb)
encodeTicket
in class AbstractJournalRM
nFile
- a file number (aka a file index)of
- a value offsetcb
- a value length