Interface KVLargeObject
- All Known Subinterfaces:
KVStore
A LOB is stored as a sequence of chunks whose sizes are optimized for the underlying storage system. The chunks constituting a LOB may not all be the same size. Individual chunk sizes are chosen automatically by the system based upon its knowledge of the underlying storage architecture and hardware. Splitting a LOB into chunks permits low latency operations across mixed work loads with values of varying sizes. The stream based APIs serve to insulate the application from the actual representation of the LOB in the underlying storage system.
The methods used to read and write LOBs are not atomic.
Relaxing the atomicity requirement permits distribution of chunks across the
entire store. It's the application's responsibility to coordinate operations
on a LOB. The implementation will make a good faith effort to detect
concurrent modification of an LOB and throw
ConcurrentModificationException
when it detects such
concurrency conflicts but does not guarantee that it will detect all such
conflicts. The safe course of action upon encountering this exception in the
context of conflicting write operations is to delete the LOB and replace it
with a new value after fixing the application level coordination issue that
provoked the exception.
Failures during a LOB write operation result in the creation of a
partial
LOB. The LOB value of a partial
LOB is in
some intermediate state, where it cannot be read by the application;
attempts to getLOB
on a partial LOB will result in a
PartialLOBException
. A partial LOB resulting from an incomplete
putLOB
, deleteLOB
or appendLOB
operation can be repaired by retrying the corresponding failed
putLOB
, deleteLOB
or appendLOB
operation. Or it can be deleted and a new key/value pair can be created in
its place. The documentation associated with individual LOB methods
describes their behavior when invoked on partial LOBs in greater detail.
LOBs, due to their representation as a sequence of chunks, must be accessed
exclusively via the LOB APIs defined in this interface. The family of
KVStore.get
methods when applied to a LOB key will be presented
with a value that is internal to the KVS implementation and cannot be used
directly by the application.
Keys associated with LOBs must have a trailing suffix string (as defined by
KVStoreConfig.getLOBSuffix()
) at the end of their final Key component.
This requirement permits non-LOB methods to check for inadvertent
modifications to LOB objects.
All methods in this class verify that the key used to access LOBs meets this
trailing suffix requirement and throw IllegalArgumentException
if the verification fails. The use of the name lobKey
for the
key argument in the method signatures below emphasizes this requirement.
Here is a summary of LOB related key checks performed across all methods:
- All non-LOB write operations check for the absence of the LOB suffix as part of the other key validity checks. If the check fails it will result in a IllegalArgumentException.
- All non-LOB read operations return the associated opaque value used internally to construct a LOB stream.
- All LOB write and read operations in this interface check for the presence of the LOB suffix. If the check fails it will result in an IllegalArgumentException.
Example:
The following simplified code fragment loads an mp3 file named "f1.mp3" into
the store associating it with the key "f1.lob". Note that this interface is
a superinterface for KVStore
, so to access the LOB methods
you simply create and use a KVStore handle.
File file = new File("f1.mp3"); FileInputStream fis = new FileInputStream(file); Version version = store.putLOB(Key.createKey("f1.lob"), fis, Durability.COMMIT_WRITE_NO_SYNC, 5, TimeUnit.SECONDS);
The following simplified code fragment retrieves the LOB that was loaded above and computes its size:
InputStreamVersion istreamVersion = store.getLOB(Key.createKey("f1.lob"), Consistency.NONE_REQUIRED, 5, TimeUnit.SECONDS); InputStream stream = istreamVersion.getInputStream(); int byteCount = 0; while (stream.read() != -1) { byteCount++; }
- Since:
- 2.0
-
Nested Class Summary
Modifier and TypeInterfaceDescriptionstatic enum
The enumeration defines the states associated with a LOB. -
Method Summary
Modifier and TypeMethodDescriptionappendLOB
(Key lobKey, InputStream lobAppendStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) Appends to a value of an existing LOB key/value pair.boolean
deleteLOB
(Key lobKey, Durability durability, long lobTimeout, TimeUnit timeoutUnit) Deletes the LOB associated with the key.getLOB
(Key lobKey, Consistency consistency, long lobTimeout, TimeUnit timeoutUnit) Returns an InputStream representing the underlying LOB value associated with the key.putLOB
(Key lobKey, InputStream lobStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) Put a key/LOB value pair, inserting new value or overwriting an existing pair as appropriate.putLOBIfAbsent
(Key lobKey, InputStream lobStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) Put a key/LOB value pair, but only if the key either has no value or has a partially inserted or deleted LOB value present.putLOBIfPresent
(Key lobKey, InputStream lobStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) Put a key/LOB value pair, but only if a complete value for the given key is present.
-
Method Details
-
putLOB
Version putLOB(Key lobKey, InputStream lobStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) throws PartialLOBException, ConcurrentModificationException, IOException Put a key/LOB value pair, inserting new value or overwriting an existing pair as appropriate. If a new key/LOB value pair was successfully inserted or updated, the (non-null) version of the new KV pair is returned. Failures result in one of the exceptions listed below being thrown.The value associated with the large object (LOB) is obtained by reading the
InputStream
associated with thelobStream
parameter. The stream must be positioned at the first byte of the LOB value and must return -1 after the last byte of the LOB has been fetched. For best performance the stream should support an efficient implementation ofInputStream.read(byte[], int, int)
. The stream implementation is not required to support theInputStream.mark(int)
andInputStream.reset()
methods, that is,InputStream.markSupported()
may return false. If the methods are supported, they may be used during internal retry operations. Such retry operations are otherwise transparent to the application.This method, like all LOB methods, is not atomic. Failures (like the loss of network connectivity) while an insert operation is in progress may result in a partially inserted LOB.
If the method detects a partially inserted LOB, it will skip reading the bytes that were already loaded and resume insertion of the LOB after ensuring that the trailing bytes in the partial LOB match the ones supplied by the
lobStream
. The number of bytes that are matched is determined by the configuration parameterKVStoreConfig.getLOBVerificationBytes()
. If the trailing bytes do not match, or the stream does not skip to the requested location, it throws anIllegalArgumentException
.A partially deleted LOB is deleted in its entirety and replaced with the new key/value pair.
- Parameters:
lobKey
- the key associated with the LOB.lobStream
- the stream of bytes representing the LOB as described earlier.durability
- the durability associated with the operation. If null, thedefault durability
is used.lobTimeout
- is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, theKVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit)
value is used.timeoutUnit
- is the unit of the timeout parameter and may be null only if timeout is zero.- Returns:
- the Version associated with the newly inserted LOB
- Throws:
PartialLOBException
- if it is invoked on a partially updated LOB.ConcurrentModificationException
- if it detects that an attempt was made to modify the object while the insertion was in progress.IOException
- if one is generated by thelobStream
.- See Also:
-
putLOBIfAbsent
Version putLOBIfAbsent(Key lobKey, InputStream lobStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) throws PartialLOBException, ConcurrentModificationException, IOException Put a key/LOB value pair, but only if the key either has no value or has a partially inserted or deleted LOB value present. Returns null if the lobKey is associated with a complete LOB, and throws PartialLOBException if the LOB has been partially appended. Its behavior is otherwise identical toputLOB
. Like theputLOB
operation, it will resume the insertion if it encounters a partially inserted LOB.- Parameters:
lobKey
- the key associated with the LOB.lobStream
- the stream of bytes representing the LOB as described earlier.durability
- the durability associated with the operation. If null, thedefault durability
is used.lobTimeout
- is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, theKVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit)
value is used.timeoutUnit
- is the unit of the timeout parameter and may be null only if timeout is zero.- Returns:
- the version of the new value, or null if an existing value is present and the put is unsuccessful.
- Throws:
PartialLOBException
- if it is invoked on a partially appended LOB.ConcurrentModificationException
- if it detects that an attempt was made to modify the object while the insertion was in progress.IOException
- if one is generated by thelobStream
- See Also:
-
putLOBIfPresent
Version putLOBIfPresent(Key lobKey, InputStream lobStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) throws PartialLOBException, ConcurrentModificationException, IOException Put a key/LOB value pair, but only if a complete value for the given key is present. If thelobKey
is absent it returns null. Its behavior is otherwise identical toputLOB
. Like theputLOB
operation, it will resume the insertion if it encounters a partially inserted LOB.- Parameters:
lobKey
- the key associated with the LOB.lobStream
- the stream of bytes representing the LOB as described earlier.durability
- the durability associated with the operation. If null, thedefault durability
is used.lobTimeout
- is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, theKVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit)
value is used.timeoutUnit
- is the unit of the timeout parameter and may be null only if timeout is zero.- Returns:
- the version of the new value, or null if no existing value is present and the put is unsuccessful.
- Throws:
PartialLOBException
- if it is invoked on apartially appended
LOB.ConcurrentModificationException
- if it detects that an attempt was made to modify the object while the insertion was in progress.IOException
- if one is generated by thelobStream
- See Also:
-
getLOB
InputStreamVersion getLOB(Key lobKey, Consistency consistency, long lobTimeout, TimeUnit timeoutUnit) throws PartialLOBException, ConcurrentModificationException Returns an InputStream representing the underlying LOB value associated with the key.An attempt to access a partial LOB will result in a
PartialLOBException
being thrown.The returned input stream can be read to obtain the value associated with the LOB. The application can use the InputStream method
InputStream.skip(long)
in conjunction withInputStream.mark(int)
andInputStream.reset()
to read random byte ranges within the LOB.Reading the input stream can result in various exceptions like
ConsistencyException
,RequestTimeoutException
,ConcurrentModificationException
orFaultException
, etc. All such exceptions are wrapped inIOException
. Specialized stream readers can useThrowable.getCause()
to examine the underlying cause and take appropriate action. The application must ensure that the KVStore handle is not closed before the contents of the returned InputStream have been read. Such a premature close will result in an IOException when the stream is subsequently read.- Parameters:
lobKey
- the key used to lookup the key/value pair.consistency
- determines the consistency associated with the read used to lookup and read the value via the returned stream object. If null, thedefault consistency
is used. Note that Consistency.Version cannot be used to access a LOB that's striped across multiple partitions; it will result in an IllegalArgumentException being thrown.lobTimeout
- is an upper bound on the time interval for retrieving a chunk or its associated metadata. A best effort is made not to exceed the specified limit. If zero, theKVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit)
value is used. Note that this timeout also applies to read operations performed on the returned stream object.timeoutUnit
- is the unit of the timeout parameter, and may be null only if timeout is zero.- Returns:
- the input stream and version associated with the key, or null if there is no LOB associated with the key.
- Throws:
PartialLOBException
- if it is invoked on a partial LOBConcurrentModificationException
- if it detects that an attempt was made to modify the object while the operation was in progress- See Also:
-
deleteLOB
boolean deleteLOB(Key lobKey, Durability durability, long lobTimeout, TimeUnit timeoutUnit) throws ConcurrentModificationException Deletes the LOB associated with the key. This method can be used to delete partial LOBs.- Parameters:
lobKey
- the key associated with the LOB.durability
- the durability associated with the operation. If null, thedefault durability
is used.lobTimeout
- is an upper bound on the time taken for deleting each chunk. A best effort is made not to exceed the specified limit. If zero, theKVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit)
value is used.timeoutUnit
- is the unit of the timeout parameter, and may be null only if timeout is zero.- Returns:
- true if the delete is successful, or false if no existing value is present. Note that the method will return true if a partial LOB was deleted.
- Throws:
ConcurrentModificationException
- if it detects that an attempt was made to modify the object while the operation was in progress.- See Also:
-
appendLOB
Version appendLOB(Key lobKey, InputStream lobAppendStream, Durability durability, long lobTimeout, TimeUnit timeoutUnit) throws PartialLOBException, ConcurrentModificationException, IOException Appends to a value of an existing LOB key/value pair. If the append was successful, the (non-null) version of the modified KV pair is returned. This method is most efficient for large granularity append operations where the value being appended is 128K bytes or greater in size.The value to be appended to the large object (LOB) is obtained by reading the
InputStream
associated with thelobAppendStream
parameter. The stream must be positioned at the first byte of the value to be appended and must return -1 after the last byte of the LOB has been fetched. For best performance the stream should support an efficient implementation ofInputStream.read(byte[], int, int)
. The stream implementation is not required to support theInputStream.mark(int)
andInputStream.reset()
methods, that is,InputStream.markSupported()
may return false. If the methods are supported, they may be used during internal retry operations. Such retry operations are otherwise transparent to the application.This method, like all LOB methods, is not atomic. Failures (like the loss of network connectivity) while an append operation is in progress may result in a partially appended LOB. The append operation can be resumed by repeating the
appendLOB
operation.If the method detects a partially appended LOB, it will skip reading the bytes that were already appended and resume appending to the LOB after ensuring that the trailing bytes in the partial LOB match the ones supplied by the
lobAppendStream
. The number of bytes that are matched is determined by the configuration parameterKVStoreConfig.getLOBVerificationBytes()
. If the trailing bytes do not match, or the stream does not skip to the requested location, it throws anIllegalArgumentException
.- Parameters:
lobKey
- the key associated with the existing LOB.lobAppendStream
- the stream of bytes representing just the value to be appended.durability
- the durability associated with the operation. If null, thedefault durability
is used.lobTimeout
- is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, theKVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit)
value is used.timeoutUnit
- is the unit of the timeout parameter and may be null only if timeout is zero.- Returns:
- the Version associated with the updated LOB
- Throws:
PartialLOBException
- if it is invoked on a partially insertedLOB or partially deleted LOB.ConcurrentModificationException
- if it detects that an attempt was made to modify the object while the insertion was in progress.IOException
- if one is generated by thelobStream
.- Since:
- 2.1.55
- See Also:
-