Package oracle.kv.lob

Interface KVLargeObject

All Known Subinterfaces:

KVStore

public interface KVLargeObject

The KVLargeObject interface defines the operations used to read and write Large Objects (LOBs) such as audio and video files. As a general rule, any object larger than 1 MB is a good candidate for representation as a LOB. The LOB API permits access to large values, without having to materialize the value in its entirety by providing streaming APIs for reading and writing these objects.

A LOB is stored as a sequence of chunks whose sizes are optimized for the underlying storage system. The chunks constituting a LOB may not all be the same size. Individual chunk sizes are chosen automatically by the system based upon its knowledge of the underlying storage architecture and hardware. Splitting a LOB into chunks permits low latency operations across mixed work loads with values of varying sizes. The stream based APIs serve to insulate the application from the actual representation of the LOB in the underlying storage system.

The methods used to read and write LOBs are not atomic. Relaxing the atomicity requirement permits distribution of chunks across the entire store. It's the application's responsibility to coordinate operations on a LOB. The implementation will make a good faith effort to detect concurrent modification of an LOB and throw ConcurrentModificationException when it detects such concurrency conflicts but does not guarantee that it will detect all such conflicts. The safe course of action upon encountering this exception in the context of conflicting write operations is to delete the LOB and replace it with a new value after fixing the application level coordination issue that provoked the exception.

Failures during a LOB write operation result in the creation of a partial LOB. The LOB value of a partial LOB is in some intermediate state, where it cannot be read by the application; attempts to getLOB on a partial LOB will result in a PartialLOBException. A partial LOB resulting from an incomplete putLOB, deleteLOB or appendLOB operation can be repaired by retrying the corresponding failed putLOB, deleteLOB or appendLOB operation. Or it can be deleted and a new key/value pair can be created in its place. The documentation associated with individual LOB methods describes their behavior when invoked on partial LOBs in greater detail.

LOBs, due to their representation as a sequence of chunks, must be accessed exclusively via the LOB APIs defined in this interface. The family of KVStore.get methods when applied to a LOB key will be presented with a value that is internal to the KVS implementation and cannot be used directly by the application.

Keys associated with LOBs must have a trailing suffix string (as defined by KVStoreConfig.getLOBSuffix()) at the end of their final Key component. This requirement permits non-LOB methods to check for inadvertent modifications to LOB objects.

All methods in this class verify that the key used to access LOBs meets this trailing suffix requirement and throw IllegalArgumentException if the verification fails. The use of the name lobKey for the key argument in the method signatures below emphasizes this requirement.

Here is a summary of LOB related key checks performed across all methods:

• All non-LOB write operations check for the absence of the LOB suffix as part of the other key validity checks. If the check fails it will result in a IllegalArgumentException.
• All non-LOB read operations return the associated opaque value used internally to construct a LOB stream.
• All LOB write and read operations in this interface check for the presence of the LOB suffix. If the check fails it will result in an IllegalArgumentException.

Example:

The following simplified code fragment loads an mp3 file named "f1.mp3" into the store associating it with the key "f1.lob". Note that this interface is a superinterface for KVStore, so to access the LOB methods you simply create and use a KVStore handle.

 File file = new File("f1.mp3");

 FileInputStream fis = new FileInputStream(file);
 Version version = store.putLOB(Key.createKey("f1.lob"),
                                fis,
                                Durability.COMMIT_WRITE_NO_SYNC,
                                5, TimeUnit.SECONDS);
 

The following simplified code fragment retrieves the LOB that was loaded above and computes its size:

 InputStreamVersion istreamVersion =
     store.getLOB(Key.createKey("f1.lob"),
                  Consistency.NONE_REQUIRED,
                  5, TimeUnit.SECONDS);

 InputStream stream = istreamVersion.getInputStream();
 int byteCount = 0;
 while (stream.read() != -1) {
     byteCount++;
 }
 

Since:

2.0

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static class	KVLargeObject.LOBState	The enumeration defines the states associated with a LOB.

Method Summary

Modifier and Type	Method	Description
Version	appendLOB(Key lobKey, java.io.InputStream lobAppendStream, Durability durability, long lobTimeout, java.util.concurrent.TimeUnit timeoutUnit)	Appends to a value of an existing LOB key/value pair.
boolean	deleteLOB(Key lobKey, Durability durability, long lobTimeout, java.util.concurrent.TimeUnit timeoutUnit)	Deletes the LOB associated with the key.
InputStreamVersion	getLOB(Key lobKey, Consistency consistency, long lobTimeout, java.util.concurrent.TimeUnit timeoutUnit)	Returns an InputStream representing the underlying LOB value associated with the key.
Version	putLOB(Key lobKey, java.io.InputStream lobStream, Durability durability, long lobTimeout, java.util.concurrent.TimeUnit timeoutUnit)	Put a key/LOB value pair, inserting new value or overwriting an existing pair as appropriate.
Version	putLOBIfAbsent(Key lobKey, java.io.InputStream lobStream, Durability durability, long lobTimeout, java.util.concurrent.TimeUnit timeoutUnit)	Put a key/LOB value pair, but only if the key either has no value or has a partially inserted or deleted LOB value present.
Version	putLOBIfPresent(Key lobKey, java.io.InputStream lobStream, Durability durability, long lobTimeout, java.util.concurrent.TimeUnit timeoutUnit)	Put a key/LOB value pair, but only if a complete value for the given key is present.

Method Detail

putLOB

Version putLOB(Key lobKey,
               java.io.InputStream lobStream,
               Durability durability,
               long lobTimeout,
               java.util.concurrent.TimeUnit timeoutUnit)
        throws PartialLOBException,
               java.util.ConcurrentModificationException,
               java.io.IOException

Put a key/LOB value pair, inserting new value or overwriting an existing pair as appropriate. If a new key/LOB value pair was successfully inserted or updated, the (non-null) version of the new KV pair is returned. Failures result in one of the exceptions listed below being thrown.

The value associated with the large object (LOB) is obtained by reading the InputStream associated with the lobStream parameter. The stream must be positioned at the first byte of the LOB value and must return -1 after the last byte of the LOB has been fetched. For best performance the stream should support an efficient implementation of InputStream.read(byte[], int, int). The stream implementation is not required to support the InputStream.mark(int) and InputStream.reset() methods, that is, InputStream.markSupported() may return false. If the methods are supported, they may be used during internal retry operations. Such retry operations are otherwise transparent to the application.

This method, like all LOB methods, is not atomic. Failures (like the loss of network connectivity) while an insert operation is in progress may result in a partially inserted LOB.

If the method detects a partially inserted LOB, it will skip reading the bytes that were already loaded and resume insertion of the LOB after ensuring that the trailing bytes in the partial LOB match the ones supplied by the lobStream. The number of bytes that are matched is determined by the configuration parameter KVStoreConfig.getLOBVerificationBytes(). If the trailing bytes do not match, or the stream does not skip to the requested location, it throws an IllegalArgumentException.

A partially deleted LOB is deleted in its entirety and replaced with the new key/value pair.

Parameters:

lobKey - the key associated with the LOB.

lobStream - the stream of bytes representing the LOB as described earlier.

durability - the durability associated with the operation. If null, the default durability is used.

lobTimeout - is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, the KVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit) value is used.

timeoutUnit - is the unit of the timeout parameter and may be null only if timeout is zero.

Returns:

the Version associated with the newly inserted LOB

Throws:

PartialLOBException - if it is invoked on a partially updated LOB.

java.util.ConcurrentModificationException - if it detects that an attempt was made to modify the object while the insertion was in progress.

java.io.IOException - if one is generated by the lobStream.

See Also:

Write exceptions

putLOBIfAbsent

Version putLOBIfAbsent(Key lobKey,
                       java.io.InputStream lobStream,
                       Durability durability,
                       long lobTimeout,
                       java.util.concurrent.TimeUnit timeoutUnit)
                throws PartialLOBException,
                       java.util.ConcurrentModificationException,
                       java.io.IOException

Put a key/LOB value pair, but only if the key either has no value or has a partially inserted or deleted LOB value present. Returns null if the lobKey is associated with a complete LOB, and throws PartialLOBException if the LOB has been partially appended. Its behavior is otherwise identical to putLOB. Like the putLOB operation, it will resume the insertion if it encounters a partially inserted LOB.

Parameters:

lobKey - the key associated with the LOB.

lobStream - the stream of bytes representing the LOB as described earlier.

durability - the durability associated with the operation. If null, the default durability is used.

lobTimeout - is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, the KVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit) value is used.

timeoutUnit - is the unit of the timeout parameter and may be null only if timeout is zero.

Returns:

the version of the new value, or null if an existing value is present and the put is unsuccessful.

Throws:

PartialLOBException - if it is invoked on a partially appended LOB.

java.util.ConcurrentModificationException - if it detects that an attempt was made to modify the object while the insertion was in progress.

java.io.IOException - if one is generated by the lobStream

See Also:

putLOB(oracle.kv.Key, java.io.InputStream, oracle.kv.Durability, long, java.util.concurrent.TimeUnit), Write exceptions

putLOBIfPresent

Version putLOBIfPresent(Key lobKey,
                        java.io.InputStream lobStream,
                        Durability durability,
                        long lobTimeout,
                        java.util.concurrent.TimeUnit timeoutUnit)
                 throws PartialLOBException,
                        java.util.ConcurrentModificationException,
                        java.io.IOException

Put a key/LOB value pair, but only if a complete value for the given key is present. If the lobKey is absent it returns null. Its behavior is otherwise identical to putLOB. Like the putLOB operation, it will resume the insertion if it encounters a partially inserted LOB.

Parameters:

lobKey - the key associated with the LOB.

lobStream - the stream of bytes representing the LOB as described earlier.

durability - the durability associated with the operation. If null, the default durability is used.

lobTimeout - is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, the KVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit) value is used.

timeoutUnit - is the unit of the timeout parameter and may be null only if timeout is zero.

Returns:

the version of the new value, or null if no existing value is present and the put is unsuccessful.

Throws:

PartialLOBException - if it is invoked on a partially appended LOB.

java.util.ConcurrentModificationException - if it detects that an attempt was made to modify the object while the insertion was in progress.

java.io.IOException - if one is generated by the lobStream

See Also:

putLOB(oracle.kv.Key, java.io.InputStream, oracle.kv.Durability, long, java.util.concurrent.TimeUnit), Write exceptions

getLOB

InputStreamVersion getLOB(Key lobKey,
                          Consistency consistency,
                          long lobTimeout,
                          java.util.concurrent.TimeUnit timeoutUnit)
                   throws PartialLOBException,
                          java.util.ConcurrentModificationException

Returns an InputStream representing the underlying LOB value associated with the key.

An attempt to access a partial LOB will result in a PartialLOBException being thrown.

The returned input stream can be read to obtain the value associated with the LOB. The application can use the InputStream method InputStream.skip(long) in conjunction with InputStream.mark(int) and InputStream.reset() to read random byte ranges within the LOB.

Reading the input stream can result in various exceptions like ConsistencyException, RequestTimeoutException, ConcurrentModificationException or FaultException, etc. All such exceptions are wrapped in IOException. Specialized stream readers can use Throwable.getCause() to examine the underlying cause and take appropriate action. The application must ensure that the KVStore handle is not closed before the contents of the returned InputStream have been read. Such a premature close will result in an IOException when the stream is subsequently read.

Parameters:

lobKey - the key used to lookup the key/value pair.

consistency - determines the consistency associated with the read used to lookup and read the value via the returned stream object. If null, the default consistency is used. Note that Consistency.Version cannot be used to access a LOB that's striped across multiple partitions; it will result in an IllegalArgumentException being thrown.

lobTimeout - is an upper bound on the time interval for retrieving a chunk or its associated metadata. A best effort is made not to exceed the specified limit. If zero, the KVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit) value is used. Note that this timeout also applies to read operations performed on the returned stream object.

timeoutUnit - is the unit of the timeout parameter, and may be null only if timeout is zero.

Returns:

the input stream and version associated with the key, or null if there is no LOB associated with the key.

Throws:

PartialLOBException - if it is invoked on a partial LOB

java.util.ConcurrentModificationException - if it detects that an attempt was made to modify the object while the operation was in progress

See Also:

Read exceptions

deleteLOB

boolean deleteLOB(Key lobKey,
                  Durability durability,
                  long lobTimeout,
                  java.util.concurrent.TimeUnit timeoutUnit)
           throws java.util.ConcurrentModificationException

Deletes the LOB associated with the key. This method can be used to delete partial LOBs.

Parameters:

lobKey - the key associated with the LOB.

durability - the durability associated with the operation. If null, the default durability is used.

lobTimeout - is an upper bound on the time taken for deleting each chunk. A best effort is made not to exceed the specified limit. If zero, the KVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit) value is used.

timeoutUnit - is the unit of the timeout parameter, and may be null only if timeout is zero.

Returns:

true if the delete is successful, or false if no existing value is present. Note that the method will return true if a partial LOB was deleted.

Throws:

java.util.ConcurrentModificationException - if it detects that an attempt was made to modify the object while the operation was in progress.

See Also:

Write exceptions

appendLOB

Version appendLOB(Key lobKey,
                  java.io.InputStream lobAppendStream,
                  Durability durability,
                  long lobTimeout,
                  java.util.concurrent.TimeUnit timeoutUnit)
           throws PartialLOBException,
                  java.util.ConcurrentModificationException,
                  java.io.IOException

Appends to a value of an existing LOB key/value pair. If the append was successful, the (non-null) version of the modified KV pair is returned. This method is most efficient for large granularity append operations where the value being appended is 128K bytes or greater in size.

The value to be appended to the large object (LOB) is obtained by reading the InputStream associated with the lobAppendStream parameter. The stream must be positioned at the first byte of the value to be appended and must return -1 after the last byte of the LOB has been fetched. For best performance the stream should support an efficient implementation of InputStream.read(byte[], int, int). The stream implementation is not required to support the InputStream.mark(int) and InputStream.reset() methods, that is, InputStream.markSupported() may return false. If the methods are supported, they may be used during internal retry operations. Such retry operations are otherwise transparent to the application.

This method, like all LOB methods, is not atomic. Failures (like the loss of network connectivity) while an append operation is in progress may result in a partially appended LOB. The append operation can be resumed by repeating the appendLOB operation.

If the method detects a partially appended LOB, it will skip reading the bytes that were already appended and resume appending to the LOB after ensuring that the trailing bytes in the partial LOB match the ones supplied by the lobAppendStream. The number of bytes that are matched is determined by the configuration parameter KVStoreConfig.getLOBVerificationBytes(). If the trailing bytes do not match, or the stream does not skip to the requested location, it throws an IllegalArgumentException.

Parameters:

lobKey - the key associated with the existing LOB.

lobAppendStream - the stream of bytes representing just the value to be appended.

durability - the durability associated with the operation. If null, the default durability is used.

lobTimeout - is an upper bound on the time taken for storing each chunk. A best effort is made not to exceed the specified limit. If zero, the KVStoreConfig.getLOBTimeout(java.util.concurrent.TimeUnit) value is used.

timeoutUnit - is the unit of the timeout parameter and may be null only if timeout is zero.

Returns:

the Version associated with the updated LOB

Throws:

PartialLOBException - if it is invoked on a partially insertedLOB or partially deleted LOB.

java.util.ConcurrentModificationException - if it detects that an attempt was made to modify the object while the insertion was in progress.

java.io.IOException - if one is generated by the lobStream.

Since:

2.1.55

See Also:

Write exceptions