com.oracle.bmc.hdfs.store

Class BmcParallelReadAheadFSInputStream

java.lang.Object

java.io.InputStream

org.apache.hadoop.fs.FSInputStream

com.oracle.bmc.hdfs.store.BmcFSInputStream

com.oracle.bmc.hdfs.store.BmcParallelReadAheadFSInputStream

All Implemented Interfaces:

Closeable, AutoCloseable, PositionedReadable, Seekable

public class BmcParallelReadAheadFSInputStream
extends BmcFSInputStream

FSInputStream implementation that reads ahead to cache chunks of data using parallel ranged GET requests. Like in-memory, but memory usage is limited making it suitable for all files.

Field Summary

Fields inherited from class com.oracle.bmc.hdfs.store.BmcFSInputStream

closed, currentPosition, FIRST_READ_WINDOW_SIZE, firstRead, objectStorage, readMaxRetries, requestBuilder, retryPolicy, sourceInputStream, statistics, status

Constructor Summary

Constructors

Constructor and Description
BmcParallelReadAheadFSInputStream(com.oracle.bmc.objectstorage.ObjectStorage objectStorage, FileStatus status, Supplier<com.oracle.bmc.objectstorage.requests.GetObjectRequest.Builder> requestBuilder, int readMaxRetries, FileSystem.Statistics statistics, ExecutorService executor, int ociReadAheadBlockSize, int readAheadBlockCount)

Method Summary

Modifier and Type	Method and Description
void	close()
protected long	doSeek(long position) Perform the requested seek operation.
int	read()
int	read(byte[] buffer, int offset, int length)
int	read(long position, byte[] buffer, int offset, int length)
protected int	readAtPosition(long position, byte[] buffer, int offset, int length) Reads a specified number of bytes from the input stream at the given absolute position, without changing the current file offset.
void	readFully(long position, byte[] buffer, int offset, int length)
String	toString()

Methods inherited from class com.oracle.bmc.hdfs.store.BmcFSInputStream

available, checkNotClosed, getPos, getSourceInputStream, getStatistics, readAllBytes, readAllBytes, retryPolicy, seek, seekToNewSource, setSourceInputStream, validateState, wrap

Methods inherited from class org.apache.hadoop.fs.FSInputStream

readFully, validatePositionedReadArgs

Methods inherited from class java.io.InputStream

mark, markSupported, read, reset, skip

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.fs.PositionedReadable

maxReadSizeForVectorReads, minSeekForVectorReads, readVectored

Constructor Detail

BmcParallelReadAheadFSInputStream

public BmcParallelReadAheadFSInputStream(com.oracle.bmc.objectstorage.ObjectStorage objectStorage,
                                         FileStatus status,
                                         Supplier<com.oracle.bmc.objectstorage.requests.GetObjectRequest.Builder> requestBuilder,
                                         int readMaxRetries,
                                         FileSystem.Statistics statistics,
                                         ExecutorService executor,
                                         int ociReadAheadBlockSize,
                                         int readAheadBlockCount)

Method Detail

read

public int read()
         throws IOException

Overrides:

read in class BmcFSInputStream

Throws:

IOException

read

public int read(long position,
                byte[] buffer,
                int offset,
                int length)
         throws IOException

Specified by:

read in interface PositionedReadable

Overrides:

read in class FSInputStream

Throws:

IOException

read

public int read(byte[] buffer,
                int offset,
                int length)
         throws IOException

Overrides:

read in class BmcFSInputStream

Throws:

IOException

readFully

public void readFully(long position,
                      byte[] buffer,
                      int offset,
                      int length)
               throws IOException

Specified by:

readFully in interface PositionedReadable

Overrides:

readFully in class FSInputStream

Throws:

IOException

readAtPosition

protected int readAtPosition(long position,
                             byte[] buffer,
                             int offset,
                             int length)
                      throws IOException

Reads a specified number of bytes from the input stream at the given absolute position, without changing the current file offset. This method is intended to provide a mechanism for reading bytes from a specific position without affecting the current position in the stream.

Parameters:

position - the absolute position in the input stream from where the data should be read.

buffer - the buffer into which the data is read.

offset - the start offset in the buffer at which the data should be written.

length - the maximum number of bytes to be read from the input stream.

Returns:

the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.

Throws:

IOException - if an I/O error occurs during the read operation.

doSeek

protected long doSeek(long position)
               throws IOException

Description copied from class: BmcFSInputStream

Perform the requested seek operation. Note, if the subclass changes the input stream or closes it, a new one must be provided and set using BmcFSInputStream.setSourceInputStream(InputStream) before returning. The input stream that was originally created (and wrapped by BmcFSInputStream.wrap(InputStream) can be obtained from BmcFSInputStream.getSourceInputStream().

Specified by:

doSeek in class BmcFSInputStream

Parameters:

position - The position to seek to.

Returns:

The new position after seeking.

Throws:

IOException - if the operation could not be completed

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class BmcFSInputStream

Throws:

IOException

toString

public String toString()

Overrides:

toString in class FSInputStream