Package com.nt.udc.util

Class ObjectStorageFile

java.lang.Object
com.nt.udc.util.ObjectStorageFile
Direct Known Subclasses:
NARKeyStorageFile, NARStorageFile, TimeStampedRecordStorageFile
public abstract class ObjectStorageFile extends Object
This class implements the basics of file-based Object storage. For the most part, it is similar to a Hashtable class, except that the Object is not stored in memory - it is stored and accessed from disk. The Object must be able to convert to/from a byte[] object in order to use this class. It is highly recommended that this NOT be done with an ObjectStream because of the performance implications. Also, the key for a given Object is the offset at which it sits in the file. If you would like to use a different key, use the FileHashtable class with this class, as that class maintains an Object key -> offset table. You may also create your own key -> offset mapping through use of the FileMap interface.

This class can also restore an old table, as defined in the constructor.

Implementation:

Since the table is on disk, any new or updated Objects must be appended to the file. This causes the file to grow rather large, seeing that an old or outdated Object cannot be removed from the table. A "removed" or "outdated" Object is simply marked for later removal, during the "compression" stage. The "compression" takes place when a pre-defined number of "outdated" Objects have been reached. This number must be defined in the constructor, and is known as the compression threshold. Keep in mind that a lower compression threshold causing compression to happen more often, but results in the disk table being smaller. Likewise, a larger compression threshold causing a less frequent compression, thereby improving performance, but allowing a larger disk table.

This class must be extended to handle the specifics of the Objects it is storing.

Through use of the "setupWalker" and "walkFile" methods, a user may step through the file by valid Object. This means that all "outdated" Objects will be skipped. Note that the walker supports only one active walk at a time, and the user must use "setupWalker" to reset the walker. It will not reset on its own.

  • Field Details

    • objectFile

      protected RandomAccessFile objectFile
      RandomAccessFile that actually stores the Objects.

    • OBJ_HEADER

      public static final int OBJ_HEADER
      See Also:

    • allocator

      protected ObjectStorageAllocator allocator

    • currentStorageSize

      protected int currentStorageSize

    • VALID_OBJ

      protected static final int VALID_OBJ
      See Also:

    • INVALID_OBJ

      protected static final int INVALID_OBJ
      See Also:

    • time_used

      protected long time_used

    • MAX_PARTITION_SIZE

      protected static final long MAX_PARTITION_SIZE
      See Also:

    • partitioned

      protected boolean partitioned

    • headers

      protected byte[] headers

    • invalid_flags

      protected static final byte[] invalid_flags

    • valid_flags

      protected static final byte[] valid_flags

    • headers_no_flag

      protected byte[] headers_no_flag

    • maxPartitionSize

      protected long maxPartitionSize

  • Constructor Details

    • ObjectStorageFile

      public ObjectStorageFile(int count, File fileName, boolean newFile)
      Constructor for the storage manager.
      Parameters:
      count - Compression threshold. (see class description)
      fileName - File name to use for the table.
      newFile - Is this file name for a new table, or should this restore an old table?

  • Method Details

    • setPartition

      public void setPartition(long partitionSize)

    • getObjectFile

      public File getObjectFile()
      Returns the File object associated with this ObjectStorageFile.
      Returns:
      File, representing this ObjectStorageFile

    • add

      public long add(Object obj) throws IOException
      Adds a new Object to the table.
      Parameters:
      obj - Add this Object to the table.
      Returns:
      Offset at which this Object exists.
      Throws:
      IOException

    • get

      public Object get(long offset) throws IOException
      Get the Object at the given offset.
      Parameters:
      offset - to retrieve the Object.
      Returns:
      The desired Object.
      Throws:
      IOException

    • remove

      public Object remove(long offset, boolean bRetValue) throws IOException
      Remove the Object at the given offset.
      Parameters:
      offset - Offset of the Object.
      bRetValue - Whether it will return the removed object or not
      Throws:
      IOException

    • removeDelayedParsing

      public byte[] removeDelayedParsing(long offset, boolean bRetValue) throws IOException
      Same as remove(), except that objectFromBytes() is not called. This is exclusively for use by multi-threaded EP and AP.
      Parameters:
      offset -
      bRetValue -
      Returns:
      The byte[] that should have been passed to objectFromBytes() to convert to original object with.
      Throws:
      IOException

    • update

      public long update(Object obj, long offset) throws IOException
      Update a prior Object with a new Object.
      Parameters:
      obj - New Object.
      offset - Offset of the old Object.
      Returns:
      Offset of the new Object.
      Throws:
      IOException

    • restoreFile

      public long[] restoreFile() throws IOException
      Rebuilds the storage file offsets from a existing table storage file.
      Returns:
      An array of longs, representing the new offsets.
      Throws:
      IOException

    • open

      public void open() throws IOException
      Opens a storage file.
      Throws:
      IOException

    • close

      public void close() throws IOException
      Closes a storage file.
      Throws:
      IOException

    • drop

      public void drop()

    • isNewFile

      public boolean isNewFile()
      Returns whether this is a new storage file.
      Returns:
      true, if this is using a new storage file

    • getNumObjects

      public int getNumObjects() throws IOException
      Returns the number of Objects in the storage file.
      Returns:
      Number of Objects in the storage file.
      Throws:
      IOException

    • getValidOffsets

      protected long[] getValidOffsets() throws IOException
      Returns a list of offsets for all the valid Objects in the file.
      Returns:
      array of valid offsets
      Throws:
      IOException

    • setupWalker

      protected void setupWalker() throws IOException
      Sets up the file walker to start at the beginning of the file.
      Throws:
      IOException

    • setupWalker

      protected void setupWalker(long offset) throws IOException
      Sets up the file walker to start at the given offset.
      Throws:
      IOException

    • walkFile

      protected long walkFile() throws IOException
      Walks the file to the next valid Object, and returns its offset.
      Returns:
      offset of new valid Object, or -1 if no valid offset was found
      Throws:
      IOException

    • getSomeObjects

      public Object[] getSomeObjects(int numObjects) throws IOException
      Returns a given set of Objects that exist in the file. Subsequent calls to this method will eventually return all the Objects in the file. Note that getting ALL the Objects at once, and storing them in memory, may cause "out of memory" issues. Therefore, it is recommended that you deal with the Objects a little bit at a time. When using this method, be sure to call it until ALL of the Objects have been retrieved. The method will only reset its pointer to the top of the file once all of the current Objects have been retrieved.
      Parameters:
      numObjects - Number of Objects to return.
      Returns:
      Set of Objects from the file. Null, if no more Objects exist.
      Throws:
      IOException

    • toString

      public String toString()
      Converts the storage file to String representation.
      Overrides:
      toString in class Object
      Returns:
      Storage table, in String representation.

    • toString

      public void toString(OutputStream os) throws IOException
      Writes the String representation of this storage file to the given OutputStream.
      Parameters:
      os - OutputStream to write to.
      Throws:
      IOException

    • allocatorReport

      public String allocatorReport()

    • addBytes

      protected long addBytes(byte[] newBytes, ObjectStorageInfo info) throws IOException
      Adds a set of bytes to the file, and returns the offset where they are stored.
      Parameters:
      newBytes - Bytes to store
      Returns:
      offset of bytes
      Throws:
      IOException

    • getBytes

      protected byte[] getBytes(long offset) throws IOException
      Returns the bytes at the given offset.
      Parameters:
      offset - offset to retrieve the bytes
      Returns:
      bytes at the offset
      Throws:
      IOException

    • markDeletedBytes

      protected void markDeletedBytes(long offset) throws IOException
      Throws:
      IOException

    • removeBytes

      protected byte[] removeBytes(long offset) throws IOException
      Removes the bytes at the given offset, and returns them.
      Parameters:
      offset - offset of bytes to remove
      Returns:
      bytes at offset
      Throws:
      IOException

    • isEmpty

      public boolean isEmpty()

    • objectFromBytes

      protected abstract Object objectFromBytes(byte[] bytes)
      This method should return an Object, as created by the given bytes. If, for whatever reason, the Object could not be created from these bytes, this method should return null.
      Parameters:
      bytes - byte[] used to create the Object
      Returns:
      A valid Object, created from the bytes.

    • objectToBytes

      protected abstract byte[] objectToBytes(Object obj)
      This method should return the Object in its byte[] format.
      Parameters:
      obj - Object to convert to bytes.
      Returns:
      bytes of the Object