Package oracle.kv.hadoop.hive.table

Class TableSerDe

java.lang.Object
org.apache.hadoop.hive.serde2.AbstractSerDe
oracle.kv.hadoop.hive.table.TableSerDe
All Implemented Interfaces:
Deserializer, SerDe, Serializer
public class TableSerDe extends AbstractSerDe
Concrete implementation of TableSerDeBase that performs deserialization and/or serialization of data loaded into a KVStore via the PrimaryKey based Table API. - Note on Logging - Two loggers are currently employed by this class:
  • One logger based on Log4j version 1, accessed via the org.apache.commons.logging wrapper.
  • One logger based on the Log4j2 API.
Two loggers are necessary because Hive2 employs the Log4j2 logging mechanism, whereas logging in Big Data SQL 4.0 is still based on Log4j version 1. As a result, when one executes a Hive query and wishes to log output from this class, the Log4j2-based logger specified by this class must be added to the Hive2 logging configuration file. On the other hand, to log output from this class when executing a Big Data SQL query, the Log4j v1 logger must be added to the Big Data SQL logging configuration file. In the future, when Big Data SQL changes its logging mechanims from Log4j v1 to Log4j2, this class should be changed to employ only the Log4j2-based logger.

  • Field Details

    • USER_SECURITY_DIR

      protected static final String USER_SECURITY_DIR

    • hiveRow

      protected List<Object> hiveRow

    • kvMapWritable

      protected MapWritable kvMapWritable

  • Constructor Details

    • TableSerDe

      public TableSerDe()

  • Method Details

    • validateParams

      protected void validateParams(Properties tbl) throws SerDeException
      Verifies that the names and types of the fields in the KV Store table correctly map to the names and types of the Hive table against which the Hive query is to be executed. Note that this method assumes that both the KVStore parameters and the serde parameters have been initialized. If a mismatch is found between KV Store fields and Hive columns, then a SerDeException will be thrown with a descriptive message.
      Throws:
      SerDeException

    • createObjectInspector

      protected ObjectInspector createObjectInspector() throws SerDeException
      Throws:
      SerDeException

    • deserialize

      public Object deserialize(Writable field) throws SerDeException
      Deserializes the given Writable parameter and returns a Java Object representing the contents of that parameter. The field parameter of this method references a field from a row of the KVStore table having the name specified by the tableName field of this class. Thus, the Object returned by this method references the contents of that table field.
      Specified by:
      deserialize in interface Deserializer
      Parameters:
      field - The Writable object containing a serialized object from a row of the KVStore table with name specified by tableName.
      Returns:
      A Java object representing the contents in the given table field parameter.
      Throws:
      SerDeException

    • serialize

      public Writable serialize(Object obj, ObjectInspector objectInspector) throws SerDeException
      Serialize the given Object by navigating inside the Object with the given ObjectInspector. The given Object references a Hive row and the return value is an instance of Writable that references a KVStore table fieldName:fieldValue pair.
      Specified by:
      serialize in interface Serializer
      Parameters:
      obj - The Object whose contents are examined and from which the return value is constructed.
      objectInspector - The object to use to navigate the given Object's contents.
      Returns:
      A Writable object representing the contents in the given Object to seriaize.
      Throws:
      SerDeException

    • initialize

      public void initialize(Configuration job, Properties tableProperties, Properties partitionProperties) throws SerDeException
      The original org.apache.hadoop.hive.serde2.AbstractSerDe.initialize method from Hive 1 was a 2-arg method. It is still defined in Hive 2, but is deprecated. We override the new, 3-arg method defined in Hive 2. But we currently don't yet do anything with the partitionProperties.
      Overrides:
      initialize in class AbstractSerDe
      Throws:
      SerDeException

    • initialize

      public void initialize(Configuration job, Properties tbl) throws SerDeException
      Specified by:
      initialize in interface Deserializer
      Specified by:
      initialize in interface Serializer
      Specified by:
      initialize in class AbstractSerDe
      Throws:
      SerDeException

    • getSerDeStats

      public SerDeStats getSerDeStats()
      Returns statistics collected when deserializing and/or serializing.
      Specified by:
      getSerDeStats in interface Deserializer
      Specified by:
      getSerDeStats in interface Serializer
      Specified by:
      getSerDeStats in class AbstractSerDe

    • getObjectInspector

      public ObjectInspector getObjectInspector() throws SerDeException
      Returns the ObjectInspector that can be used to navigate through the internal structure of the Object returned from the deserialize method.
      Specified by:
      getObjectInspector in interface Deserializer
      Specified by:
      getObjectInspector in class AbstractSerDe
      Throws:
      SerDeException

    • getSerializedClass

      public Class<? extends Writable> getSerializedClass()
      Return the Writable class returned by the serialize method; which is used to initialize the SequenceFile header.
      Specified by:
      getSerializedClass in interface Serializer
      Specified by:
      getSerializedClass in class AbstractSerDe

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • setStore

      protected void setStore(KVStore testStore)
      For testing only; to support the use of a mocked store.

    • resetStaticFields

      protected void resetStaticFields()
      For testing only. Used by the TableSerDe tests to clear state between each test, so the tests don't interfere with each other.