public class TableStorageHandler extends org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler
org.apache.hadoop.hive.ql.metadata.HiveStoragePredicateHandler.DecomposedPredicate
Modifier and Type | Field and Description |
---|---|
protected int |
batchSize |
protected Consistency |
consistency |
protected Direction |
direction |
protected String |
fieldRangeProperty |
protected JobConf |
jobConf |
protected String[] |
kvHadoopHosts |
protected String[] |
kvHelperHosts |
protected String |
kvStoreName |
protected int |
maxBatches |
protected int |
maxRequests |
protected String |
primaryKeyProperty |
protected String |
tableName |
protected long |
timeout |
protected TimeUnit |
timeoutUnit |
Constructor and Description |
---|
TableStorageHandler() |
Modifier and Type | Method and Description |
---|---|
void |
configureInputJobProperties(org.apache.hadoop.hive.ql.plan.TableDesc tableDesc,
Map<String,String> jobProperties)
Creates a configuration for job input.
|
void |
configureOutputJobProperties(org.apache.hadoop.hive.ql.plan.TableDesc tableDesc,
Map<String,String> jobProperties)
Using semantics identical to the semantics of the
configureInputJobProperties method, creates a
configuration for job output. |
void |
configureTableJobProperties(org.apache.hadoop.hive.ql.plan.TableDesc tableDesc,
Map<String,String> jobProperties)
Although this method was originally intended to configure
properties for a job based on the definition of the source or
target table the job accesses, this method is now deprecated
in Hive.
|
org.apache.hadoop.hive.ql.metadata.HiveStoragePredicateHandler.DecomposedPredicate |
decomposePredicate(JobConf jobConfig,
org.apache.hadoop.hive.serde2.Deserializer deserializer,
org.apache.hadoop.hive.ql.plan.ExprNodeDesc predicate)
Method required by the HiveStoragePredicateHandler interface.
|
Class<? extends InputFormat> |
getInputFormatClass() |
Class<? extends OutputFormat> |
getOutputFormatClass() |
Class<? extends org.apache.hadoop.hive.serde2.SerDe> |
getSerDeClass() |
configureJobConf, getAuthorizationProvider, getConf, getMetaHook, setConf, toString
protected String kvStoreName
protected String[] kvHelperHosts
protected String[] kvHadoopHosts
protected String tableName
protected String primaryKeyProperty
protected String fieldRangeProperty
protected Direction direction
protected Consistency consistency
protected long timeout
protected TimeUnit timeoutUnit
protected int maxRequests
protected int batchSize
protected int maxBatches
protected JobConf jobConf
public Class<? extends InputFormat> getInputFormatClass()
getInputFormatClass
in interface org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
getInputFormatClass
in class org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler
public Class<? extends OutputFormat> getOutputFormatClass()
getOutputFormatClass
in interface org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
getOutputFormatClass
in class org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler
public Class<? extends org.apache.hadoop.hive.serde2.SerDe> getSerDeClass()
getSerDeClass
in interface org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
getSerDeClass
in class org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler
public void configureInputJobProperties(org.apache.hadoop.hive.ql.plan.TableDesc tableDesc, Map<String,String> jobProperties)
JobContext.getConfiguration
) with the properties that
may be needed by the handler's bundled artifacts; for example, the
InputFormat
class, the SerDe
class, etc.
returned by this handler. Any key value pairs set in the
jobProperties
argument are guaranteed to be set in the
job's configuration object; and any "context" information associated
with the job can be retrieved from the given TableDesc
parameter.
Note that implementations of this method must be idempotent. That
is, when this method is invoked more than once with the same
tableDesc
values for a given job, the key value pairs
in jobProperties
, as well as any external state set
by this method, should be the same after each invocation. How this
invariant guarantee is achieved is left as an implementation detail;
although to support this guarantee, changes should only be made
to the contents of jobProperties
, but never to
tableDesc
.
configureInputJobProperties
in interface org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
configureInputJobProperties
in class org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler
public void configureOutputJobProperties(org.apache.hadoop.hive.ql.plan.TableDesc tableDesc, Map<String,String> jobProperties)
configureInputJobProperties
method, creates a
configuration for job output. For more detail, refer to the
description of the configureInputJobProperties
method.configureOutputJobProperties
in interface org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
configureOutputJobProperties
in class org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler
public void configureTableJobProperties(org.apache.hadoop.hive.ql.plan.TableDesc tableDesc, Map<String,String> jobProperties)
configureInputJobProperties
and configureOutputJobProperties
should be used
instead.configureTableJobProperties
in interface org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
configureTableJobProperties
in class org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler
public org.apache.hadoop.hive.ql.metadata.HiveStoragePredicateHandler.DecomposedPredicate decomposePredicate(JobConf jobConfig, org.apache.hadoop.hive.serde2.Deserializer deserializer, org.apache.hadoop.hive.ql.plan.ExprNodeDesc predicate)
This method validates the components of the given predicate and ultimately produces the following artifacts:
Note that because the Hive built-in IndexPredicateAnalyzer produces only predicates that consist of 'AND' statements, and which correspond to PrimaryKey based or IndexKey based predicates, if the Hive built-in analyzer does not produce a predicate to push, then a custom analyzer that extends the capabilities of the Hive built-in analyzer is employed. This extended analyzer handles statements that the built-in analyzer does not handle. Additionally, whereas the built-in analyzer populates a List of Hive IndexSearchConditions corresponding to the filtering criteria of the predicate to push, the extended analyzer populates an ArrayDeque in which the top (first element) of the Deque is a Hive object consisting of all the components of the original input predicate, but with 'invalid' operators replaced with 'valid' operators; for example, with 'IN list' replaced with 'OR' statements.
In each case, translateSearchConditions constructs the appropriate Hive predicate to push from the contents of the given Collection; either List of IndexSearchCondition, or ArrayDeque.
decomposePredicate
in interface org.apache.hadoop.hive.ql.metadata.HiveStoragePredicateHandler
Copyright (c) 2011, 2017 Oracle and/or its affiliates. All rights reserved.