oracle.spatial.hadoop.vector.mapred.job

Class Categorization<K,V>

java.lang.Object

org.apache.hadoop.conf.Configured

oracle.spatial.hadoop.vector.mapred.job.BaseJob<K,V>

oracle.spatial.hadoop.vector.mapred.job.IndexedInputBaseJob<K,V>

oracle.spatial.hadoop.vector.mapred.job.Categorization<K,V>

Type Parameters:

K - the type of the input keys

V - the type of the input values

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

Categorization

public class Categorization<K,V>
extends IndexedInputBaseJob<K,V>

This class can be used to run or configure a job for categorizing records of an input dataset into features of a hierarchical dataset.

The result will be a summary file for each hierarchical level containing the number of records from the non-hierarchical dataset per hierarchy level feature.

The categorization can be performed determining a spatial interaction specified by a spatial operation or using MVSuggest.

Field Summary

Fields

Modifier and Type	Field and Description
protected MVSService	mvsService

Fields inherited from class oracle.spatial.hadoop.vector.mapred.job.BaseJob

argsp, inputDataSet, jarClass, jobRegEntryPath, proxyIDS

Constructor Summary

Constructors

Constructor and Description
Categorization()

Method Summary

Methods

Modifier and Type	Method and Description
void	configure(org.apache.hadoop.mapred.JobConf jobConf) Validates and adds the current parameters to the job configuration
java.lang.String	getCmdOptions() Gets a description of the arguments expected from command line.
java.util.Map<java.lang.String,java.lang.Object>	getCurrentCmdArgs(org.apache.hadoop.conf.Configuration conf) Returns the current driver properties in a map where each key-value is a name and value of a command line argument.
org.apache.hadoop.fs.Path[]	getHierarchyDataPaths() Gets the paths to hierarchy data catalogs.
org.apache.hadoop.fs.Path	getHierarchyIndexPath() Gets the HDFS path where the hierarchy data index is located or should be placed once it is generated
java.lang.Class<? extends HierarchyInfo>	getHierarchyInfoClass() Gets the HierarchyInfo implementing class which describes the current hierarchy
protected InputDataSetCmdArgsParserHandler	getInputDataSetCmdParserHandler(org.apache.hadoop.conf.Configuration conf) Gets the current instance of InputDataSetCmdArgsParserHandler used to parse command line parameters for the input data set
protected InputDataSetConfiguratorHandler	getInputDataSetConfiguratorHandler(org.apache.hadoop.conf.Configuration conf) Returns the current instance of InputDataSetConfiguratorHandler used to configure the input data set
MVSConfig	getMvsConfig() Gets the MVSuggest configuration
SpatialOperationConfig	getSpatialOperationConfig() Gets the configuration of the spatial operation to be performed
static <K,V> void	main(java.lang.String[] args)
void	processArgs(java.lang.String[] args, org.apache.hadoop.conf.Configuration conf) Extracts and validates arguments from the command line
int	run(java.lang.String[] args)
void	setHierarchyDataPaths(org.apache.hadoop.fs.Path[] hierarchyDataPaths) Sets the paths to hierarchy data catalogs.
void	setHierarchyIndexPath(org.apache.hadoop.fs.Path hieararchyDataIndexPath) Sets the HDFS path where the hierarchy data index is located or should be placed once it is generated
void	setHierarchyInfoClass(java.lang.Class<? extends HierarchyInfo> hierarchyInfoClass) Sets the HierarchyInfo implementing class which describes the current hierarchy
void	setMvsConfig(MVSConfig mvsConfig) Sets the MVSuggest configuration
void	setSpatialOperationConfig(SpatialOperationConfig spatialOperationConfig) Sets the configuration of the spatial operation to be performed.

Methods inherited from class oracle.spatial.hadoop.vector.mapred.job.IndexedInputBaseJob

getIndexMetadataDir, getIndexName, getProxyIDS, isInputIndex, setIndexMetadataDir, setIndexName, setInputIndex, updateInputDataSet

Methods inherited from class oracle.spatial.hadoop.vector.mapred.job.BaseJob

addJobRegistryEntry, addJobRegistryEntry, addJobRegistryEntry, configure, createJob, createJob, createJob, createJob, createJobConf, createJobConf, createJobConf, getCmdOptionsWithInputDataSets, getCmdOptionsWithInputDataSets, getCurrentCmdArgsAsString, getCurrentCmdArgsAsString, getInput, getInputDataSet, getInputFormatClass, getJarClass, getOutput, getRecordInfoProviderClass, getSpatialConfig, runJob, runJob, setInput, setInputDataSet, setInputFormatClass, setJarClass, setOutput, setRecordInfoProviderClass, setSpatialConfig

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

mvsService

protected MVSService mvsService

Constructor Detail

Categorization

public Categorization()

Method Detail

getInputDataSetConfiguratorHandler

protected InputDataSetConfiguratorHandler getInputDataSetConfiguratorHandler(org.apache.hadoop.conf.Configuration conf)

Description copied from class: BaseJob

Returns the current instance of InputDataSetConfiguratorHandler used to configure the input data set

Specified by:

getInputDataSetConfiguratorHandler in class BaseJob<K,V>

Parameters:

conf - a job configuration

Returns:

an instance of InputDataSetConfiguratorHandler

getInputDataSetCmdParserHandler

protected InputDataSetCmdArgsParserHandler getInputDataSetCmdParserHandler(org.apache.hadoop.conf.Configuration conf)

Description copied from class: BaseJob

Gets the current instance of InputDataSetCmdArgsParserHandler used to parse command line parameters for the input data set

Specified by:

getInputDataSetCmdParserHandler in class BaseJob<K,V>

Parameters:

conf - a job configuration

Returns:

an instance of InputDataSetCmdArgsParserHandler

setHierarchyInfoClass

public void setHierarchyInfoClass(java.lang.Class<? extends HierarchyInfo> hierarchyInfoClass)

Sets the HierarchyInfo implementing class which describes the current hierarchy

Parameters:

hierarchyInfoClass - a subclass of HierarchyInfo

getHierarchyInfoClass

public java.lang.Class<? extends HierarchyInfo> getHierarchyInfoClass()

Gets the HierarchyInfo implementing class which describes the current hierarchy

Returns:

a subclass of HierarchyInfo

setSpatialOperationConfig

public void setSpatialOperationConfig(SpatialOperationConfig spatialOperationConfig)

Sets the configuration of the spatial operation to be performed. The query window is not needed.

Parameters:

spatialOperationConfig - a spatial operation configuration.

getSpatialOperationConfig

public SpatialOperationConfig getSpatialOperationConfig()

Gets the configuration of the spatial operation to be performed

Returns:

a spatial operation configuration

setHierarchyIndexPath

public void setHierarchyIndexPath(org.apache.hadoop.fs.Path hieararchyDataIndexPath)

Sets the HDFS path where the hierarchy data index is located or should be placed once it is generated

Parameters:

hieararchyDataIndexPath - an HDFS path

getHierarchyIndexPath

public org.apache.hadoop.fs.Path getHierarchyIndexPath()

Gets the HDFS path where the hierarchy data index is located or should be placed once it is generated

Returns:

an HDFS path

setHierarchyDataPaths

public void setHierarchyDataPaths(org.apache.hadoop.fs.Path[] hierarchyDataPaths)

Sets the paths to hierarchy data catalogs. The paths are expected to be sorted in ascending way according to the hierarchy level each path represents. For example: path[0] is the path to the catalog of the hierarchy level 1, path[1] is the path for hierarchy level 2 and so on.

Parameters:

hierarchyDataPaths - an array of paths

getHierarchyDataPaths

public org.apache.hadoop.fs.Path[] getHierarchyDataPaths()

Gets the paths to hierarchy data catalogs. The paths are expected to be sorted in ascending way according to the hierarchy level each path represents. For example: path[0] is the path to the catalog of the hierarchy level 1, path[1] is the path for hierarchy level 2 and so on.

Returns:

an array of paths

setMvsConfig

public void setMvsConfig(MVSConfig mvsConfig)

Sets the MVSuggest configuration

Parameters:

mvsConfig - the MVSuggest configuration

getMvsConfig

public MVSConfig getMvsConfig()

Gets the MVSuggest configuration

Returns:

the MVSuggest configuration

processArgs

public void processArgs(java.lang.String[] args,
               org.apache.hadoop.conf.Configuration conf)
                 throws java.lang.Exception

Description copied from class: BaseJob

Extracts and validates arguments from the command line

Overrides:

processArgs in class BaseJob<K,V>

Parameters:

args - arguments from the command line

conf - the job configuration

Throws:

java.lang.Exception

getCurrentCmdArgs

public java.util.Map<java.lang.String,java.lang.Object> getCurrentCmdArgs(org.apache.hadoop.conf.Configuration conf)

Description copied from class: BaseJob

Returns the current driver properties in a map where each key-value is a name and value of a command line argument. By printing this information it is possible to know how to execute a similar job from command line

Overrides:

getCurrentCmdArgs in class BaseJob<K,V>

Parameters:

conf - a job configuration

Returns:

a map which entries are of type <String,String> or <String, Map<String, Object>> for multiple input data sets. For nested maps, the entry types are the same than the enclosing map

getCmdOptions

public java.lang.String getCmdOptions()

Description copied from class: BaseJob

Gets a description of the arguments expected from command line.

Overrides:

getCmdOptions in class BaseJob<K,V>

Returns:

a text description

configure

public void configure(org.apache.hadoop.mapred.JobConf jobConf)
               throws java.lang.Exception

Description copied from class: BaseJob

Validates and adds the current parameters to the job configuration

Overrides:

configure in class BaseJob<K,V>

Parameters:

jobConf - the job configuration

Throws:

java.lang.Exception

run

public int run(java.lang.String[] args)
        throws java.lang.Exception

Throws:

java.lang.Exception

main

public static <K,V> void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception