oracle.spatial.hadoop.vector.mapreduce.job

Class Partitioning

java.lang.Object

org.apache.hadoop.conf.Configured

oracle.spatial.hadoop.vector.mapred.job.BaseJob<java.lang.Object,java.lang.Object>

oracle.spatial.hadoop.vector.mapred.job.MultipleInputsJob

oracle.spatial.hadoop.vector.mapred.job.Partitioning

oracle.spatial.hadoop.vector.mapreduce.job.Partitioning

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class Partitioning
extends Partitioning

This class can be used to run or configure a job for spatially partitioning one or more data sets.
Partitioning can be performed using only a fraction of the input data size by setting the Partitioning.setSamplingRatio(double) property (by default it is set to 0.1).
The job's output is a partitioning file, a GeoJSON file containing the generated partitions with information such as the partition geometry and number of elements in the partition.

Field Summary

Fields inherited from class oracle.spatial.hadoop.vector.mapred.job.Partitioning

defaultNumReduceTasks, HV_RESULT_FOLDER, hvResultPath, PARTITION_RESULT_FILE, partitionsPath, SAMPLE_FILE_NAME, samplePath, samplingRatio

Fields inherited from class oracle.spatial.hadoop.vector.mapred.job.MultipleInputsJob

inputDataSets, miConf

Fields inherited from class oracle.spatial.hadoop.vector.mapred.job.BaseJob

argsp, inputDataSet, jarClass, jobRegEntryPath, proxyIDS

Constructor Summary

Constructors

Constructor and Description
Partitioning()

Method Summary

Methods

Modifier and Type	Method and Description
void	configure(org.apache.hadoop.mapreduce.Job job) Validates and adds the current parameters to the job configuration
protected AbstractInputDataSet	createSampleInputDataSet(org.apache.hadoop.conf.Configuration conf) Samples the input data and create a file containing the sampled geometries
protected InputDataSetCmdArgsParserHandler	getInputDataSetCmdParserHandler(org.apache.hadoop.conf.Configuration conf) Gets the current instance of InputDataSetCmdArgsParserHandler used to parse command line parameters for the input data set
protected InputDataSetConfiguratorHandler	getInputDataSetConfiguratorHandler(org.apache.hadoop.conf.Configuration conf) Returns the current instance of InputDataSetConfiguratorHandler used to configure the input data set
static void	main(java.lang.String[] args)
int	run(java.lang.String[] args)
boolean	runFullPartitioningProcess(org.apache.hadoop.mapreduce.Job job) Runs the full partitioning process.

Methods inherited from class oracle.spatial.hadoop.vector.mapred.job.Partitioning

clean, configure, defineGlobalBounds, getCmdOptions, getCurrentCmdArgs, getCurrentCmdArgsAsString, getHVResultPartPath, getPartitionsPath, getPathsLength, getSamplingRatio, processArgs, runFullPartitioningProcess, setSamplingRatio

Methods inherited from class oracle.spatial.hadoop.vector.mapred.job.MultipleInputsJob

addInputDataSet, asMultiInputDataSet, configureInputs, configureInputs, getInputListCmdOptions, getInputs, getMultipleInputDataSetsParams, removeInputDataSet, setInputDataSets, updateInputDataSet

Methods inherited from class oracle.spatial.hadoop.vector.mapred.job.BaseJob

addJobRegistryEntry, addJobRegistryEntry, addJobRegistryEntry, createJob, createJob, createJob, createJob, createJobConf, createJobConf, createJobConf, getCmdOptionsWithInputDataSets, getCmdOptionsWithInputDataSets, getCurrentCmdArgsAsString, getInput, getInputDataSet, getInputFormatClass, getJarClass, getOutput, getRecordInfoProviderClass, getSpatialConfig, runJob, runJob, setInput, setInputDataSet, setInputFormatClass, setJarClass, setOutput, setRecordInfoProviderClass, setSpatialConfig

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

Partitioning

public Partitioning()

Method Detail

getInputDataSetConfiguratorHandler

protected InputDataSetConfiguratorHandler getInputDataSetConfiguratorHandler(org.apache.hadoop.conf.Configuration conf)

Description copied from class: BaseJob

Returns the current instance of InputDataSetConfiguratorHandler used to configure the input data set

Overrides:

getInputDataSetConfiguratorHandler in class Partitioning

Parameters:

conf - a job configuration

Returns:

an instance of InputDataSetConfiguratorHandler

getInputDataSetCmdParserHandler

protected InputDataSetCmdArgsParserHandler getInputDataSetCmdParserHandler(org.apache.hadoop.conf.Configuration conf)

Description copied from class: BaseJob

Gets the current instance of InputDataSetCmdArgsParserHandler used to parse command line parameters for the input data set

Overrides:

getInputDataSetCmdParserHandler in class Partitioning

Parameters:

conf - a job configuration

Returns:

an instance of InputDataSetCmdArgsParserHandler

configure

public void configure(org.apache.hadoop.mapreduce.Job job)
               throws java.lang.Exception

Description copied from class: BaseJob

Validates and adds the current parameters to the job configuration

Overrides:

configure in class BaseJob<java.lang.Object,java.lang.Object>

Parameters:

job - a job instance

Throws:

java.lang.Exception

createSampleInputDataSet

protected AbstractInputDataSet createSampleInputDataSet(org.apache.hadoop.conf.Configuration conf)
                                                 throws java.io.IOException

Description copied from class: Partitioning

Samples the input data and create a file containing the sampled geometries

Overrides:

createSampleInputDataSet in class Partitioning

Parameters:

conf - the job configuration

Returns:

an AbstractInputDataSet pointing to the sampled data

Throws:

java.io.IOException

runFullPartitioningProcess

public boolean runFullPartitioningProcess(org.apache.hadoop.mapreduce.Job job)
                                   throws java.lang.Exception

Runs the full partitioning process. It is not needed to call configure() when calling this method

Parameters:

job - a job

Returns:

true if the job finished successfully

Throws:

java.lang.Exception

run

public int run(java.lang.String[] args)
        throws java.lang.Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Overrides:

run in class Partitioning

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception