oracle.spatial.hadoop.vector.partition

Class PartitioningUtils

java.lang.Object

oracle.spatial.hadoop.vector.partition.PartitioningUtils

public class PartitioningUtils
extends java.lang.Object

Provides miscellaneous utility functions for working with partitions.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PartitioningUtils.PartitionCounters Counters used by mapreduce components when calculating hilbert values

Constructor Summary

Constructors

Constructor and Description
PartitioningUtils()

Method Summary

Modifier and Type	Method and Description
static int	calculateNumReduceTasks(PartitioningResult partitioning) Calculates how many reduce tasks are needed to process the partitions defined in a partitioning result file
static int	calculateNumReduceTasks(org.apache.hadoop.fs.Path partResPath, org.apache.hadoop.conf.Configuration conf) Calculates how many reduce tasks are needed to process the partitions defined in a partitioning result file
static java.util.Map<java.lang.String,java.lang.Integer>	createPartitionsToReducersMap(PartitioningResult partitioning, int numReducers) Maps partitions to reduce tasks so the reduce tasks contain approximately the same number of elements to process
static oracle.spatial.util.RTree	indexPartitions(PartitioningResult pr, SpatialOperationConfig spatialOpConf, org.apache.hadoop.conf.Configuration conf) Takes a partitioning result and returns all contained partitions indexed in an r-tree
static PartitioningResult	loadPartitioningResult(org.apache.hadoop.fs.Path partResPath, org.apache.hadoop.conf.Configuration conf) Returns an object representation of a partitioning result file
static PartitioningResult	loadPartitioningResultFromDistCache(org.apache.hadoop.conf.Configuration conf) Returns an object representation of a partitioning result file contained in the distributed cache.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitioningUtils

public PartitioningUtils()

Method Detail

indexPartitions

public static oracle.spatial.util.RTree indexPartitions(PartitioningResult pr,
                                                        SpatialOperationConfig spatialOpConf,
                                                        org.apache.hadoop.conf.Configuration conf)
                                                 throws java.lang.Exception

Takes a partitioning result and returns all contained partitions indexed in an r-tree

Parameters:

pr - partitioning result

spatialOpConf - if provided, the indexed partitions geometries are optimized for the spatial operation

conf - a job's configuration

Returns:

a spatial index containing the partitions

Throws:

java.lang.Exception

loadPartitioningResultFromDistCache

public static PartitioningResult loadPartitioningResultFromDistCache(org.apache.hadoop.conf.Configuration conf)
                                                              throws java.io.IOException

Returns an object representation of a partitioning result file contained in the distributed cache.

Parameters:

conf -

Returns:

Throws:

java.io.IOException

loadPartitioningResult

public static PartitioningResult loadPartitioningResult(org.apache.hadoop.fs.Path partResPath,
                                                        org.apache.hadoop.conf.Configuration conf)
                                                 throws java.io.IOException

Returns an object representation of a partitioning result file

Parameters:

partResPath -

conf -

Returns:

Throws:

java.io.IOException

calculateNumReduceTasks

public static int calculateNumReduceTasks(org.apache.hadoop.fs.Path partResPath,
                                          org.apache.hadoop.conf.Configuration conf)
                                   throws java.io.IOException

Calculates how many reduce tasks are needed to process the partitions defined in a partitioning result file

Parameters:

partResPath -

conf -

Returns:

Throws:

java.io.IOException

calculateNumReduceTasks

public static int calculateNumReduceTasks(PartitioningResult partitioning)

Calculates how many reduce tasks are needed to process the partitions defined in a partitioning result file

Parameters:

partitioning -

Returns:

createPartitionsToReducersMap

public static java.util.Map<java.lang.String,java.lang.Integer> createPartitionsToReducersMap(PartitioningResult partitioning,
                                                                                              int numReducers)

Maps partitions to reduce tasks so the reduce tasks contain approximately the same number of elements to process

Parameters:

partitioning -

numReducers -

Returns: