oracle.spatial.spark.vector.index

Class DistributedSpatialIndex<T>

java.lang.Object

oracle.spatial.spark.vector.index.DistributedSpatialIndex<T>

Type Parameters:

T - the generic type of the indexed spatial RDD

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

QuadTreeDistIndex

public abstract class DistributedSpatialIndex<T>
extends java.lang.Object
implements java.io.Serializable

A distributed spatial index for a spatial RDD. An implementation of this class creates and handles a spatial index for a spatial RDD which uses to speed up spatial searches.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected static java.lang.String	BOOTSTRAP_FILE
protected java.lang.Class<T>	type

Constructor Summary

Constructors

Constructor and Description
DistributedSpatialIndex(java.lang.Class<T> type) Creates an empty instance which will accept spatial RDDs of the given generic type

Method Summary

Methods

Modifier and Type	Method and Description
abstract void	cache() Persist the index' internal RDD with the default storage level
static <T> DistributedSpatialIndex<T>	createIndex(org.apache.spark.api.java.JavaSparkContext sc, SpatialJavaRDD<T> rdd, SpatialPartitioningConfiguration partConf) Spatially index a spatial RDD using the specified DistributedSpatialIndex implementing class
abstract SpatialJavaRDD<T>	filter(org.apache.spark.api.java.function.Function<T,java.lang.Boolean> f, SpatialOperationConfig spatialOpConf) Returns a new spatial RDD containing only the elements that satisfy both, the filtering function and the spatial operation
abstract <R> org.apache.spark.api.java.JavaRDD<R>	flatMap(org.apache.spark.api.java.function.FlatMapFunction<T,R> f, SpatialOperationConfig spatialOpConf) Returns a new RDD by first spatially filtering the RDD elements using the spatial operation given by spatialOpConf, then a function is applied to all the remaining elements.
protected abstract void	index(org.apache.spark.api.java.JavaSparkContext sc, SpatialJavaRDD<T> rdd, SpatialPartitioningConfiguration partConf) Spatially indexes an RDD.
abstract <T2,R> org.apache.spark.api.java.JavaRDD<R>	join(SpatialJavaRDD<T2> rdd2, org.apache.spark.api.java.function.Function2<T,T2,R> f, SpatialOperationConfig spatialOpConf) Joins the records of the current index' data set with another spatial data set based on a spatial relationship between both data set's records
static <T> DistributedSpatialIndex<T>	load(org.apache.spark.api.java.JavaSparkContext sc, java.lang.String pathStr) Reads an existing persisted index from the given path
abstract void	persist(org.apache.spark.storage.StorageLevel storageLevel) Sets the index' internal RDD's storage level to persist its values across operations after the first time it is computed.
protected abstract void	read(org.apache.spark.api.java.JavaSparkContext sc, java.lang.String pathStr) Loads a persisted index from the given path.
void	save(org.apache.spark.api.java.JavaSparkContext sc, java.lang.String pathStr) Stores the current index as a file in the given path
abstract void	unpersist() Marks the index' internal RDD as non-persistent
protected abstract void	write(org.apache.spark.api.java.JavaSparkContext sc, java.lang.String pathStr) Writes the current index to the given path

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

type

protected java.lang.Class<T> type

BOOTSTRAP_FILE

protected static final java.lang.String BOOTSTRAP_FILE

See Also:

Constant Field Values

Constructor Detail

DistributedSpatialIndex

public DistributedSpatialIndex(java.lang.Class<T> type)

Creates an empty instance which will accept spatial RDDs of the given generic type

Parameters:

type - the generic type of the spatial RDD to be indexed

Method Detail

index

protected abstract void index(org.apache.spark.api.java.JavaSparkContext sc,
                              SpatialJavaRDD<T> rdd,
                              SpatialPartitioningConfiguration partConf)

Spatially indexes an RDD. This method is called by the Vecto API and is not meant to be called by users.

Parameters:

sc - an existing Spark context

rdd - a spatial RDD

partConf - the indexing and partitioning configuration

write

protected abstract void write(org.apache.spark.api.java.JavaSparkContext sc,
                              java.lang.String pathStr)
                       throws java.lang.Exception

Writes the current index to the given path

Parameters:

sc - an existing Spark context

pathStr - a path on either, a local or distributed file system

Throws:

java.lang.Exception

read

protected abstract void read(org.apache.spark.api.java.JavaSparkContext sc,
                             java.lang.String pathStr)
                      throws java.lang.Exception

Loads a persisted index from the given path. The persisted index is expected to be of the same implementation class than the current instance

Parameters:

sc - an existing Spark context

pathStr - a path on either, a local or distributed file system

Throws:

java.lang.Exception

filter

public abstract SpatialJavaRDD<T> filter(org.apache.spark.api.java.function.Function<T,java.lang.Boolean> f,
                                         SpatialOperationConfig spatialOpConf)
                                  throws java.lang.Exception

Returns a new spatial RDD containing only the elements that satisfy both, the filtering function and the spatial operation

Parameters:

f - a filtering function

spatialOpConf - the spatial operation configuration to spatially filter the RDD

Returns:

a new SpatialJavaRDD

Throws:

java.lang.Exception

flatMap

public abstract <R> org.apache.spark.api.java.JavaRDD<R> flatMap(org.apache.spark.api.java.function.FlatMapFunction<T,R> f,
                                                                 SpatialOperationConfig spatialOpConf)
                                                          throws java.lang.Exception

Returns a new RDD by first spatially filtering the RDD elements using the spatial operation given by spatialOpConf, then a function is applied to all the remaining elements.

Parameters:

f - a function to apply to each element

spatialOpConf - the spatial operation configuration to spatially filter the RDD

Returns:

a new RDD

Throws:

java.lang.Exception

join

public abstract <T2,R> org.apache.spark.api.java.JavaRDD<R> join(SpatialJavaRDD<T2> rdd2,
                                                                 org.apache.spark.api.java.function.Function2<T,T2,R> f,
                                                                 SpatialOperationConfig spatialOpConf)
                                                          throws java.lang.Exception

Joins the records of the current index' data set with another spatial data set based on a spatial relationship between both data set's records

Parameters:

f - a tuple processor function which receives a tuple of records from both data sets which match the spatial join condition

rdd2 - a spatial RDD to join with

spatialOpConf - a spatial operation configuration defining the spatial relationship to join records

Returns:

a new RDD

Throws:

java.lang.Exception

persist

public abstract void persist(org.apache.spark.storage.StorageLevel storageLevel)

Sets the index' internal RDD's storage level to persist its values across operations after the first time it is computed.

Parameters:

storageLevel -

unpersist

public abstract void unpersist()

Marks the index' internal RDD as non-persistent

cache

public abstract void cache()

Persist the index' internal RDD with the default storage level

save

public void save(org.apache.spark.api.java.JavaSparkContext sc,
                 java.lang.String pathStr)
          throws java.lang.Exception

Stores the current index as a file in the given path

Parameters:

sc - an existing Spark context

pathStr - a path on either, a local or distributed file system

Throws:

java.lang.Exception

load

public static <T> DistributedSpatialIndex<T> load(org.apache.spark.api.java.JavaSparkContext sc,
                                                  java.lang.String pathStr)
                                           throws java.lang.Exception

Reads an existing persisted index from the given path

Parameters:

sc - an existing Spark context

pathStr - a path on either, a local or distributed file system

Returns:

Throws:

java.lang.Exception

createIndex

public static <T> DistributedSpatialIndex<T> createIndex(org.apache.spark.api.java.JavaSparkContext sc,
                                                         SpatialJavaRDD<T> rdd,
                                                         SpatialPartitioningConfiguration partConf)
                                                  throws java.lang.Exception

Spatially index a spatial RDD using the specified DistributedSpatialIndex implementing class

Parameters:

sc - an existing Spark context

rdd - a spatial RDD

partConf - Indexing and partitioning configuration. It determines the concrete index implementation to be created.

Returns:

an instance of DistributedSpatialIndex

Throws:

java.lang.Exception