SpatialJavaRDD (Oracle Big Data Spatial and Graph Vector Analysis Java API Reference)

java.lang.Object
- org.apache.spark.api.java.JavaRDD<T>
- - oracle.spatial.spark.vector.rdd.SpatialJavaRDD<T>

Type Parameters:

T -

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.JavaRDDLike<T,org.apache.spark.api.java.JavaRDD<T>>
```
public class SpatialJavaRDD<T>
extends org.apache.spark.api.java.JavaRDD<T>
```
This class represents a spatially enabled RDD. A SpatialJavaRDD encapsulates an existing RDD and adds spatial transformations and functions.

Spatial information is extracted from the source RDD records using an implementation of SparkRecordInfoProvider provided by the user. The SparkRecordInfoProvider is expected to return the geometry of spatial records.

The following example shows how to create a SpatialJavaRDD from a text RDD, using a SparkRecordInfo which extracts the spatial information from text records
```
 
 JavaRDD<String> rdd = sc.textFile("someFile.txt");
 SparkRecordInfoProvider recordInfoProvider = new MySparkRecordInfoProvider();
 SpatialJavaRDD<String> spatialRDD = SpatialJavaRDD.fromJavaRDD(rdd, recordInfoProvider, String.class);
 
 
```
The previous example creates a SpatialJavaRDD whose records are of type String as the source RDD. Every time a spatial operation is performed, the provided SparkRecordInfoProvider instance will be used to extract the spatial information. For some data set formats, like JSON, extracting spatial information may require parsing each record every time the spatial information is needed for a spatial operation and it may be executed several times if multiple spatial transformations are applied.

A way to ensure records will be parsed only once by the SparkRecordInfoProvider is to create a SpatialJavaRDD which records are of type SparkRecordInfo. A SparkRecordInfo holds spatial information and any desired additional data from the source RDD records. The SpatialJavaRDD static method fromJavaRDD(JavaRDD, SparkRecordInfoProvider) can be used to create JavaSpatialRDD of type SparkRecordInfo.

Spatial versions of existing transformations are provided. Spatial transformations take the following parameters:
- A lambda provided by the user to further filter or transform records. This is the same lambda function passed to the non-spatial version of the transformation
- An instance of SpatialOperationConfig which contains the information of the spatial operation to be performed to filter records.
See Also:

Serialized Form

Method Summary

Methods
Modifier and Type	Method and Description
`long`	`count()`
`org.apache.spark.sql.DataFrame`	`createSpatialDataFrame(org.apache.spark.sql.SQLContext sqlContext, java.util.List<java.lang.String> fieldsList)` Creates data frame based on the SpatialJavaRDD
`SpatialTransformationContext<T>`	`createSpatialTransformationContext()` Creates an instance of `SpatialTransformationContext`
`SpatialTransformationContext<T>`	`createSpatialTransformationContext(SpatialOperationConfig spatialOperationConf)` Creates an instance of `SpatialTransformationContext` associated to the given `SpatialOperationConfig`.
`SpatialJavaRDD<T>`	`filter(org.apache.spark.api.java.function.Function<T,java.lang.Boolean> f, SpatialOperationConfig spatialOpConf)` Returns a new spatial RDD containing only the elements that satisfy both, the filtering function and the spatial operation
`<R> org.apache.spark.api.java.JavaRDD<R>`	`flatMap(org.apache.spark.api.java.function.Function<T,java.lang.Iterable<R>> m, SpatialOperationConfig spatialOpConf)` Returns a new RDD by first spatially filtering the RDD elements using the spatial operation given by spatialOpConf, then a function is applied to all the remaining elements.
`static <T> SpatialJavaRDD<SparkRecordInfo>`	`fromJavaRDD(org.apache.spark.api.java.JavaRDD<T> rdd, SparkRecordInfoProvider<T> recordInfoProvider)` Creates a spatial RDD of type SparkRecordInfo from the given rdd.
`static <T> SpatialJavaRDD<T>`	`fromJavaRDD(org.apache.spark.api.java.JavaRDD<T> rdd, SparkRecordInfoProvider<T> recordInfoProvider, java.lang.Class<T> recordType)` Creates a spatial RDD from the given java rdd.
`static <T> SpatialJavaRDD<T>`	`fromRDD(org.apache.spark.rdd.RDD<T> rdd, SparkRecordInfoProvider<T> recordInfoProvider)` Creates a spatial RDD from the given rdd.
`double[]`	`getMBR()` Gets the minimum bounding rectangle of the RDD
`SparkRecordInfoProvider<T>`	`getRecordInfoProvider()` Gets the RDD's `SparkRecordInfoProvider` instance
`java.lang.Class<T>`	`getType()` Gets the type of the records in the RDD
`java.util.List<scala.Tuple2<java.lang.Double,T>>`	`nearestNeighbors(oracle.spatial.geometry.JGeometry qryWindow, int k, double tol)` Returns the k elements which are closest to the given query window

Methods inherited from class org.apache.spark.api.java.JavaRDD
aggregate, cache, cartesian, checkpoint, classTag, coalesce, coalesce, collect, collectAsync, collectPartitions, context, countApprox, countApprox, countApproxDistinct, countAsync, countByValue, countByValueApprox, countByValueApprox, distinct, distinct, filter, first, flatMap, flatMapToDouble, flatMapToPair, fold, foreach, foreachAsync, foreachPartition, foreachPartitionAsync, fromRDD, getCheckpointFile, getStorageLevel, glom, groupBy, groupBy, id, intersection, isCheckpointed, isEmpty, iterator, keyBy, map, mapPartitions, mapPartitions, mapPartitionsToDouble, mapPartitionsToDouble, mapPartitionsToPair, mapPartitionsToPair, mapPartitionsWithIndex, mapPartitionsWithIndex$default$2, mapToDouble, mapToPair, max, min, name, partitions, persist, pipe, pipe, pipe, randomSplit, randomSplit, rdd, reduce, repartition, sample, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, setName, sortBy, splits, subtract, subtract, subtract, take, takeAsync, takeOrdered, takeOrdered, takeSample, takeSample, toArray, toDebugString, toLocalIterator, top, top, toRDD, toString, treeAggregate, treeAggregate, treeReduce, treeReduce, union, unpersist, unpersist, wrapRDD, zip, zipPartitions, zipWithIndex, zipWithUniqueId

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Method Detail
  - fromJavaRDD
```
public static <T> SpatialJavaRDD<T> fromJavaRDD(org.apache.spark.api.java.JavaRDD<T> rdd,
                                                SparkRecordInfoProvider<T> recordInfoProvider,
                                                java.lang.Class<T> recordType)
```
    Creates a spatial RDD from the given java rdd. The given recordInfoProvider is used to extract spatial data from the the source rdd records which are expected to be of the type specified by recordType.
    
    Parameters:
    
    rdd - an existing JavaRDD
    
    recordInfoProvider - an implementation of SparkRecordInfoProvider
    
    recordType - the type of the source rdd records
    
    Returns:
    
    a new SpatialJavaRDD instance
  - fromRDD
```
public static <T> SpatialJavaRDD<T> fromRDD(org.apache.spark.rdd.RDD<T> rdd,
                                            SparkRecordInfoProvider<T> recordInfoProvider)
```
    Creates a spatial RDD from the given rdd. The given recordInfoProvider is used to extract spatial data from the the source rdd records.
    
    Parameters:
    
    rdd - an existing RDD
    
    recordInfoProvider - an implementation of SparkRecordInfoProvider
    
    Returns:
    
    a new SpatialJavaRDD instance
  - fromJavaRDD
```
public static <T> SpatialJavaRDD<SparkRecordInfo> fromJavaRDD(org.apache.spark.api.java.JavaRDD<T> rdd,
                                                              SparkRecordInfoProvider<T> recordInfoProvider)
```
    Creates a spatial RDD of type SparkRecordInfo from the given rdd. The given recordInfoProvider is used to extract spatial data and any other data from the the source rdd records.
    
    Parameters:
    
    rdd - an existing RDD
    
    recordInfoProvider - an implementation of SparkRecordInfoProvider
    
    Returns:
    
    a new SpatialJavaRDD instance
  - getRecordInfoProvider
```
public SparkRecordInfoProvider<T> getRecordInfoProvider()
```
    Gets the RDD's SparkRecordInfoProvider instance
    
    Returns:
    
    a SparkRecordInfoProvider instance
  - getType
```
public java.lang.Class<T> getType()
```
    Gets the type of the records in the RDD
    
    Returns:
    
    The class type of the RDD's records
  - createSpatialTransformationContext
```
public SpatialTransformationContext<T> createSpatialTransformationContext(SpatialOperationConfig spatialOperationConf)
```
    Creates an instance of SpatialTransformationContext associated to the given SpatialOperationConfig.
    
    Parameters:
    
    spatialOperationConf - a spatial operation used to filter records
    
    Returns:
    
    an instance of SpatialTransformationContext
  - createSpatialTransformationContext
```
public SpatialTransformationContext<T> createSpatialTransformationContext()
```
    Creates an instance of SpatialTransformationContext
    
    Returns:
    
    an instance of SpatialTransformationContext
  - filter
```
public SpatialJavaRDD<T> filter(org.apache.spark.api.java.function.Function<T,java.lang.Boolean> f,
                                SpatialOperationConfig spatialOpConf)
```
    Returns a new spatial RDD containing only the elements that satisfy both, the filtering function and the spatial operation
    
    Parameters:
    
    f - a filtering function
    
    spatialOpConf - a spatial operation used to filter records
    
    Returns:
    
    a new SpatialJavaRDD
  - flatMap
```
public <R> org.apache.spark.api.java.JavaRDD<R> flatMap(org.apache.spark.api.java.function.Function<T,java.lang.Iterable<R>> m,
                                                        SpatialOperationConfig spatialOpConf)
```
    Returns a new RDD by first spatially filtering the RDD elements using the spatial operation given by spatialOpConf, then a function is applied to all the remaining elements.
    
    Parameters:
    
    m - a function to apply to each element
    
    spatialOpConf - a spatial operation used to filter records
    
    Returns:
    
    a new RDD
  - nearestNeighbors
```
public java.util.List<scala.Tuple2<java.lang.Double,T>> nearestNeighbors(oracle.spatial.geometry.JGeometry qryWindow,
                                                                         int k,
                                                                         double tol)
```
    Returns the k elements which are closest to the given query window
    
    Parameters:
    
    qryWindow - a geometry from where the nearest neighbors will be calculated
    
    k - the number of neighbors
    
    tol - the tolerance used
    
    Returns:
    
    a list of the k nearest neighbors
  - getMBR
```
public double[] getMBR()
```
    Gets the minimum bounding rectangle of the RDD
    
    Returns:
    
    a double array containing the following elements: minX, minY, maxX, maxY
  - count
```
public long count()
```
    Specified by:
    
    count in interface org.apache.spark.api.java.JavaRDDLike<T,org.apache.spark.api.java.JavaRDD<T>>
    
    Overrides:
    
    count in class org.apache.spark.api.java.JavaRDD<T>
  - createSpatialDataFrame
```
public org.apache.spark.sql.DataFrame createSpatialDataFrame(org.apache.spark.sql.SQLContext sqlContext,
                                                             java.util.List<java.lang.String> fieldsList)
```
    Creates data frame based on the SpatialJavaRDD
    
    Parameters:
    
    sqlContext - the sqlContext
    
    fieldsList - the extra fields to include. The geometry will always be the first field.
    
    Returns:
    
    the data frame

Class SpatialJavaRDD<T>

Method Summary

Methods inherited from class org.apache.spark.api.java.JavaRDD

Methods inherited from class java.lang.Object

Method Detail

fromJavaRDD

fromRDD

fromJavaRDD

getRecordInfoProvider

getType

createSpatialTransformationContext

createSpatialTransformationContext

filter

flatMap

nearestNeighbors

getMBR

count

createSpatialDataFrame