oracle.pgx.api.spark2

Class PgxSparkContext

java.lang.Object

<any>

oracle.pgx.api.spark2.PgxSparkContext

public class PgxSparkContext
extends <any>

NOTE: Because this class is designed to be used only in a client mode, some of transitive libraries would not be included in server's library package. So, do NOT import this file in other files which can be ran in a server.

Constructor Summary

Constructors

Constructor and Description
PgxSparkContext(JavaSparkContext sc, PgxSession session)
PgxSparkContext(SQLContext sqlc, PgxSession session)

Method Summary

Modifier and Type	Method and Description
static <any>	createDataFrame(SQLContext sqlContext, <any> rowRdd, java.util.List<Pair<java.lang.String,java.lang.String>> scheme)
PgxGraph	read(<any> vertexDataset, <any> edgeDataset, java.lang.String graphName) Read a PGX frame by using the two provided datasets as vertex and edge tables.
<V,E> PgxGraph	read(java.lang.Class<V> vertexBeanClass, <any> vertexRdd, java.lang.Class<E> edgeBeanClass, <any> edgeRdd, java.lang.String graphName) Blocking version of #readAsync(Class, JavaRDD, Class, JavaRDD, String).
PgxGraph	read(GraphTableScheme vScheme, <any> vertexRdd, GraphTableScheme eScheme, <any> edgeRdd, java.lang.String graphName) Deprecated.
PgxFuture<PgxGraph>	readAsync(<any> vertexDataset, <any> edgeDataset, java.lang.String graphName) Read two table Datasets and store them into a Pgx backend as a PgxGraph.
<V,E> PgxFuture<PgxGraph>	readAsync(java.lang.Class<V> vertexBeanClass, <any> vertexRdd, java.lang.Class<E> edgeBeanClass, <any> edgeRdd, java.lang.String graphName) Read two table JavaRDDs and store them into a Pgx backend as a PgxGraph.
PgxFuture<PgxGraph>	readAsync(GraphTableScheme vScheme, <any> vertexRdd, GraphTableScheme eScheme, <any> edgeRdd, java.lang.String graphName) Deprecated.
Pair<<any>,<any>>	writeToDataframes(StructType vStruct, StructType eStruct, java.lang.String graphName)
PgxFuture<Pair<<any>,<any>>>	writeToDataframesAsync(GraphTableScheme vStruct, GraphTableScheme eStruct, java.lang.String graphName)
PgxFuture<Pair<<any>,<any>>>	writeToDataframesAsync(StructType vStruct, StructType eStruct, java.lang.String graphName) read a graph stored in Pgx and store as two table Datasets in Spark.
<V,E> Pair<<any>,<any>>	writeToJavaRdds(java.lang.Class<V> vertexBeanClass, java.lang.Class<E> edgeBeanClass, java.lang.String graphName)
<V,E> Pair<<any>,<any>>	writeToJavaRdds(java.lang.Class<V> vertexBeanClass, java.lang.Object vRowFactory, java.lang.Class<E> edgeBeanClass, java.lang.Object eRowFactory, java.lang.String graphName)
Pair<<any>,<any>>	writeToJavaRdds(TwoTablesSparkGraphConfig graphConfig, java.lang.String graphName)
<V,E> PgxFuture<Pair<<any>,<any>>>	writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass, java.lang.Class<E> edgeBeanClass, java.lang.String graphName) read a graph stored in Pgx and store as two table JavaRDDs in Spark.
<V,E> PgxFuture<Pair<<any>,<any>>>	writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass, java.lang.Object vRowFactory, java.lang.Class<E> edgeBeanClass, java.lang.Object eRowFactory, java.lang.String graphName) read a graph stored in Pgx and store as two table JavaRDDs in Spark.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PgxSparkContext

public PgxSparkContext(JavaSparkContext sc,
                       PgxSession session)

PgxSparkContext

public PgxSparkContext(SQLContext sqlc,
                       PgxSession session)

Method Detail

createDataFrame

public static <any> createDataFrame(SQLContext sqlContext,
                                    <any> rowRdd,
                                    java.util.List<Pair<java.lang.String,java.lang.String>> scheme)

read

public PgxGraph read(<any> vertexDataset,
                     <any> edgeDataset,
                     java.lang.String graphName)
              throws java.lang.InterruptedException,
                     java.util.concurrent.ExecutionException

Read a PGX frame by using the two provided datasets as vertex and edge tables.

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

read

public <V,E> PgxGraph read(java.lang.Class<V> vertexBeanClass,
                           <any> vertexRdd,
                           java.lang.Class<E> edgeBeanClass,
                           <any> edgeRdd,
                           java.lang.String graphName)
                    throws java.util.concurrent.ExecutionException,
                           java.lang.InterruptedException,
                           java.lang.instrument.IllegalClassFormatException,
                           java.lang.NoSuchMethodException

Blocking version of #readAsync(Class, JavaRDD, Class, JavaRDD, String). Calls #readAsync(Class, JavaRDD, Class, JavaRDD, String) and waits for returned PgxFuture to complete.

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

read

@Deprecated
public PgxGraph read(GraphTableScheme vScheme,
                                 <any> vertexRdd,
                                 GraphTableScheme eScheme,
                                 <any> edgeRdd,
                                 java.lang.String graphName)
                          throws java.lang.InterruptedException,
                                 java.util.concurrent.ExecutionException

Deprecated.

Blocking version of #readAsync(GraphTableScheme, JavaRDD, GraphTableScheme, JavaRDD, String). Calls #readAsync(GraphTableScheme, JavaRDD, GraphTableScheme, JavaRDD, String) and waits for returned PgxFuture to complete.

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

readAsync

public PgxFuture<PgxGraph> readAsync(<any> vertexDataset,
                                     <any> edgeDataset,
                                     java.lang.String graphName)
                              throws java.util.concurrent.ExecutionException,
                                     java.lang.InterruptedException

Read two table Datasets and store them into a Pgx backend as a PgxGraph.

vertexDataset must have following table schema:

• 1-st column: vertex ID (String, Integer or Long)
• 2-nd ~ columns: vertex properties

edgeDataset must have following table schema:

• 1-st column: source vertex ID (String, Integer or Long)
• 2-nd column: destination vertex ID (String, Integer or Long)
• 3-rd ~ columns: edge properties

Parameters:

vertexDataset - a Dataset object contains vertex data

edgeDataset - a Dataset object contains edge data

graphName - a name of graph to be loaded

Returns:

PgxGraph for the given Datasets

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

readAsync

public <V,E> PgxFuture<PgxGraph> readAsync(java.lang.Class<V> vertexBeanClass,
                                           <any> vertexRdd,
                                           java.lang.Class<E> edgeBeanClass,
                                           <any> edgeRdd,
                                           java.lang.String graphName)
                                    throws java.lang.instrument.IllegalClassFormatException,
                                           java.lang.NoSuchMethodException

Read two table JavaRDDs and store them into a Pgx backend as a PgxGraph.

Type Parameters:

V - a class to represent a vertex tuple

E - a class to represent an edge tuple

Parameters:

vertexBeanClass - a JavaBean class definition for vertexRdd

vertexRdd - JavaRDD<V>

edgeBeanClass - a JavaBean class definition for edgeRdd

edgeRdd - JavaRDD<E>

graphName - a name of graph to be loaded

Returns:

PgxGraph for the given JavaRDDs

Throws:

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

readAsync

@Deprecated
public PgxFuture<PgxGraph> readAsync(GraphTableScheme vScheme,
                                                 <any> vertexRdd,
                                                 GraphTableScheme eScheme,
                                                 <any> edgeRdd,
                                                 java.lang.String graphName)

Deprecated.

Read two table JavaRDDs and store them into a Pgx backend as a PgxGraph.

Parameters:

vScheme - vertex table schema

vertexRdd - JavaRDD<GraphVertexTableTuple>

eScheme - edge table schema

edgeRdd - JavaRDD<GraphEdgeTableTuple>

graphName - a name of graph to be loaded

Returns:

PgxGraph for the given JavaRDDs

writeToDataframes

public Pair<<any>,<any>> writeToDataframes(StructType vStruct,
                                           StructType eStruct,
                                           java.lang.String graphName)
                                    throws java.util.concurrent.ExecutionException,
                                           java.lang.InterruptedException

Parameters:

vStruct - schema of vertices

eStruct - schema of edges

graphName - the name of the graph in the Pgx side to export into a vertex Dataset and a edge Dataset

Returns:

a Pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

writeToDataframesAsync

public PgxFuture<Pair<<any>,<any>>> writeToDataframesAsync(GraphTableScheme vStruct,
                                                           GraphTableScheme eStruct,
                                                           java.lang.String graphName)
                                                    throws java.util.concurrent.ExecutionException,
                                                           java.lang.InterruptedException

Throws:

java.util.concurrent.ExecutionException

java.lang.InterruptedException

writeToDataframesAsync

public PgxFuture<Pair<<any>,<any>>> writeToDataframesAsync(StructType vStruct,
                                                           StructType eStruct,
                                                           java.lang.String graphName)
                                                    throws java.util.concurrent.ExecutionException,
                                                           java.lang.InterruptedException

read a graph stored in Pgx and store as two table Datasets in Spark.

Parameters:

vStruct - a StructType of a node Dataset to be stored

eStruct - a StructType of a edge Dataset to be stored

graphName - a name of graph stored in Pgx backend

Returns:

a PgxFuture instance for a pair of a vertex table and an edge table

Throws:

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

writeToJavaRdds

public <V,E> Pair<<any>,<any>> writeToJavaRdds(java.lang.Class<V> vertexBeanClass,
                                               java.lang.Class<E> edgeBeanClass,
                                               java.lang.String graphName)
                                        throws java.util.concurrent.ExecutionException,
                                               java.lang.InterruptedException,
                                               java.lang.instrument.IllegalClassFormatException,
                                               java.lang.NoSuchMethodException

Throws:

java.util.concurrent.ExecutionException

java.lang.InterruptedException

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

writeToJavaRdds

public <V,E> Pair<<any>,<any>> writeToJavaRdds(java.lang.Class<V> vertexBeanClass,
                                               java.lang.Object vRowFactory,
                                               java.lang.Class<E> edgeBeanClass,
                                               java.lang.Object eRowFactory,
                                               java.lang.String graphName)
                                        throws java.util.concurrent.ExecutionException,
                                               java.lang.InterruptedException,
                                               java.lang.instrument.IllegalClassFormatException,
                                               java.lang.NoSuchMethodException

Throws:

java.util.concurrent.ExecutionException

java.lang.InterruptedException

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

writeToJavaRdds

public Pair<<any>,<any>> writeToJavaRdds(TwoTablesSparkGraphConfig graphConfig,
                                         java.lang.String graphName)
                                  throws java.util.concurrent.ExecutionException,
                                         java.lang.InterruptedException

Parameters:

graphConfig - an instance of TwoTablesSparkGraphConfig which represent schemes of graph tables

graphName - the name of the graph in the Pgx side want to store in the Spark side

Returns:

a Pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

writeToJavaRddsAsync

public <V,E> PgxFuture<Pair<<any>,<any>>> writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass,
                                                               java.lang.Class<E> edgeBeanClass,
                                                               java.lang.String graphName)
                                                        throws java.util.concurrent.ExecutionException,
                                                               java.lang.InterruptedException,
                                                               java.lang.instrument.IllegalClassFormatException,
                                                               java.lang.NoSuchMethodException

read a graph stored in Pgx and store as two table JavaRDDs in Spark.

Type Parameters:

V - the type of a vertex object

E - the type of an edge object

Parameters:

vertexBeanClass - a JavaBean class definition for V

edgeBeanClass - a JavaBean class definition for E

graphName - a name of graph stored in Pgx backend

Returns:

a PgxFuture instance for a pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

writeToJavaRddsAsync

public <V,E> PgxFuture<Pair<<any>,<any>>> writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass,
                                                               java.lang.Object vRowFactory,
                                                               java.lang.Class<E> edgeBeanClass,
                                                               java.lang.Object eRowFactory,
                                                               java.lang.String graphName)
                                                        throws java.util.concurrent.ExecutionException,
                                                               java.lang.InterruptedException,
                                                               java.lang.instrument.IllegalClassFormatException,
                                                               java.lang.NoSuchMethodException

read a graph stored in Pgx and store as two table JavaRDDs in Spark.

Type Parameters:

V - the type of a vertex object

E - the type of an edge object

Parameters:

vertexBeanClass - a JavaBean class definition for V

vRowFactory - a instance of row factory. It must has a public member function which is named as "newInstance" with no argument an returns a instance of V

edgeBeanClass - a JavaBean class definition for E

eRowFactory - a instance of row factory. It must has a public member function which is named as "newInstance" with no argument an returns a instance of E

graphName - a name of graph stored in Pgx backend

Returns:

a PgxFuture instance for a pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException