oracle.pgx.api.spark1

Class PgxSparkContext

java.lang.Object

<any>

oracle.pgx.api.spark1.PgxSparkContext

public class PgxSparkContext
extends <any>

NOTE: Because this class is designed to be used only in a client mode, some of transitive libraries would not be included in server's library package. So, do NOT import this file in other files which can be ran in a server.

Constructor Summary

Constructors

Constructor and Description
PgxSparkContext(JavaSparkContext sc, PgxSession session)
PgxSparkContext(SQLContext sqlc, PgxSession session)

Method Summary

Modifier and Type	Method and Description
static DataFrame	createDataFrame(SQLContext sqlContext, <any> rowRdd, java.util.List<Pair<java.lang.String,java.lang.String>> scheme)
<V,E> PgxGraph	read(java.lang.Class<V> vertexBeanClass, <any> vertexRdd, java.lang.Class<E> edgeBeanClass, <any> edgeRdd, java.lang.String graphName) Blocking version of #readAsync(Class, JavaRDD, Class, JavaRDD, String).
PgxGraph	read(DataFrame vertexDataFrame, DataFrame edgeDataFrame, java.lang.String graphName) Blocking version of read(DataFrame, DataFrame, String).
PgxGraph	read(GraphTableScheme vScheme, <any> vertexRdd, GraphTableScheme eScheme, <any> edgeRdd, java.lang.String graphName) Deprecated.
<V,E> PgxFuture<PgxGraph>	readAsync(java.lang.Class<V> vertexBeanClass, <any> vertexRdd, java.lang.Class<E> edgeBeanClass, <any> edgeRdd, java.lang.String graphName) Read two table JavaRDDs and store them into a Pgx backend as a PgxGraph.
PgxFuture<PgxGraph>	readAsync(DataFrame vertexDataFrame, DataFrame edgeDataFrame, java.lang.String graphName) Read two table DataFrames and store them into a Pgx backend as a PgxGraph.
PgxFuture<PgxGraph>	readAsync(GraphTableScheme vScheme, <any> vertexRdd, GraphTableScheme eScheme, <any> edgeRdd, java.lang.String graphName) Deprecated.
Pair<DataFrame,DataFrame>	writeToDataframes(StructType vStruct, StructType eStruct, java.lang.String graphName)
PgxFuture<Pair<DataFrame,DataFrame>>	writeToDataframesAsync(GraphTableScheme vStruct, GraphTableScheme eStruct, java.lang.String graphName)
PgxFuture<Pair<DataFrame,DataFrame>>	writeToDataframesAsync(StructType vStruct, StructType eStruct, java.lang.String graphName) read a graph stored in Pgx and store as two table DataFrames in Spark.
<V,E> Pair<<any>,<any>>	writeToJavaRdds(java.lang.Class<V> vertexBeanClass, java.lang.Class<E> edgeBeanClass, java.lang.String graphName)
<V,E> Pair<<any>,<any>>	writeToJavaRdds(java.lang.Class<V> vertexBeanClass, java.lang.Object vRowFactory, java.lang.Class<E> edgeBeanClass, java.lang.Object eRowFactory, java.lang.String graphName)
Pair<<any>,<any>>	writeToJavaRdds(TwoTablesSparkGraphConfig graphConfig, java.lang.String graphName)
<V,E> PgxFuture<Pair<<any>,<any>>>	writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass, java.lang.Class<E> edgeBeanClass, java.lang.String graphName) read a graph stored in Pgx and store as two table JavaRDDs in Spark.
<V,E> PgxFuture<Pair<<any>,<any>>>	writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass, java.lang.Object vRowFactory, java.lang.Class<E> edgeBeanClass, java.lang.Object eRowFactory, java.lang.String graphName) read a graph stored in Pgx and store as two table JavaRDDs in Spark.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PgxSparkContext

public PgxSparkContext(JavaSparkContext sc,
                       PgxSession session)

PgxSparkContext

public PgxSparkContext(SQLContext sqlc,
                       PgxSession session)

Method Detail

createDataFrame

public static DataFrame createDataFrame(SQLContext sqlContext,
                                        <any> rowRdd,
                                        java.util.List<Pair<java.lang.String,java.lang.String>> scheme)

read

public <V,E> PgxGraph read(java.lang.Class<V> vertexBeanClass,
                           <any> vertexRdd,
                           java.lang.Class<E> edgeBeanClass,
                           <any> edgeRdd,
                           java.lang.String graphName)
                    throws java.util.concurrent.ExecutionException,
                           java.lang.InterruptedException,
                           java.lang.instrument.IllegalClassFormatException,
                           java.lang.NoSuchMethodException

Blocking version of #readAsync(Class, JavaRDD, Class, JavaRDD, String). Calls #readAsync(Class, JavaRDD, Class, JavaRDD, String) and waits for returned PgxFuture to complete.

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

read

public PgxGraph read(DataFrame vertexDataFrame,
                     DataFrame edgeDataFrame,
                     java.lang.String graphName)
              throws java.lang.InterruptedException,
                     java.util.concurrent.ExecutionException

Blocking version of read(DataFrame, DataFrame, String). Calls readAsync(DataFrame, DataFrame, String) and waits for returned PgxFuture to complete.

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

read

@Deprecated
public PgxGraph read(GraphTableScheme vScheme,
                                 <any> vertexRdd,
                                 GraphTableScheme eScheme,
                                 <any> edgeRdd,
                                 java.lang.String graphName)
                          throws java.lang.InterruptedException,
                                 java.util.concurrent.ExecutionException

Deprecated.

Blocking version of #readAsync(GraphTableScheme, JavaRDD, GraphTableScheme, JavaRDD, String). Calls #readAsync(GraphTableScheme, JavaRDD, GraphTableScheme, JavaRDD, String) and waits for returned PgxFuture to complete.

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

readAsync

public <V,E> PgxFuture<PgxGraph> readAsync(java.lang.Class<V> vertexBeanClass,
                                           <any> vertexRdd,
                                           java.lang.Class<E> edgeBeanClass,
                                           <any> edgeRdd,
                                           java.lang.String graphName)
                                    throws java.lang.instrument.IllegalClassFormatException,
                                           java.lang.NoSuchMethodException

Read two table JavaRDDs and store them into a Pgx backend as a PgxGraph.

Type Parameters:

V - a class to represent a vertex tuple

E - a class to represent an edge tuple

Parameters:

vertexBeanClass - a JavaBean class definition for vertexRdd

vertexRdd - JavaRDD<V>

edgeBeanClass - a JavaBean class definition for edgeRdd

edgeRdd - JavaRDD<E>

graphName - a name of graph to be loaded

Returns:

PgxGraph for the given JavaRDDs

Throws:

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

readAsync

public PgxFuture<PgxGraph> readAsync(DataFrame vertexDataFrame,
                                     DataFrame edgeDataFrame,
                                     java.lang.String graphName)
                              throws java.util.concurrent.ExecutionException,
                                     java.lang.InterruptedException

Read two table DataFrames and store them into a Pgx backend as a PgxGraph.

vertexDataFrame must have following table schema:

• 1-st column: vertex ID (String, Integer or Long)
• 2-nd ~ columns: vertex properties

edgeDataFrame must have following table schema:

• 1-st column: source vertex ID (String, Integer or Long)
• 2-nd column: destination vertex ID (String, Integer or Long)
• 3-rd ~ columns: edge properties

Parameters:

vertexDataFrame - a DataFrame object contains vertex data

edgeDataFrame - a DataFrame object contains edge data

graphName - a name of graph to be loaded

Returns:

PgxGraph for the given DataFrames

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

readAsync

@Deprecated
public PgxFuture<PgxGraph> readAsync(GraphTableScheme vScheme,
                                                 <any> vertexRdd,
                                                 GraphTableScheme eScheme,
                                                 <any> edgeRdd,
                                                 java.lang.String graphName)

Deprecated.

Read two table JavaRDDs and store them into a Pgx backend as a PgxGraph.

Parameters:

vScheme - vertex table schema

vertexRdd - JavaRDD<GraphVertexTableTuple>

eScheme - edge table schema

edgeRdd - JavaRDD<GraphEdgeTableTuple>

graphName - a name of graph to be loaded

Returns:

PgxGraph for the given JavaRDDs

writeToDataframes

public Pair<DataFrame,DataFrame> writeToDataframes(StructType vStruct,
                                                   StructType eStruct,
                                                   java.lang.String graphName)
                                            throws java.util.concurrent.ExecutionException,
                                                   java.lang.InterruptedException

Parameters:

vStruct -

eStruct -

graphName - the name of the graph in the Pgx side want to store in the Spark side

Returns:

a Pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

writeToDataframesAsync

public PgxFuture<Pair<DataFrame,DataFrame>> writeToDataframesAsync(GraphTableScheme vStruct,
                                                                   GraphTableScheme eStruct,
                                                                   java.lang.String graphName)
                                                            throws java.util.concurrent.ExecutionException,
                                                                   java.lang.InterruptedException

Throws:

java.util.concurrent.ExecutionException

java.lang.InterruptedException

writeToDataframesAsync

public PgxFuture<Pair<DataFrame,DataFrame>> writeToDataframesAsync(StructType vStruct,
                                                                   StructType eStruct,
                                                                   java.lang.String graphName)
                                                            throws java.util.concurrent.ExecutionException,
                                                                   java.lang.InterruptedException

read a graph stored in Pgx and store as two table DataFrames in Spark.

Parameters:

vStruct - a StructType of a node DataFrame to be stored

eStruct - a StructType of a edge DataFrame to be stored

graphName - a name of graph stored in Pgx backend

Returns:

a PgxFuture instance for a pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

writeToJavaRdds

public <V,E> Pair<<any>,<any>> writeToJavaRdds(java.lang.Class<V> vertexBeanClass,
                                               java.lang.Class<E> edgeBeanClass,
                                               java.lang.String graphName)
                                        throws java.util.concurrent.ExecutionException,
                                               java.lang.InterruptedException,
                                               java.lang.instrument.IllegalClassFormatException,
                                               java.lang.NoSuchMethodException

Throws:

java.util.concurrent.ExecutionException

java.lang.InterruptedException

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

writeToJavaRdds

public <V,E> Pair<<any>,<any>> writeToJavaRdds(java.lang.Class<V> vertexBeanClass,
                                               java.lang.Object vRowFactory,
                                               java.lang.Class<E> edgeBeanClass,
                                               java.lang.Object eRowFactory,
                                               java.lang.String graphName)
                                        throws java.util.concurrent.ExecutionException,
                                               java.lang.InterruptedException,
                                               java.lang.instrument.IllegalClassFormatException,
                                               java.lang.NoSuchMethodException

Throws:

java.util.concurrent.ExecutionException

java.lang.InterruptedException

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

writeToJavaRdds

public Pair<<any>,<any>> writeToJavaRdds(TwoTablesSparkGraphConfig graphConfig,
                                         java.lang.String graphName)
                                  throws java.util.concurrent.ExecutionException,
                                         java.lang.InterruptedException

Parameters:

graphConfig - an instance of TwoTablesSparkGraphConfig which represent schemes of graph tables

graphName - the name of the graph in the Pgx side want to store in the Spark side

Returns:

a Pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

writeToJavaRddsAsync

public <V,E> PgxFuture<Pair<<any>,<any>>> writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass,
                                                               java.lang.Class<E> edgeBeanClass,
                                                               java.lang.String graphName)
                                                        throws java.util.concurrent.ExecutionException,
                                                               java.lang.InterruptedException,
                                                               java.lang.instrument.IllegalClassFormatException,
                                                               java.lang.NoSuchMethodException

read a graph stored in Pgx and store as two table JavaRDDs in Spark.

Type Parameters:

V - the type of a vertex object

E - the type of an edge object

Parameters:

vertexBeanClass - a JavaBean class definition for V

edgeBeanClass - a JavaBean class definition for E

graphName - a name of graph stored in Pgx backend

Returns:

a PgxFuture instance for a pair of a vertex table and an edge table

Throws:

java.util.concurrent.ExecutionException

java.lang.InterruptedException

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException

writeToJavaRddsAsync

public <V,E> PgxFuture<Pair<<any>,<any>>> writeToJavaRddsAsync(java.lang.Class<V> vertexBeanClass,
                                                               java.lang.Object vRowFactory,
                                                               java.lang.Class<E> edgeBeanClass,
                                                               java.lang.Object eRowFactory,
                                                               java.lang.String graphName)
                                                        throws java.util.concurrent.ExecutionException,
                                                               java.lang.InterruptedException,
                                                               java.lang.instrument.IllegalClassFormatException,
                                                               java.lang.NoSuchMethodException

read a graph stored in Pgx and store as two table JavaRDDs in Spark.

Type Parameters:

V - the type of a vertex object

E - the type of an edge object

Parameters:

vertexBeanClass - a JavaBean class definition for V

vRowFactory - a instance of row factory. It must has a public member function which is named as "newInstance" with no argument an returns a instance of V

edgeBeanClass - a JavaBean class definition for E

eRowFactory - a instance of row factory. It must has a public member function which is named as "newInstance" with no argument an returns a instance of E

graphName - a name of graph stored in Pgx backend

Returns:

a PgxFuture instance for a pair of a vertex table and an edge table

Throws:

java.lang.InterruptedException - if the caller thread gets interrupted while waiting for completion.

java.util.concurrent.ExecutionException - if any exception occurred during asynchronous execution. The actual exception will be nested.

java.lang.instrument.IllegalClassFormatException

java.lang.NoSuchMethodException