Handling Partitioned Graph Configurations and Provider Configurations in Applications

This document discusses how to handle partitioned graph configurations, as well as provider configurations programmatically.

Java APIs To Parse Partitioned Graph Configuration Files

PGX provides a factory class in the oracle.pgx.config package you can use to parse partitioned graph configuration files. This factory is automatically used to deserialize contained provider configurations from a range of data-sources. Use the following API to acquire a config factory for partitioned graphs:

PartitionedGraphConfigFactory factory = GraphConfigFactory.forPartitioned();

Additionally the general-purpose factory can also be used for partitioned graphs:

GraphConfigFactory factory = GraphConfigFactory.forAnyFormat();

Partitioned graph configuration factories are similar to those for non-partitioned graph configurations, with some differences illustrated in this page.

PGX 20.1.1 limitation

It is currently not possible to construct a graph configuration from properties.

See the Javadocs for the full set of APIs.

Java APIs To Build Partitioned Graph Configuration

PGX provides builder classes in the oracle.pgx.config package which can be used to construct graph configuration and provider configuration objects programmatically without the need of any JSON or properties file:

Example 1

Build a simple graph configuration object with two vertex providers and one edge provider:

PartitionedGraphConfigBuilder configBuilder = GraphConfigBuilder.forPartitioned().setName("simpleGraph");

FileEntityProviderConfigBuilder vertexProviderBuilder1 = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("simpleVertexProvider1")
    .setKeyColumn(1)
    .addUri("simple1.csv");

FileEntityProviderConfigBuilder vertexProviderBuilder2 = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("simpleVertexProvider2")
    .setKeyColumn(1)
    .addUri("simple2.csv");

FileEntityProviderConfigBuilder edgeProviderBuilder = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("simpleEdgeProvider")
    .addUri("simple.csv")
    .setSourceVertexProvider("simpleVertexProvider1")
    .setDestinationVertexProvider("simpleVertexProvider2")
    .setKeyColumn(1)
    .setSourceColumn(2)
    .setDestinationColumn(3);

PartitionedGraphConfig graphConfig = configBuilder
    .addVertexProvider(vertexProviderBuilder1)
    .addVertexProvider(vertexProviderBuilder2)
    .addEdgeProvider(edgeProviderBuilder)
    .build();

Example 2

Build a typical graph configuration object with 2 vertex providers and 1 edge provider. This example illustrates how to load from different provider formats, how to use vertex and edge keys as IDs and how to specify properties.

PartitionedGraphConfigBuilder configBuilder = GraphConfigBuilder.forPartitioned().setName("typicalGraph");

// load one vertex provider from CSV
FileEntityProviderConfigBuilder vertexProviderBuilder1 = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("typicalVertexProvider1")
    .addUri("typical1.csv")
    .setSeparator(",")
    .hasHeader(true)
    .setKeyColumn(1)
    .addProperty("prop1", PropertyType.STRING, "", 2);

// load another vertex provider from PGB
FileEntityProviderConfigBuilder vertexProviderBuilder2 = new FileEntityProviderConfigBuilder(ProviderFormat.PGB)
    .setName("typicalVertexProvider2")
    .addUri("typical2.pgb")
    .addProperty("prop2", PropertyType.LOCAL_DATE);

// load the edge provider from CSV
FileEntityProviderConfigBuilder edgeProviderBuilder = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("typicalEdgeProvider")
    .addUri("typical.csv")
    .setSourceVertexProvider("typicalVertexProvider1")
    .setDestinationVertexProvider("typicalVertexProvider2")
    .setKeyColumn(1)
    .setSourceColumn(2)
    .setDestinationColumn(3)
    .addProperty("cost", PropertyType.DOUBLE, 0d, 4)
    .createKeyMapping(true)
    .setErrorHandlingOnMissingVertex(OnMissingVertex.IGNORE_EDGE);

GraphConfig graphConfig = configBuilder
    .setVertexProviders(vertexProviderBuilder1, vertexProviderBuilder2)
    .addEdgeProvider(edgeProviderBuilder)
    .setVertexIdStrategy(IdStrategy.KEYS_AS_IDS)
    .setEdgeIdStrategy(IdStrategy.KEYS_AS_IDS)
    .build();

Example 3

Build a complex graph configuration object with 5 vertex providers and 3 edge providers with different formats and combinations. This example illustrates how to load multiple providers from different formats, how to specify properties and other configuration fields. Also the graph will be loaded with no vertex/edge IDs.

PartitionedGraphConfigBuilder configBuilder = GraphConfigBuilder.forPartitioned().setName("complexGraph");

// Vertex Providers
FileEntityProviderConfigBuilder personVertexProvider = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("Person")
    .setUris("person1.csv", "person2.csv", "person3.csv", "person4.csv")
    .setKeyColumn(1)
    .addProperty("name", PropertyType.STRING, "", 2)
    .addProperty("age", PropertyType.INTEGER, 0, 3);

FileEntityProviderConfigBuilder transactionVertexProvider = new FileEntityProviderConfigBuilder(ProviderFormat.PGB)
    .setName("Transaction")
    .addUri("transaction.pgv")
    .setKeyColumn(1)
    .addProperty("creationDate", PropertyType.LOCAL_DATE);

FileEntityProviderConfigBuilder bankAccountVertexProvider = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("BankAccount")
    .setUris("bank1.csv", "bank2.csv")
    .hasHeader(true)
    .setKeyColumn("VID")
    .addProperty("money", PropertyType.DOUBLE);

FileEntityProviderConfigBuilder houseVertexProvider = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("House")
    .setUris("house1.csv", "house2.csv", "house3.csv")
    .setSeparator("|")
    .setKeyColumn(1)
    .addProperty("sqm", PropertyType.FLOAT, 0f, 2);

FileEntityProviderConfigBuilder carVertexProvider = new FileEntityProviderConfigBuilder(ProviderFormat.PGB)
    .setName("Car")
    .addUri("car.pgb")
    .addProperty("numDoors", PropertyType.INTEGER)
    .addProperty("year", PropertyType.INTEGER);

//Edge Providers
RdbmsEntityProviderConfigBuilder carOutrunsCarEdgeProvider = new RdbmsEntityProviderConfigBuilder()
    .setName("CarOutrunsCar")
    .setKeyColumn("EID")
    .setSourceColumn("SVID")
    .setDestinationColumn("DVID")
    .setJdbcUrl("jdbc:oracle:thin:@localhost:1521:rdbmsod")
    .setUsername("scott")
    .setPassword("tiger")
    .setSourceVertexProvider("Car")
    .setDestinationVertexProvider("Car")
    .addProperty("differenceInMillis", PropertyType.LONG)
    .addProperty("sourceTimeInMillis", PropertyType.LONG)
    .addProperty("destinationTimeInMillis", PropertyType.LONG)
    .createKeyMapping(true);

FileEntityProviderConfigBuilder personLivesInHouseEdgeProvider = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("PersonLivesInHouse")
    .addUri("personLivesInHouse.csv")
    .setSourceVertexProvider("Person")
    .setDestinationVertexProvider("House")
    .hasHeader(true)
    .setKeyColumn("EID")
    .setSourceColumn("person_key")
    .setDestinationColumn("house_key")
    .addProperty("movedSince", PropertyType.TIMESTAMP);


FileEntityProviderConfigBuilder bankAccountPaidTransactionEdgeProvider = new FileEntityProviderConfigBuilder(ProviderFormat.CSV)
    .setName("BankAccountPaidTransaction")
    .setSeparator(",")
    .hasHeader(true)
    .setUris("bankAccountPaidTransaction1.csv", "bankAccountPaidTransaction2.csv")
    .setSourceVertexProvider("Car")
    .setDestinationVertexProvider("Car")
    .setKeyColumn("EID")
    .setSourceColumn("car_source_key")
    .setDestinationColumn("car_dest_key")
    .setErrorHandlingOnMissingVertex(OnMissingVertex.ERROR);

GraphConfig graphConfig = configBuilder
    .setEdgeProviders(carOutrunsCarEdgeProvider, personLivesInHouseEdgeProvider, bankAccountPaidTransactionEdgeProvider)
    .setVertexProviders(personVertexProvider, transactionVertexProvider, bankAccountVertexProvider, houseVertexProvider, carVertexProvider)
    .setVertexIdStrategy(IdStrategy.NO_IDS)
    .setEdgeIdStrategy(IdStrategy.NO_IDS)
    .build();

See the Javadocs for the full set of APIs.