15.4 Exporting a Graph into Multiple Files

You can store a graph into multiple files using the store method. Most parameters are the same, as if storing to a single file. However, the main difference lies in specifying how to partition the data.

You can partition the data in either of the following two ways:

• specifying a FileGraphStoringConfig (see Table 15-21 for more information)
• specifying a base path and the number of partitions

Export into Multiple Files Using FileGraphStoringConfig

You can specify a more detailed way of creating the multiple partitions used to store the graph by using the FileGraphStoringConfig. You can create a FileGraphStoringConfig object using a FileGraphStoringConfigBuilder.

For example, the following code specifies that the storing should be done into four partitions using the specified base path and using zero as the initial index for the partitioning. It also contains the file extension to use for vertex files and for edge files and finally it sets comma as the delimiter to be used when storing the graph data:

FileGraphStoringConfig storingConfig = new FileGraphStoringConfigBuilder(basePath) //
  .setNumPartitions(4) //
  .setInitialPartitionIndex(0) //
  .setVertexExtension(vertexExtension) //
  .setEdgeExtension(edgeExtension) //
  .setDelimiter(',') //
  .build();

You can also partition all tables equally using the numPartitions parameter. This implies that all tables are exported into the same number of files.

If you do not want to partition the tables equally, you can either create one PartitionedGraphConfig which contains for each provider a FileGraphStoringConfig (see Table 15-21) or we can use a version of store() that takes two maps of FileGraphStoringConfigs, one for the vertex tables and one for the edge tables.

For the first option, you can create for each vertex and edge table a FileGraphStoringConfig and put it into a FileEntityProviderConfig using setStoringOptions in the builder of FileEntityProviderConfig. The providers are then added to the PartitionedGraphConfig as edge and vertex providers using addVertexProvider() and addEdgeProvider() in the builder of PartitionedGraphConfig. Later you can use the store() method which takes the PartitionedGraphConfig as parameter.

The second option creates for every edge and vertex table a storing configuration, adds those into a vertex provider and an edge provider map and calls the corresponding store() method with these maps as parameters.

For example:

FileGraphStoringConfig vertexStoringConfig1 = new FileGraphStoringConfigBuilder(basePath + "_vertexTable1") //
  .setNumPartitions(4) //
  .setInitialPartitionIndex(0) //
  .setVertexExtension(vertexExtension) //
  .setDelimiter(',') //
  .build();

FileGraphStoringConfig vertexStoringConfig2 = new FileGraphStoringConfigBuilder(basePath + "_vertexTable2") //
  .setNumPartitions(4) //
  .setInitialPartitionIndex(0) //
  .setVertexExtension(vertexExtension) //
  .setDelimiter(',') //
  .build();

FileGraphStoringConfig edgeStoringConfig1 = new FileGraphStoringConfigBuilder(basePath + "_edgeTable1") //
  .setNumPartitions(4) //
  .setInitialPartitionIndex(0) //
  .setEdgeExtension(edgeExtension) //
  .setDelimiter(',') //
  .build();

Map<String, FileGraphStoringConfig> vertexStoringConfigs = new HashMap<>();
vertexStoringConfigs.put("vertexTable1", vertexStoringConfig1);
vertexStoringConfigs.put("vertexTable2", vertexStoringConfig2);

Map<String, FileGraphStoringConfig> edgeStoringConfigs = new HashMap<>();
edgeStoringConfigs.put("edgeTable1", edgeStoringConfig);

Export into Multiple Files without FileGraphStoringConfig

If you only need to specify how many partitions are required and the base name to be used, it is simpler to use store() method by only specifying those parameters. Following this procedure, the graph server (PGX) will use defaults for the other fields. See Table 15-21 for more information on default values.

Export into Multiple Files Using a Graph Configuration Object

An alternate way for exporting into multiple files is by creating a FileGraphStoringConfig and putting it into a Graph Configuration object using setStoringOptions in its builder, and then using the corresponding version of the store() method.