oracle.pg.hbase

Class OraclePropertyGraphUtils

java.lang.Object

oracle.pg.common.OraclePropertyGraphUtilsBase

oracle.pg.hbase.OraclePropertyGraphUtils

All Implemented Interfaces:

MesgConsts

public class OraclePropertyGraphUtils
extends OraclePropertyGraphUtilsBase

This class contains utility functions to help manage property graphs stored in an Apache HBase Database

Field Summary

Fields inherited from interface oracle.pg.common.MesgConsts

ERR_ARG_CANNOT_BE_NULL, ERR_CODE_NOT_IN_LIST, ERR_DOP_CONSTRAINT, ERR_E_AUTO_IDX_MUST_SOLR_OR_LUCENE, ERR_E_AUTO_IDX_NOT_EXIST, ERR_E_DS_MUSTBE_FILE_OR_IS, ERR_E_MAX_LINE_CONSTRAINT, ERR_EDGE_DIR_NOT_BOTH, ERR_EDGE_WORK_TBL_NOT_CLEAN, ERR_GRAPH_NAME_CANT_BENULL, ERR_HIT_EXCEPTION, ERR_HIT_INTERRUPT, ERR_HIT_IO, ERR_HIT_OPGE, ERR_HIT_PARSEE, ERR_ID_CANNOT_BE_NULL, ERR_IDX_KEY_CANNOT_BE_NULL, ERR_IDX_KEY_NOT_NULL, ERR_IDX_KEYS_NOT_NULL, ERR_IN_DS_EDGE_CANTBE_NULL, ERR_IN_DS_VERT_CANTBE_NULL, ERR_IN_E_FLAT_DOESNT_EXIST, ERR_IN_V_FLAT_DOESNT_EXIST, ERR_INPUT_LIST_NOT_NULL, ERR_INPUT_NOT_NULL, ERR_INVALID_CHANGE_ACTION, ERR_INVALID_NUM_FIELDS, ERR_INVALID_NUM_NOARG, ERR_INVALID_NUMBER, ERR_INVALID_VAL, ERR_KEY_CANNOT_BE_ID, ERR_KEY_CANNOT_BE_NULL, ERR_KEY_CANNOT_BE_NULL_EMPTY, ERR_KEY_NOT_SET_TO, ERR_NO_ELEM_TO_BE_CONSUMED, ERR_NO_OP_SUPPORTED, ERR_NOT_IMPLEMENTED_YET, ERR_NOT_ORACLE_V_E, ERR_NUM_PART_CONSTRAINT, ERR_OFFSET_CONSTRAINT, ERR_OFFSET_E_MUST_BE_POS, ERR_OFFSET_V_MUST_BE_POS, ERR_OPT_FLAG_EDGE_INVALID, ERR_OPT_FLAG_VERT_INVALID, ERR_ORA_TEXT_INVALID_KEY, ERR_ORA_TEXT_ONLY_AUTO, ERR_OUTPUT_LIST_NOT_NULL, ERR_PG_ALREADY_EXISTS, ERR_PG_NOT_EMPTY, ERR_PG_NOT_NULL, ERR_SIZE_MUST_BE_POS, ERR_TIMEOUT_TAB_CREATION, ERR_TXT_IDX_NOT_FOUND, ERR_TYPE_ID_NOT_RECO, ERR_UNSUPPORTED_VAL_TYP, ERR_USER_REQ_OP_CANCEL, ERR_V_AUTO_IDX_MUST_SOLR_OR_LUCENE, ERR_V_AUTO_IDX_NOT_EXIST, ERR_V_DS_MUSTBE_FILE_OR_IS, ERR_V_MAX_LINE_CONSTRAINT, ERR_VALUE_CANNOT_BE_NULL, INFO_EMP_K_NO_P_VAL, INFO_NUL_K_NO_P_VAL, TYPE_DT_BOOL, TYPE_DT_BYTE, TYPE_DT_CHAR, TYPE_DT_DATE, TYPE_DT_DOUBLE, TYPE_DT_EMPTY, TYPE_DT_FLOAT, TYPE_DT_INTEGER, TYPE_DT_JSON, TYPE_DT_LONG, TYPE_DT_SERI, TYPE_DT_SHORT, TYPE_DT_SPATIAL, TYPE_DT_STRING, TYPE_DT_URI

Constructor Summary

Constructors

Constructor and Description
OraclePropertyGraphUtils()

Method Summary

Modifier and Type	Method and Description
static byte[]	bigIntegerToByteArray(BigInteger key) Parse a bigInteger into a byte array of length 16.
static String	byteArrayToStr(String marker, byte[] ba) Returns a string representation of the specified byte array
static String	bytesToHex(byte[] bytes) Parse a byte array into a Hex-based string.
static void	copyGraph(org.apache.hadoop.conf.Configuration conf, String graphName, String newGraphName) Copy the property graph (including its associated tables) to the specified graph name (if it is not empty).
static void	dropPropertyGraph(org.apache.hadoop.conf.Configuration conf, String szGraphName) Drops the specified property graph (including its associated tables).
static boolean	existsGraph(org.apache.hadoop.conf.Configuration conf, String graphName) Checks if the specified property graph already exists in the database
static void	exportEdgeFlatFile(OraclePropertyGraph pg, OutputStream osEdge, int dop, boolean append, boolean closeStream) This function exports the edges of a given property graph into OPE flat file
static void	exportFlatFiles(OraclePropertyGraph pg, OutputStream osVertex, OutputStream osEdge, boolean append, int dop) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportFlatFiles(OraclePropertyGraph pg, OutputStream osVertex, OutputStream osEdge, int dop, boolean append, boolean closeStreams) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportFlatFiles(OraclePropertyGraph pg, String vertexFileName, String edgeFileName, boolean append) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportFlatFiles(OraclePropertyGraph pg, String vertexFileName, String edgeFileName, int dop, boolean append) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportVertexFlatFile(OraclePropertyGraph pg, OutputStream osVertex, int dop, boolean append, boolean closeStream) This function exports the vertices of a given property graph into OPV flat file
static List<String>	getGraphNames(org.apache.hadoop.conf.Configuration conf) List all the names of the property graphs stored in the given database
static byte[][]	getHexSplitsWithEndpoints(String startKey, String endKey, int numRegions) Generates an array with the splits (including the end points) for the specified start and end keys.
static void	printByteArray(String marker, byte[] ba, PrintStream ps) Prints a string representation of the byte array to the specified PrintStream
static void	quietlyCloseConnection(org.apache.hadoop.hbase.client.HConnection connection) Quietly close an HConnection object handling a connection to an Apache HBase Database
static void	quietlyCloseConnections(org.apache.hadoop.hbase.client.HConnection[] connections) Quietly close an array of HConnection objects handling connections to an Apache HBase Database.
static void	renameGraph(org.apache.hadoop.conf.Configuration conf, String graphName, String newGraphName) Renames the property graph (including its associated tables) to the specified graph name (if it does not exists or is not empty).
static boolean	sameSubArray(byte[] ba1, byte[] ba2, int iStart, int iLen) Verifies if two byte arrays are equals from the specified start and length.

Methods inherited from class oracle.pg.common.OraclePropertyGraphUtilsBase

compareFiles, compareValueObjects, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPV, convertCSV2OPV, convertCSV2OPV, convertCSV2OPV, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPV, escape, exportEdges, exportFlatFiles, exportFlatFiles, exportFlatFiles, exportGML, exportGML, exportGraphML, exportGraphML, exportGraphSON, exportGraphSON, exportVertices, getDatatypeAsPropertyType, handleGZipInputStream, importGML, importGML, importGraphML, importGraphML, importGraphSON, importGraphSON, listify, outputEdgeRecord, outputVertexRecord, prepareConfigEdgePropertiesFromOPVFile, prepareConfigVertexPropertiesFromOPVFile, quietlyClose, quietlyClose, quietlyClose, quietlyCloseResultSet, quietlyCloseResultSetAndStmt, quietlyCloseResultSetAndStmt, quietlyCloseStmt, quietlyShutdown, setExceptionHandlers, size, splitAndLoad, unescape, validateSources, writeValue

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OraclePropertyGraphUtils

public OraclePropertyGraphUtils()

Method Detail

bigIntegerToByteArray

public static byte[] bigIntegerToByteArray(BigInteger key)

Parse a bigInteger into a byte array of length 16.

Parameters:

key - the big integer to be parsed

Returns:

a byte array of length 16, representing the number.

byteArrayToStr

public static String byteArrayToStr(String marker,
                                    byte[] ba)

Returns a string representation of the specified byte array

Parameters:

marker - an additional marker to be printed with the byte array

ba - the byte array to be printed

Returns:

a string representation of the byte array.

bytesToHex

public static String bytesToHex(byte[] bytes)

Parse a byte array into a Hex-based string.

Parameters:

bytes - the byte array to be parsed.

Returns:

a Hex-based string

copyGraph

public static void copyGraph(org.apache.hadoop.conf.Configuration conf,
                             String graphName,
                             String newGraphName)
                      throws IOException,
                             Exception

Copy the property graph (including its associated tables) to the specified graph name (if it is not empty). This operation will not copy existing indices into the new graph.

Parameters:

conf - the Configuration object with all the HBase connection information.

graphName - the name of the property graph to be copied

newGraphName - the new name of the graph

Throws:

IOException

Exception

dropPropertyGraph

public static void dropPropertyGraph(org.apache.hadoop.conf.Configuration conf,
                                     String szGraphName)
                              throws IOException,
                                     Exception

Drops the specified property graph (including its associated tables). This operation cannot be undone.

Parameters:

conf - the Configuration object with all the HBase connection information.

szGraphName - the name of the graph to drop.

Throws:

IOException

Exception

existsGraph

public static boolean existsGraph(org.apache.hadoop.conf.Configuration conf,
                                  String graphName)
                           throws IOException

Checks if the specified property graph already exists in the database

Parameters:

conf - the Configuration object with all the HBase connection information.

graphName - the name of the property graph

Throws:

IOException

exportEdgeFlatFile

public static void exportEdgeFlatFile(OraclePropertyGraph pg,
                                      OutputStream osEdge,
                                      int dop,
                                      boolean append,
                                      boolean closeStream)
                               throws IOException,
                                      FileNotFoundException,
                                      InterruptedException

This function exports the edges of a given property graph into OPE flat file

Parameters:

pg - the property graph that will be exported into flat files

osEdge - an OutputStream object for the edge flat file

dop - the degree of parallelism used when writing the data into the flat file. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat file removing all existing data.

closeStream - whether to close the OutputStream or not

Throws:

IOException

FileNotFoundException

InterruptedException

exportFlatFiles

public static void exportFlatFiles(OraclePropertyGraph pg,
                                   OutputStream osVertex,
                                   OutputStream osEdge,
                                   boolean append,
                                   int dop)
                            throws IOException,
                                   FileNotFoundException,
                                   InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

osVertex - an OutputStream object for the vertex flat file

osEdge - an OutputStream object for the edge flat file

dop - the degree of parallelism used when writing the data into the flat files. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat files removing all existing data.

Throws:

IOException

FileNotFoundException

InterruptedException

exportFlatFiles

public static void exportFlatFiles(OraclePropertyGraph pg,
                                   OutputStream osVertex,
                                   OutputStream osEdge,
                                   int dop,
                                   boolean append,
                                   boolean closeStreams)
                            throws IOException,
                                   FileNotFoundException,
                                   InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

osVertex - an OutputStream object for the vertex flat file

osEdge - an OutputStream object for the edge flat file

dop - the degree of parallelism used when writing the data into the flat files. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat files removing all existing data.

closeStreams - whether to close the OutputStreams or not

Throws:

IOException

FileNotFoundException

InterruptedException

exportFlatFiles

public static final void exportFlatFiles(OraclePropertyGraph pg,
                                         String vertexFileName,
                                         String edgeFileName,
                                         boolean append)
                                  throws IOException,
                                         FileNotFoundException,
                                         InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

vertexFileName - the name of the vertex flat file e.g. graph.opv

edgeFileName - the name of the edge flat file e.g. graph.ope

append - if false, overwrite the existing flat files removing all existing data.

Throws:

InterruptedException

IOException

FileNotFoundException

exportFlatFiles

public static final void exportFlatFiles(OraclePropertyGraph pg,
                                         String vertexFileName,
                                         String edgeFileName,
                                         int dop,
                                         boolean append)
                                  throws IOException,
                                         FileNotFoundException,
                                         InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

vertexFileName - the name of the vertex flat file e.g. graph.opv

edgeFileName - the name of the edge flat file e.g. graph.ope

dop - the degree of parallelism used when writing the data into the flat files. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat files removing all existing data.

Throws:

InterruptedException

IOException

FileNotFoundException

exportVertexFlatFile

public static void exportVertexFlatFile(OraclePropertyGraph pg,
                                        OutputStream osVertex,
                                        int dop,
                                        boolean append,
                                        boolean closeStream)
                                 throws IOException,
                                        FileNotFoundException,
                                        InterruptedException

This function exports the vertices of a given property graph into OPV flat file

Parameters:

pg - the property graph that will be exported into flat files

osVertex - an OutputStream object for the vertex flat file

dop - the degree of parallelism used when writing the data into the flat file. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat file removing all existing data.

closeStream - whether to close the OutputStream or not

Throws:

IOException

FileNotFoundException

InterruptedException

getGraphNames

public static List<String> getGraphNames(org.apache.hadoop.conf.Configuration conf)

List all the names of the property graphs stored in the given database

Parameters:

conf - the Configuration object with all the HBase connection information.

Returns:

a list of Strings with the names of the property graphs in the database.

getHexSplitsWithEndpoints

public static byte[][] getHexSplitsWithEndpoints(String startKey,
                                                 String endKey,
                                                 int numRegions)

Generates an array with the splits (including the end points) for the specified start and end keys.

Parameters:

startKey - the start key of the splits as a byte array.

endKey - the end key of the splits as a byte array.

numRegions - the number of splits to define.

Returns:

an array of byte[] with the split keys. The size of this array is the numRegions + 1 (as startKey and endKey are included in the array).

printByteArray

public static void printByteArray(String marker,
                                  byte[] ba,
                                  PrintStream ps)

Prints a string representation of the byte array to the specified PrintStream

Parameters:

marker - an additional marker to be printed with the byte array

ba - the byte array to be printed

ps - a PrintStream object where the byte array will be printed

quietlyCloseConnection

public static void quietlyCloseConnection(org.apache.hadoop.hbase.client.HConnection connection)

Quietly close an HConnection object handling a connection to an Apache HBase Database

Parameters:

connection - the HConnection to be closed.

quietlyCloseConnections

public static void quietlyCloseConnections(org.apache.hadoop.hbase.client.HConnection[] connections)

Quietly close an array of HConnection objects handling connections to an Apache HBase Database.

Parameters:

connections - the array of HConnection to be closed.

renameGraph

public static void renameGraph(org.apache.hadoop.conf.Configuration conf,
                               String graphName,
                               String newGraphName)
                        throws IOException,
                               Exception

Renames the property graph (including its associated tables) to the specified graph name (if it does not exists or is not empty). This operation will not take into account existing indices. Indices should be removed before calling this operation and re-executed using the new graph instance.

Parameters:

conf - the Configuration object with all the HBase connection information.

graphName - the name of the property graph to be renamed

newGraphName - the new name of the property graph.

Throws:

IOException

Exception

sameSubArray

public static boolean sameSubArray(byte[] ba1,
                                   byte[] ba2,
                                   int iStart,
                                   int iLen)

Verifies if two byte arrays are equals from the specified start and length.

Parameters:

ba1 - the first byte array to compare

ba2 - the second byte array to compare

iStart - the start of the sub-array to compare

iLen - the length of the sub-array to compare

Returns:

true if the sub-arrays are equal, else return false.