oracle.pg.hbase

Class OraclePropertyGraphUtils

java.lang.Object

oracle.pg.common.OracleConvertersUtilsBase

oracle.pg.common.OraclePropertyGraphUtilsBase

oracle.pg.hbase.OraclePropertyGraphUtils

All Implemented Interfaces:

oracle.pg.common.messages.MesgConsts

public class OraclePropertyGraphUtils
extends OraclePropertyGraphUtilsBase

This class contains utility functions to help manage property graphs stored in an Apache HBase Database

Field Summary

Fields inherited from interface oracle.pg.common.messages.MesgConsts

ERR_ARG_CANNOT_BE_NULL, ERR_CODE_NOT_IN_LIST, ERR_DOP_CONSTRAINT, ERR_E_AUTO_IDX_MUST_SOLR_OR_LUCENE, ERR_E_AUTO_IDX_NOT_EXIST, ERR_E_DS_MUSTBE_FILE_OR_IS, ERR_E_MAX_LINE_CONSTRAINT, ERR_EDGE_DIR_NOT_BOTH, ERR_EDGE_WORK_TBL_NOT_CLEAN, ERR_GRAPH_NAME_CANT_BENULL, ERR_HIT_EXCEPTION, ERR_HIT_INTERRUPT, ERR_HIT_IO, ERR_HIT_OPGE, ERR_HIT_PARSEE, ERR_ID_CANNOT_BE_NULL, ERR_IDX_KEY_CANNOT_BE_NULL, ERR_IDX_KEY_NOT_NULL, ERR_IDX_KEYS_NOT_NULL, ERR_IN_DS_EDGE_CANTBE_NULL, ERR_IN_DS_VERT_CANTBE_NULL, ERR_IN_E_FLAT_DOESNT_EXIST, ERR_IN_V_FLAT_DOESNT_EXIST, ERR_INPUT_LIST_NOT_NULL, ERR_INPUT_NOT_NULL, ERR_INVALID_CHANGE_ACTION, ERR_INVALID_NUM_FIELDS, ERR_INVALID_NUM_NOARG, ERR_INVALID_NUMBER, ERR_INVALID_VAL, ERR_KEY_CANNOT_BE_ID, ERR_KEY_CANNOT_BE_NULL, ERR_KEY_CANNOT_BE_NULL_EMPTY, ERR_KEY_NOT_SET_TO, ERR_NO_ELEM_TO_BE_CONSUMED, ERR_NO_OP_SUPPORTED, ERR_NOT_IMPLEMENTED_YET, ERR_NOT_ORACLE_V_E, ERR_NUM_PART_CONSTRAINT, ERR_OFFSET_CONSTRAINT, ERR_OFFSET_E_MUST_BE_POS, ERR_OFFSET_V_MUST_BE_POS, ERR_OPT_FLAG_EDGE_INVALID, ERR_OPT_FLAG_VERT_INVALID, ERR_ORA_TEXT_INVALID_KEY, ERR_ORA_TEXT_ONLY_AUTO, ERR_OUTPUT_LIST_NOT_NULL, ERR_PG_ALREADY_EXISTS, ERR_PG_NOT_EMPTY, ERR_PG_NOT_NULL, ERR_SIZE_MUST_BE_POS, ERR_TIMEOUT_TAB_CREATION, ERR_TXT_IDX_NOT_FOUND, ERR_TYPE_ID_NOT_RECO, ERR_UNSUPPORTED_VAL_TYP, ERR_USER_REQ_OP_CANCEL, ERR_V_AUTO_IDX_MUST_SOLR_OR_LUCENE, ERR_V_AUTO_IDX_NOT_EXIST, ERR_V_DS_MUSTBE_FILE_OR_IS, ERR_V_MAX_LINE_CONSTRAINT, ERR_VALUE_CANNOT_BE_NULL, INFO_EMP_K_NO_P_VAL, INFO_NUL_K_NO_P_VAL

Constructor Summary

Constructors

Constructor and Description
OraclePropertyGraphUtils()

Method Summary

Modifier and Type	Method and Description
static byte[]	bigIntegerToByteArray(java.math.BigInteger key) Parse a bigInteger into a byte array of length 16.
static java.lang.String	byteArrayToStr(java.lang.String marker, byte[] ba) Returns a string representation of the specified byte array
static java.lang.String	bytesToHex(byte[] bytes) Parse a byte array into a Hex-based string.
static void	copyGraph(org.apache.hadoop.conf.Configuration conf, java.lang.String graphName, java.lang.String newGraphName) Copy the property graph (including its associated tables) to the specified graph name (if it is not empty).
static void	dropPropertyGraph(org.apache.hadoop.conf.Configuration conf, java.lang.String szGraphName) Drops the specified property graph (including its associated tables).
static void	dropTable(org.apache.hadoop.hbase.client.Admin admin, java.lang.String tableName) Drops the specified table This operation cannot be undone.
static boolean	existsGraph(org.apache.hadoop.conf.Configuration conf, java.lang.String graphName) Checks if the specified property graph already exists in the database
static boolean	existsTable(org.apache.hadoop.hbase.client.Admin admin, org.apache.hadoop.hbase.TableName tableName) Checks if the specified property graph already exists in the database
static void	exportEdgeFlatFile(OraclePropertyGraph pg, java.io.OutputStream osEdge, int dop, boolean append, boolean closeStream) This function exports the edges of a given property graph into OPE flat file
static void	exportFlatFiles(OraclePropertyGraph pg, java.io.OutputStream osVertex, java.io.OutputStream osEdge, boolean append, int dop) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportFlatFiles(OraclePropertyGraph pg, java.io.OutputStream osVertex, java.io.OutputStream osEdge, int dop, boolean append, boolean closeStreams) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportFlatFiles(OraclePropertyGraph pg, java.lang.String vertexFileName, java.lang.String edgeFileName, boolean append) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportFlatFiles(OraclePropertyGraph pg, java.lang.String vertexFileName, java.lang.String edgeFileName, int dop, boolean append) This function exports a given property graph into two flat files: one for vertices and the other for edges.
static void	exportVertexFlatFile(OraclePropertyGraph pg, java.io.OutputStream osVertex, int dop, boolean append, boolean closeStream) This function exports the vertices of a given property graph into OPV flat file
static java.util.List<java.lang.String>	getGraphNames(org.apache.hadoop.conf.Configuration conf) List all the names of the property graphs stored in the given database
static byte[][]	getHexSplitsWithEndpoints(java.lang.String startKey, java.lang.String endKey, int numRegions) Generates an array with the splits (including the end points) for the specified start and end keys.
static void	printByteArray(java.lang.String marker, byte[] ba, java.io.PrintStream ps) Prints a string representation of the byte array to the specified PrintStream
static void	quietlyCloseConnection(org.apache.hadoop.hbase.client.Connection connection) Quietly close an Connection object handling a connection to an Apache HBase Database
static void	quietlyCloseConnections(org.apache.hadoop.hbase.client.Connection[] connections) Quietly close an array of Connection objects handling connections to an Apache HBase Database.
static void	renameGraph(org.apache.hadoop.conf.Configuration conf, java.lang.String graphName, java.lang.String newGraphName) Renames the property graph (including its associated tables) to the specified graph name (if it does not exists or is not empty).
static boolean	sameSubArray(byte[] ba1, byte[] ba2, int iStart, int iLen) Verifies if two byte arrays are equals from the specified start and length.

Methods inherited from class oracle.pg.common.OraclePropertyGraphUtilsBase

castAsOracleConnection, compareFiles, compareValueObjects, exportEdgeFlatFile, exportEdges, exportFlatFiles, exportFlatFiles, exportFlatFiles, exportFlatFiles, exportVertexFlatFile, exportVertices, getDatatypeAsPropertyType, getPropertiesFromFile, getPropertyType, getPropertyValueAsStr, handleGZipInputStream, isDateValue, isNumericValue, listify, outputEdgeRecord, outputVertexRecord, outputVertexRecord, prepareConfigEdgePropertiesFromOPVFile, prepareConfigVertexPropertiesFromOPVFile, quietlyClose, quietlyClose, quietlyClose, quietlyClose, quietlyClose, quietlyCloseResultSet, quietlyCloseResultSetAndStmt, quietlyCloseResultSetAndStmt, quietlyCloseStmt, quietlyShutdown, saveInputStreamIntoFile, size, size, unescape, validateSources

Methods inherited from class oracle.pg.common.OracleConvertersUtilsBase

convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPE, convertCSV2OPV, convertCSV2OPV, convertCSV2OPV, convertCSV2OPV, convertCSV2OPV, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPE, convertRDBMSTable2OPV, convertRDBMSTable2OPV, convertRDBMSTable2OPV, convertRDBMSTable2OPV

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OraclePropertyGraphUtils

public OraclePropertyGraphUtils()

Method Detail

bigIntegerToByteArray

public static byte[] bigIntegerToByteArray(java.math.BigInteger key)

Parse a bigInteger into a byte array of length 16.

Parameters:

key - the big integer to be parsed

Returns:

a byte array of length 16, representing the number.

byteArrayToStr

public static java.lang.String byteArrayToStr(java.lang.String marker,
                                              byte[] ba)

Returns a string representation of the specified byte array

Parameters:

marker - an additional marker to be printed with the byte array

ba - the byte array to be printed

Returns:

a string representation of the byte array.

bytesToHex

public static java.lang.String bytesToHex(byte[] bytes)

Parse a byte array into a Hex-based string.

Parameters:

bytes - the byte array to be parsed.

Returns:

a Hex-based string

copyGraph

public static void copyGraph(org.apache.hadoop.conf.Configuration conf,
                             java.lang.String graphName,
                             java.lang.String newGraphName)
                      throws java.io.IOException,
                             java.lang.Exception

Copy the property graph (including its associated tables) to the specified graph name (if it is not empty). This operation will not copy existing indices into the new graph.

Parameters:

conf - the Configuration object with all the HBase connection information.

graphName - the name of the property graph to be copied

newGraphName - the new name of the graph

Throws:

java.io.IOException

java.lang.Exception

dropPropertyGraph

public static void dropPropertyGraph(org.apache.hadoop.conf.Configuration conf,
                                     java.lang.String szGraphName)
                              throws java.io.IOException,
                                     java.lang.Exception

Drops the specified property graph (including its associated tables). This operation cannot be undone.

Parameters:

conf - the Configuration object with all the HBase connection information.

szGraphName - the name of the graph to drop.

Throws:

java.io.IOException

java.lang.Exception

dropTable

public static void dropTable(org.apache.hadoop.hbase.client.Admin admin,
                             java.lang.String tableName)
                      throws java.io.IOException

Drops the specified table This operation cannot be undone.

Parameters:

admin - the Admin object with all the HBase connection information.

tableName - the name of the table to drop.

Throws:

java.io.IOException

java.lang.Exception

existsGraph

public static boolean existsGraph(org.apache.hadoop.conf.Configuration conf,
                                  java.lang.String graphName)
                           throws java.io.IOException

Checks if the specified property graph already exists in the database

Parameters:

conf - the Configuration object with all the HBase connection information.

graphName - the name of the property graph

Throws:

java.io.IOException

existsTable

public static boolean existsTable(org.apache.hadoop.hbase.client.Admin admin,
                                  org.apache.hadoop.hbase.TableName tableName)
                           throws java.io.IOException

Checks if the specified property graph already exists in the database

Parameters:

admin - the Admin object with all the HBase connection information.

tableName - the name of the property graph

Throws:

java.io.IOException

exportEdgeFlatFile

public static void exportEdgeFlatFile(OraclePropertyGraph pg,
                                      java.io.OutputStream osEdge,
                                      int dop,
                                      boolean append,
                                      boolean closeStream)
                               throws java.io.IOException,
                                      java.io.FileNotFoundException,
                                      java.lang.InterruptedException

This function exports the edges of a given property graph into OPE flat file

Parameters:

pg - the property graph that will be exported into flat files

osEdge - an OutputStream object for the edge flat file

dop - the degree of parallelism used when writing the data into the flat file. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat file removing all existing data.

closeStream - whether to close the OutputStream or not

Throws:

java.io.IOException

java.io.FileNotFoundException

java.lang.InterruptedException

exportFlatFiles

public static void exportFlatFiles(OraclePropertyGraph pg,
                                   java.io.OutputStream osVertex,
                                   java.io.OutputStream osEdge,
                                   boolean append,
                                   int dop)
                            throws java.io.IOException,
                                   java.io.FileNotFoundException,
                                   java.lang.InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

osVertex - an OutputStream object for the vertex flat file

osEdge - an OutputStream object for the edge flat file

dop - the degree of parallelism used when writing the data into the flat files. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat files removing all existing data.

Throws:

java.io.IOException

java.io.FileNotFoundException

java.lang.InterruptedException

exportFlatFiles

public static void exportFlatFiles(OraclePropertyGraph pg,
                                   java.io.OutputStream osVertex,
                                   java.io.OutputStream osEdge,
                                   int dop,
                                   boolean append,
                                   boolean closeStreams)
                            throws java.io.IOException,
                                   java.io.FileNotFoundException,
                                   java.lang.InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

osVertex - an OutputStream object for the vertex flat file

osEdge - an OutputStream object for the edge flat file

dop - the degree of parallelism used when writing the data into the flat files. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat files removing all existing data.

closeStreams - whether to close the OutputStreams or not

Throws:

java.io.IOException

java.io.FileNotFoundException

java.lang.InterruptedException

exportFlatFiles

public static final void exportFlatFiles(OraclePropertyGraph pg,
                                         java.lang.String vertexFileName,
                                         java.lang.String edgeFileName,
                                         boolean append)
                                  throws java.io.IOException,
                                         java.io.FileNotFoundException,
                                         java.lang.InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

vertexFileName - the name of the vertex flat file e.g. graph.opv

edgeFileName - the name of the edge flat file e.g. graph.ope

append - if false, overwrite the existing flat files removing all existing data.

Throws:

java.lang.InterruptedException

java.io.IOException

java.io.FileNotFoundException

exportFlatFiles

public static final void exportFlatFiles(OraclePropertyGraph pg,
                                         java.lang.String vertexFileName,
                                         java.lang.String edgeFileName,
                                         int dop,
                                         boolean append)
                                  throws java.io.IOException,
                                         java.io.FileNotFoundException,
                                         java.lang.InterruptedException

This function exports a given property graph into two flat files: one for vertices and the other for edges.

Parameters:

pg - the property graph that will be exported into flat files

vertexFileName - the name of the vertex flat file e.g. graph.opv

edgeFileName - the name of the edge flat file e.g. graph.ope

dop - the degree of parallelism used when writing the data into the flat files. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat files removing all existing data.

Throws:

java.lang.InterruptedException

java.io.IOException

java.io.FileNotFoundException

exportVertexFlatFile

public static void exportVertexFlatFile(OraclePropertyGraph pg,
                                        java.io.OutputStream osVertex,
                                        int dop,
                                        boolean append,
                                        boolean closeStream)
                                 throws java.io.IOException,
                                        java.io.FileNotFoundException,
                                        java.lang.InterruptedException

This function exports the vertices of a given property graph into OPV flat file

Parameters:

pg - the property graph that will be exported into flat files

osVertex - an OutputStream object for the vertex flat file

dop - the degree of parallelism used when writing the data into the flat file. This will use multiple threads to optimize writing performance.

append - if false, overwrite the existing flat file removing all existing data.

closeStream - whether to close the OutputStream or not

Throws:

java.io.IOException

java.io.FileNotFoundException

java.lang.InterruptedException

getGraphNames

public static java.util.List<java.lang.String> getGraphNames(org.apache.hadoop.conf.Configuration conf)

List all the names of the property graphs stored in the given database

Parameters:

conf - the Configuration object with all the HBase connection information.

Returns:

a list of Strings with the names of the property graphs in the database.

getHexSplitsWithEndpoints

public static byte[][] getHexSplitsWithEndpoints(java.lang.String startKey,
                                                 java.lang.String endKey,
                                                 int numRegions)

Generates an array with the splits (including the end points) for the specified start and end keys.

Parameters:

startKey - the start key of the splits as a byte array.

endKey - the end key of the splits as a byte array.

numRegions - the number of splits to define.

Returns:

an array of byte[] with the split keys. The size of this array is the numRegions + 1 (as startKey and endKey are included in the array).

printByteArray

public static void printByteArray(java.lang.String marker,
                                  byte[] ba,
                                  java.io.PrintStream ps)

Prints a string representation of the byte array to the specified PrintStream

Parameters:

marker - an additional marker to be printed with the byte array

ba - the byte array to be printed

ps - a PrintStream object where the byte array will be printed

quietlyCloseConnection

public static void quietlyCloseConnection(org.apache.hadoop.hbase.client.Connection connection)

Quietly close an Connection object handling a connection to an Apache HBase Database

Parameters:

connection - the Connection to be closed.

quietlyCloseConnections

public static void quietlyCloseConnections(org.apache.hadoop.hbase.client.Connection[] connections)

Quietly close an array of Connection objects handling connections to an Apache HBase Database.

Parameters:

connections - the array of Connection to be closed.

renameGraph

public static void renameGraph(org.apache.hadoop.conf.Configuration conf,
                               java.lang.String graphName,
                               java.lang.String newGraphName)
                        throws java.io.IOException,
                               java.lang.Exception

Renames the property graph (including its associated tables) to the specified graph name (if it does not exists or is not empty). This operation will not take into account existing indices. Indices should be removed before calling this operation and re-executed using the new graph instance.

Parameters:

conf - the Configuration object with all the HBase connection information.

graphName - the name of the property graph to be renamed

newGraphName - the new name of the property graph.

Throws:

java.io.IOException

java.lang.Exception

sameSubArray

public static boolean sameSubArray(byte[] ba1,
                                   byte[] ba2,
                                   int iStart,
                                   int iLen)

Verifies if two byte arrays are equals from the specified start and length.

Parameters:

ba1 - the first byte array to compare

ba2 - the second byte array to compare

iStart - the start of the sub-array to compare

iLen - the length of the sub-array to compare

Returns:

true if the sub-arrays are equal, else return false.