Changes in This Release for Oracle Big Data Spatial and Graph

Big Data Spatial and Graph includes the following changes to the product in Release 2.4.

• Changes for Release 2.5
  
• Changes for Release 2.4
  

Changes for Release 2.5

The following changes apply to Release 2.5 of Big Data Spatial and Graph.

• Spark Vector API Changes for Release 2.5
  
• Multimedia Analytics Feature Deprecated
  

Spark Vector API Changes for Release 2.5

The following features have been added to the Spark Vector API for Big Data Spatial and Graph 2.5:

• Count action added to the Spatial Index

Multimedia Analytics Feature Deprecated

The multimedia analytics feature of Big Data Spatial and Graph is deprecated in this release and may be desupported in a future release.  There is no replacement for the multimedia analytics features.

The multimedia analytics feature is currently documented in Using Multimedia Analytics.

Changes for Release 2.4

The following changes apply to Release 2.4 of Big Data Spatial and Graph.

• In-Memory Analyst (PGX) - Related Changes
  
• Spark Vector API Changes for Release 2.4
  
• Vector REST API Additions
  
• SpatialViewer Changes
  

In-Memory Analyst (PGX) - Related Changes

The following changes relate to the in-memory analyst (PGX) capabilities in Big Data Spatial and Graph.

• New PGX Built-in Algorithms for Cycle Detection
  
• Temporal Data Types Support in PGX
  
• PGX Java API Improvements
  
• New Features in PGQL
  
• PGX Loader Improvements
  
• PGX Distributed Engine Improvements
  
• PGX Deprecations
  

New PGX Built-in Algorithms for Cycle Detection

Release 2.4 of the in-memory analyst (PGX) introduces two in-memory algorithms for finding cycles: a robust version, which always scans the whole graph by performing several DFS traversals, and a lightweight version, which will perform just one single DFS traversal for the task. The lightweight version is faster, but may not explore the whole graph and could thus fail to detect some cycles.

You can use the new algorithms through the analyst.findCycle() API.

Temporal Data Types Support in PGX

The in-memory analyst (PGX) in Release 2.4 gives you more precise control on time-related properties, with support for five temporal data types that map directly to the five temporal types in SQL as well as to the new Java 8 date-time types. The date property type is now deprecated and replaced by local_date, time, timestamp, time_with_timezone, and timestamp_with_timezone. The new types are supported both in the PGX API and in PGQL, as the following table summarizes.

Table -1 Temporal Data Types Support in PGX

Type	PGX property type	Example plain text	Example PGQL literal	PGQL ResultSet API
TIMESTAMP WITH TIMEZONE	timestamp_with_timezone	"2017-08-18 20:15:00+08"	TIMESTAMP '2017-08-18 20:15:00+08'	java.time.OffsetDateTime getTimestampWithTimezone(..)
TIMESTAMP	timestamp	"2017-08-18 20:15:00"	TIMESTAMP '2017-08-18 20:15:00'	java.time.LocalDateTime getTimestamp(..)
TIME WITH TIMEZONE	time_with_timezone	"20:15:00+08"	TIME '20:15:00+08'	java.time.OffsetTime getTimeWithTimezone(..)
TIME	time	"20:15:00"	TIME '20:15:00'	java.time.LocalTime getTime(..)
DATE	local_date	"2017-08-18"	DATE '2017-08-18'	java.time.LocalDate getDate(..)

PGX Java API Improvements

Release 2.4 introduces several additions and improvements in the PGX Java API:

• Added Java API for getting all session private graphs and getting a graph by its name (PgxSession#getGraphs(), PgxSession#getGraph(String)).

• Added API for checking whether a graph has vertex/edge labels (PgxGraph#hasVertexLabels, PgxGraph#hasEdgeLabel.

• The GraphConfig builders can now copy values from existing GraphConfigs and it is now possible to remove properties from a GraphConfig builder. (copyFrom(GraphConfig), copyBaseFrom(GraphConfig), removeVertexProperty(String), removeEdgeProperty(String)).

• Added API for retrieving a random edge (PgxGraph#getRandomEdge)).

New Features in PGQL

Release 2.4 introduces several new features in PGQL, including the following.

Prepared Statements

Prepared statements provide a way to safeguard your application from query injection. The use of prepared statements can also speed up query execution as queries do not need to get recompiled every time their bind values change. PGQL uses the question mark symbol (?) to indicate a bind variable. Values for the bind variables are then assigned through the PreparedStatement API

Undirected Edge Queries

PGQL has now support for undirected edge queries, which can be used to query undirected graphs or ignoring edge direction in directed graphs. These two use cases are illustrated in the following two queries:

SELECT d1.name WHERE (d1:Device) -[:connects_to]- (d2:Device), d1.name = 'LoadTransformer 2533'
SELECT m.name WHERE (n:Person) -[:follows]- (m:Person) , n.name = 'Bono'

The first query matches undirected edges labeled connects_to, the second query matches all people that follow or are followed by a person named 'Bono'.

Other Additions and Improvements in PGQL

• PGQL now has an all_different(a, b, c, ...) function , which allows to specify that a set of values (typically vertices or edges) are all different from each other.

• Support for greater than, greater than equal, less than, and less than equal for comparing String values (also works for filter expressions in the Java API).

• Added support for constraints on vertices in PATH patterns, as in the following example. Previously, only constraints on edges in PATH patterns were supported. For example:

  PATH connects_to_high_volt_dev := (:Device) -> (:Device WITH voltage > 35000) SELECT ...

PGX Loader Improvements

The PGX graph loader in Release 2.4 has extended capabilities:

• The Apache Spark loader now supports Spark 2.X through the oracle.pgx.api.spark2.PgxSparkContext class. Loading from Spark 1.x is still possible using the class in oracle.pgx.api.spark1.

• Columns names are now configurable when loading from the Oracle RDBMS in two- tables format.

• The two- tables format now supports string, integer, and long as vertex ID types.

• Added support for directly loading compressed (gzip) graph data without the need to unpack the archives first.

PGX Distributed Engine Improvements

The in-memory analyst (PGX) distributed graph processing execution engine included in Release 2.4 includes several improvements:

• PGX.D now supports top-k and bottom-k for string properties.

• Fixed a bug concering NULL values (Oracle bug 25491165).

• Added support for edge properties of vector type.

• Extended the supported endpoints in the client-server API: added support for rename(), getNeighbours(), getEdges(), getRandomVertex(), getRandomEdge(), getSource(), and getDestination().

PGX Deprecations

The following are now deprecated.

• PgxSparkContext for in the oracle.pgx.api is now deprecated. Use the class in the oracle.pgx.api.spark1 package instead.

• The REST endpoint /core/graph/<graphname>/randomNode is deprecated. Call /core/graph/<graphname/randomEntity instead

• The graph configuration fields for Spark skip_nodes and skip_edges are deprecated. Use graph loading configuration fields loading.skip_vertices and loading.skip_edges instead.

• The graph configuration methods isSkipNodes() and isSkipEdges() are deprecated. Use the skipVertexLoading() and skipEdgeLoading() methods instead.

• The SALSA algorithm algorithms/link_prediction/salsa_deprecated.gm is deprecated. Use algorithms/ranking_and_walking/salsa.gm instead.

• The CALLER_THREAD PoolType is deprecated.

• The REST endpoint /core/analysis/<analysisId> with a targetPool is deprecated. Use the workloadCharacteristics field instead

• The use of the path finding filter argument type is deprecated.

• The property type DATE is deprecated. Use LOCAL_DATE, TIME, TIMESTAMP, TIME_WITH_TIMEZONE or TIMESTAMP_WITH_TIMEZONE instead.

• The REST endpoint GET /core/graph/<graphname>/query is deprecated. Use POST to /core/graph/<graphname>/query with query and semantic options in the JSON payload

• In PGQL, user-defined pattern matching semantic (i.e., ISOMORPHISM / HOMOMORPHSIM) is deprecated. Homomorphism remains the default semantic, but isomorphic constraints should now be specified using either the new built-in PGQL function all_different(v1, v2, ...) or using non-equality constraints (for example, v1 != v2). The deprecations are as follows:

  • The method PgxGraph.queryPgql(String, PatternMatchingSemantic) (use PgxGraph.queryPgql(String) instead)

  • The method PgxSession.setPatternMatchingSemantic(..)

  • The configuration field pattern_matching_semantic

Spark Vector API Changes for Release 2.4

The following capabilities have been added to the Spark Vector API for Release 2.4:

• Spatial transformations for the Spark Streaming API:

  • Filter, flatMap, nearestNeighbors

  • Java and Scala APIs

• Spatial join using two spatial indexes

• GeoEnrich transformation for the Streaming API

Vector REST API Additions

The following APIs are available:

• Vector Hadoop REST API with the following available operations:

  • List/create/delete a spatial index

  • Filter spatially the records using a spatial index

  • Categorization, clustering, binning

• Vector Spark REST API with the following available operations:

  • List/create/delete a spatial index

  • Filter spatially the records using a spatial index

SpatialViewer Changes

The Big Data Spatial Image Server has been integrated into the Oracle Big Data SpatialViewer web application. SpatialViewer uses Oracle JET, which provides a rich set of UI components.