Go to main content
1/15
Contents
List of Figures
List of Tables
Title and Copyright Information
Preface
Audience
Documentation Accessibility
Related Documents
Conventions
Changes in This Release for Oracle Big Data Spatial and Graph User's Guide and Reference
Integration of Apache Spark with the In-Memory Analyst
Improved CSV/Relational Tables to .opv/.ope Converter
Improved Support for Querying Property Graph Data
Vertex Label Support by Treating Value of a Designated Property as Label
Mocking Framework for Testing Raster Processing
Added Support for Apache Solr 5.2.x
Improved Support for Converting Multiple Data Sources (Relational or CSV) into a Single Property Graph
1
Big Data Spatial and Graph Overview
1.1
About Big Data Spatial and Graph
1.2
Spatial Features
1.3
Property Graph Features
1.3.1
Property Graph Sizing Recommendations
1.4
Multimedia Analytics Features
1.5
Installing Oracle Big Data Spatial and Graph on an Oracle Big Data Appliance
1.6
Installing and Configuring the Big Data Spatial Image Processing Framework
1.6.1
Getting and Compiling the Cartographic Projections Library
1.6.2
Installing the Image Processing Framework for Oracle Big Data Appliance Distribution
1.6.3
Installing the Image Processing Framework for Other Distributions (Not Oracle Big Data Appliance)
1.6.3.1
Prerequisites for Installing the Image Processing Framework for Other Distributions
1.6.3.2
Installing the Image Processing Framework for Other Distributions
1.6.4
Post-installation Verification of the Image Processing Framework
1.6.4.1
Image Loading Test Script
1.6.4.2
Image Processor Test Script (Mosaicking)
1.6.4.3
Single-Image Processor Test Script
1.6.4.4
Image Processor DEM Test Script
1.7
Installing and Configuring the Big Data Spatial Image Server
1.7.1
Installing and Configuring the Image Server for Oracle Big Data Appliance
1.7.1.1
Prerequisites for Performing a Manual Installation
1.7.1.2
Installing Dependencies on the Image Server Web on an Oracle Big Data Appliance
1.7.1.3
Configuring the Environment for Big Data Appliance
1.7.2
Installing and Configuring the Image Server Web for Other Systems (Not Big Data Appliance)
1.7.2.1
Prerequisites for Installing the Image Server on Other Systems
1.7.2.2
Installing the Image Server Web on Other Systems
1.7.2.3
Configuring the Environment for Other Systems
1.7.3
Post-Installation Verification Example for the Image Server Console
1.7.3.1
Loading Images from the Local Server to the HDFS Hadoop Cluster
1.7.3.2
Creating a Mosaic Image and Catalog
1.7.3.3
Creating a Mosaic Directly from the Globe
1.7.3.4
Removing Identical Rasters
1.7.4
Using the Provided Image Server Web Services
1.8
Installing the Oracle Big Data Spatial Hadoop Vector Console
1.8.1
Assumptions and Prerequisite Libraries
1.8.1.1
Assumptions
1.8.1.2
Prerequisite Libraries
1.8.2
Installing the Spatial Hadoop Vector Console on Oracle Big Data Appliance
1.8.3
Installing the Spatial Hadoop Vector Console for Other Systems (Not Big Data Appliance)
1.8.4
Configuring the Spatial Hadoop Vector Console on Oracle Big Data Appliance
1.8.5
Configuring the Spatial Hadoop Vector Console for Other Systems (Not Big Data Appliance)
1.9
Installing Property Graph Support on a CDH Cluster or Other Hardware
1.9.1
Apache HBase Prerequisites
1.9.2
Property Graph Installation Steps
1.9.3
About the Property Graph Installation Directory
1.9.4
Optional Installation Task for In-Memory Analyst Use
1.9.4.1
Installing and Configuring Hadoop
1.9.4.2
Running the In-Memory Analyst on Hadoop
1.10
Installing and Configuring Multimedia Analytics Support
1.10.1
Assumptions and Libraries for Multimedia Analytics
1.10.2
Transcoding Software (Options)
2
Using Big Data Spatial and Graph with Spatial Data
2.1
About Big Data Spatial and Graph Support for Spatial Data
2.1.1
What is Big Data Spatial and Graph on Apache Hadoop?
2.1.2
Advantages of Oracle Big Data Spatial and Graph
2.1.3
Oracle Big Data Spatial Features and Functions
2.1.4
Oracle Big Data Spatial Files, Formats, and Software Requirements
2.2
Oracle Big Data Vector and Raster Data Processing
2.2.1
Oracle Big Data Spatial Raster Data Processing
2.2.2
Oracle Big Data Spatial Vector Data Processing
2.3
Oracle Big Data Spatial Hadoop Image Processing Framework for Raster Data Processing
2.3.1
Image Loader
2.3.2
Image Processor
2.3.3
Image Server
2.4
Loading an Image to Hadoop Using the Image Loader
2.4.1
Image Loading Job
2.4.2
Input Parameters
2.4.3
Output Parameters
2.5
Processing an Image Using the Oracle Spatial Hadoop Image Processor
2.5.1
Image Processing Job
2.5.2
Input Parameters
2.5.2.1
Catalog XML Structure
2.5.2.2
Mosaic Definition XML Structure
2.5.3
Job Execution
2.5.4
Processing Classes and ImageBandWritable
2.5.4.1
Location of the Classes and Jar Files
2.5.5
Map Algebra Operations
2.5.6
Output
2.6
Loading and Processing an Image Using the Oracle Spatial Hadoop Raster Processing API
2.7
Using the Oracle Spatial Hadoop Raster Mocking Framework to Test Raster Processing
2.8
Oracle Big Data Spatial Vector Analysis
2.8.1
Multiple Hadoop API Support
2.8.2
Spatial Indexing
2.8.2.1
Spatial Indexing Class Structure
2.8.2.2
Configuration for Creating a Spatial Index
2.8.2.3
Spatial Index Metadata
2.8.2.4
Input Formats for a Spatial Index
2.8.2.5
Support for GeoJSON and Shapefile Formats
2.8.2.6
Removing a Spatial Index
2.8.3
Using MVSuggest
2.8.4
Spatial Filtering
2.8.4.1
Filtering Records
2.8.4.2
Filtering Using the Input Format
2.8.5
Classifying Data Hierarchically
2.8.5.1
Changing the Hierarchy Level Range
2.8.5.2
Controlling the Search Hierarchy
2.8.5.3
Using MVSuggest to Classify the Data
2.8.6
Generating Buffers
2.8.7
Spatial Binning
2.8.8
Spatial Clustering
2.8.9
Spatial Join
2.8.10
Spatial Partitioning
2.8.11
RecordInfoProvider
2.8.11.1
Sample RecordInfoProvider Implementation
2.8.11.2
LocalizableRecordInfoProvider
2.8.12
HierarchyInfo
2.8.12.1
Sample HierarchyInfo Implementation
2.8.13
Using JGeometry in MapReduce Jobs
2.8.14
Support for Different Data Sources
2.8.15
Job Registry
2.8.16
Tuning Performance Data of Job Running Times Using the Vector Analysis API
2.9
Oracle Big Data Spatial Vector Hive Analysis
2.9.1
HiveRecordInfoProvider
2.9.2
Using the Hive Spatial API
2.9.3
Using Spatial Indexes in Hive
2.10
Using the Oracle Big Data Spatial and Graph Vector Console
2.10.1
Creating a Spatial Index Using the Console
2.10.2
Exploring the Indexed Spatial Data
2.10.3
Running a Categorization Job Using the Console
2.10.4
Viewing the Categorization Results
2.10.5
Saving Categorization Results to a File
2.10.6
Creating and Deleting Templates
2.10.7
Configuring Templates
2.10.8
Running a Clustering Job Using the Console
2.10.9
Viewing the Clustering Results
2.10.10
Saving Clustering Results to a File
2.10.11
Running a Binning Job Using the Console
2.10.12
Viewing the Binning Results
2.10.13
Saving Binning Results to a File
2.10.14
Running a Job to Create an Index Using the Command Line
2.10.15
Running a Job to Create a Categorization Result
2.10.16
Running a Job to Create a Clustering Result
2.10.17
Running a Job to Create a Binning Result
2.10.18
Running a Job to Perform Spatial Filtering
2.10.19
Running a Job to Get Location Suggestions
2.10.20
Running a Job to Perform a Spatial Join
2.10.21
Running a Job to Perform Partitioning
2.10.22
Using Multiple Inputs
2.11
Using Oracle Big Data Spatial and Graph Image Server Console
2.11.1
Loading Images to an HDFS Hadoop Cluster to Create a Mosaic
3
Integrating Big Data Spatial and Graph with Oracle Database
3.1
Using Oracle SQL Connector for HDFS with Delimited Text Files
3.2
Using Oracle SQL Connector for HDFS with Hive Tables
3.3
Using Oracle SQL Connector for HDFS with Files Generated by Oracle Loader for Hadoop
3.3.1
Creating HDFS Data Pump Files or Delimited Text Files
3.3.2
Creating the SQL Connector for HDFS
3.4
Integrating HDFS, Oracle NoSQL, and Apache HBase Spatial Data with Oracle Database Using Oracle Big Data SQL
3.4.1
Creating Oracle External Tables for HDFS Files with Big Data SQL
3.4.2
Creating Oracle External Tables Using Hive Tables with Big Data SQL
4
Configuring Property Graph Support
4.1
Tuning the Software Configuration
4.1.1
Tuning Apache HBase for Use with Property Graphs
4.1.1.1
Modifying the Apache HBase Configuration
4.1.1.2
Modifying the Java Memory Settings
4.1.2
Tuning Oracle NoSQL Database for Use with Property Graphs
5
Using Property Graphs in a Big Data Environment
5.1
About Property Graphs
5.1.1
What Are Property Graphs?
5.1.2
What Is Big Data Support for Property Graphs?
5.1.2.1
In-Memory Analyst
5.1.2.2
Data Access Layer
5.1.2.3
Storage Management
5.1.2.4
RESTful Web Services
5.2
About Property Graph Data Formats
5.2.1
GraphML Data Format
5.2.2
GraphSON Data Format
5.2.3
GML Data Format
5.2.4
Oracle Flat File Format
5.3
Getting Started with Property Graphs
5.4
Using Java APIs for Property Graph Data
5.4.1
Overview of the Java APIs
5.4.1.1
Oracle Big Data Spatial and Graph Java APIs
5.4.1.2
TinkerPop Blueprints Java APIs
5.4.1.3
Apache Hadoop Java APIs
5.4.1.4
Oracle NoSQL Database Java APIs
5.4.1.5
Apache HBase Java APIs
5.4.2
Parallel Loading of Graph Data
5.4.2.1
Parallel Data Loading Using Partitions
5.4.2.2
Parallel Data Loading Using Fine-Tuning
5.4.2.3
Parallel Data Loading Using Multiple Files
5.4.2.4
Parallel Retrieval of Graph Data
5.4.2.5
Using an Element Filter Callback for Subgraph Extraction
5.4.2.6
Using Optimization Flags on Reads over Property Graph Data
5.4.2.7
Adding and Removing Attributes of a Property Graph Subgraph
5.4.2.8
Getting Property Graph Metadata
5.4.3
Opening and Closing a Property Graph Instance
5.4.3.1
Using Oracle NoSQL Database
5.4.3.2
Using Apache HBase
5.4.4
Creating Vertices
5.4.5
Creating Edges
5.4.6
Deleting Vertices and Edges
5.4.7
Reading a Graph from a Database into an Embedded In-Memory Analyst
5.4.8
Specifying Labels for Vertices
5.4.9
Building an In-Memory Graph
5.4.10
Dropping a Property Graph
5.4.10.1
Using Oracle NoSQL Database
5.4.10.2
Using Apache HBase
5.5
Managing Text Indexing for Property Graph Data
5.5.1
Configuring a Text Index for Property Graph Data
5.5.2
Using Automatic Indexes for Property Graph Data
5.5.3
Using Manual Indexes for Property Graph Data
5.5.4
Executing Search Queries Over Property Graph Text Indexes
5.5.5
Handling Data Types
5.5.5.1
Appending Data Type Identifiers on Apache Lucene
5.5.5.2
Appending Data Type Identifiers on SolrCloud
5.5.6
Uploading a Collection's SolrCloud Configuration to Zookeeper
5.5.7
Updating Configuration Settings on Text Indexes for Property Graph Data
5.5.8
Using Parallel Query on Text Indexes for Property Graph Data
5.5.9
Using Native Query Objects on Text Indexes for Property Graph Data
5.5.10
Using Native Query Results on Text Indexes for Property Graph Data
5.6
Querying Property Graph Data
5.7
Support for Secure Oracle NoSQL Database
5.8
Implementing Security on Graphs Stored in Apache HBase
5.9
Using the Groovy Shell with Property Graph Data
5.10
Exploring the Sample Programs
5.10.1
About the Sample Programs
5.10.2
Compiling and Running the Sample Programs
5.10.3
About the Example Output
5.10.4
Example: Creating a Property Graph
5.10.5
Example: Dropping a Property Graph
5.10.6
Examples: Adding and Dropping Vertices and Edges
5.11
Oracle Flat File Format Definition
5.11.1
About the Property Graph Description Files
5.11.2
Vertex File
5.11.3
Edge File
5.11.4
Encoding Special Characters
5.11.5
Example Property Graph in Oracle Flat File Format
5.11.6
Converting an Oracle Database Table to an Oracle-Defined Property Graph Flat File
5.11.7
Converting CSV Files for Vertices and Edges to Oracle-Defined Property Graph Flat Files
5.12
Example Python User Interface
6
Using the In-Memory Analyst
6.1
Reading a Graph into Memory
6.1.1
Connecting to an In-Memory Analyst Server Instance
6.1.2
Using the Shell Help
6.1.3
Providing Graph Metadata in a Configuration File
6.1.4
Reading Graph Data into Memory
6.1.4.1
Read a Graph Stored in Apache HBase into Memory
6.1.4.2
Read a Graph Stored in Oracle NoSQL Database into Memory
6.1.4.3
Read a Graph Stored in the Local File System into Memory
6.2
Reading Custom Graph Data
6.2.1
Creating a Simple Graph File
6.2.2
Adding a Vertex Property
6.2.3
Using Strings as Vertex Identifiers
6.2.4
Adding an Edge Property
6.3
Storing Graph Data on Disk
6.3.1
Storing the Results of Analysis in a Vertex Property
6.3.2
Storing a Graph in Edge-List Format on Disk
6.4
Executing Built-in Algorithms
6.4.1
About the In-Memory Analyst
6.4.2
Running the Triangle Counting Algorithm
6.4.3
Running the Pagerank Algorithm
6.5
Creating Subgraphs
6.5.1
About Filter Expressions
6.5.2
Using a Simple Filter to Create a Subgraph
6.5.3
Using a Complex Filter to Create a Subgraph
6.5.4
Using a Vertex Set to Create a Bipartite Subgraph
6.6
Starting the In-Memory Analyst Server
6.6.1
Configuring the In-Memory Analyst Server
6.7
Deploying to Jetty
6.7.1
About the Authentication Mechanism
6.8
Deploying to Apache Tomcat
6.9
Deploying to Oracle WebLogic Server
6.9.1
Installing Oracle WebLogic Server
6.9.2
Deploying the In-Memory Analyst
6.9.3
Verifying That the Server Works
6.10
Connecting to the In-Memory Analyst Server
6.10.1
Connecting with the In-Memory Analyst Shell
6.10.1.1
About Logging HTTP Requests
6.10.2
Connecting with Java
6.10.3
Connecting with an HTTP Request
6.11
Using the In-Memory Analyst in Distributed Mode
6.12
Reading and Storing Data in HDFS
6.12.1
Reading Data from HDFS
6.12.2
Storing Graph Snapshots in HDFS
6.12.3
Compiling and Running a Java Application in Hadoop
6.13
Running the In-Memory Analyst as a YARN Application
6.13.1
Starting and Stopping In-Memory Analyst Services
6.13.1.1
Configuring the In-Memory Analyst YARN Client
6.13.1.2
Starting a New In-Memory Analyst Service
6.13.1.3
About Long-Running In-Memory Analyst Services
6.13.1.4
Stopping In-Memory Analyst Services
6.13.2
Connecting to In-Memory Analyst Services
6.13.3
Monitoring In-Memory Analyst Services
6.14
Using the In-Memory Analyst to Analyze Graph Data in Apache Spark
7
Using Multimedia Analytics
7.1
About Multimedia Analytics
7.2
Face Recognition Using the Multimedia Analytics Framework
7.2.1
Training to Detect Faces
7.2.2
Selecting Faces to be Used for Training
7.2.3
Detecting Faces in Videos
7.2.4
Detecting Faces in Images
7.2.5
Working with Oracle NoSQL Database
7.2.6
Working with Apache HBase
7.2.7
Examples and Training Materials for Detecting Faces
7.3
Configuration Properties for Multimedia Analytics
7.4
Using the Multimedia Analytics Framework with Third-Party Software
7.5
Displaying Images in Output
A
Third-Party Licenses for Bundled Software
A.1
Apache Licensed Code
A.2
ANTLR 3
A.3
AOP Alliance
A.4
Apache Commons CLI
A.5
Apache Commons Codec
A.6
Apache Commons Collections
A.7
Apache Commons Configuration
A.8
Apache Commons IO
A.9
Apache Commons Lang
A.10
Apache Commons Logging
A.11
Apache Commons VFS
A.12
Apache fluent
A.13
Apache Groovy
A.14
Apache htrace
A.15
Apache HTTP Client
A.16
Apache HTTPComponents Core
A.17
Apache Jena
A.18
Apache Log4j
A.19
Apache Lucene
A.20
Apache Tomcat
A.21
Apache Xerces2
A.22
Apache xml-commons
A.23
Argparse4j
A.24
check-types
A.25
Cloudera CDH
A.26
cookie
A.27
Fastutil
A.28
functionaljava
A.29
GeoNames Data
A.30
Geospatial Data Abstraction Library (GDAL)
A.31
Google Guava
A.32
Google Guice
A.33
Google protobuf
A.34
int64-native
A.35
Jackson
A.36
Jansi
A.37
JCodec
A.38
Jettison
A.39
JLine
A.40
Javassist
A.41
json-bignum
A.42
Jung
A.43
Log4js
A.44
MessagePack
A.45
Netty
A.46
Node.js
A.47
node-zookeeper-client
A.48
OpenCV
A.49
rxjava-core
A.50
Slf4j
A.51
Spoofax
A.52
Tinkerpop Blueprints
A.53
Tinkerpop Gremlin
A.54
Tinkerpop Pipes
B
Hive Spatial Functions
B.1
ST_AnyInteract
B.2
ST_Area
B.3
ST_AsWKB
B.4
ST_AsWKT
B.5
ST_Buffer
B.6
ST_Contains
B.7
ST_ConvexHull
B.8
ST_Distance
B.9
ST_Envelope
B.10
ST_Geometry
B.11
ST_Inside
B.12
ST_Length
B.13
ST_LineString
B.14
ST_MultiLineString
B.15
ST_MultiPoint
B.16
ST_MultiPolygon
B.17
ST_Point
B.18
ST_Polygon
B.19
ST_Simplify
B.20
ST_SimplifyVW
B.21
ST_Volume
Scripting on this page enhances content navigation, but does not change the content in any way.