Contents

List of Examples

List of Figures

List of Tables

Title and Copyright Information

Preface

• Audience
• Documentation Accessibility
• Related Documents
• Conventions

Part I Oracle Big Data Spatial and Graph Overview

1 Big Data Spatial and Graph Overview

• 1.1 About Big Data Spatial and Graph
• 1.2 Spatial Features
• 1.3 Property Graph Features
  • 1.3.1 Property Graph Sizing Recommendations
• 1.4 Installing Oracle Big Data Spatial and Graph on an Oracle Big Data Appliance
• 1.5 Installing and Configuring the Big Data Spatial Image Processing Framework
  • 1.5.1 Installing Image Processing Framework for Oracle Big Data Appliance Distribution
  • 1.5.2 Installing the Image Processing Framework for Other Distributions (Not Oracle Big Data Appliance)
    • 1.5.2.1 Prerequisites for Installing the Image Processing Framework for Other Distributions
    • 1.5.2.2 Installing the Image Processing Framework for Other Distributions
    • 1.5.2.3 Configuring the environment
  • 1.5.3 Post-installation Verification of the Image Processing Framework
    • 1.5.3.1 Image Loading Test Script
    • 1.5.3.2 Image Processor Test Script
• 1.6 Installing and Configuring the Big Data Spatial Image Server
  • 1.6.1 Installing and Configuring the Image Server for Oracle Big Data Appliance
    • 1.6.1.1 Prerequisites for installing Image Server on Oracle Big Data Appliance
    • 1.6.1.2 Installing Image Server Web on an Oracle Big Data Appliance
    • 1.6.1.3 Configuring the Environment
  • 1.6.2 Installing and Configuring the Image Server Web for Other Systems (Not Big Data Appliance)
    • 1.6.2.1 Prerequisites for Installing the Image Server on Other Systems
    • 1.6.2.2 Installing the Image Server Web on Other Systems
    • 1.6.2.3 Configuring the Environment
  • 1.6.3 Post-installation Verification Example for the Image Server Console
    • 1.6.3.1 Loading images from the local server to HDFS Hadoop cluster
    • 1.6.3.2 Creating a mosaic image and catalog
• 1.7 Installing Oracle Big Data Spatial Hadoop Vector Console
  • 1.7.1 Assumptions and Prerequisite Libraries
    • 1.7.1.1 Assumptions
    • 1.7.1.2 Prerequisite Libraries
  • 1.7.2 Installing Spatial Hadoop Vector Console on Oracle Big Data Appliance
  • 1.7.3 Installing Spatial Hadoop Vector Console for Other Systems (Not Big Data Appliance)
  • 1.7.4 Configuring Spatial Hadoop Vector Console on Oracle Big Data Appliance
  • 1.7.5 Configuring Spatial Hadoop Vector Console for Other Systems (Not Big Data Appliance)
• 1.8 Installing Property Graph Support on a CDH Cluster or Other Hardware
  • 1.8.1 Apache HBase Prerequisites
  • 1.8.2 Property Graph Installation Steps
  • 1.8.3 About the Property Graph Installation Directory
  • 1.8.4 Optional Installation Task for In-Memory Analytics
    • 1.8.4.1 Installing and Configuring Hadoop
    • 1.8.4.2 Running In-Memory Analytics on Hadoop

Part II Big Data Spatial and Graph on Apache Hadoop

2 Using Big Data Spatial and Graph with Spatial Data

• 2.1 About Big Data Spatial and Graph Support for Spatial Data
  • 2.1.1 What is Big Data Spatial and Graph on Apache Hadoop?
  • 2.1.2 Advantages of Oracle Big Data Spatial and Graph
  • 2.1.3 Oracle Big Data Spatial Features and Functions
  • 2.1.4 Oracle Big Data Spatial Files, Formats, and Software Requirements
• 2.2 Oracle Big Data Vector and Raster Data Processing
  • 2.2.1 Oracle Big Data Spatial Raster Data Processing
  • 2.2.2 Oracle Big Data Spatial Vector Data Processing
• 2.3 Oracle Big Data Spatial Hadoop Image Processing Framework for Raster Data Processing
  • 2.3.1 Image Loader
  • 2.3.2 Image Processor
  • 2.3.3 Image Server
• 2.4 Loading an Image to Hadoop Using the Image Loader
  • 2.4.1 Image Loading Job
  • 2.4.2 Input Parameters
  • 2.4.3 Output Parameters
• 2.5 Processing an Image Using the Oracle Spatial Hadoop Image Processor
  • 2.5.1 Image Processing Job
  • 2.5.2 Input Parameters
    • 2.5.2.1 Catalog XML Structure
    • 2.5.2.2 Mosaic definition XML Structure
  • 2.5.3 Job Execution
  • 2.5.4 Processing Classes and ImageBandWritable
    • 2.5.4.1 Location of the Classes and Jar Files
  • 2.5.5 Output
• 2.6 Oracle Big Data Spatial Vector Analysis
  • 2.6.1 Spatial Indexing
    • 2.6.1.1 Spatial Indexing Class Structure
    • 2.6.1.2 Configuration for Creating a Spatial Index
    • 2.6.1.3 Input Formats for Spatial Index
    • 2.6.1.4 MVSuggest for Locating Records
  • 2.6.2 Spatial Filtering
    • 2.6.2.1 Filtering Records
  • 2.6.3 Classifying Data Hierarchically
    • 2.6.3.1 Changing the Hierarchy Level Range
    • 2.6.3.2 Controlling the Search Hierarchy
    • 2.6.3.3 Using MVSuggest to Classify the Data
  • 2.6.4 Generating Buffers
  • 2.6.5 RecordInfoProvider
    • 2.6.5.1 Sample RecordInfo Implementation
    • 2.6.5.2 LocalizableRecordInfoProvider
  • 2.6.6 HierarchyInfo
    • 2.6.6.1 Sample HierarchyInfo Implementation
  • 2.6.7 Using JGeometry in MapReduce Jobs
  • 2.6.8 Tuning Performance Data of Job Running Times using Vector Analysis API
• 2.7 Using Oracle Big Data Spatial and Graph Vector Console
  • 2.7.1 Creating a Spatial Index Using the Console
  • 2.7.2 Running a Hierarchy Job Using the Console
  • 2.7.3 Viewing the Index Results
  • 2.7.4 Creating Results Manually
  • 2.7.5 Creating and Deleting templates
  • 2.7.6 Configuring Templates
  • 2.7.7 Running a Job to Create an Index Using the Command Line
  • 2.7.8 Running a Job to Perform a Spatial Filtering
  • 2.7.9 Running a Job to Create a Hierarchy Result
  • 2.7.10 Running a Job to Generate Buffer
• 2.8 Using Oracle Big Data Spatial and Graph Image Server Console
  • 2.8.1 Loading Images to HDFS Hadoop Cluster to Create a Mosaic

Part III Property Graphs

3 Configuring Property Graph Support

• 3.1 Tuning the Software Configuration
  • 3.1.1 Tuning Apache HBase for Use With Property Graphs
    • 3.1.1.1 Modifying the Apache HBase Configuration
    • 3.1.1.2 Modifying the Java Memory Settings
  • 3.1.2 Tuning Oracle NoSQL Database for Use with Property Graphs

4 Using Property Graphs in a Big Data Environment

• 4.1 About Property Graphs
  • 4.1.1 What Are Property Graphs?
  • 4.1.2 What Is Big Data Support for Property Graphs?
    • 4.1.2.1 Analytics Layer
    • 4.1.2.2 Data Access Layer
    • 4.1.2.3 Storage Management
    • 4.1.2.4 RESTful Web Services
• 4.2 Getting Started With Property Graphs
• 4.3 About Property Graph Data Formats
  • 4.3.1 GraphML Data Format
  • 4.3.2 GraphSON Data Format
  • 4.3.3 GML Data Format
  • 4.3.4 Oracle Flat File Format
• 4.4 Using Java APIs for Property Graph Data
  • 4.4.1 Overview of the Java APIs
    • 4.4.1.1 Oracle Big Data Spatial and Graph Java APIs
    • 4.4.1.2 TinkerPop Blueprints Java APIs
    • 4.4.1.3 Apache Hadoop Java APIs
    • 4.4.1.4 Oracle NoSQL Database Java APIs
    • 4.4.1.5 Apache HBase Java APIs
  • 4.4.2 Opening and Closing a Property Graph Instance
    • 4.4.2.1 Using Oracle NoSQL Database
    • 4.4.2.2 Using Apache HBase
  • 4.4.3 Creating the Vertices
  • 4.4.4 Creating the Edges
  • 4.4.5 Deleting the Vertices and Edges
  • 4.4.6 Dropping a Property Graph
    • 4.4.6.1 Using Oracle NoSQL Database
    • 4.4.6.2 Using Apache HBase
• 4.5 Managing Text Indexing for Property Graph Data
  • 4.5.1 Using Automatic Indexes with the Apache Lucene Search Engine
  • 4.5.2 Using Manual Indexes with the SolrCloud Search Engine
  • 4.5.3 Handling Data Types
    • 4.5.3.1 Appending Data Type Identifiers on Apache Lucene
    • 4.5.3.2 Appending Data Type Identifiers on SolrCloud
  • 4.5.4 Uploading a Collection's SolrCloud Configuration to Zookeeper
  • 4.5.5 Updating Configuration Settings on Text Indexes for Property Graph Data
• 4.6 Property Graph Support for Secure Oracle NoSQL Database
• 4.7 Using the Groovy Shell with Property Graph Data
• 4.8 Exploring the Sample Programs
  • 4.8.1 About the Sample Programs
  • 4.8.2 Compiling and Running the Sample Programs
  • 4.8.3 About the Example Output
  • 4.8.4 Example: Creating a Property Graph
  • 4.8.5 Example: Dropping a Property Graph
  • 4.8.6 Examples: Adding and Dropping Vertices and Edges
• 4.9 Oracle Flat File Format Definition
  • 4.9.1 About the Property Graph Description Files
  • 4.9.2 Vertex File
  • 4.9.3 Edge File
  • 4.9.4 Encoding Special Characters
  • 4.9.5 Example Property Graph in Oracle Flat File Format
• 4.10 Example Python User Interface

5 Using In-Memory Analytics

• 5.1 Reading a Graph into Memory
  • 5.1.1 Starting the In-Memory Analytics Shell
  • 5.1.2 Using the Shell Help
  • 5.1.3 Providing Graph Metadata in a Configuration File
  • 5.1.4 Reading Graph Data into Memory
    • 5.1.4.1 Read a Graph Stored in Apache HBase into Memory
    • 5.1.4.2 Read a Graph Stored in Oracle NoSQL Database into Memory
    • 5.1.4.3 Read a Graph Stored in the Local File System into Memory
• 5.2 Reading Custom Graph Data
  • 5.2.1 Creating a Simple Graph File
  • 5.2.2 Adding a Vertex Property
  • 5.2.3 Using Strings as Node Identifiers
  • 5.2.4 Adding an Edge Property
• 5.3 Storing Graph Data on Disk
  • 5.3.1 Storing the Results of Analysis in a Node Property
  • 5.3.2 Storing a Graph in Edge-List Format on Disk
• 5.4 Executing Built-in Algorithms
  • 5.4.1 About In-Memory Analytics
  • 5.4.2 About the Analyst Interface
  • 5.4.3 Reading the Graph
  • 5.4.4 Running the Triangle Counting Algorithm
  • 5.4.5 Running the Pagerank Algorithm
• 5.5 Creating Subgraphs
  • 5.5.1 About Filter Expressions
  • 5.5.2 Using a Simple Filter to Create a Subgraph
  • 5.5.3 Using a Complex Filter to Create a Subgraph
  • 5.5.4 Using a Node List to Create a Bipartite Subgraph
• 5.6 Deploying to Jetty
  • 5.6.1 About the Authentication Mechanism
• 5.7 Deploying to Apache Tomcat
• 5.8 Deploying to Oracle WebLogic Server
  • 5.8.1 Installing Oracle WebLogic Server
  • 5.8.2 Deploying In-Memory Analytics
  • 5.8.3 Verifying That the Server Works
• 5.9 Connecting to the In-Memory Analytics Server
  • 5.9.1 Connecting with the In-Memory Analytics Shell
    • 5.9.1.1 About Logging HTTP Requests
  • 5.9.2 Connecting with Java
  • 5.9.3 Connecting with an HTTP Request
• 5.10 Reading and Storing Data in HDFS
  • 5.10.1 Loading Data from HDFS
  • 5.10.2 Storing Graph Snapshots in HDFS
  • 5.10.3 Compiling and Running a Java Application in Hadoop
• 5.11 Running In-Memory Analytics as a YARN Application
  • 5.11.1 Starting and Stopping In-Memory Analytics Services
    • 5.11.1.1 Configuring the In-Memory Analytics YARN Client
    • 5.11.1.2 Starting a New In-Memory Analytics Service
    • 5.11.1.3 About Long-Running In-Memory Analytics Services
    • 5.11.1.4 Stopping In-Memory Analytics Services
  • 5.11.2 Connecting to In-Memory Analytics Services
  • 5.11.3 Monitoring In-Memory Analytics Services
• 5.12 Using the Java API Inside the In-Memory Analytics Shell

A Third-Party Licenses for Bundled Software

• A.1 Apache Licensed Code
• A.2 ANTLR 3
• A.3 AOP Alliance
• A.4 Apache Commons CLI
• A.5 Apache Commons Codec
• A.6 Apache Commons Collections
• A.7 Apache Commons Configuration
• A.8 Apache Commons IO
• A.9 Apache Commons Lang
• A.10 Apache Commons Logging
• A.11 Apache fluent
• A.12 Apache Groovy
• A.13 Apache htrace
• A.14 Apache HTTP Client
• A.15 Apache HTTPComponents Core
• A.16 Apache Jena
• A.17 Apache Log4j
• A.18 Apache Lucene
• A.19 Apache Xerces2
• A.20 Apache xml-commons
• A.21 Cloudera CDH
• A.22 Fastutil
• A.23 GeoNames Data
• A.24 Geospatial Data Abstraction Library (GDAL)
• A.25 Google Guava
• A.26 Google Guice
• A.27 Google protobuf
• A.28 Jackson
• A.29 Jansi
• A.30 Jettison
• A.31 JLine
• A.32 Javassist
• A.33 Jung
• A.34 MessagePack
• A.35 Netty
• A.36 Slf4j
• A.37 Tinkerpop Blueprints
• A.38 Tinkerpop Gremlin
• A.39 Tinkerpop Pipes