Table of Contents
- Title and Copyright Information
- 1 Overview
-
2
Install
- 2.1 Planning Your Installation
- 2.2 Installing GoldenGate Stream Analytics
- 2.3 Configuring the Metadata Store
- 2.4 Initializing Metadata Store
- 2.5 Jetty Properties File
- 2.6 Adjusting Jetty Threadpool
- 2.7 Integrating Stream Analytics with Oracle GoldenGate
- 2.8 Maven Setting for GoldenGate Big Data Handlers
- 2.9 GoldenGate Stream Analytics Hardware Requirements for Enterprise Deployment
- 2.10 Retaining https and Disabling http
- 2.11 Setting up Runtime for GoldenGate Stream Analytics Server
- 2.12 Validating Data Flow to GoldenGate Stream Analytics
- 2.13 Terminating GoldenGate Stream Analytics
- 2.14 Upgrading GoldenGate Stream Analytics
-
3
Configure
- 3.1 Configure Runtime Environment
- 3.2 Configure Users
-
4
Manage
-
4.1
Connections
-
4.1.1
Create Connections
- 4.1.1.1 Creating a Connection to ADW or ATP
- 4.1.1.2 Creating a Connection to AWS S3
- 4.1.1.3 Creating a Connection to Coherence
- 4.1.1.4 Creating a Connection to Druid
- 4.1.1.5 Creating a Connection to Elasticsearch
- 4.1.1.6 Creating a Connection to GoldenGate
- 4.1.1.7 Creating a Connection to HBase
- 4.1.1.8 Creating a Connection to HDFS
- 4.1.1.9 Creating a Connection to Hive
- 4.1.1.10 Creating a connection to Ignite Cache
- 4.1.1.11 Creating a Connection to JMS
- 4.1.1.12 Creating a Connection to Kafka
- 4.1.1.13 Creating a Connection to Microsoft Azure Data Lake-Gen2
- 4.1.1.14 Creating a Connection to MongoDB
- 4.1.1.15 Creating a Connection to MySQL Database
- 4.1.1.16 Creating a Connection to OCI Object Store
- 4.1.1.17 Creating a Connection to ONS
- 4.1.1.18 Creating a Connection to Oracle AQ
- 4.1.1.19 Creating a Connection to Oracle Database
- 4.1.1.20 Creating a Connection to OSS
- 4.1.2 Manage Connections
-
4.1.1
Create Connections
- 4.2 Streams
-
4.3
References
- 4.3.1 Create References
-
4.3.2
Manage References
-
4.3.2.1
Coherence Reference
- 4.3.2.1.1 Configuring Extend Proxy on the Coherence Server
- 4.3.2.1.2 Limitations of Coherence as Reference
- 4.3.2.1.3 Loading Number Type Data on Coherence Cache
- 4.3.2.1.4 Data Mapping in Coherence Reference Map Type
- 4.3.2.1.5 Data Mapping in Coherence Reference POJO Type
- 4.3.2.1.6 Datatypes Supported in Correlation Conditions
- 4.3.2.1.7 Sample POJO Cache Loading in Coherence
- 4.3.2.1.8 Sample POJO Class
-
4.3.2.1
Coherence Reference
-
4.4
Targets
-
4.4.1
Create Targets
- 4.4.1.1 Creating an AWS S3 Target
- 4.4.1.2 Creating an Azure DataLake Gen-2 Target
- 4.4.1.3 Creating a Coherence Target
- 4.4.1.4 Creating a Database Target
- 4.4.1.5 Creating an Elasticsearch Target
- 4.4.1.6 Creating an HBase Target
- 4.4.1.7 Creating HDFS Target
- 4.4.1.8 Creating a Hive Target
- 4.4.1.9 Creating an Ignite Cache Target
- 4.4.1.10 Creating a JMS Target
- 4.4.1.11 Creating a Kafka Target
- 4.4.1.12 Creating a MongoDB Target
- 4.4.1.13 Creating a Network File System (NFS) Target
- 4.4.1.14 Creating a Notification Target
- 4.4.1.15 Creating an OCI Object Store Target
- 4.4.1.16 Creating an OSS Target
- 4.4.1.17 Creating a REST Target
- 4.4.2 Manage Targets
-
4.4.1
Create Targets
- 4.5 Pipelines
- 4.6 GoldenGate Change Stream
- 4.7 Embedded Ignite Cache
- 4.8 Ignite Cluster on OCI GGSA
- 4.9 GGBD Cluster on OCI GGSA
-
4.1
Connections
-
5
Transform
-
5.1
Adding Stages to a Pipeline
- 5.1.1 Adding a Query Stage
- 5.1.2 Adding a Filter to a Query Stage
- 5.1.3 Adding a Summary to a Query Stage
- 5.1.4 Adding a Summary with Group By
- 5.1.5 Adding a Query Group Stage
- 5.1.6 Adding a Rule Stage
- 5.1.7 Adding a Pattern Stage
- 5.1.8 Adding a Scoring Stage
- 5.1.9 Adding a Target Stage
- 5.1.10 Adding a Custom CQL Stage
- 5.2 Correlating Streams and References
-
5.3
Applying Window Functions to a Stream
- 5.3.1 Applying a Time Window with Slide
- 5.3.2 Applying a Time Window without Slide
- 5.3.3 Applying a Row Window with Slide
- 5.3.4 Applying a Row Window without Slide
- 5.3.5 Applying a window with current year, month, day, or hour
- 5.3.6 Applying your own Window using Field from Payload
- 5.3.7 Applying a Row window with Partition without Range
- 5.3.8 Applying a Row Window with Partition with Range without Slide
- 5.3.9 Applying a Row Window with Partition with Slide and Range
-
5.4
Applying Functions to Create a New Column
- 5.4.1 Using Bessel Functions
- 5.4.2 Using Conversion Functions
- 5.4.3 Using Date Functions
- 5.4.4 Using Geometry Functions
- 5.4.5 Using Interval Functions
-
5.4.6
Using Math Functions
- 5.4.6.1 IEEEremainder(value1, value1)
- 5.4.6.2 abs(value1)
- 5.4.6.3 acos(value1)
- 5.4.6.4 asin(value1)
- 5.4.6.5 atan(value1)
- 5.4.6.6 atan2
- 5.4.6.7 binomial(base, power)
- 5.4.6.8 bitMaskWithBitsSetFromTo(value1, value2)
- 5.4.6.9 cbrt()
- 5.4.6.10 ceil()
- 5.4.6.11 copySign()
- 5.4.6.12 cos(value1)
- 5.4.6.13 cosh(value1)
- 5.4.6.14 exp(value1, value2)
- 5.4.6.15 expm1(value1)
- 5.4.6.16 factorial(value1)
- 5.4.6.17 floor(value1)
- 5.4.6.18 GetExponent(value1)
- 5.4.6.19 getSeedAtRowColumn(value1, value2)
- 5.4.6.20 hash(value1)
- 5.4.6.21 hypot(value1, value2)
- 5.4.6.22 LeastSignificantBit(value1)
- 5.4.6.23 log(value1, value2)
- 5.4.6.24 log1(value1)
- 5.4.6.25 log10(value1)
- 5.4.6.26 log2(value1)
- 5.4.6.27 logFactorial(value1)
- 5.4.6.28 long()
- 5.4.6.29 longFactorial(value1)
- 5.4.6.30 minimum(value1, value2)
- 5.4.6.31 mod(value1, value2)
- 5.4.6.32 mostSignificantBit(value1)
- 5.4.6.33 nextAfter(value1, value2)
- 5.4.6.34 nextDown(value1, value2)
- 5.4.6.35 nextUp(value1)
- 5.4.6.36 pow(value1, value2)
- 5.4.6.37 rint(value1)
- 5.4.6.38 round(value1)
- 5.4.6.39 scalb(
- 5.4.6.40 signum(value1)
- 5.4.6.41 sin(value1)
- 5.4.6.42 sinh(value1)
- 5.4.6.43 sqrt(value1)
- 5.4.6.44 stirlingCorrection(value1)
- 5.4.6.45 tan(value1)
- 5.4.6.46 tanh(value1)
- 5.4.6.47 toDegrees(value1)
- 5.4.6.48 toRadians(value1)
- 5.4.6.49 ulp(value1)
- 5.4.7 Using Null-related Functions
-
5.4.8
Using Statistical Functions
- 5.4.8.1 beta1(value1, value2, value3)
- 5.4.8.2 betacomplemented(value1, value2, value3)
- 5.4.8.3 binomial2(value1, value2, value3)
- 5.4.8.4 binomialcomplemented(value1, value2, value3)
- 5.4.8.5 chiSquare(value1, value2)
- 5.4.8.6 chiSquareComplemented(value1, value2)
- 5.4.8.7 errorFunction(value1)
- 5.4.8.8 errorFunctionComplemented(value1)
- 5.4.8.9 gamma(value1, value2, value3)
- 5.4.8.10 gammacomplemented(value1, value2, value3)
- 5.4.8.11 incompleteBeta(value1, value2, value3)
- 5.4.8.12 incompleteGamma(value1, value2)
- 5.4.8.13 incompleteGammaComplement(value1, value2)
- 5.4.8.14 logGamma(value1)
- 5.4.8.15 negativeBinomial(value1, value2, value3)
- 5.4.8.16 negativeBinomialComplemented(value1, value2, value3)
- 5.4.8.17 normal(value1, value2, value3)
- 5.4.8.18 normalInverse(value1)
- 5.4.8.19 poisson(value1, value2)
- 5.4.8.20 poissonComplemented(value1, value2)
- 5.4.8.21 studentT(value1, value2)
- 5.4.8.22 studentTInverse(value1, value2)
-
5.4.9
Using String Functions
- 5.4.9.1 coalesce(value1,... )
- 5.4.9.2 Concat(value1,...)
- 5.4.9.3 indexof(value1, value2)
- 5.4.9.4 initcap(value1)
- 5.4.9.5 length(value1)
- 5.4.9.6 like(string, pattern)
- 5.4.9.7 lower(value1)
- 5.4.9.8 lpad(value1, value2, value3)
- 5.4.9.9 ltrim(value1, value2)
- 5.4.9.10 replace(string, match, replacement)
- 5.4.9.11 rpad(value1, value2, value3)
- 5.4.9.12 rtrim(value1, value2)
- 5.4.9.13 substr()
- 5.4.9.14 substring(string, from, to)
- 5.4.9.15 translate(expression, from_string, to_string)
- 5.4.9.16 upper(value1)
- 5.5 Adding Custom Functions and Custom Stages
- 5.6 Writing CQL Queries
-
5.1
Adding Stages to a Pipeline
-
6
Analyze
-
6.1
Using Geofences for Location-based Analytics
- 6.1.1 Selecting a Tile Layer
- 6.1.2 Managing Geofences using the Map Editor
- 6.1.3 Importing a Geofence from a Database
-
6.1.4
Using Spatial Patterns in Pipeline Stages
- 6.1.4.1 Clearing Objects Outside a Geo Fence
- 6.1.4.2 Tracking Objects using a Geo Fence
- 6.1.4.3 Getting Direction of a Moving Object
- 6.1.4.4 Obtaining Geographic Coordinates
- 6.1.4.5 Calculating Distance between Objects in a Stream
- 6.1.4.6 Calculating Distance between Objects in Two Streams
- 6.1.4.7 Creating Geo Fence
- 6.1.4.8 Monitoring Proximity between Objects in a Stream
- 6.1.4.9 Monitoring Proximity between Objects in Two Streams
- 6.1.4.10 Obtaining the Proximity of an Object from a Geo Fence
- 6.1.4.11 Finding Nearest Place using the Geographical Coordinates
- 6.1.4.12 Finding Nearest Place Details using the Geographical Coordinates
- 6.1.4.13 Determining Average Speed
-
6.2
Transforming and Analyzing Data using Patterns
- 6.2.1 Adding a Pattern Stage
- 6.2.2 Detecting Missing Events
- 6.2.3 Calculating Quantile Value
- 6.2.4 Identifying Correlation between Two Numeric Patterns
- 6.2.5 Detecting Duplicate Events
- 6.2.6 Eliminating Duplicate Events
- 6.2.7 Detecting Event Value Changes
- 6.2.8 Detecting Data Field Value Changes
- 6.2.9 Monitoring Sequence of Events
- 6.2.10 Outputting Highest Value Events
- 6.2.11 Outputting Lowest Value Events
- 6.2.12 Monitoring Invariably Increasing Numeric Values
- 6.2.13 Monitoring Invariably Decreasing Numeric Values
- 6.2.14 Identifying the Missing First Event in a Sequence
- 6.2.15 Identifying the Second Missing Event in a Sequence
- 6.2.16 Analyzing Data using Double Bottom Charts
- 6.2.17 Analyzing Data using Double Top Charts
- 6.2.18 Correlating Current and Previous Events
- 6.2.19 Delaying Delivery of Events to Downstream Node
- 6.2.20 Outputting Contents to Downstream Node
- 6.2.21 Outputting Unexpired Contents to Downstream Node
- 6.2.22 Merging Two Streams having Identical Shapes
- 6.2.23 Joining Flows with Streams and References
- 6.2.24 Transforming Events into JSON
- 6.2.25 Transforming a Single Event from a Stage into Multiple Events
- 6.2.26 Merging Two Continuous Events into a Single Event
- 6.2.27 Applying OML Models to get the Scoring of Events (Preview Feature)
- 6.2.28 Detecting Contiguous Events
- 6.2.29 Creating Pivot Columns
- 6.3 Using Machine Learning Models for Scoring and Prediction
- 6.4 Integrating with Druid Timeseries Database for Realtime Interactive Analytics
-
6.1
Using Geofences for Location-based Analytics
- 7 Visualize
- 8 Monitor
- 9 Reference
-
10
Troubleshoot
- 10.1 Pipeline Debug and Monitoring Metrics
-
10.2
Common Issues and Remedies
- 10.2.1 Pipeline
-
10.2.2
Pipeline
- 10.2.2.1 Pipelines are not running as expected
- 10.2.2.2 GGSA Pipeline getting Terminated
- 10.2.2.3 Live Table Shows Listening Events with No Events in the Table
- 10.2.2.4 Live Table Still Shows Starting Pipeline
- 10.2.2.5 Time-out Exception in the Spark Logs when you Unpublish a Pipeline
- 10.2.2.6 Piling up of Queued Batches in HA mode
- 10.2.2.7 Null Record from Summary in Query Stage
- 10.2.3 Stream
- 10.2.4 Connection
- 10.2.5 Target
- 10.2.6 Geofence
- 10.2.7 Cube
- 10.2.8 Dashboard
- 10.2.9 Live Output
- 10.2.10 Pipeline Deployment Failure