About Stream Analytics

Create custom operational dashboards that provide real time monitoring and analyses of event streams using OCI GoldenGate Stream analytics. Identify events of interest, run queries against the event streams in real time, or raise alerts based on your analysis.

Related Topics

Stream analytics concepts

Start with the basics. Familiarize yourself with the following concepts:

  • Connection: Stores the connectivity information for a source or target technology.
  • Stream: A continuous flow of dynamic data.
  • Pipeline: The workflow data from source to target.
  • Business logic: Various filters and functions you can apply to a pipeline to obtain the precise data you want to analyze.
  • Publishing: Makes the pipeline available to all Stream analytics users and sends data to targets.

Supported connections

Learn about what types of connections are supported by OCI GoldenGate Stream Analytics.

OCI GoldenGate Stream Analytics supports the following source technology types:

Note:

You can also create Coherence, Ignite, and Java Message Server (JMS) connections directly within the Stream Analytics console.

Stream Analytics supports the following target technology types:

Note:

You can also create Amazon S3, Azure Data Lake Storage, Coherence, Hadoop File Storage (HDFS), Ignite, JMS, and MongoDB connections directly within the Stream Analytics console.

Stream analytics limitations

While OCI GoldenGate Stream Analytics appears the same as Oracle Stream Analytics, there are certain features that are not supported in the OCI version. Pay careful attention to notes in the Oracle Stream Analytics that inform you of whether or not a feature is supported in OCI GoldenGate Stream Analytics.

Metering and billing for Stream Analytics deployments

Ensure that you review the information in Metering and billing for OCI GoldenGate deployments about Oracle Compute Unit (OCPU) selection and scaling.

OCI GoldenGate Stream Analytics OCPU usage is calculated based on the following factors:

  • Stream Analytics console
  • Number of Streaming pipelines
  • Ignite cluster
  • GoldenGate Big Data cluster

Before calculating the number of OCPUs you need, let's first review how many compute units each Stream Analytics resource requires. 1 OCPU is equal to 2 compute units (vCPUs). 1 vCPU is equal to 1000 millicores (1000m).

The following table lists example Stream Analytics pipeline settings and the calculated number of OCPUs required.
Pipeline Driver Executor Total vCPUs OCPUs billed
Pipeline A 500m 1 x 500m 1000m 1
Pipeline B 500m 2 x 500m 1500m 1
Pipeline C 500m 4 x 500m 2500m 2
Pipeline D 600m 2 x 700m 2000m 1
Pipeline E 1000m 2 x 1000m 3000m 2

You can configure the Driver and Executor settings as needed for each pipeline in the Stream Analytics console.

The following table lists example Stream Analytics resource configurations based on the number of pipelines (from the above table) and the calcuated number of OCPUs required.

Stream Analytics console Number of pipelines Streaming pipelines Ignite cluster GoldenGate for Big Data cluster OCPUs billed
1000m 1 x Pipeline A 1000m 0 0 1
1000m 3 x Pipeline A 3000m 0 0 2
1000m 1 x Pipeline B 1500m 0 0 2
1000m 1 x Pipeline B 1500m 2 x 500m 500m 2
1000m 1 x Pipeline A

1 x Pipeline B

 2500m 2 x 500m 500m 3
1000m 2 x Pipeline A

1 x Pipeline B

 3500m 2 x 500m 500m 3

The Stream Analytics console requires 1000m. Each streaming pipeline requires additional millicores depending on their settings. The Ignite cluster, if activated, requires a minimum of 2 cluster instances. You can configure the millicore limit for both Ignite and GoldenGate Big Data clusters in the Stream Analytics console. When added together, you can determine the total number of OCPUs that you need to select when creating your Stream Analytics deployment.

If you're unsure, you can start with 2 or 3 OCPUs, and then review the OCPU consumption metrics on the deployment details page and adjust accordingly.