4 Working with Explorations

This chapter explains about explorations in Oracle Stream Analytics.

This chapter contains the following sections:

• Understanding Explorations

• Creating an Exploration

• Editing an Exploration

• Configuring a Target

• Understanding a Draft Exploration

• Publishing an Exploration

• Unpublishing an Exploration

• Exporting an Exploration

• Reusing an Exploration

• Marking an Exploration as Favorite

• Deleting an Exploration

• Restoring an Exploration.

4.1 Understanding Explorations

Explorations are Oracle Stream Analytics Event Processing applications that allow you to apply business logic to the event flows. Explorations are the visual representation of the data streamed from a source. The data in the exploration is represented visually in the form of:

• Live Output Stream - Live Output Stream shows the streaming data shapes in a tabular form.

  Figure 4-1 Sample Live Output Stream

  
  Description of "Figure 4-1 Sample Live Output Stream"

• Line Charts - Line charts are the primary way of visualizing time-series data. The X-axis is used to represent the time movement and allows natural scrolling of the data as time moves forward. The Y-axis is used to see the change of the dependent variable, that is, variable of interest, as time moves.

  Figure 4-2 Sample Line Chart

  
  Description of "Figure 4-2 Sample Line Chart"

• Scatter Charts - When the time dimension is not important, then data may be better understood by looking at a scatter plot. In this case, the user can assign distinct event properties to both the X-axis and Y-axis. However, values on both axes need to be numeric variables. Remember that the X-axis represents the explanatory variable and the Y-axis the response variable. Hence, the properties that indicate response or calculated field are better candidates to automatically assign to the Y-axis.

  In the case of time-series, it is obvious that new values enter in the right side of the graph and old values exit in the left side. However, this behavior does not translate to scatter plots, as the new point may show up anywhere.

  Figure 4-3 Sample Scatter Chart

  
  Description of "Figure 4-3 Sample Scatter Chart"

Explorations allow you to:

• Use one or more streams and references as sources of events and data

• Correlate multiple event and data sources

• Filter events and data

• Group events and data

• Specify the time/event -based windows for aggregation functions

• Specify aggregation functions to be used in summaries

• Review incoming events (before you apply the logic) and resulting events (after you apply the logic) in tabular and graph forms

• Add targets

• Define or apply functions to the data in the live output stream

• Define Oracle Business Rules to the live data in the output stream

4.1.1 Metadata

Every exploration has the following set of metadata:

• Name

• Tags

• Description

• Created By

• Created At

• Updated By

• Updated At

• Is Favorite.

Tags, Description, and being Favorite are specified explicitly in the Exploration Editor. Creation and update information is persisted automatically.

Figure 4-4 Exploration Metadata

Description of "Figure 4-4 Exploration Metadata"

4.1.2 Correlations

When an exploration has more than one data source associated with it, a logical relation must be established between the data sources. This logical relation is known as correlation.

Oracle Stream Analytics allows the data sources to be joined using correlation rules. When all the sources are correlated, they provide a single event stream that can be further sent to a Target. Only fields of compatible types from different non-correlated sources may be used as operands in the correlation.

If your streams and references are not correlated, the exploration is in an invalid state and will remain so until you correlate the sources. The exploration will not be updated on the server until you correlate.

4.1.3 Filters

The conditions with which the output can be filtered are known as Filters. When you apply filters to the output, only the events that match ALL or ANY of the filters are displayed. You can filter events by comparing a left operand with a right operand using the following operators:

• equal to

• not equal to

• lower than

• lower than or equal to

• greater than

• greater than or equal to

• contains

• not contains

• is null

• not null.

Left operand can be any one of the following:

• a field of any source in the exploration

• a summary that is defined in the Summaries area.

Right operand can be any one of the following:

• a constant of a type that is compatible with the type of left operand

• a field or a summary of a field from a source of the left operand.

Add a Condition

To add a condition:

1. Click the Filter Field and select any field from the data source.

2. Click the Filter Operator and select the appropriate operator.

3. Enter a suitable value in the input field.

4. Select an appropriate datatype if the populated datatype is not correct.

   A condition id added to the business rule.

5. Select match All or Match Any based on you requirement to filter.

Add a Group

Groups can be added to add nested queries.The events that match "AND" and "OR" conditions can be displayed:

To add a group:

1. Click the Filter Field and select any field from the data source.

2. Click the Filter Operator and select the appropriate operator.

3. Enter a suitable value in the input field.

4. Select an appropriate datatype if the populated datatype is not correct.

   A group id added to the business rule applying the logical operator OR to added the condition..

5. Select match All or Match Any based on you requirement to filter.

4.1.4 Summaries

You can aggregate functions (Summaries) to fields and apply grouping by any combination of fields. The only exception is that a field cannot have both summary and be included in a Group. When you define summaries and/or grouping, all other non-aggregate and non-grouping fields will be automatically excluded from the results shape.

When you define a summary or grouping, the only fields allowed in the output are fields from Summaries and Group By area. When you define a summary, it is automatically shown in the results table and filters area is provided with an option to filter by this newly created Summary. When you remove or change the summary, if it is used in the filters and/or in output, you must confirm this action as it will affect the filter and/or output.

4.2 Creating an Exploration

This section explains how to create an exploration.

To create an exploration:

1. Navigate to Catalog.

2. Select Exploration in the Create New Item menu.

3. Enter information in the Create Exploration dialog. The Create Exploration screen is as shown below.

   Figure 4-5 Create Exploration Dialog

   
   Description of "Figure 4-5 Create Exploration Dialog"

   1. Enter a name for the exploration in the Name field. This is a mandatory field.

   2. Provide a description that helps users understand about this exploration in the Description field. This is an optional field.

   3. Enter a tag with which you want to tag the exploration in the Tags field. You can provide multiple tags. This is an optional field.

   4. Select a stream as the source for the exploration in the Source field. This can be another exploration that is in published state. This is a mandatory field. You can select multiple streams.

      Note:

      You cannot create an exploration based only on a reference as the source. You must start with a stream and then add reference(s) as source later.

      Attention

      When you are using Spark as the runtime, you cannot select multiple streams for the exploration. Only a single stream is supported.

4. Click Create to complete the creation of an exploration. The new exploration opens in the Exploration Editor. The newly created exploration also appears in the Catalog.

   You can continue to add further to the newly created exploration in this editor:

   • Correlation — You must add correlations between sources if the exploration uses more than one source.

   • Summaries — You can add summary of the exploration data and then group the data accordingly. This is optional. See Summaries.

   • Filters — You can add filters to further filter the data based on the summaries you have added to the exploration. This is optional. See Filters.

   • Business Rules — You can add business rules to the data of your exploration. Business rules build conditional logic into the applications. This is optional.

   • Expressions — You can add expressions to the streaming exploration data. The expressions help you in populating new data fields in the stream by performing calculations on the existing data. Adding expressions is optional

5. You can perform the actions on the columns in the Live Output Stream.

   • Hide — hide any selected column

   • Remove — remove any selected column

   • Rename — rename any selected column

   • Function — add an expression to any selected columns

   • Restore — restore any removed columns. This option appears only when you have removed any column.

     Figure 4-6 Actions You Can perform on the Individual Columns in the Live Output Stream

     
     Description of "Figure 4-6 Actions You Can perform on the Individual Columns in the Live Output Stream"

6. Click Cancel at any point of time to abort the operation.

4.3 Editing an Exploration

When you create an exploration, the exploration opens up in the Exploration Editor. You have the option to edit the parameters of the exploration. You can edit or update existing explorations. You can add or modify the data sources for the exploration.

To edit an exploration:

1. Navigate to the Catalog.

2. Identify the exploration that you want to edit or update. Click the name string of the exploration or the exploration icon to open it for editing. The Exploration Editor is as shown below.

   Figure 4-7 Exploration Editor

   
   Description of "Figure 4-7 Exploration Editor"

3. Click anywhere in the Sources field to add or remove data sources. If you have more than one data source, the sources must be correlated.

4. Define the Correlations if the exploration has more than one data source.

   Note:

   Your changes to the exploration are not applied until the sources are correlated. An error message is generated when you try to save the changes without correlating the sources.

5. Click Add Summary in the Summaries area to add a summary to the exploration.

6. Click Add a Filter in the Filters area to add filters to the output. The filters further filter the streaming data based on the summaries you have added.

7. Click Properties in the Live Output Stream area to configure the properties of the output. You can indicate which fields must appear in the output stream.

   • You can also add the Timestamp to the output stream. The timestamp at which the streaming data was captured appears against each row of live output stream. This is a toggle button and you can use it to switch ON or OFF the timestamp in the live data.

     Note:

     Only the fields that appear in the live output stream can be sent to the target.

8. Configure the number of graphs you want to be displayed for the output stream in the Charts area.

9. Click Window Drawer icon, available to the right of the screen, to specify time and event based windows used in processing your stream.

   1. Select a value in the Range drop-down list and specify a numerical value. This value specifies the range of time period for the stream of data. The range is defined in units like now, nanoseconds, microseconds, milliseconds, seconds, minutes, hours, or events.

      Now indicates that the range is as narrow as 1 nanosecond and unbounded indicates a range since the server has started. When the range is anything other than Now, another field, Evaluation frequency appears.

   2. Select a value in the Evaluation frequency drop-down list and specify a numerical value. This value defines how often the system produces the result events.

      Note:

      Evaluation frequency cannot be greater than Range.

10. Click the pencil icon at the top-right corner to view the information of the exploration.

    1. Click Edit Info to update Name, Description, or Tags of the exploration.

       Figure 4-8 Update Information of Exploration

       
       Description of "Figure 4-8 Update Information of Exploration"

    2. Click Done after updating the information.

11. Click Undo Changes icon at the top-right corner to undo the changes you have made. The Undo is applicable only on the following changes:

    • add/edit/delete summary

    • group by

    • add/edit/delete filter

    • specifying range.

    Note:

    Undo operation is not available for publish action. To undo a publish, you must unpublish the exploration.

    Figure 4-9 Undo Changes Icon

    
    Description of "Figure 4-9 Undo Changes Icon"

    Note:

    Clicking Undo once, reverses only the last made change.

12. Click Redo Changes icon at the top-right corner to re-apply the changes you have made. The Redo is applicable only on the following changes:

    • add/edit/delete summary

    • group by

    • add/edit/delete filter

    • specifying range.

    Figure 4-10 Redo Changes Icon

    
    Description of "Figure 4-10 Redo Changes Icon"

    Note:

    Clicking Redo once, re-applies only the last made change.

13. Click the User Assistant icon at the top-right corner to view explanation about the Exploration Editor. This icon acts as a toggle button.

    Figure 4-11 User Assistant Icon

    
    Description of "Figure 4-11 User Assistant Icon"

Note:

When you edit an exploration, you must publish it again for the changes to be implemented.

4.4 Configuring a Target

Every exploration must be configured with a target to send the details downstream. Unless the exploration is published, you cannot send the events downstream to the configured target.

To configure a target:

1. Click Configure a Target in the Exploration Editor. The Configure a Target dialog appears.

   Figure 4-12 Configure a Target Dialog

   
   Description of "Figure 4-12 Configure a Target Dialog"
2. Select the Destination Type in the Configure a Target dialog.

   ---

   Destination Type	Data Fields
   CSV File	If the destination type is CSV File, you must enter data in the following fields: File Name - name of the CSV file. This is a mandatory field. Append - indicates that you do not want to re-create the file and want to add events to the existing file (generally a non-empty file) Under Oracle Stream Analytics event structure, two folders/directories are created under the server name: FileStorage <username> - the file name you specify for the CSV file goes under this directory.
   HTTP Publisher	If the destination type is HTTP Publisher, you must enter data in the following fields: Server URL - the location of the HTTP Publisher. This is a mandatory field. If you use Oracle Stream Analytics's own publisher, the URL is: http://host:port/pubsub. Channel - the channel of the HTTP Publisher. This is a mandatory field. Username - username to access the HTTP Publisher Password - password to access the HTTP Publisher
   EDN	If the destination type is EDN, you must enter data in the following fields: URL - the location of the EDN. This is a mandatory field. Username - username to access the EDN. This is a mandatory field. Password - password to access the EDN. This is a mandatory field. Get Event Definitions - retrieves the event definitions. Select Shape - select a shape from the drop-down list. Target Property - target shape property. Output Stream Property - select a corresponding shape for the exploration property, or No mapping if you do want to map.
   JMS	If the destination type is JMS, you must enter data in the following fields: URL - the location of JMS. This is a mandatory field. Username - username to access the JMS Password - password to access the JMS JNDI Name - the JNDI queue name of JMS
   REST	If the destination type is REST, you must enter data in the following fields: Target URL - the location of the REST service the exploration will invoke with the resulting messages. This is a mandatory field.

   ---

3. Click Finish in the Configure a Target screen to configure the exploration.
4. Click Cancel at any point of time to abort the operation.

4.5 Understanding a Draft Exploration

When you create an exploration, it is in draft state by default. The draft exploration is not available to other users of the application. An exploration moves to a published state when it is published.

When a published exploration is deleted or unpublished, the exploration moves to a draft state.

A draft exploration has the following distinctive features:

• it gets undeployed from the runtime once the Exploration Editor is closed

• it is not sending events into the target even if there is a target. You have to publish the exploration before events appear in the target.

4.6 Publishing an Exploration

You must publish an exploration to make the exploration available for all the users of Oracle Stream Analytics. A published exploration:

• can be used as a source for another exploration

• will be visible to other users of the application

• is in running state until another user unpublishes or deletes the exploration

• can have targets applied.

To publish an exploration:

1. Navigate to the Catalog.

2. Identify the exploration you want to publish and click its name or icon.

3. Select Publish from the Actions menu.

   Figure 4-13 Actions Menu

   
   Description of "Figure 4-13 Actions Menu"

   The exploration gets published and you can see a confirmation message. When the exploration is published, it is indicated in the Catalog as shown in the figure below.

   Figure 4-14 Published Exploration

   
   Description of "Figure 4-14 Published Exploration"

   You can publish/re-publish an exploration when no consumers are using it, in the following scenarios:

   • Re-publish a published exploration with no changes

   • Publish a published exploration after modifying its details (a published exploration moves to Draft status when it is modified). After publishing, the draft status changes to published status.

   • Re-publish an exploration that has a target configured.

   When more than one user tries to publish the exploration after modifications, a conflict arises. The following conflict resolution actions help you in resolving the conflict:

   1. Click Overwrite to overwrite the published version with the changes you have made.

   2. Click Discard to discard your changes and go to the latest published version of the exploration.

   3. Click Clone to publish the exploration under a different name, retaining changes from multiple users.

Note:

If there are any users using a published exploration, you cannot re-publish that exploration until you clear the dependencies.

4.7 Unpublishing an Exploration

You can unpublish an exploration that has already been published.

Note:

If there are users using a published exploration, you cannot un-publish that exploration until you clear all the dependencies.

To unpublish an exploration:

1. Navigate to the Catalog.
2. Identify the published exploration you want to configure and click it.
3. Select Unpublish from the Actions menu.

   The exploration gets unpublished and you can see a confirmation message. When the exploration is unpublished, the Published status changes to Draft in the Catalog.

   Figure 4-15 Unpublished Exploration

   
   Description of "Figure 4-15 Unpublished Exploration"

Note:

You cannot un-publish a published exploration when it is being used by any user.

4.8 Exporting an Exploration

Oracle Stream Analytics allows you to export an exploration as a JAR file and download it for reuse. If you try to export an exploration that is not published, you get an error message.

You can import the exported exploration into JDeveloper and develop the application. However, you cannot import an application or exploration into Oracle Stream Analytics.

Remember

You cannot export an exploration when you are using Spark as the runtime environment.

To export an exploration:

1. Navigate to the Catalog.

2. Identify the published exploration you want to export and click it.

3. Select Export from the Actions menu.

   Figure 4-16 Export Exploration

   
   Description of "Figure 4-16 Export Exploration"

   The Export Exploration dialog is as shown below:

   Figure 4-17 Export Exploration Dialog

   
   Description of "Figure 4-17 Export Exploration Dialog"

4. Review the details of the exploration under Export List. Resource is the name of the exploration and its dependencies to be exported. Type is the type of the resource (exploration/source).

5. Edit/update the exploration bundle details under Advanced Options if required.

   1. Enter a value in the Bundle ID field. This is an OSGI Bundle-SymbolicName parameter which is unique OSGI Bundle and will be the name of the exported JAR file.

   2. Enter a value in the Bundle Name field. This is the name with which the exploration will be exported.

6. Click Export to complete exporting the exploration. The exploration is saved as a jar file. You can save the JAR file to any location.

   Figure 4-18 Save Exported Exploration

   
   Description of "Figure 4-18 Save Exported Exploration"

7. Click Cancel at any point of time to abort the operation.

If an exploration uses any target or sources with specified user credentials (EDN, JMS, HTTP pubsub), then the exported exploration contains the credentials with encrypted passwords. Passwords are encrypted with the server encryption key. Usually, different servers use different encryption keys. As a result, you must update the encrypted passwords or convert them to plain text passwords before deploying the exploration to a server. If you do not do this, an encryption error appears.

4.9 Reusing an Exploration

You can re-use an existing exploration as a source for another exploration.

Remember

You cannot export an exploration when you are using Spark as the runtime environment.

4.10 Marking an Exploration as Favorite

You can mark an exploration as Favorite. Marking an exploration as Favorite allows you to locate and navigate such explorations easily even if you have numerous items on the catalog.

To mark an exploration as favorite:

1. Identify the exploration you want to mark as favorite in the Catalog.
2. Click the Favorite icon to the right of the exploration to mark it as a favorite. This icon acts as a toggle button.

   Figure 4-19 Favorite Icon

   
   Description of "Figure 4-19 Favorite Icon"
3. Click the Favorite icon to remove the exploration from favorites.

4.11 Deleting an Exploration

You can delete an exploration if you are sure you do not require the exploration anymore. When you delete a published exploration, it moves into a draft state. When an exploration is available in both draft and published state, the delete operation first deletes the draft exploration. When you invoke delete operation for the second time on the same exploration, the exploration gets deleted.

To delete an exploration:

1. Identify the exploration you want to delete in the Catalog.
2. Click the Delete icon to the right of the exploration to delete it.

   Figure 4-20 Delete Icon

   
   Description of "Figure 4-20 Delete Icon"

   A confirmation dialog appears. If you select OK, the exploration will be deleted.

4.12 Restoring an Exploration

An exploration goes to an invalid state in any of the following scenarios:

• if the deployment was removed from the Visualizer

• the EDN source could not start because of a network issue

• a CSV source could not start because of the file absence

• the corresponding Event Processing application was undeployed or paused on the Event Processing Server

The above stated scenarios are a few amongst the reasons why an exploration can go into an invalid state.

Invalid state indicates that there is no deployment for the object, either a source or a published exploration.

You can restore an exploration. The restore operation tries to deploy an object in invalid state with the information available in Oracle Stream Analytics. For example, there is an exploration E1 that uses the source S1, and both of them are in invalid state. When you invoke a restore operation on the exploration, the Oracle Stream Analytics tries to deploy both the stream and exploration in sequence.

To restore an exploration:

1. Identify the exploration that is in invalid state in the Catalog. The exploration in invalid state is indicated by a red cross icon.

   Figure 4-21 Exploration in Invalid State

   
   Description of "Figure 4-21 Exploration in Invalid State"
2. Click the Restore icon to the right of the exploration to fix the deployment problems and restore the exploration.

   Figure 4-22 Restore Icon

   
   Description of "Figure 4-22 Restore Icon"

   A notification appears about the result of restore operation.