4 Working with Streams

This chapter explains the details about streams, creating streams, and editing streams.

This chapter contains the following sections:

4.1 Understanding Streams

An event stream is an Oracle Stream Analytics Event Processing adapter node producing events of a single type.

In Oracle Stream Analytics object taxonomy, an event stream is an entity responsible for receiving events from the outside world. There are different stream types based on different communications technology or messaging system, such as, for example, REST or JMS or Kafka.

4.1.1 Metadata

Every stream has the following set of metadata:

  • Name

  • Tags

  • Description

  • Created By

  • Created At

  • Updated By

  • Updated At

  • Is Favorite.

4.2 Creating a Stream

A stream is required for the users to create an exploration. This section explains how to create a stream. The Create Stream screen is shown below.

Figure 4-1 Create Stream - Source Details Dialog

Description of Figure 4-1 follows
Description of "Figure 4-1 Create Stream - Source Details Dialog"

To create a stream:

  1. Navigate to Catalog.
  2. Select Stream in the Create New Item menu.
  3. Enter a name for the stream in the Name field. This is a mandatory field.
  4. Provide a description that helps users understand about the stream in the Description field. This is an optional field.
  5. Enter a tag with which you want to tag the stream in the Tag field. You can provide multiple tags. This is an optional field. You can separate the multiple tags by commas, Enter key, or a space.

    Note:

    The tags you enter must represent the industry verticals supported by Oracle Stream Analytics. For more information about pre-existing tags, see Home Page.

  6. Select a stream type that you want to use for the stream from the Source Type field. This is a mandatory field.

    Note:

    If you select EDN as the source type, you can work only with an XML element that has a flat structure. The XML schema does not support any other schemas. The Stream wizard does not display all other types of EDN.

  7. Select Create Exploration with this source (Launch Exploration Editor) if you want to launch the Exploration Editor utilizing this stream as the primary source.
  8. Click Next. Based on the source type you have selected, you must enter data in the following fields on the Type Properties screen:

    Figure 4-2 Create Stream - Type Properties Dialog

    Description of Figure 4-2 follows
    Description of "Figure 4-2 Create Stream - Type Properties Dialog"

    Table 4-1 Source Types for Streams

    Source Type Data Fields

    CSV File

    If the source type is CSV File, you must enter data in the following fields:

    • Source Path - either upload a local CSV file or provide a URL to the location where the file resides. This is a mandatory field.

    • Event Interval - the time interval at which the event refreshes. The time is in milliseconds. Default value is 1000 milliseconds, which is 1 second.

    • Initial Delay (ms) - the time interval before which the data streaming starts. The time is in milliseconds. Default value is 0.

    • Loop the CSV File - the option that allows you to keep the CSV source always running by automatically restarting it from the beginning of the file when the end of the file is reached.

    HTTP Subscriber

    If the source type is HTTP Subscriber, you must enter data in the following fields:

    • Connections - the connection of the type HTTP Subscriber. This is a mandatory field.

    • Password - password to access the HTTP Subscriber. This is a mandatory field.

    EDN

    If the source type is EDN, you must enter data in the following fields:

    • Connections - a connection which is of type EDN. This is a mandatory field.

    JMS

    If the source type is JMS, you must enter data in the following fields:

    • Connections - URL to access the JMS. This is a mandatory field.

    • JNDI name - JNDI name (queue or topic) to access the JMS. This is a mandatory field.

    REST

    If the source type is REST, you must enter data in the following fields:

    • Context Path - the context path of the REST Adapter. This is a mandatory field.
    Coherence cache

    If the source type is Coherence cache, you must enter data in the following fields:

    • Connections - a connection which is of type coherence cache. This is mandatory field.

    • Cache name — the name of the cache.
    Kafka

    If the source type is Kafka, you must enter data in the following fields:

    • Connections - a connection which is of type Kafka. This is a mandatory field.

    • Topic name — a topic name to be used in the stream. This topic name should have been defined at the time of creating the connection. This is a mandatory field.

    • Group Id — a group id to be used for the stream. This id should have been defined at the time of creating the connection. This is an optional field.
    MQTT

    If the source type is MQTT, you must enter data in the following fields:

    • Connections - a connection which is of type MQTT. This is a mandatory field.
    Twitter

    If the source type is Twitter, you must enter data in the following fields:

    • Connections - connections of type Twitter. This is a mandatory field.

    • Hashtags — the hashtags separated by commas that you want to include in your output stream. This is an optional field.

    • Languages — the languages separated by commas you want to include in your output stream. This is an optional field.

    • Users — list of Twitter user IDs separated by commas that you want to include in your output stream. This is an optional field.

    At-least one of Hashtags or Users property needs to be specified.

    Note:

    The logical operator OR is used when you specify both Hashtags and Users. For more information, see https://dev.twitter.com/streaming/reference/post/statuses/filter.
    Event generator

    If the source type is Event generator, you must enter data in the following fields:

    • Interval (ms) - the time interval between messages in milli seconds. This is an optional field.

    • Lowest number — the lowest generated numeric value. This is an optional field.

    • Highest number — the highest generated numeric value. This is an optional field.

    • Precision — the number of decimal places. This is an optional field.

    • Number of strings — the number of pre-generated strings. This is an optional field.

    • Lowest number of words — the lowest number of words in the generated string. This is an optional field.

    • Highest number of words — the highest number of words in the generated string. This is an optional field.
  9. Click Next. The Create Stream - Shape screen appears.

    Figure 4-3 Create Stream - Shape Dialog

    Description of Figure 4-3 follows
    Description of "Figure 4-3 Create Stream - Shape Dialog"
  10. Perform Manual Mapping or Select a Shape on the Shape screen.

    When you perform manual mapping, you need to decide the best data type that is suitable for the value represented by each of the fields in the data source. The data types can be String, Byte, Integer, Big int, Float, Double, Big decimal, Boolean, Timestamp.

    Note:

    Providing a shape name is optional. If the shape is not specified, then this shape will not be available for reuse in the drop-down list.

    If you Select Shape, the pre-existing configuration is used for your stream.

  11. Click Create to create the stream. The new stream appears in the Catalog.
  12. Click Cancel at any time to abort the operation.

4.3 Editing a Stream

After you create a stream, Oracle Stream Analytics provides you with an opportunity to edit the stream.

You cannot edit the stream when:

  • there are published explorations using the stream

  • you open the stream dialog from within the exploration.

To edit a stream:

  1. Click the stream you want to edit from the Catalog.
  2. Click Edit in the source dialog at the top.

    Figure 4-4 Edit Source Dialog

    Description of Figure 4-4 follows
    Description of "Figure 4-4 Edit Source Dialog"

    Note:

    The behavior of Edit at each section level in the current release is to launch the Edit Stream wizard. This implies that the Edit Stream - Source Details screen is the default screen you see.
  3. Update the data you want to edit in the stream dialog.

    Attention

    A data type of Geometry for the fields in the data source is not available when you are using Spark runtime.

    Note:

    The logical operator OR is used when you specify both Hashtags and Users for a stream of type Twitter. For more information, see https://dev.twitter.com/streaming/reference/post/statuses/filter.
  4. Select Loop the CSV file on the Edit Stream - Type Properties dialog. This feature allows you to keep the CSV source always running by automatically restarting it from the beginning of the file when the end of the file is reached.

    Figure 4-5 Edit Stream — Loop the CSV File Feature

    Description of Figure 4-5 follows
    Description of "Figure 4-5 Edit Stream — Loop the CSV File Feature"

    Note:

    This feature is available only for the CSV source and not for any other type of source or target.
  5. Click Save when you are done with updating or editing the stream.
  6. Click Cancel at any point of time to abort the operation.
  7. Click Done in the stream dialog to complete the edit operation.

4.4 Restoring a Stream

A stream goes to an invalid state if the stream or the corresponding Event Processing application is undeployed in the Visualizer. Oracle Stream Analytics allows you to restore a stream that is in an invalid state.

To restore an exploration:

  1. Identify the stream that is in invalid state in the Catalog. The stream in invalid state is indicated by a red cross icon.
  2. Click the Restore icon to the right of the stream to fix the deployment problems and restore the stream.

    Figure 4-6 Restore Icon

    Description of Figure 4-6 follows
    Description of "Figure 4-6 Restore Icon"

    A notification appears about the result of restore operation.

4.5 Marking a Stream as Favorite

You can mark a stream as Favorite. Marking a stream as Favorite allows you to locate and navigate such streams easily even if you have numerous items on the catalog.

To mark a stream as favorite:

  1. Identify the stream you want to mark as favorite in the Catalog.
  2. Click the Favorite icon to the right of the table row for that stream to mark it as a favorite. This icon acts as a toggle button.
  3. Click the Favorite button again to remove the stream from favorites.

4.6 Deleting a Stream

You can delete a stream if you are sure you do not require the stream anymore. If a stream has dependencies, it cannot be deleted. A stream is said to have dependencies if it is part of a published exploration being used by any user.

To delete a stream with dependencies, you must delete the dependencies first.

To delete a stream:

  1. Identify the stream you want to delete in the Catalog.
  2. Click Delete Item option from the Actions menu to the right of the stream to delete it.

    A confirmation dialog appears. If you select Delete, the stream will be deleted. Click Cancel if you do not want to delete the stream for any reason.