This chapter walks through building a basic Oracle Stream Analytics application. The steps include explanations of key Oracle Stream Analytics application programming concepts.
This chapter covers the following topics:
The basic Oracle Stream Analytics application models a simple stock trade alert system. The application receives example data about stock trades, monitors the data for certain characteristics, and based on the results, prints some of the data to the console. The following illustration shows the finished event processing network (EPN) diagram for the application:
This walkthrough requires that you have downloaded and installed the 12c version of Oracle Stream Analytics including Oracle JDeveloper and the Oracle Stream Analytics JDeveloper plug-in. Follow the installation instructions that come with the download to ensure you have the correct setup.
Make sure you set the JAVA_HOME
variable to point to JDK7_u55 or above and set the PATH
variable to point to the bin
directory under your JDK installation:
export JAVA_HOME=<path to installation directory> export PATH=${JAVA_HOME}/bin:${PATH}
In this walkthrough, the installation directory is /Oracle/Middleware/my_oep/
.
Optionally, you can set the WLEVS_HOME
variable to point to the installation directory. The Oracle Stream Analytics JDeveloper plug-in uses this variable to detect the local Oracle Stream Analytics server.
Note:
This walkthrough introduces features specific to Oracle Stream Analytics and assumes that you are familiar with basic Java programming.
In Oracle JDeveloper an application is the highest level in the control structure. An application is a view of all the objects you need while you work. An application keeps track of all your projects while you develop programs.A project is a logical container for a set of files that define an Oracle JDeveloper program or portion of a program. A project can contain files that represent different tiers of a multi-tier application or different subsystems of a complex application. Project files can reside in any directory and still be contained within a single project.
Start Oracle JDeveloper
Go to/Oracle/Middleware/my_oep/jdeveloper/jdev/bin
.
Type ./jdev
.
The Select Role dialog displays.
In the Select Role dialog, select Studio Developer (All Features) and click OK.
Wait a few moments while Oracle JDeveloper starts.
Create the TradeReport Application
The TradeReport application contains the Projects and Applications Resources windows. The Projects window lists the TradeReport project. The TradeReport project contains an OEP Content
folder with the spring
and wlevs
subfolders. On the right side of Oracle JDeveloper under IDE Connections is the Resources window.
Projects Window
spring subfolder that contains the TradeReport.context.xml
assembly file. The assembly file conforms to the Spring framework and contains the contents and structure of the TradeReport EPN.
The assembly file also contains the default configuration for each EPN stage. This default configuration cannot be changed at run time without redeploying the application. As you add and connect stages on the EPN diagram, Oracle JDeveloper captures your work in this file. You can also edit this file manually.
Note:
The EPN assembly file XML schema extends the Spring framework configuration file. See the Spring website at http://www.springsource.org/spring-framework
.
wlevs subfolder that contains the default processor.xml
configuration file. The files in the wlevs
folder describe components with configurations that can be edited at runtime with Oracle Stream Analytics Visualizer. As you use Oracle JDeveloper to create components, you can place their configurations in the processor.xml
file or specify another component configuration file to group component types in the same file. You can also edit configuration files manually.
EPN diagram The EPN diagram represents the components that make up the application. Event data enters your application from the left of the diagram, and moves to the right from stage to stage.
The EPN diagram shows a graphical representation of the underlying EPN configuration. When you add a component to the EPN, Oracle JDeveloper writes information to the TradeReport.context.xml
assembly file and the configuration file.
MANIFEST.MF that describes the contents of the OSGi bundle that you deploy to the Oracle Stream Analytics server.
Resources Window
The Resources window, which is on the right side of Oracle JDeveloper under IDE Connections, provides information about running server connections.
Within an Oracle Stream Analytics application, every event has an event type. The event type is a structure that defines a particular kind of event data in terms of the set of values the event can take, and the operations that can be performed on that data.
Oracle Stream Analytics supports several data structures for creating a new event type. These data structures are JavaBean
classes, tuples, and java.util.Map
classes. A JavaBean
class is the best practice structure for new event types and is used in this walkthrough to define trade events.
As raw event data comes into the Oracle Stream Analytics application, the application binds that data to an event of a particular event type. You define the event type in terms of the set of data it can hold and the required type for each data in the set.
In this walkthrough, the event data comes into the application from a CSV file in consistent rows of comma-separated values as follows:
IBM,15.5,3.333333333,3000,15 SUN,10.8,-1.818181818,5000,11 ORCL,14.1,0.714285714,6000,14 GOOG,30,-6.25,4000,32 YHOO,7.8,-2.5,1000,8
The data columns are not labeled in the CSV file, but if they were labeled, they would have the corresponding Java data type shown in Table 4-1. The Java data types that define an event type are referred to as properties in Oracle Stream Analytics.
Table 4-1 Mapping Event Data to Event Types
Possible Columns | Java Data Type |
---|---|
Stock Symbol |
String |
Price per share |
Double |
Percent change |
Double |
Volume of shares transacted |
Integer |
Last price |
Double |
Create the TradeEvent JavaBean
Select the TradeReport project.
Oracle JDeveloper highlights the TradeReport project.
Select File > New > From Gallery.
The New Gallery dialog displays.
In the New Gallery dialog, select General in the left panel, Java Class in the right panel, and click OK.
The Create Java Class dialog displays.
In the Create Java Class dialog, enter TradeEvent in the Name field, and enter or review the following default settings:
Name: TradeEvent Package: tradereport Extends: java.lang.Object Access Modifiers: public Other Modifiers: <None> Constructors from Superclass: checked Implement Abstract Methods: checked
Click OK.
Oracle JDeveloper adds the tradereport.TradeEvent
JavaBean class to the Project under the Application Sources folder. The stub code displays in the Oracle JDeveloper center window in its own tab:
package tradereport; public class TradeEvent { public TradeEvent() { super (); } }
Create Private Variables and Accessor Methods
In the TradeEvent class, add private variables for each of the properties (Java data types) as shown in the following example.
package tradereport; public class TradeEvent { // One variable for each field in the event data. private String symbol; private Double price; private Double lastPrice; private Double percChange; private Integer volume; public TradeEvent () { super(); } }
To generate the accessor methods, right click anywhere in the source editor.
The source editor pop-up menu displays.
In the source editor pop-up menu, select Generate Accessors.
The Generate Accessors dialog displays.
In the Generate Accessors dialog, click the Select All button and click OK.
Close the TradeEvent.java tab and save the file.
Configure the TradeEvent Event Type
In the TradeReport project under the OEP Content folder, double-click the EPN Diagram.
The EPN diagram displays in the center window and is empty.
Below the EPN diagram select the Event Types tab.
The Event Type Definitions window displays the TradeReport.context.xml
folder with Add (+) and Delete (x) buttons at the top.
In the Event Type Definitions window, select the TradeReport.context.xml folder and click Add.
Controls to define the event type display below Event Type Details on the right.
In the Type Name field enter TradeEvent.
The event type name does not have to be similar to the JavaBean class name, but by making them similar, it is easier to track which event types go with which classes.
Select the Properties defined in Java bean radio button and enter or use search to the name of the JavaBean in the Class box.
The name of the JavaBean is tradereport.TradeEvent.
Note:
The Properties defined declaratively radio button is for defining events as tuples.
Close the EPN diagram editor and save the file.
View the EPN Assembly File
Notice that Oracle Stream Analytics manages event types in an event type repository, and that the TradeEvent
event type contains (maps to) the tradereport.TradeEvent
class.
Adapters manage data flowing into and out of the EPN. This example uses a csvgen adapter that works with the load generator utility to simulate a data feed to test your application. The load generator reads an ASCII file that contains the sample data feed information and sends each line of data in order to a port. The csvgen adapter listens for data at the same port. The csvgen adapter logic translates data read from the CSV file into an event that has the TradeEvent
event type.
Note:
Before you deploy an application to the final production environment, you must switch to an input adapter that can read the type of incoming data your application will receive in production.
In this procedure, you declare the adapter and set its properties. When completed, the EPN diagram displays the adapter to create the first stage in your TradeReport EPN.
Create the csvgen Adapter and Set Its Properties
A channel is a conduit that uses logic to transfer events from one stage in the EPN to the next stage. In this step, you add a channel to carry newly generated events from the StockTradeCSVAdapter to the next stage.
Create the AdapterOutputChannel
With the EPN Diagram open, go to Basic Components and drag the Channel component to an empty space on the EPN diagram.
The New Channel dialog displays.
In New Channel dialog, enter the following values:
Channel ID: AdapterOutputChannel. Event Type: TradeEvent.
Click OK.
The AdapterOutputChannel component displays on the EPN diagram.
The AdapterOutputChannel conveys events of type TradeEvent
to the next stage in the EPN diagram. Recall that the TradeEvent
event type is implemented with the TradeEvent JavaBean
class.
Connect the Adapter to the Channel
Next you add a listener event sink that receives trade events from the channel and checks the information in those events. A listener event sink is a Java class that implements logic to listen for and work on trade events. This type of Java class is also called a listener Java class.
The following procedure shows you how to create a listener event sink that listens for trade events, gets the stock symbol and trade volume information, and prints the stock symbol and trade volume information to the console.
Create the Listener Event Sink
Select the TradeReport project and select File > New > Java Class.
In the Create Java Class dialog, enter TradeListener in the Name field and review the following settings:
Name: TradeListener Package: tradereport Extends: java.lang.Object Access Modifiers: public Other Modifiers: <NONE> Constructors from Superclass: Checked Implement Abstract Methods: Checked
In the Create Java Class dialog under the Implements area, click the Add (+) button to select the interface your listener needs to implement to be an event sink.
In the Class Browser dialog, use either the Search tab or the Hierarchy tab to locate the com.bea.wlevs.ede.api.StreamSink class.
Under Matching Class, highlight the com.bea.wlevs.ede.api.StreamSink class and click OK.
You return to the Create Java Class dialog.
In the Create Java Class dialog, click OK.
Oracle JDeveloper adds the TradeListener JavaBean to the project under the Application Sources folder. The stub code displays in the Oracle JDeveloper middle panel.
package tradereport; import com.bea.wlevs.ede.api.EventRejectedException; import com.bea.wlevs.ede.api.StreamSink; public class TradeListener implements StreamSink { public TradeListener() { super(); } @Override public void onInsertEvent(Object object) throws EventRejectedException { // TODO Implement this method } }
In the TradeListener
class, edit the onInsertEvent
method as follows:
@Override public void onInsertEvent(Object event) throws EventRejectedException { if (event instanceof TradeEvent){ String symbolProp = ((TradeEvent) event).getSymbol(); Integer volumeProp = ((TradeEvent) event).getVolume(); System.out.println(symbolProp + ":" + volumeProp); } }
The onInsertEvent
method listens for trade events, and when it hears a TradeEvent
, it calls the tradereport.TradeEvent
get
methods to get the stock symbol and the trade volume, and to print the stock symbol and trade volume information to the console.
Close the TradeListener.java tab and save the file.
Add the Event Sink to the EPN Diagram as an Event Bean
Note:
There are no configuration file entries for the channel beyond the default configuration. You can edit the processor.xml
file to customize the channel configuration or create a separate configuration file, such as channel.xml
, for channels and add custom channel configuration to it. See Add Configuration Settings to a Component.
Next add an Oracle CQL processor to filter events based on certain criteria. The Oracle CQL processor goes between AdapterOutputChannel and an output channel that you create in the next section.
The Oracle CQL processor contains Oracle CQL code that you write. The Oracle CQL code queries the events sent to the processor from AdapterOutputChannel. The query retrieves only those trade events that have a volume that is greater than 4000. Oracle Stream Analytics passes the retrieved events to the output channel, which then sends the events to ListenerBean for processing. Recall that ListenerBean listens for trade events, gets the stock symbol and trade volume information, and prints the stock symbol and trade volume information to the console.
The CQL query selects the symbol and volume properties from each incoming trade
event, tests the volume property for a value higher than 4000, and outputs a set of 1 qualifying event at a time. The NOW
operator creates a window of time that contains the event that happened at the last tick of the system.
<query id="GetHighVolume"><![CDATA[ select trade.symbol, trade.volume from AdapterOutputChannel [now] as trade where trade.volume > 4000 ]]> </query>
Add a GetHighVolume Processor and Query
To deploy the example application for testing, perform the following actions:
Create an Oracle Stream Analytics Domain
To create a domain, start the Oracle Stream Analytics Configuration wizard:
Start the Configuration Wizard:
On Windows, navigate to \Oracle\Middleware\my_oep\oep\common\bin\ and type config.cmd.
On UNIX, navigate to /Oracle/Middleware/my_oep/oep/common/bin
and type ./config.sh.
The Configuration wizard Welcome screen displays.
On the Welcome screen, click Next.
The Choose Create or Update Domain screen displays.
On the Choose Create or Update Domain screen, select Create a new OEP domain and click Next.
The Create or Update Domain screen displays.
In the Create or Update Domain screen, in the User Name field, enter oepadmin, and enter and confirm the password, welcome1.
Click Next, accept the Configure Server defaults, and click Next.
The Configure Domain Identity Keystore screen displays.
In the Configure Domain Identity Keystore screen, enter and confirm the password welcome1 and click Next.
The Configuration Options screen displays.
In the Configuration Options screen, click Next to not perform any optional configuration.
The Create OEP Domain screen displays.
In the Create OEP Domain screen, enter basicapp_domain and make a note of its location.
The location will be something like /Oracle/Middleware/my_oep/user_projects/domains
.
Click Create, and after a few moments, click Done.
Start the Oracle Stream Analytics Server
Go to /Oracle/Middleware/my_oep/user_projects/domains/basicapp_domain/defaultserver
.
Execute the appropriate startup script:
On Windows:
prompt> startwlevs.cmd
On UNIX:
prompt> ./startwlevs.sh
The terminal window displays messages as the server starts. When you see, <The application context for "com.bea.wlevs.dataservices" was started successfully
>, the Oracle Stream Analytics server is ready.
Create an Oracle Stream Analytics Server Connection
Select File > New > From Gallery.
The New Gallery dialog displays.
In the New Gallery dialog under Categories > General, select Connections.
In the New Gallery dialog under Items, select OEP Connection, and click OK.
The Create OEP Server Connection dialog displays.
In the Create OEP Server Connection dialog, provide the following information:
Connection will be created in: IDE Connections: Selected Remote OEP Server: Not checked OEP Server Connection Name: OEPBasicAppConnection OEP Server Home Path: /Oracle/Middleware/my_oep/ Use Default Values: Unchecked. OEP Server Projects Directory: user_projects/domains/ basicapp_domain/defaultserver Use Default Values: Checked Host: 127.0.0.1 Port: 9002 Use Default Values: Unchecked Username: oepadmin User Password: welcome1 Additional Parameters for OEP Server: blank
In the Create OEP Server Connection dialog, click Test Connection.
If you see Success in the area below the Test Connection button, you entered the information correctly. If you see errors, correct them and test again until you see Success.
When you see the Success message, click OK.
Create a Deployment Profile
Right-click the TradeReport project and select Deploy > New Deployment Profile.
The Create Deployment Profile dialog displays.
In the Create Deployment Profile dialog, provide the following values.
Profile Type: OEP Project Deployment Profile.
Note:
Make sure you select the correct Profile Type, which is OEP Project Deployment Profile.
Deployment Profile Name: basicapp_profile.
Click OK.
The Deployment Properties dialog displays.
In the Deployment Properties dialog, verify the information:
Connection to Local OEP Server: OEPBasicAppConnection (127.0.0.1:9002) Symbolic Name: TradeReport.TradeReport Bundle Name: TradeReport.TradeReport Bundle Version: 1.0.0 OSGi JAR file: /home/<username>/jdeveloper/mywork/TradeReport/TradeReport/deploy/basicapp_profile.jar.
Click OK.
Deploy the Application
Right-click the TradeReport project.
The context menu displays.
In the context menu, select Deploy > basicapp_profile.
The Deployment Action dialog displays.
In the Deployment Action dialog, select Deploy OSGi bundle to target platform.
Click Next.
The Summary dialog displays.
In the Summary dialog, confirm the information.
Click Finish.
In the Deployment - Log panel at the bottom of the middle panel, messages indicate the successful deployment.
In the Resources window on the right side under IDE Connections, navigate to OEP Server > OEPBasicAppConnection > Applications.
The BasicApplication.BasicApp[Running} connection displays.
A deployment profile creates an OSGi bundle that contains the required library JAR file.
The load generator enables you to load test data so that you can see how your Oracle Stream Analytics application behaves when it is deployed into production.
Normally, you start the load generator after you deploy the application. However, you can start the load generator before you deploy, but you will get a message that there is no listener on port 9200. The message goes away after you deploy application.
View the Test Data
In the text editor of your choice, open the StockData.csv file included with Oracle Stream Analytics installation.
By default, the file is at the following location:
/Oracle/Middleware/my_oep/oep/utils/load-generator/StockData.csv
.
Take a look at the StockData.csv file, which contains comma-separated values in rows where each row represents a trade.
Note:
The order of the event properties in the StockData.csv
file must match the order of event properties specified in Create the csvgen Adapter and Set Its Properties.
Verify the Load Generator Properties
In the text editor of your choice, open the StockData.prop files included with Oracle Stream Analytics installation.
By default, the files are at the following location:
/Oracle/Middleware/my_oep/oep/utils/load-generator/StockData.prop
.
In the StockData.prop file, verify the following properties:
test.csvDataFile
: The name of the CSV file that the load generator reads. For this example, the value is StockData.csv.
test.port
: The port number to which the load generator sends event data. This should be the port value you specified when you configured the CSV adapter, which is 9200.
test.packetType
: The type of data format that the load generator will handle. For this example, the value is CSV.
The load generator requires the test.csvDataFile
and test.port
properties. The other properties are optional, but you need to set at least test.packetType
so that the load generator knows that your input is in CSV form.
Close the StockData.prop file and save if you made any changes.
Start the Load Generator
Run the load generator with the StockData.prop
properties file:
On Windows:
prompt> runloadgen.cmd StockData.prop
On UNIX:
prompt> ./runloadgen.sh StockData.prop
When you are finished with the example, you can stop the load generator and the Oracle Stream Analytics server.
Stop the Load Generator
Change directory to /Oracle/Middleware/my_oep/oep/utils/load-generator.
Type Ctrl-c
Stop the Server