|Oracle® Complex Event Processing Getting Started
11g Release 1 (11.1.1)
Part Number E14476-03
This section contains information on the following subjects:
Oracle CEP (formally known as the WebLogic Event Server) is a Java server for the development and deployment of high-performance event driven applications. It is a lightweight Java application container based on Equinox OSGi, with shared services, including the Oracle CEP Service Engine, which provides a rich, declarative environment based on Oracle Continuous Query Language (Oracle CQL) - a query language based on SQL with added constructs that support streaming data - to improve the efficiency and effectiveness of managing business operations. Oracle CEP supports ultra-high throughput and microsecond latency using JRockit Real Time and provides Oracle CEP Visualizer and Oracle CEP IDE for Eclipse developer tooling for a complete real time end-to-end Java Event-Driven Architecture (EDA) development platform.
Oracle CEP has the capability of deploying user Java code (POJOs) which contain the business logic. Running the business logic within Oracle CEP provides a highly tuned framework for time and event driven applications.
Figure 1-1 provides a high level view of an event-driven system.
Figure 1-1 Example Event -Driven System
An event-driven system is generally comprised of several event sources, the real-time event-driven applications, and event sinks. Oracle CEP server and the Oracle CEP applications you deploy to it comprises the event-driven applications. The event sources generate streams of ordinary event data. The Oracle CEP applications listen to the event streams, process these events, and generate notable events. Event sinks receive the notable events.
Event sources, event-driven applications, and event sinks are de-coupled from each other; one can add or remove any of these components without causing changes to the other components. This is a key attribute of event-driven architectures.
Event-driven applications are rule-driven. In Oracle CEP, rules are expressed as queries using the Oracle Continuous Query Language (Oracle CQL). These queries are persisted to a data store and are used for processing the inbound stream of events and generating the outbound stream of events. Queries typically perform filtering and aggregation functions to discover and extract notable events from the inbound event streams. As a result, the number of outbound events is generally much lower than that of the inbound events.
Oracle CEP is a middleware for the development of event-driven applications. An Oracle CEP application is essentially an event-driven application.
Next, consider the application itself, which is hosted by the Oracle CEP server, a light-weight container as shown in Figure 1-2.
Figure 1-2 Oracle CEP Application
An Oracle CEP application is typically composed of the following main component types:
Adapters interface directly to the inbound and outbound stream and relation sources and sinks. Adapters understand the inbound and outbound protocol, and are responsible for converting the event data into a normalized form that can be queried by a processor. Adapters forward the normalized event data into channels or outbound stream and relation sinks.
Beans register to listen to the output channel, and are triggered by the insertion of a new event into the output channel. This user code is generally a plain-old-Java-object (POJO). The user application makes use of a set of external services, such as JMS, Web services, and file writers, to forward the generated events to external event sinks.
Event Beans register to listen to the output channel, and is triggered by the insertion of a new event into the output channel. This user code uses the Oracle CEP event bean API so that the bean can be managed by Oracle CEP.
Adapter - Channel - Business Logic POJO
Scenario: no processing is needed; only adaptation from proprietary protocol to some normalized model.
Adapter - Channel - Processor - Channel - Business Logic POJO
Scenario: straight through processing to user code.
Adapter - Channel - Processor - Channel - Business Logic POJO - Channel - Processor - Channel - Business Logic POJO
Scenario: two layers of event processing; the first processor creates causality between events and the second processor aggregates events into complex (notable) events.
EPNs have two important attributes:
Event processing networks can be used to create a hierarchy of processing agents, and thus achieve very complex processing of events. Each layer of the EPN aggregates events of its layer into complex events that become simple events in the layer above it.
Event processing networks improve integrability, that is, the quality of having separately developed components work correctly together. For example, one can add user code and reference to external services at several places in the network.
The use cases for Oracle CEP span a variety of businesses and applications. Just a few of these diverse use cases include:
Financial: Algorithmic Trading
Automate stock trading based on market movement. Sample query: if, within any 20 second window, StockB rises by more than 2% and StockA does not, then automatically buy StockA.
For an example, see Section 3.10, "Signal Generation Example".
Transportation: Security and Fraud Detection
Discover fraudulent activity by detecting patterns among events. Sample query: if a single ID card is used twice in less than 5 seconds to gain access to a city's subway system, alert security for piggybacking.
Energy and Telecommunications: Alarm Correlation
Reduce false positive alarms. Sample query: When 15 alarms are received within any 5 second window, but less than 5 similar alarms detected within 30 seconds, then do nothing.
Health Care: Patient Monitoring
Monitor the vital signs of a patient and perform some task if a particular event happens. Sample query: When a change in medication is followed by a rise in blood pressure within 20% of maximum allowable for this patient within any 10 second window, alert nearest nurse.
The following list summarizes the main features of Oracle CEP:
New in 11g Release 1 (11.1.1): Oracle Continuous Query Language (Oracle CQL) is a query language based on SQL with added constructs that support streaming data. Using Oracle CQL, you can express queries on data streams to perform complex event processing (CEP) using Oracle CEP. Oracle CQL is scalable and comprehensive. It provides a wide range of operators (including extensive window operators), functions (including built-in, Colt, and
java.lang.Math functions), and statements. Oracle CQL supersedes EPL.
New in 11g Release 1 (11.1.1): Support for Coherence clustering and distributed cache.
New in 11g Release 1 (11.1.1): Oracle CEP Visualizer query constructor, query plan generator, and support for Coherence clustering and distributed cache.
Event Caching—Applications can optionally publish or consume events to and from a cache to increase the availability of the events and increase the performance of their applications.
Event Record and Playback—The event repository feature of Oracle CEP allows you to record events flowing through an EPN and store them so you can later play back the events.
Built-in HTTP Publish-Subscribe Adapters—The three built-in HTTP publish-subscribe adapters allow an application to easily to publish (locally and remotely) and subscribe to an HTTP publish-subscribe server channel.
Built-in JMS Adapters—The two JMS adapters (inbound and outbound) allow you to send and receive messages to and from a JMS queue, respectively, from your application without writing any Java code
Oracle CEP Visualizer—A Web 2.0 application that consumes data Oracle CEP, displays it in a useful and intuitive way to system administrators and operators, and, for specified tasks, accepts data that is then passed back to Oracle CEP so as to change it configuration
Multi-server domains (sometimes referred to as clustering)—Oracle CEP now allows multiple servers to be logically connected together for the purposes of management, and physically connected using a shared User Datagram Protocol (UDP) multicast address and port.
An application server that supports deployment of Plain Old Java applications (POJOs), or Spring applications, for handling large volumes of streaming data with low latency requirements.
Oracle CEP applications are developed and deployed as event driven applications, that is, a set of custom Spring tags is used to define the event processing network in the EPN assembly file, which extends the standard Spring context file, of your application.
The application server contains a set of real time services that include a complex event processor (CEP), adapters, and streams. The server is highly tuned for high message throughput and low latency and deterministic behavior.
The complex event processor is a high performance, continuous query engine for processing high volumes of streaming data. It has full support for filtering, correlation, and aggregation of streaming data from one or more streams.
The Event Processing Language (EPL), an SQL-like language that allows event data from streams to be declaratively filtered, correlated, aggregated, and merged, with the ability to insert results into other streams for further downstream processing. You define the EPL rules either in an XML file that configures the complex event processor or programmatically using APIs. Oracle CQL supersedes EPL.
An Adapter SDK that provides all the tools you need to create adapters that listen to incoming data feeds.
A set of product samples that show both a simple Hello World scenario to get you started and more complex foreign exchange and algorithmic trading scenarios to showcase additional features of Oracle CEP.
A load generator utility that simulates a data feed, useful for testing your application without needing to connect to a live data feed.
A monitoring service that includes pre-built instrumentation for measuring throughput and latency at the component level.
A static and dynamic configuration framework. Static configuration is performed using XML files; dynamic configuration is performed by accessing configuration and runtime MBeans using JMX and with the command-line utility
Oracle CEP is built on the Oracle microServices Architecture (mSA) which uses an OSGi-based framework to manage services provided by modules or feature sets. Oracle mSA provides the following services:
Jetty, an HTTP container for running servlets.
javax.sql.DataSource implementation and thin JDBC drivers for accessing a relational database.
Logging and debugging.
Authentication and authorization security.
For detailed information on the platforms that Oracle CEP supports, see:
You can find the installation program appropriate for your platform here:
http://www.oracle.com/technology/software/products/middleware/htdocs/111110_fmw.html. For more information, see Section 2.2, "Installation Overview".
Oracle CEP IDE for Eclipse is targeted specifically to programmers that want to develop Oracle CEP applications as Figure 1-3 shows.
Figure 1-3 Oracle CEP IDE for Eclipse
The Oracle CEP IDE for Eclipse is a set of plugins for the Eclipse IDE designed to help develop, deploy, and debug Oracle CEP applications.
The key features of Oracle CEP IDE for Eclipse are:
Project creation wizards and templates to quickly get started building event driven applications.
Advanced editors for source files including Java and XML files common to Oracle CEP applications.
Integrated server management to seamlessly start, stop, and deploy to Oracle CEP server instances all from within the IDE.
Event Processing Network (EPN) visual design views for orienting and navigating in event processing applications.
Integrated support for the Oracle CEP Visualizer so you can use the Oracle CEP Visualizer from within the IDE (see Section 1.8, "Oracle CEP Visualizer").
For details, see:
Oracle provides an advanced run-time administration console called the Oracle CEP Visualizer as Figure 1-4 shows.
Figure 1-4 Oracle CEP Visualizer
Using Oracle CEP Visualizer, you can manage, tune, and monitor Oracle CEP server domains and the Oracle CEP applications you deploy to them all from a browser. Oracle CEP Visualizer provides a variety of sophisticated run-time administration tools, including support for Oracle CQL and EPL rule maintenance and creation.
For details, see Oracle CEP Visualizer User's Guide
This version of Oracle CEP includes the following new features:
Section 1.9.1, "High Availability and Scalability"
Section 1.9.2, "Record and Playback of Events"
Section 1.9.4, "Visualizer Administration Console"
Section 1.9.5, "Oracle Continuous Query Language (Oracle CQL)"
In this release, Oracle CEP provides new high availability input and output adapters that allow you to build Oracle CEP applications with varying high availability quality of service options.
"Understanding High Availability" in the Oracle CEP Developer's Guide for Eclipse
"Introduction to Multi-Server Domains" in the Oracle CEP Administrator's Guide.
In this release, Oracle CEP provides new scalability components you can use with or without high availability to help scale applications.
The event repository feature of Oracle CEP allows you to record events flowing through an event processing network (EPN) and store them so you can later play back the events. The event repository is configured per stage, such as a processor or channel.
In this release, Oracle CEP server uses the Berkeley database as the event store.
In this release, Oracle CEP IDE for Eclipse provides:
Support for Eclipse 3.5 and WTP 3.1.
Application library support: see "Managing the Application Library" in the Oracle CEP Developer's Guide for Eclipse.
Configuration or assembly file creation: see "How to Create a Component Configuration File Using Oracle CEP IDE for Eclipse" in the Oracle CEP Developer's Guide for Eclipse.
Creation of local or remote Oracle CEP server
Deploying applications to multi-server domain by group name: "How to Deploy an Application to an Oracle CEP Server" in the Oracle CEP Developer's Guide for Eclipse.
Ctrl-Click hyperlink navigation: see "Hyperlinking" in the Oracle CEP Developer's Guide for Eclipse.
You can delete a node from the Event Processing Network (EPN) by selecting it and clicking the Delete key.
In this release, Oracle CEP Visualizer provides:
Support for the new high availability architecture.
Event trace and injection: see "Testing Applications With the Event Inspector Service and Stream Visualizer" in the Oracle CEP Visualizer User's Guide
Configuration version management: see "Managing Configuration History" in the Oracle CEP Visualizer User's Guide.
Oracle Continuous Query Language (Oracle CQL) is a query language based on SQL with added constructs that support streaming data. Using Oracle CQL, you can express queries on data streams to perform complex event processing (CEP) using Oracle CEP.
In this release, Oracle CQL includes:
Data cartridges: see "Introduction to Data Cartridges" in the Oracle CEP CQL Language Reference.
TABLE clause: see "Data Cartridge TABLE Query" in the Oracle CEP CQL Language Reference.
DIFFERENCE USING clause: see "Detecting Differences in Query Results" in the Oracle CEP CQL Language Reference.
ALIASES element: see "Defining Aliases Using the Aliases Element" in the Oracle CEP CQL Language Reference.
Complex predicates for queries against external relations (both cache and database): now you can create queries that use logical operators (such as
NOT) in a predicate sub-clause and arithmetic operators on the comparison attributes in a sub-clause.
For example, the following query is now supported:
SELECT * FROM S[NOW], MyExternalRelation WHERE S.C1 + S.C2 = MyExternalRelation.C1 * F(S.C1)
For a query that joins against a relational database table, Oracle CQL supports all kinds of complex predicates. For more information, see "Configuring an Oracle CQL Processor Table Source" in the Oracle CEP Developer's Guide for Eclipse.
For a query that joins against a cache, you must observe some restrictions. For more information, see "Creating Joins Against the Cache" in the Oracle CEP CQL Language Reference.
In this release, the Oracle CEP Developer's Guide for Eclipse includes instructions on how to integrate an Oracle CEP application with Web services.
See "Configuring Web Services" in the Oracle CEP Developer's Guide for Eclipse.
In this release, both inbound and outbound JMS adapters can use transacted sessions. That is, you can configure a JMS adapter with a session-transacted child element for both inbound and outbound JMS adapters.
See "Configuring the JMS Adapters" in the Oracle CEP Developer's Guide for Eclipse.
In this release, Oracle CEP includes a new sample: the Oracle Spatial data cartridge sample.
To support clustering and a multi-server environment, the directory structure of the out-of-the-box and sample domains has changed.
Previously, all domain files were located directly under the domain directory, such as
ORACLE_CEP_HOME refers to the top-level installation directory of Oracle CEP.
In this release, the domain directory contains one or more sub-directories that correspond to a single server; this directory contains all the configuration files for that server. For example, the HelloWorld domain contains a single server whose configuration files are located in the following directory:
The new Configuration Wizard for creating domains follows this updated directory structure.
Install Oracle CEP 11g Release 1 (11.1.1).
Run the samples from their respective domains.
Understand how the sample applications have been programmed by viewing the source and configuration files and then building them from their respective source directories.
Create your own Oracle CEP domain.
Create a new Oracle CEP application and deploy it to your new domain.
See "Overview of Creating Oracle CEP Applications" in the Oracle CEP Developer's Guide for Eclipse.
Pose questions and find solutions using the Oracle CEP forum: