Databases are best equipped to run queries over finite stored data sets. However, many modern applications require long-running queries over continuous unbounded sets of data. By design, a stored data set is appropriate when significant portions of the data are queried repeatedly and updates are relatively infrequent. In contrast, data streams represent data that is changing constantly, often exclusively through insertions of new elements. It is either unnecessary or impractical to operate on large portions of the data multiple times. Many types of applications generate data streams as opposed to data sets, including sensor data applications, financial tickers, network performance measuring tools, network monitoring and traffic management applications, and clickstream analysis tools. Managing and processing data for these types of applications involves building data management and querying capabilities with a strong temporal focus.

To address this requirement, Oracle SOA Suite provides Oracle Complex Event Processing (Oracle CEP), a data management infrastructure that supports the notion of streams of structured data records together with stored relations. Oracle CEP is included with Oracle SOA Suite, but available as a separate download. It is optimized to handle very large volumes of events, such as those found in bank transactions, that cannot be managed by Oracle BAM. In addition, Oracle Complex Event Processing can perform complex correlations and pattern matching.

For a composite application to use Oracle CEP, an Oracle Mediator publishes business events to the Event Delivery Network (EDN). At runtime, the Oracle CEP Service Engine subscribes to these events. The Oracle CEP Service Engine executes a Continuous Query Language (Oracle CQL) query and searches for patterns in event streams. Oracle CQL is a query language based on SQL with added constructs that support streaming data. Using Oracle CQL, you can express queries on data streams to perform complex event processing. The Oracle CEP Service Engine listens on these streams, caches all the necessary individual, seemingly unrelated events and tries to correlate them into specific patterns. The data provided from complex event processing queries can then be used in Oracle BAM.

The potential applications of Oracle CEP are numerous, from electronic trading and risk management to intrusion detection and compliance monitoring.

Specifically, Oracle CEP provides the following functional areas:

Data and Event Sources: An Oracle CEP event source identifies a producer of data that your Oracle CQL queries operate on. Event sources include data feeds such as wire services and stock tickers, sensors such as temperature, motion, or radio frequency identification (RFID) detectors, and other devices. Oracle CEP provides a variety of adapters that connect such real-world event sources to your Oracle CQL queries. Oracle CEP adapters support the following event sources: JMS, HTTP publisher/subscriber, and file.
Context Creation: Oracle CEP offers a variety of sliding window operators and views (subqueries) that allow you to define the temporal or semantic context in which filtering, correlation and aggregation, and pattern matching takes place. Using windows and views, you can define contexts such as events in the last 5 minutes or events for a particular customer, and so on. Oracle CQL provides a variety of sliding windows, including: range-based (time or constant value), tuple-based, and partitioned. In addition, you can easily define custom window operators.
Filtering: Using Oracle CQL, you can specify queries that select on any of the attributes of the events offered by event sources. You use such queries to filter the event sources to obtain events of interest. Oracle CQL provides a rich set of operators, expressions, conditions, and statements for this purpose.
Correlation and Aggregation: Using Oracle CQL, you can perform advanced statistical and arithmetic operations on the attributes of the events offered by event sources. Oracle CQL provides: single-row functions that return a single result row for every row of a queried stream or view; aggregate functions that return a single aggregate result based on a group of tuples, rather than on a single tuple; statistical and advanced arithmetic operations based on the Colt open source libraries for high performance scientific and technical computing; and statistical and advanced arithmetic operations based on the java.lang.Math class. In addition, you can easily define custom single-row and aggregate functions.
Pattern Matching: Using the Oracle CQL MATCH_RECOGNIZE condition, you can succinctly express complex pattern matching operations for a wide variety of tasks such as algorithmic trading, double-bottom detection, non-event detection, and so on. The following example detects if perishable food is exposed to temperatures of 25 C or higher for more than 5 minutes.

 <query id="detectPerish"><![CDATA[
    select its.itemId from ItemTempStream
    MATCH_RECOGNIZE (
        PARTITION BY
            itemId
        MEASURES
            A.itemId as itemId
        PATTERN (A B* C)
        DEFINE
            A  AS  (A.temp >= 25) and ,
            B  AS  ((B.temp >= 25) and
                    (B.element_time - A.element_time < INTERVAL "0 00:00:05:00" DAY TO SECOND)),
            C  AS  ((C.temp >= 25) and 
                    C.element_time - A.element_time >= INTERVAL "0 00:00:05:00" DAY TO SECOND)
    )) as its
]]></query>

Complex Event Sinks: An Oracle CQL event sink identifies a consumer of Oracle CQL query results. That is, a consumer of notable events that Oracle CQL queries have extracted from event sources by executing filtering, correlation and aggregation, and pattern matching within various contexts. Typically, notable events are fewer in number (and much higher in value) than the events offered by event sources. Oracle CEP adapters support the following event sinks: JMS, HTTP publisher/subscriber, file, and event beans. Event beans are Plain Old Java Objects (POJO) that contain the business logic you want executed when certain notable events occur.