11 Managing Oracle Real-Time Decisions

Oracle RTD uses the J2EE industry standard Java Management Extensions (JMX) to configure and monitor the operation of Oracle RTD. This chapter describes the architecture of Oracle RTD and how to manage Oracle RTD through the Oracle RTD JMX MBeans.

Oracle RTD uses the J2EE industry standard Java Management Extensions (JMX) to configure and monitor the operation of Oracle RTD. You can use the JMX management tool JConsole to manage Oracle RTD. Third-party JMX implementations may also be used to manage Oracle RTD. This document assumes you are using JConsole.

JMX MBeans manage various aspects of Oracle RTD, including logging and Inline Service configuration. They can also be used to assign Oracle RTD specific permissions to security roles and users. Security roles and users are both managed by the J2EE container.

Oracle RTD includes several Services. All of the Oracle RTD services are described later in this section; from the point of view of managing Oracle RTD, the most significant Oracle RTD services are the following:

Decision Service: Runs Inline Services and integrates to enterprise operational processes.

Learning Service: Maintains analytic, self-learning models that underlie Inline Services.

Query Service: Handles Decision Center and Decision Manager model queries.

Deployments of Oracle RTD are often done across multiple servers as well as in clusters to enhance performance in high transaction environments. A relational database is used by each of these Services for retention of code, transactional data and configurations.

This chapter contains the following topics:

11.1 Oracle RTD Architecture

This section describes the terms and options used for Oracle RTD components and the management of Oracle RTD processes. This section includes the following topics:

11.1.1 Definition of an Oracle RTD Cluster

An Oracle RTD cluster is defined as the set of all running RTD.ear instances (Oracle RTD applications) using a common database instance through the data source registered under the JNDI name SDDS. It is a collection of RTD.ear instances, with hub-and-spoke coordination from the database.

11.1.2 Terminology

Node = a running RTD.ear application (also referred to as an Oracle RTD server or Oracle RTD instance)

Service = a class of objects on the cluster providing a given service interface, for example, the Learning Service

Service instance = an instance of a Service type or class

Service acronyms (used throughout this chapter):

  • BA = Batch Agent

  • BM = Batch Manager

  • DS = Decision Service

  • LS = Learning Service

  • QS = Query Service

Java API = the published Java API for ILS authors unless otherwise specified

Cluster stack = the Java code and data structures implementing Oracle RTD Services and remote procedures calls (RPCs)

Application = an Inline Service (also known as an ILS)

Platform Java = the Oracle RTD system exclusive of Application Java

11.1.3 Oracle RTD Cluster, Nodes and Services

An Oracle RTD cluster has one or more Nodes (JVM processes, each running an RTD.ear application instance).

Each node offers one or more Services, with Service life-cycle managed by the Cluster stack of the node.

11.1.4 Java EE Clusters and Their Relationship to Oracle RTD Clusters

Any number of Java EE clusters may overlap the Oracle RTD cluster, under the following conditions:

  • All Oracle RTD cluster nodes receive the same SDDS data source definition

  • All Oracle RTD cluster nodes are in the same security realm (Java EE users and roles)

Any Java EE cluster strategy for server availability may be used:

  • Passive secondaries may be used provided that the passive secondary has the same JVM system properties as the primary

  • Server-down detection and automatic reboot by the Java EE cluster may be used

11.1.5 Required JVM System Properties for Oracle RTD Cluster Participation

Each node in an Oracle RTD cluster must be assigned two invariant JVM system properties, which must be unique to the node across the cluster:

rtd.instanceName - Name of the node. Only the following characters are allowed: alphanumerics, underscore, dash, period.

rtd.HTTPHostAndPort (in the format <host>:<port>) - RPC host and port for this node.

11.1.6 Scale Out

Addition of a node to the Oracle RTD cluster can be done at any time without interrupting existing nodes or Services. The new node must fulfill the following conditions:

11.1.7 Scale Back and Fail-Over

Removal of a node from the cluster, either through shutdown of the application server or removal of the RTD.ear deployment on an application server, can be performed at any time without interruption to the other nodes.

If the node removed was the active LS and/or BM, another node on which the service is enabled will be activated by the system, ensuring high availability. The same will happen if a node goes down unexpectedly. If no other server has LS or BM enabled, the cluster will have no active LS or BM instance going forward.

11.1.8 Overview of Oracle RTD Services

There are five Oracle RTD services that can be enabled and disabled by Oracle RTD system administrators:

  • Learning Service (LS) - The Oracle RTD learning service in which all learning and prediction model creation takes place.

  • Decision Service (DS) - The integrative service facade for data capture and decisioning.

  • Batch Manager (BM) - The coordinator for batch operations.

  • Batch Agent (BA) - Both the API entry point to the batch sub-system and the executor for batch jobs.

  • Query Service - Serves model queries utilizing the currently active and archived models (functionality that was provided in previous releases by the Learning Service).

In addition, there are two implicit Oracle RTD services that are always enabled and active on all nodes:

  • Workbench (also known as Management Service) - the endpoint for ILS deployment from Decision Studio

  • Decision Center - the web application for the /ui reporting web application of Oracle RTD

Enabling and Disabling the Oracle RTD Services

Each node offers five Oracle RTD Services (BA, BM, DS, LS, and QS) that may be enabled or disabled via a JVM system property or JMX (via the Node MBean).

These five Oracle RTD Services are enabled by default on all nodes. Each node also hosts implicit system services that are always available.

An Oracle RTD Service may offer an MBean for configuration of properties of cluster scope, applicable to all Oracle RTD Service instances.

An Oracle RTD Service instance may also register its own MBean locally for performance metric visibility or instance configuration.

Each service instance exists in one of three states: disabled, enabled, or active.

The active state is always set by the system.

A user may define, per node, whether the service is disabled or enabled. If enabled, the system will decide whether to make the service active, as detailed later in this section. Enablement is defined by system property and can be overridden via JMX.

There are two kinds of services:

  1. Single-instance

    A single-instance service can only be active on one node at any given time. Servers on which a single-instance service is enabled compete to be the one node where it is active.

  2. Multi-instance

    A multi-instance service can be active on any number of nodes in a cluster. Such a service is activated by the system if it is enabled on a node.

The five standard Oracle RTD Services are classified as either single-instance or multi-instance as follows:

  • Learning Service (LS): Single-instance service

  • Decision Service (DS): Multi-instance

  • Batch Manager (BM): Single-instance

  • Batch Agent (BA): Multi-instance

  • Query Service (QS): Multi-instance

Notes on the Extent of Oracle RTD Service Enablement

Learning Service (LS) - There exists at most one active instance in the cluster. Enablement is defined by a system property and JMX. Servers on which the LS is enabled compete to be the active LS. Only one can be active at any point in time.

Decision Service (DS) - Any number of active instances may exist on the cluster (one instance per node). Enablement is defined by a system property and JMX. This service is made active (by the system) if it is enabled on a node.

Batch Manager (BM) - There exists at most one active instance in the cluster. Enablement is defined by system property and JMX. Servers on which the BM is enabled compete to be the active BM singleton. Only one can be active at any point in time.

Batch Agent (BA) - Any number of active instances may exist on the cluster (one instance per node). Enablement is defined by a system property and JMX. This service is made active (by the system) if it is enabled on a node.

Query Service (QS) - Any number of active instances may exist on the cluster. Enablement is defined by a system property and JMX. This service is made active by the system if it is enabled on a node.

[Note that the implicit services WorkBench and Decision Center are always active on all nodes.]

Service Dependencies

All services can be independently enabled or disabled on any node, using JMX or system properties.

However, both the DS and the Workbench services require BA enablement if an ILS with a batch job definition is deployed.

Note:

If you want to create and deploy an ILS with a batch job, ensure that the BA is enabled on all DS nodes (and any other node to which the ILS's are deployed).

11.1.9 Further Information About Oracle RTD Services

This section provides more details about some of the Oracle RTD Services:

11.1.9.1 Learning Service Details

Learning throughput of the learning records is now increased through the implementation of the following:

  • Parallel model learning

    When learning occurs at the end of an integration point request, or at the close of the Oracle RTD session, such that more than one model is to be updated by the learning, the updates to the models occur in parallel.

  • Read-ahead caching of learning records

    The learning server maintains an active read-ahead queue of learning records to incorporate into its learning models. It is continuously updating this queue of records from the database, in order to offer the learning algorithms the next record as fast as possible.

  • Asynchronous deletion of processed learning records

    After learning has occurred on learning records, those learning records are expunged from the database in an asynchronous manner that prevents any decrease in learning throughput in the learning server.

11.1.9.1.1 Configuration of the Learning Server

The Java system properties listed in this section direct the behavior of the learning server. They all have default values, and typically you do not need to modify these values. If you want to change any values, you must set the new values on the Learning Server JVM before starting the JVM.

1. rtd.numCPUs - to specify the number of threads to use in the parallel learning thread pool. This is recommended to be less than or equal to the number of CPU cores on the machine running the LS instance. Default: The LS instance detects the number of CPU cores on its machine and defaults the thread count to this number. You do not typically need to modify this setting.

For hardware with a very large number of CPU cores, careful testing may determine that a number smaller than the maximum number of CPU cores may achieve higher throughput.

2. rtd.LearningDispatcher.LearningDurationMS - duration of the recurring main task, during which learning records are processed. Default is 15000 (15 s).

You do not typically need to modify this setting.

3. rtd.LearningDispatcher.LearningSleepMS - sleep time between executions of the recurring main loop task. Default is 100 (100m s).

You do not typically need to modify this setting.

4. rtd.LearningDispatcher.ProcessingSize - number of Learning Records to process as a group before checking for timeout in the "Learn Learning Records" sub-task of the main loop task. Default is 20. The LS will process this many Learning Records before checking whether the timeout set in #2has been met.

You do not typically need to modify this setting.

5. rtd.LRQueueCapacity - the capacity of the LS queue the pre-reads Learning Records from the database (from the SDLearning table). Default is 10,000.

Recommendation: set to ~5% of expected learning records generated per hour. That is, if the system is expected to generate 1,000,000 learning records/hour, set the Queue capacity to 50,000.

6. rtd.LRQueueRefillRatio - Refill Ratio of the LS queue - this is the ratio of queue size/capacity that induces launch of a new reader task when size is reduced by 1 in Learning Record processing. Default is 0.5 (i.e. when polling removes a Learning Record such that size < ¾ capacity, a reader task is launched if one is not already active).

You do not typically need to modify this setting.

7. rtd.LRQueueSleepIntervalSec - Sleep time, in seconds, for the LS queue reading task. If the reading task encounters an empty SDLearning table, it sets a sleep time for the next launch of itself - on the next launch, it sleeps for this time before resuming. Default is 10 sec.

You do not typically need to modify this setting.

11.1.9.2 Query Service Details

Studies can be assigned to QS instances via JMX (and not through a JVM system property). One study can be assigned to more than one instance. This is to enable Oracle RTD systems with asymmetric study weights (that is, with one or a few very large and important studies and a number of smaller or less important studies) to dedicate appropriately sized hardware to serving queries for large studies.

Note that the setting of study assignments only through JMX prevents the need to restart any server in the QS array. Study assignment is a cluster-wide setting - that makes it possible to manage all assignments throughout the cluster from a single node (any node in the QS array). The assignment persists beyond the start and stop of each QS instance.

A QS instance is designated a "universal processor" if it has no specific study assignment. In this case it can potentially handle queries for any study. A QS instance that is explicitly assigned a study is called a "dedicated processor."

The preference for processing a query related to a given study is given to a dedicated QS instance that is explicitly assigned that study.

The cluster maintains a partial mapping of studies to the instances that should process queries pertaining to those studies, and monitors which is the "current processor" for a a study.

There is a strong preference for "dedicated over universal" processing: if a universal processor is currently handling queries for a study (because no dedicated processor was previously available), and a dedicated processor comes online, all subsequent queries for that study are directed to the dedicated processor. If several processors dedicated to that study become available, the QS instance with the highest available memory among the dedicated processors for the study will handle the query.

When there is no current processor for the study, it will be processed by the QS node with the highest available memory among the dedicated processors for that study; if no dedicated processors for it exist a universal processor with the highest amount of available memory will be selected to handle the query. The handler of the query will be designated as the current processor for the query.

Upon receipt of a collection of queries, a QS instance determines, for each query, which QS instance should handle that query, as described previously. If the determined QS instance happens to be the current instance, no forwarding occurs. In theory, a collection of queries can therefore be dispatched to multiple QS instances. In practice, however, a collection of queries typically will pertain to one study and therefore be handled by a single QS instance, local or remote.

The assignment of studies to QS instances is achieved through the JMX Query Service MBean, which exposes the list of all QS instances in the cluster. System administrators can add a comma-separated list of study names beside each QS instance name to make study assignments. The keyword "*" indicates that the QS instance can service all queries. "*" is the default for all QS instances. For more information, see Section 11.4.11, "About OracleRTD > Cluster > Query Service."

11.2 Overview of Oracle RTD and System Properties

All configuration in Oracle RTD is exposed in MBeans.

Types of properties

  • Cluster-scoped properties pertain to the cluster as a whole. They apply to all nodes. Each node sees and uses the same value for each cluster-scoped property.

  • Cluster-scoped properties which are locally overrideable have cluster-level defaults that apply to each node unless they are overridden on that node.

  • Node-scoped properties apply to a given node only and do not have a cluster-wide default.

  • Node-scoped immutable properties are set as system properties and may be exposed as read-only MBean properties.

Any property that may be defined locally for a node may be defined by JVM system property. All RTD JVM system properties now conform to the syntax "rtd.X", with the name following the "rtd." prefix. Any property set by JVM system property is invariant and node-scoped, with 1 exception - service enablement (see below)

Overrides through JMX are persisted and thereby maintain their values through node restarts.

Service Enablement

All 5 services (DS, LS, BA, BM, QS) are enabled by default on all nodes in the cluster.

Any service may be disabled at node boot by setting the JVM system property "rtd.<name>Enabled=false" where name is the (concatenated) name of the service ("LearningService", "BatchAgent", "BatchManager", "DecisionService", or "QueryService").

The JVM system property defines the service enablement state at node boot only. This state may be updated via JMX at any time. For example, a node on which the DS was disabled can have the DS enabled via JMX (in the Node MBean, see Section 11.4.12, "About OracleRTD > Cluster > [Node].").

This provides for service migration across the cluster without node restarts. Service enablement state is not persisted - at node reboot, the service is again enabled by default, unless overridden with a JVM system property.

11.3 Accessing JConsole

The steps to access JConsole depend on which application server you are using.

This section contains the following topics:

11.3.1 Accessing JConsole for WebLogic

To access JConsole for WebLogic:

  1. Run JDK_HOME/bin/jconsole.

    For example:

    C:\Program Files\Java\jdk1.7\bin\jconsole.exe

    If you have more than one JDK on your computer, it does not matter which JDK_HOME you use. You can run JConsole from any JDK.

    Note:

    If you are running JConsole on UNIX, you must first add JAVA_HOME/bin to the beginning of the PATH operating system environment variable.

  2. Select Remote Process.

  3. In the Remote Process box, enter (using the format <hostname>:<port>) - the name of the host where Real-Time Decision Server is running and the JMX remote port number.For User Name and Password, enter the credentials you specified when you set up JConsole. See Section 3.1.5, "Configuring Oracle RTD Security in WebLogic" for more information.

11.3.2 Accessing JConsole for WebSphere

To access JConsole for WebSphere:

  1. On the computer where WebSphere is installed, run the batch script to start JConsole. See Section 3.12.2, "Setting Up a Batch File for JConsole" for more information.

  2. Log in as the JConsole user you created when you set up JConsole (for example, jmx_admin).

    Note:

    When Oracle RTD is running on WebSphere, JConsole must be run on the same computer as the WebSphere server, by default. This requirement is due to WebSphere limitations.

11.4 About JMX MBean Operations and Attributes

Oracle RTD MBeans can be accessed through the OracleRTD folder in the JConsole MBeans tab. These MBeans can be used to manage various aspects of Oracle RTD. Each MBean consists of attributes and operations that can be used for informational and administration purposes.

The OracleRTD MBeans break down into the following top-level categories:

Figure 11-1 shows an example of the OracleRTD MBeans as they appear in JConsole (the specific attributes displayed there are for the Decision Service Instance MBean, which appears under the Local Node category).

Figure 11-1 Example of OracleRTD MBeans

This image is described in the surrounding text.

11.4.1 Cluster Category

The Cluster category contains:

  • The Service type MBeans: Batch Agent, Batch Manager, Learning Service, Decision Service, and Query Service - described in later sections.

  • The Cluster Properties MBean, providing access to all cluster-scoped non-service properties.

  • The Security Manager MBean, providing cluster-wide security configuration.

  • One MBean for each node in the cluster, termed a "Node" MBean. Shown in Figure 11-1 are two: RTD_server1 and RTD_server2.

11.4.2 Inline Services Category

The Inline Services category contains:

  • The Inline Service Manager, providing a cluster-wide view of the deployed ILS's on the cluster.

  • One MBean for each deployed ILS. Figure 11-1 shows the MBean for the CrossSellBatchTest ILS.

11.4.3 Learning Server Category

The Learning Server category contains:

  • One MBean for each study, which displays a list of the populated models for the study.

11.4.4 Local Node Category

The Local Node category contains:

  • The Batch Agent Instance MBean, if the BA is running on the node. This MBean provides access to instance-specific properties.

  • The Decision Service Instance MBean, if the DS is running on the node. Figure 11-1 shows an example of a Decision Service Instance MBean. The properties in blue font are editable.

  • The Node Properties bean, providing access to node-scoped non-service-instance properties.

11.4.5 Study Category

The Study category contains:

  • One MBean for each populated model in each study.

11.4.6 About OracleRTD > Cluster > Batch Agent

The Batch Agent service MBean reveals the locations of the running Batch Agent instances.

This image is described in the surrounding text.

The Cluster > Batch Agent MBean has the following attributes:

Attribute Description

Locations

Locations of the running Batch Agent instances.

11.4.7 About OracleRTD > Cluster > Batch Manager

The Batch Manager service MBean displays all batch names, all active batch jobs in the cluster, and the location of the Batch Manager singleton instance.

This image is described in the surrounding text.

The Cluster > Batch Manager MBean has the following attributes:

Attribute Description

ActiveBatches

List of brief status information for all batch jobs currently running, paused, or waiting to run. The list could be empty.

BatchNames

List of batches registered with the batch framework.

Location

Location of the Batch Manager singleton instance.

11.4.8 About OracleRTD > Cluster > Cluster Properties

The Cluster Properties MBean provides access to the following updateable cluster-wide properties:

This image is described in the surrounding text.

The Cluster > Cluster Properties MBean has the following attributes:

Attribute Description

LoggingLevel

The cluster-wide logging level for Oracle RTD platform code. Default: INFO.

This is a cluster-scoped property, set in the Cluster Properties MBean, which may also be overridden at the local level using the Node Properties MBean or JVM system property "rtd.LoggingLevel."

This is a dynamic re-configuration property - restart of the cluster or local node is not required.

If updated, the new value must be one of ERROR, DEBUG, WARN or INFO, ignoring case. Any other input value is ignored.

In previous releases, this was known as SDLogging.Priority.

InlineServiceLoggingLevel

The cluster-wide logging level for logging performed by an ILS using the function logDebug(), logInfo(), and so on. Default: INFO.

This is a cluster-scoped property, set in the Cluster Properties MBean, which may also be overridden at the local level using the Node Properties MBean or JVM system property "rtd.InlineServiceLoggingLevel."

This is a dynamic re-configuration property - restart of the cluster or local node is not required.

If updated, the new value must be one of ERROR, DEBUG, WARN or INFO, ignoring case. Any other input value is ignored.

In previous releases, this was known as SDLogging.InlineServicePriority.

OperationalDataCleanupPeriod

The frequency, in hours (decimals are accepted), at which the system cleans up database tables containing operational data: choice history, statistics.

Default is 2.0.

A cluster-scoped property, not overridden at the local level.

This is a dynamic re-configuration property - restart of the cluster is not required.

WorkerThreadPoolSize

The thread pool size for system tasks (not for normal Integration Point request processing).

This is a cluster-scoped property, set in the Cluster Properties MBean, which may also be overridden at the local level using the Node Properties MBean or JVM system property "rtd.WorkerThreadPoolSize."

Default: 6.

This is not a dynamic re-configuration property - restart of the cluster or local node is required.

The Operations include a restoreDefaultProperties() operation to restore the system defaults for these properties.

11.4.9 About OracleRTD > Cluster > Decision Service

The Decision Service MBean provides access to the following properties:

This image is described in the surrounding text.

The Cluster > Decision Service MBean has the following attributes:

Attribute Description

ApplicationPollingInterval

The frequency, in seconds, that DS polls for new application deployments.

A cluster-scoped property, not overridden at the local level. Updateable in the Decision Service MBean.

Default: 60.

This is a dynamic reconfiguration property - restart of the cluster is not required.

DecisionServiceAddress

The proxy or direct DS node address to which DC directs calls.

A cluster-scoped property, not overridden at the local level.

Updateable in the Decision ServiceMBean.

Default: http://myHost:8080. This is a dynamic reconfiguration property - restart of the cluster is not required.

GenerateJSessionIdCookie

Set to true to have Decision Service associate Integration Point requests with HTTP sessions, thus causing the Web container to generate container-specific session-affinity cookies.

Cluster-scoped.

No local override.

Default: false.

GenerateServerIdCookie

Set to true to have Decision Service supply a cookie named ORA_RTD_DS_ServerID identifying the DS session serving the current Integration Point request.

Cluster-scoped.

No local override.

Default: false.

IntegrationPointMaxConcurrentJobs

The maximum number of concurrent integration point requests in any DS node in the cluster.

A cluster-scoped property, set in the Decision Service MBean, which may also be overridden at the local level using the Decision Service Instance MBean or JVM system property "rtd.IntegrationPointMaxConcurrentJobs."

Default: 0, indicating no maximum.

This is a dynamic reconfiguration property - restart of the cluster or local node is not required.

IntegrationPointQueueSize

The request wait queue size in the DS.

A cluster-scoped property, set in the Decision Service MBean, which may also be overridden at the local level using the Decision Service Instance MBean or JVM system property "rtd. IntegrationPointQueueSize".

Default: 140.

This is a dynamic reconfiguration property - restart of the cluster or local node is not required.

IntegrationPointRequestTimeout

The time out period, in mSec, for an integration point request into any DS node in the cluster.

A cluster-scoped property, not overridden at the local level.

Updateable in the Decision Service MBean.

Default: 2000.

This is a dynamic reconfiguration property - restart of the cluster or local node is not required.

In previous releases, this was known as IntegrationPointGuaranteedRequestTimeout.

Locations

The locations of the running DS instances in the cluster.

ManageSessionAffinity

Whether the cluster manages session affinity with DS request forwarding.

Default is TRUE.

A cluster-scoped property, not overridden at the local level.

Updateable in the Decision Service MBean.

This is a dynamic reconfiguration property - restart of the cluster is not required.

In previous releases, this was known as DSManagesSessionAffinity.

RestrictDSClients

Controls whether the hosts that can send DS requests are restricted to a fixed list of trusted IP addresses.

If false, all clients are allowed.

If true (the default), localhost is implicitly allowed, plus those clients whose IP addresses appear in TrustedDSClients (a semicolon delimited list).

By default,TrustedDSClients is empty, that is, only localhost is allowed as a client.

Default: True.

See also TrustedDSClients.

SessionIdleTimeoutSeconds

The period, in seconds, after which sessions time out with inactivity.

A cluster-scoped property, not overridden at the local level.

Updateable in the Decision Service MBean.

Default: 1800.

This is a dynamic reconfiguration property - restart of the cluster is not required.

In previous releases, the equivalent attribute was known as DSSessionIdleTimeoutMillisec, with value in mSec.

TrustedDSClients

The IP address list (semi-colon separated) of trusted DS clients. localhost is always trusted.

A cluster-scoped property, not overridden at the local level.

Updateable in the Decision Service MBean.

Default: null (no trusted clients other than localhost).

This is a dynamic re-configuration property - restart of the cluster is not required.

See also RestrictDSClients.

The Operations include a restoreDefaultProperties() operation to restore the system defaults for these properties.

11.4.10 About OracleRTD > Cluster > Learning Service

The Learning Service MBean provides access to the following properties:

This image is described in the surrounding text.

The Cluster > Learning Service MBean has the following attributes:

Attribute Description

Location

The location of the running LS instance in the cluster.

StudyNames

A list of all the actively learning studies running on the Learning Server.

Note: You can view the models of a study by clicking on the study under OracleRTD> Study > [Study Name].

The Cluster > Learning Service MBean has the following operations:

deleteStudy()

Deletes this study.

StudyName The name of the study.

saveStudy()

Saves this study.

StudyName The name of the study.

saveAllStudies()

Saves all studies.

blockingModelBuild()

Process all outstanding learning records, then create and save prediction and learning models.

11.4.11 About OracleRTD > Cluster > Query Service

The Query Service MBean provides access to the following properties:

This image is described in the surrounding text.

The Cluster > Query Service MBean has the following attributes:

Attribute Description

Locations

The locations of the running QS instances in the cluster.

ModelSnapshotDSName

The JNDI name of the data source for the model snapshots.

Default: SDDS.

A cluster-wide property, not overridden at the local level.


If updated, the new value is used only after a restart of the Query Service.

ModelSnapshotMinimumAbsoluteCorrelation

Controls whether to dump all correlation rows or to set a minimum correlation value for dumping.

A cluster-wide property, not overridden at the local level.

The default value of 0.000001 prevents the dumping of very small value correlation rows. Set the value to 0 to dump all correlation rows.

If updated, the new value is used on the next model snapshot.

ModelSnapshotNumberOfBins

Controls the number of bins for model snapshots. Numeric attribute values are automatically binned, or assigned to numeric ranges. The default number of bins is 5. To achieve greater resolution of your numeric data, increase the number of bins.

A cluster-wide property, not overridden at the local level.

If updated, the new value is used on the next model snapshot.

Note that for the same numeric attribute, Oracle RTD creates different bins in different time windows. Therefore, it is unlikely that you will be able to join numeric attribute values across time windows.

StudyAssignment

Comma-separated list of QS instances and studies assigned to them (format "Server1=Study1,Study2;Server2=Study3").

The dummy study value "*" indicates that the QS instance can service all queries. "*" is the default for all QS instances.

For more information, see Section 11.1.9.2, "Query Service Details."

The Cluster > Query Service MBean has the following operation:

incrementalSnapshot()

Saves a snapshot of this study to the database (delta).

completeSnapshot()

Saves a snapshot of this study to the database (complete).

deleteSnapshot()

Removes this study's snapshot from the database.

restoreDefaultProperties()

Restores the system defaults for this MBean's properties.

11.4.12 About OracleRTD > Cluster > [Node]

Each Node in the cluster has a representative Node MBean, for example, the MBean for node RTD1:

This image is described in the surrounding text.

The Service states for each of the Oracle RTD Services (Batch Agent, Batch Manager, Decision Service, Learning Service, Query Service) on the node are displayed.

The HostAndPort of the node is displayed. This is the node's value of the invariant and required JVM system property rtd.HTTPHostAndPort.

The SSLEnabled attribute indicates whether the port specified in HostAndPort is SSL enabled. This is the value of the invariant system property rtd.SSLEnabled, which defaults to false.

The Operations of the MBean permit enablement and disablement of each of the Service types on the node, as shown:

This image is described in the surrounding text.

The Cluster > [Node] MBean has the following operations:

  • enableLearningService()

  • enableDecisionService()

  • enableBatchManager()

  • enableBatchAgent()

  • enableQueryService()

Each of these operations has the Enable parameter.

This parameter can be set to true or false to enable or disable the appropriate Oracle RTD Service.

11.4.13 About OracleRTD > Cluster > Security Manager

The Cluster > Security Manager MBean has the following operations:

assignPermission()

Assigns the specified cluster permission. See Section 5.4, "Assigning Permissions" for more information. This operation accepts the following parameters:

Role: The role to which you want to assign permissions.

permCode: The code of the permission.

listDirectPermissions()

Returns the permissions directly assigned to a specific role. This operation accepts the following parameter:

Role: The target role.

listEveryoneHavingDirectPermissions()

Returns the names of the roles that have directly assigned permissions.

listPermissionCodes()

Returns the localized name and integer code for all possible permission types. The codes are used as operands for various operations that require a permission type designation.

removeAllDirectPermissions()

Removes all permissions directly assigned for the specified role. This operation accepts the following parameter:

role: The target role.

removePermission()

Removes the specified permission. This operation accepts the following parameters:

Role: The target role name.

permCode: The code of the permission you want to remove.

revertToStandardPermissions()

Reverts to the standard permissions.

11.4.14 About OracleRTD > Inline Services > InlineServiceManager

Each deployed Inline Service is displayed under the InlineServiceManager MBean.

The Inline Services > InlineServiceManager MBean has the following attribute:

Attribute Description

InlineServices

List of deployed inline services.

The Inline Services > InlineServiceManager MBean has the following operations:

refreshInlineServiceMBeans()

Removes MBeans for applications no longer in the database, and creates MBeans for new ones.

removeAllInlineServices()

Removes all Inline Services (loaded, loadable, failed).

11.4.15 About OracleRTD > Inline Services > [Inline Service] > [Deployment State] > [Flag]

Inline Service MBeans can be viewed by choosing the name of an Inline Service, then choosing a Deployment State (for example, Development, QA, or Production), then choosing a particular flag (Failed, Inactive, or Loadable). For example: OracleRTD > Inline Services > CrossSell > Production > Loadable.

The MBeans for a specific Inline Service have the following attributes:

Attribute Description

DeploymentState

Development, QA, or Production.

Flag

Failed, Inactive, or Loadable.

LockStatus

The lock status for the Inline Service.

ServiceId

The service ID for the Inline Service.

The MBeans for a specific Inline Service have the following operations:

unlockService()

Unlocks this service.

removeService()

Stops an Inline Service in this server and removes the service from the database.

flushStatistics()

Flushes all of the statistics for this service to the database.

deleteStatistics()

Flushes and deletes all of the statistics for this service from the database.

deleteChoiceHistory()

Deletes all of the choice history for this service from the database.

deleteAllOperationalData()

Deletes all of the operational data for this service from the database. This includes choice history, statistics, and the study.

deleteStudy()

Removes the study for this service.

11.4.16 Invoking Maintenance Operations

Maintenance Operations appear in the MBean tree under an Inline Service when both of the following conditions hold:

  • The Inline Service includes one or more Maintenance Operations

  • The Inline Service has the Deployment flag set to Loadable

Each Maintenance Operation appears in both of the BroadcastAsyncOperations and DirectBlockingOperations nodes, under the Loadable entry.

Operations listed in the DirectBlockingOperations node are invoked on the local server only, and they return only after the operation has completed. The returned value will be displayed to the JConsole user in a popup dialog. If the operation has return type "void," then "null" will appear. If the operation fails for any reason, a short error message will be displayed in a JConsole popup dialog, and a more detailed report can be found in the log of that server.

Operations listed in the BroadcastAsyncOperations node are invoked across every node of a cluster. The operation returns immediately with the number of cluster members who received the broadcast. If the cluster has just one node, the operation returns 1, and the invocation is still asynchronous.

The following run-time considerations apply for Maintenance Operations:

  • Oracle RTD does not guarantee that all cluster members are notified of Maintenance Operation invocations, although usually they will be.

    For example, if one member of a cluster is down when a Maintenance Operation is invoked, there is no notification to indicate that the cluster member should run the Maintenance Operation when it comes back up.

  • Ordering of Maintenance Operations is not guaranteed. For example, if two Maintenance Operations A and B are invoked in sequence, an Inline Service may run B before it runs A, or it may even run them simultaneously.

11.4.17 About OracleRTD > Learning Server > [Study]

The Learning Server > [Study] MBeans have the following attributes:

Attribute Description

Models

Models belonging to this study.

Name

The name of the study.

The Learning Server > [Study] MBeans have the following operation:

Delete()

Deletes this study.

11.4.18 Local Node - Batch Agent Instance MBean

The Local Node > Batch Agent Instance MBean displays the running batch jobs and names on the BA instance:

This image is described in the surrounding text.

The Local Node > Batch Agent Instance MBean has the following attributes:

Attribute Description

ActiveBatches

List of all batch jobs currently running on this batch agent, paused, or waiting to run. The list could be empty.

BatchNames

List of batches registered with this batch agent.

11.4.19 About OracleRTD > Local Node > Decision Service Instance

The Local Node > Decision Service Instance MBean has the following locally overrideable attributes:

Attribute Description

CurrentReceivedRequestsForwarded

Number of requests that were forwarded from other servers to this server, and which have not yet been completely processed by this server.

CurrentRequestsForwarded

Number of requests that have been forwarded from this server to other servers, and for which no acknowledgment has yet been received to indicate that the request has been processed by the forwarded-to server.

CurrentRequestsQueued

Number of currently waiting requests, not yet running.

CurrentRequestsRunning

Number of Integration Point requests that are currently being processed by Inline Services.

CurrentSessions

Number of Decision Service sessions still open.

[The general notes (see Description) apply to the following DSPerf... attributes for this MBean.]

General notes about the DSPerf... attributes

  • All are node-scoped properties only, and can be set in the Decision Service Instance MBean or using a JVM system property.

  • All Oracle RTD JVM system properties take the prefix "rtd.", for example rtd.DSPerfCounterAppend.

  • All are dynamic reconfiguration properties - restart of the node is not required.

DSPerfCounterAppend

Whether to append to the DS performance CSV file or not. Default is TRUE.

(See also General notes about the DSPerf... attributes in previous Description.)

DSPerfCounterEnabled

Whether to enable DS performance logging to a CSV file or not. Default is FALSE.

(See also General notes about the DSPerf... attributes in previous Description.)

DSPerfCounterFileRollingEnabled

If set to TRUE, then the filename in DSPerfCounterLogFile is expected to include a date format string, such as RTD1_ds_perf_%yyyy-MM-dd_hh.csv.

At runtime, the date format string will be replaced with current date and time values. The filename will thus change as the hour changes and a new performance log file begins.

The filename MUST include a % if DSPerfCounterFileRollingEnabled is true, indicating the start of the date format string.The date format string can be anything allowed by the Java class SimpleDateFormat (http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html). Note that the date format should not be too granular - for example, if it includes a minute element, the log file will roll every minute.

DSPerfCounterLogFile

File name to write DS performance data into. Default is RTD_HOME/log/ds_perf.csv.

(See also General notes about the DSPerf... attributes in previous Description.)

DSPerfCounterLogFileInterval

The interval at which the DS performance log is written, in mSec. Default is 5000.

(See also General notes about the DSPerf... attributes in previous Description.)

IntegrationPointMaxConcurrentJobs

The maximum number of concurrently executing Integration Point requests. This should normally be set to 0, in which case the value is calculated as follows, where Math.ceil means "round up to the next higher integer:"

NumCPUs * Math.ceil(1/(1-DSRequestIOFactor)) + 5

IntegrationPointQueueSize

The maximum number of Integration Point requests that can wait to execute. If a request tries to exceed this number, the server terminates the request with a "Server Too Busy" error message. This setting should be less than or equal to the number of servlet threads allocated by the servlet container minus the configured or calculated value of IntegrationPointMaxConcurrentJobs.

MaxAllowedConcurrentRequests

Maximum number of requests that can run concurrently.

NumCPUs

Number of CPUs in this host.

PeakReceivedRequestsForwarded

Largest number of received requests forwarded.

PeakRequestsForwarded

Largest number of requests forwarded.

PeakRequestsQueued

Largest number of requests that have had to wait at any one time.

ReceivedRequestsForwarded

Total number of requests that were forwarded from other servers to this server.

RequestQueueCapacity

Maximum number of requests that can wait concurrently.

RequestsForwarded

Total number of requests forwarded to another server in the cluster.

RequestsQueued

Total number of requests that have had to wait before running.

RequestsWhenQueueFull

Total number of requests rejected because the request queue was full.

TimedOutRequests

Total number of requests that have timed out.

TotalRequests

Total number of requests seen since the server started.

TotalSessions

Total number of Decision Service sessions created.

11.4.20 About OracleRTD > Local Node > Node Properties

The Local Node > Node Properties MBean has the following node-scoped properties:

This image is described in the surrounding text.

The LoggingLevel, InlineServiceLoggingLevel and WorkerThreadPoolSize properties override, for the local node, those defined in the Cluster Properties MBean for the local node.

LogFile - The log file to use.

Default log file name is:

  • <RTD_HOME>/log/<instance_name>/rtd_server.log

This is a node-scoped property only, which may be set in the Node Properties MBean or using a JVM system property.

Updates do not require a node restart.

The previous name for this property was SDLoggingFileName.

The Operations include a restoreDefaultProperties() operation to restore the system defaults for these properties.

11.4.21 About OracleRTD > Study > [Study Name] > [Model]

The Study > [Study Name] > [Model] MBeans have the following attributes:

Attribute Description

Attributes

Names of the model attributes. The names listed here match the attribute names in the session for your Inline Service.

Name

The name of the model.

TimeWindows

List of the ranges of time that have been learned about by this particular model.

The Study > [Study Name] > [Model] MBeans have the following operations:

Delete()

Deletes this model.

DeleteAttributeValue()

Erases model data collected for a value of an attribute. This operation accepts the following parameters:

AttributeName: The name of an attribute.

Value: The value to be deleted.

DeleteAttributeValueRange()

Erases model data collected for a range of values of an attribute. This operation accepts the following parameters:

AttributeName: The name of an attribute.

HighestValue: The highest value to be deleted.

LowestValue: The lowest value to be deleted.

StartNewTimeWindow()

Closes the current time window and starts a new one. Do not use this operation in a production environment, because it may impair future model learning.

11.5 Required System Properties and Other Requirements

This section describes mandatory and optional requirements for Oracle RTD systems. Some of these apply to all supported application servers, others are application-server specific.

There are two system properties that are required for all Oracle RTD systems:

  • rtd.instanceName - Must be non-null and unique over the cluster. Only the following characters are allowed: alphanumerics, underscore, dash, period. Any others and the cluster will not start.

  • rtd.HTTPHostAndPort - Must be in <host>:<port> format - non-null and unique over the cluster.

Also required for all systems: the EMF registry must be set:

  • org.eclipse.emf.ecore.EPackage.Registry.INSTANCE=com.sigmadynamics.emf.util.SDEMFRegistry

The following are optional or application-server specific requirements:

  • On Sun JVM on WebLogic systems only, the PermGen size must be at least 256m: -XX:MaxPermSize=256m

  • Required for WebLogic systems only: weblogic.wsee.skip.async.response=true

  • Optional for all systems: sun.reflect.inflationThreshold=2147483647

    This only applies to Decision Servers. If an Inline Service uses many external rules, specifying this parameter may improve their initial load time.