13 Managing Oracle Real-Time Decisions

Oracle RTD uses the J2EE industry standard Java Management Extensions (JMX) to configure and monitor the operation of Oracle RTD. This chapter describes how to manage Oracle RTD through the Oracle RTD JMX MBeans.

Oracle RTD systems deployed into WebLogic are managed using the Fusion Middleware Control MBean Browsers in Enterprise Manager.

Note:

Some of the management properties can also be set as system properties. For details, see Section 13.9, "System Properties."

JMX MBeans manage various aspects of Oracle RTD, including logging and Inline Service configuration. They can also be used to assign Oracle RTD specific permissions to security roles and users. Security roles and users are both managed by the J2EE container.

Oracle RTD is comprised of three Services:

Deployments of Oracle RTD are often done across multiple servers as well as in clusters to enhance performance in high transaction environments. A relational database is used by each of these Services for retention of code, transactional data and configurations.

Note:

This chapter deals mainly with the management of Oracle RTD through JMX MBeans in Enterprise Manager. Administrators can also perform certain Inline Service related administrative tasks using web service calls in their own application utilities. For more information, see the section "Administration Web Service" in Oracle Fusion Middleware Platform Developer's Guide for Oracle Real-Time Decisions.

This chapter contains the following topics:

13.1 Accessing the Oracle Real-Time Decisions MBeans

Oracle RTD under WebLogic is a component of Oracle Fusion Middleware. You access the Oracle RTD JMX MBeans through the System MBean Browser that exists within Oracle Enterprise Manager Fusion Middleware Control.

To access the System MBean Browser for Oracle RTD, you must perform the following steps:

  1. Log into Fusion Middleware Control, as described in Section 2.1.1, "Logging into Fusion Middleware Control."

  2. In the left-hand Target Navigation Pane, after selecting Application Deployments, then Internal Applications, right-click the Oracle RTD node (as shown in the following image) or the Oracle RTD node within the cluster or server that contains Oracle RTD.

    Surrounding text describes em01.gif.

  3. Select System MBean Browser.

  4. To access the Oracle RTD MBeans, scroll down the System MBean Browser left-hand pane, and select Application Defined MBeans, then Oracle RTD, then the server where Oracle RTD is deployed.

The following example shows the MBean grouping categories for Oracle RTD:

Surrounding text describes smb01.gif.

For more information about MBeans and MBean Browsers, see Oracle Fusion Middleware Administrator's Guide.

13.2 About JMX MBean Operations and Attributes

Oracle RTD MBeans can be accessed through the OracleRTD folder in the System MBean Browser. These MBeans can be used to manage various aspects of Oracle RTD. Each MBean consists of attributes and operations that can be used for informational and administration purposes. The attributes and operations described in this chapter are specific to Oracle RTD.

13.3 MBeans for Oracle Real-Time Decisions Cluster-Level Management

Management at the cluster level is for items that impact the entire cluster of servers. Note that if you have only one server, there is still cluster-level management.

This section contains the following topics:

13.3.1 About OracleRTD > SDManagement > SDClusterPropertyManager

The SDManagement > SDClusterPropertyManager MBean has the following attributes:

Attribute Description

Cluster

Cluster configuration.

Deployment

Configuration for deployment.

Misc

Miscellaneous properties.

The SDManagement > SDClusterPropertyManager MBean has the following operation:

java.lang.Void restoreDefault()

Restores the default installation settings to Oracle RTD. If this command is run on a cluster, then values are restored to cluster defaults.

13.3.2 About OracleRTD > SDClusterPropertyManager > Misc

The SDClusterPropertyManager > Misc MBean has the following attributes:

Attribute Description

ArchivedModelCacheTimeToLive

Maximum time in seconds an archived model is preserved in memory. The cache of archived models is used by Discovery Explorer.

ArchivedModelCatalogRefreshInterval

Refresh interval in seconds for a catalog of archived models. The catalog is used by Discovery Explorer.

AutoFlushTimeout

Interval in seconds controlling auto flush of database write buffers. Fractional values are supported.

DBOperationLogThresholdMilliSec

All database operations that take longer than the specified threshold are logged.

DCOperationLogThresholdMilliSec

All decision center requests that take longer than the specified threshold are logged.

DSManagesSessionAffinity

Decision Service manages session affinity. When set to true, the decision service maintains a map of active session keys and, if necessary, will forward Integration Point requests to the cluster host owning the key's session. Should be disabled in single-host installations and in installations where session affinity is perfectly managed by the application server or external load balancer.

DSSessionIdleTimeoutMillisec

Decision Service session idle timeout in milliseconds.

DSStrictSessionAffinityConcurrency

How many concurrent DB requests can be outstanding, per Oracle RTD instance, to manage the strict Decision Service session directory. Keep this low for SQL Server to avoid DB-centric deadlocks. 0 means unlimited.

DatabaseComponentCloseTimeoutSeconds

How long to wait for the database provider to close an individual component (for example, each BatchUpdater) before abandoning the effort.

DatabaseShutdownTimeoutSeconds

How long to wait for the database provider to shutdown all its components before abandoning the effort.

DecisionServiceAddress

Must be set for Decision Center to be able to test integration point requests from its Interactive Integration Map when Decision Center is not co-located with Decision Service.

DisableBatchDBOperations

Boolean switch that controls batch database operations.

IntegrationPointGuaranteed
RequestTimeout

Guaranteed response time, in milliseconds, for Integration Point requests. (Service Level Guarantee). Zero means don't timeout Integration Point requests - suitable for debugging only.

ModelDSName

The JNDI name of the datasource used by the Learning Service.

ModelSnapshotDSName

The JNDI name of the datasource for the model snapshots.

ModelSnapshotMinAbsCorrelation

Controls whether to snapshot all correlation rows or to set a minimum correlation value for snapshots. The default value of 0.000001 prevents snapshots of very small value correlation rows. Set the value to 0 to snapshot all correlation rows.

ModelSnapshotNumberOfBins

Controls the number of bins for model snapshots. Numeric attribute values are automatically binned, or assigned to numeric ranges. The default number of bins is 5. To achieve greater resolution of your numeric data, increase the number of bins.

Note that for the same numeric attribute, Oracle RTD creates different bins in different time windows. Therefore, it is unlikely that you will be able to join numeric attribute values across time windows.

SystemDSName

The JNDI name of the datasource.

WorkerThreadPoolSize

The number of threads used for general purpose maintenance activities, not for normal Integration Point request processing. Maintenance activities include model maintenance, session timing, and timed-out request processing.

13.3.3 About OracleRTD > SDClusterPropertyManager > Cluster

The SDClusterPropertyManager > Cluster MBean has the following attributes:

Attribute Description

BatchManagerInitialWait

The number of milliseconds to wait when the server first starts up before trying to start the Batch Manager Service.

BatchManagerRestartWait

The number of milliseconds to wait after a computer fails or leaves the cluster before trying to restart the Batch Manager Service.

ChoiceHistoryCleanupChunkSize

The chunk size to use when deleting old choice history records.

ChoiceHistoryCleanupThrottle

A number between 0.1 and 1, inclusive. Higher throttle corresponds to higher speed.

GenerateDSCookies

Generate Decision Server HTTP Cookies. Set to true to have Decision Server associate Integration Point requests with HTTP sessions, thus causing the Web container to generate container-specific session-affinity cookies.

GenerateDSSessionIdCookie

Generate Decision Service Session-ID Cookies. Set to true to have Decision Service supply a cookie named ORTD_DS_SessionID identifying the DS session serving the current integration point request.

LearningDataStorageCleanupChunkSize

The chunk size to use when deleting old learning data storage records.

LearningDataStorageCleanupThrottle

A number between 0.1 and 1, inclusive. Higher throttle corresponds to higher speed.

LearningServiceInitialWait

The number of milliseconds to wait when the server first starts up before trying to start the Learning Service.

LearningServiceRestartWait

The number of milliseconds to wait after a computer fails or leaves the cluster before trying to restart the Learning Service.

OperationalDataCleanupPeriod

The number of hours (fractions are allowed) between cleanup of the operational data (choice history, statistics, learning data storage) in the database.

RequireIntegrationPointAuthorization

True if Integration Points can only be called by security principals granted the \\"decision_service\:normal\\" action on the containing Inline Service.

RestrictDSClients

True if the hosts that can send Decision Service requests is restricted to a fixed list of trusted IP addressees.

StatisticsCleanupChunkSize

The chunk size to use when deleting old statistic records.

StatisticsCleanupThrottle

A number between 0.1 and 1, inclusive. Higher throttle corresponds to higher speed.

TrustedDSClients

List of host IP addresses from which Decision Service requests will be accepted. Port is optional, separated from IP by ':'. Entries are separated by ';'.

Adding Trusted Decision Service Clients

If any host can send Decision Service requests, set RestrictDSClients to False (default value is True).

If you want to restrict the hosts allowed to send Decision Service requests hosts:

  • Set RestrictDSClients to True

  • For TrustedDSClients, enter a list of the host IP addresses of the client hosts allowed to send Decision Service requests

This change should be propagated to all existing instances, and should be effectively immediately.

13.3.4 About OracleRTD > SDClusterPropertyManager > Deployment

The SDClusterPropertyManager > Deployment MBean has the following attribute:

Attribute Description

AppPollingInterval

How frequently, in seconds, the AppFactory polls the SDApps table to see if there are new apps.

13.3.5 About OracleRTD > SDCluster > SDManagement

Attributes and operations at the SDCluster level are meant to manage cluster-level features.

The SDCluster > SDManagement MBean has the following attributes:

Attribute Description

BatchManager

Administers the Batch Manager Service attributes.

DeploymentStates

Allows the setup and ordering of deployment states.

InlineServiceManager:

Manages deployed Inline Services.

LearningService

Administers the Learning Service attributes.

Members

Members of the cluster. Each Member is listed through this attribute. Member is used to manage local server properties.

Properties

Cluster properties configuration.

Security

Security Manager.

13.4 MBeans for Oracle Real-Time Decisions Member-Level Management

This section provides information about management at the member level.

This section contains the following topics:

13.4.1 About OracleRTD > SDManagement > SDPropertyManager

The SDManagement > SDPropertyManager MBean has the following attributes:

Attribute Description

PerformanceMonitoring

Performance counter properties.

Misc

Miscellaneous properties.

13.4.2 About OracleRTD > SDPropertyManager > Performance Monitoring

The SDPropertyManager > Performance Monitoring MBean has the following attributes:

Attribute Description

DSPerfCounterAppend

If true, performance data is appended to an existing file, if any. Otherwise, any existing file is overwritten when the server restarts.

DSPerfCounterEnabled

Enables the writing of DS performance counters. This should not be enabled indefinitely, because the file grows without limit.

DSPerfCounterLogFile

The tab-separated CSV file into which DS performance counts are periodically appended. If MS Excel is available, ds_perf.xls, supplied in the installation's etc directory, provides a convenient view.

DSPerfCounterLogInterval

The update interval in milliseconds for DS performance counts.

For more information about using performance monitoring, see Chapter 11, "Performance Monitoring".

13.4.3 About OracleRTD > SDPropertyManager > Misc

The SDPropertyManager > Misc MBean has the following attributes:

Attribute Description

BatchAgentEnabled

Whether or not Batch Agent should run in this instance.

BatchManagerEnabled

Whether or not Batch Manager should run in this instance.

DecisionCenterEnabled

Whether or not Decision Center should run in this instance.

DecisionServiceEnabled

Whether or not Decision Service should run in this instance.

LearningServiceEnabled

Whether or not Learning Service should run in this instance.

WorkerThreadPoolSize

The number of threads used for general purpose maintenance activities, not for normal Integration Point request processing. Maintenance activities include model maintenance, session timing, and timed-out request processing.

13.4.4 About OracleRTD > Server > DecisionService

The Server > DecisionService MBean has the following read-only attributes:

Attribute Description

CurrentReceivedRequestsForwarded

Number of requests that were forwarded from other servers to this server, and which have not yet been completely processed by this server.

CurrentRequestsForwarded

Number of requests that have been forwarded from this server to other servers, and for which no acknowledgment has yet been received to indicate that the request has been processed by the forwarded-to server.

CurrentRequestsRunning

Number of Integration Point requests that are currently being processed by Inline Services.

CurrentSessions

Number of Decision Service sessions still open.

PeakReceivedRequestsForwarded

Largest number of received requests forwarded.

PeakRequestsForwarded

Largest number of requests forwarded.

RequestsForwarded

Total number of requests forwarded to another server in the cluster.

TimedOutRequests

Total number of requests that have timed out.

TotalReceivedRequestsForwarded

Total number of received requests forwarded.

TotalRequests

Total number of requests seen since the server started.

TotalRequestsForwarded

Total number of requests forwarded.

TotalSessions

Total number of Decision Service sessions created.

13.4.5 About OracleRTD > Server > SDManagement

The Server > SDManagement MBean has the following attributes:

Attribute Description

Properties

Properties configuration.

13.4.6 About OracleRTD > Server > BatchAgent

The Server > BatchAgent MBean has the following attributes:

Attribute Description

ActiveBatches

List of all batch jobs currently running on this batch agent, paused, or waiting to run. The list could be empty.

BatchNames

List of batches registered with this batch agent.

13.4.7 About OracleRTD > Server > BatchManager

The Server > BatchManager MBean has the following attributes:

Attribute Description

ActiveBatches

List of brief status information for all batch jobs currently running, paused, or waiting to run. The list could be empty.

BatchNames

List of batches registered with the batch framework.

13.4.8 About OracleRTD > Server > BatchManager > Proxy > BatchManagerProxy

The Server > BatchManager > Proxy > BatchManagerProxy MBean has the following attribute:

Attribute Description

BatchManagerLocation

The location where Batch Manager is running.

13.5 MBeans for Managing Inline Services

Use Inline Service Manager to manage the Inline Services deployed on the cluster.

This section contains the following topics:

13.5.1 About OracleRTD > SDManagement > InlineServiceManager

Each deployed Inline Service is displayed under the InlineServiceManager MBean.

The SDManagement > InlineServiceManager MBean has the following attribute:

Attribute Description

InlineServices

List of deployed inline services.

The SDManagement > InlineServiceManager MBean has the following operation:

refreshMBeans()

Removes MBeans for applications no longer in the database, and creates MBeans for new ones.

removeAllServices()

Removes all Inline Services (loaded, loadable, failed).

13.5.2 About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]

InlineServiceManager MBeans can be viewed by choosing the name of an Inline Service and a Deployment State, for example: DC_Demo.Development.

The InlineServiceManager MBeans for a specific Inline Service have the following attributes:

Attribute Description

DeploymentState

Development, QA, or Production.

LockStatus

The lock status for the Inline Service.

ServiceId

The service ID for the Inline Service.

Status

Failed, Inactive, or Loadable.

The InlineServiceManager MBeans for a specific Inline Service have the following operations:

unlockService()

Unlocks this service.

removeService()

Stops an Inline Service in this server and removes the service from the database.

flushStatistics()

Flushes all of the statistics for this service to the database.

makeLoadable()

Does a test load on this server and, if successful, marks the service loadable.

deleteStatistics()

Flushes and deletes all of the statistics for this service from the database.

deleteChoiceHistory()

Deletes all of the choice history for this service from the database.

deleteAllOperationalData()

Deletes all of the operational data for this service from the database. This includes choice history, statistics, and the study.

Note:

The prediction model data remains in memory for a short time after a user runs the deleteAllOperationalData() operation.

deleteStudy()

Removes the study for this service.

13.5.3 Invoking Maintenance Operations

Maintenance Operations appear in a node under an Inline Service when both of the following conditions hold:

  • The Inline Service includes one or more Maintenance Operations

  • The Inline Service has the Status flag set to Loadable

Each Maintenance Operation appears in both of the BroadcastAsyncOperations and DirectBlockingOperations nodes, under Maintenance Operations.

Operations listed in the DirectBlockingOperations node are invoked on the local server only, and they return only after the operation has completed. The returned value will be displayed in a popup dialog. If the operation has return type "void," then "null" will appear. If the operation fails for any reason, a short error message will be displayed in a popup dialog, and a more detailed report can be found in the log of that server.

Operations listed in the BroadcastAsyncOperations node are invoked across every node of a cluster. The operation returns immediately with the number of cluster members who received the broadcast. If the cluster has just one node, the operation returns 1, and the invocation is still asynchronous.

The following run-time considerations apply for Maintenance Operations:

  • Oracle RTD does not guarantee that all cluster members are notified of Maintenance Operation invocations, although usually they will be.

    For example, if one member of a cluster is down when a Maintenance Operation is invoked, there is no notification to indicate that the cluster member should run the Maintenance Operation when it comes back up.

  • Ordering of Maintenance Operations is not guaranteed. For example, if two Maintenance Operations A and B are invoked in sequence, an Inline Service may run B before it runs A, or it may even run them simultaneously.

13.6 MBeans for Deployment States

By viewing the OracleRTD > DeploymentStates MBeans, you can see a list of deployment states that are available on the cluster.

This section contains the following topics:

13.6.1 About OracleRTD > SDManagement > DeploymentStates

The SDManagement > DeploymentStates MBean has the following attribute:

Attribute Description

StateObjectNames

A listing of all deployment states available on the server.

13.6.2 About OracleRTD > Deployment States > [State]

MBeans for a particular Deployment State can be viewed by choosing OracleRTD > Deployment States, then choosing a Deployment State (for example, Development, QA, or Production).

Each Deployment States > [State] MBean has the following attributes:

Attribute Description

AllowHotSwapping

Allow hot swapping of Inline Services with this deployment state in Decision Service.

Id

ID of the deployment state.

Name

Name of the deployment state.

Note:

If hot swapping is enabled for a deployment state, and an Inline Service is redeployed in the state, the existing Inline Service will remain active until all existing sessions close or timeout. New sessions will be created on the newly deployed Inline Service.

13.7 MBeans for Managing Learning Services

Managing Learning Services on the cluster allows you to check the status of the learning models and perform maintenance on them.

This section contains the following topics:

13.7.1 About OracleRTD > Server > LearningService

The Server > LearningService MBean has the following attribute:

Attribute Description

Studies

A list of all Studies running on the Learning Server. The models of a Study are viewed by clicking on a Study.

13.7.2 About OracleRTD > Server > LearningService > Proxy > LearningServiceProxy

The Server > LearningService > Proxy > LearningServiceProxy MBean has the following attribute:

Attribute Description

LearningServiceLocation

The location where Learning Service is running.

13.7.3 About OracleRTD > Learning Server > [Study]

The Learning Server > [Study] MBeans have the following attributes:

Attribute Description

Models

Models belonging to this study.

Name

The name of the study.

The Learning Server > [Study] MBeans have the following operation:

CompleteSnapshot()

Saves a snapshot of this study to the database (complete).

Delete()

Deletes study data.

DeleteSnapshot()

Removes this study's snapshot from the database.

IncrementalSnapshot()

Saves a snapshot of this study to the database (delta).

13.7.4 About OracleRTD > Study > [Model.Study]

The Study > [Model.Study] MBeans have the following attributes:

Attribute Description

Attributes

Names of the model attributes. The names listed here match the attribute names in the session for your Inline Service.

Name

The name of the model.

TimeWindows

List of the ranges of time that have been learned about by this particular model.

The Study > [Study Name] > [Model] MBeans have the following operations:

Delete()

Deletes model data.

DeleteAttributeValue()

Erases model data collected for a value of an attribute. This operation accepts the following parameters:

AttributeName: The name of an attribute.

Value: The value to be deleted.

DeleteAttributeValueRange()

Erases model data collected for a range of values of an attribute. This operation accepts the following parameters:

AttributeName: The name of an attribute.

HighestValue: The highest value to be deleted.

LowestValue: The lowest value to be deleted.

StartNewTimeWindow()

Closes the current time window and starts a new one. Do not use this operation in a production environment, because it may impair future model learning.

13.8 Post-Deployment Management of Inline Services

The deployment of an Inline Service writes two types of data to the SDDB database, metadata and content.

Inline Service metadata describes the underlying elements and structure of the Inline Service, in effect, the framework of the Inline Service.

Inline Service content is data that changes at runtime, and consists of the following general types of data:

  • (A) Study (a collection of one or more models)

    Note:

    Each Inline Service's learnings are associated with a study name. If you want to redeploy an Inline Service and restart its learnings, deploy it with a new study name. Different study names can be used for Development, QA, and Production.

  • (B) Statistics

  • (C) Choice history

  • (D) Learning data

The following table shows the JMX MBeans and operations to use when you want to remove an Inline Service or to delete some or all of the content data of an Inline Service.

The Operation column shows the operation to select for the corresponding MBean-related Topic, which is described separately in this chapter. The Actions which relate to content data include one or more letter references to the content types in the preceding bulleted list.

Action Operation Topic

Removing an Inline Service

This deletes all the metadata of an Inline Service, but none of the content.

removeService()

About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]

Deleting Inline Service statistics (B)

deleteStatistics()

About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]

Deleting Inline Service choice history (C)

deleteChoiceHistory()

About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]

Deleting all Inline Service operational data (A)+(B)+(C)+(D)

This deletes all the content data of an Inline Service, but no metadata.

deleteAllOperationalData()

About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]

Deleting Inline Service study (A)

deleteStudy()

About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]

(Alternate)
Deleting Inline Service study (A)

Delete()

About OracleRTD > Learning Server > [Study]

Deleting models from an Inline Service study

This deletes a single model from a study (A).

Delete()

About OracleRTD > Study > [Model.Study]

Caution:

It is strongly advised that, prior to performing any remove or delete action, you backup the SDDB database.

13.9 System Properties

You can view and set the following system properties:

System Property Description

ModelDSName

The JNDI name of the datasource used by the Learning Service. Default=SDDS.

ModelSnapshotDSName

The JNDI name of the datasource used by the Learning Service to perform snapshots of its learning models. Default=SDDS.

RestrictClusterMembers

True if the hosts that can be in the cluster is restricted to a fixed list of trusted IP addresses.

RestrictDSClients

True if the hosts that can send Decision Service requests is restricted to a fixed list of trusted IP addresses.

SDGroupName

This is the name of the Oracle RTD cluster, as recognized by Oracle RTD's cluster management.

SystemDSName

This is the name of the Oracle RTD system datasource. Default=SDDS.

TrustedClusterMembers

List of host IP addresses that can join the cluster. Port is optional, separated from IP by ':'. Entries are separated by ';'.

TrustedDSClients

List of host IP addresses from which Decision Service requests will be accepted. Port is optional, separated from IP by ':'. Entries are separated by ';'.