13 Managing Oracle Real-Time Decisions

Oracle RTD uses the J2EE industry standard Java Management Extensions (JMX) to configure and monitor the operation of Oracle RTD.

Oracle RTD systems deployed into WebLogic are managed using the Fusion Middleware Control MBean Browsers in Enterprise Manager.

Note:

Some of the management properties can also be set as system properties. For details, see Section 13.9, "System Properties."

JMX MBeans manage various aspects of Oracle RTD, including logging and Inline Service configuration. They can also be used to assign Oracle RTD specific permissions to security roles and users. Security roles and users are both managed by the J2EE container.

Oracle RTD is comprised of three Services:

Deployments of Oracle RTD are often done across multiple servers as well as in clusters to enhance performance in high transaction environments. A relational database is used by each of these Services for retention of code, transactional data and configurations.

This chapter contains the following topics:

13.1 Accessing the Oracle Real-Time Decisions MBeans

Oracle RTD under WebLogic is a component of Oracle Fusion Middleware. You access the Oracle RTD JMX MBeans through the System MBean Browser that exists within Oracle Enterprise Manager Fusion Middleware Control.

To access the System MBean Browser for Oracle RTD, you must perform the following steps:

  1. Log into Fusion Middleware Control, as described in Section 2.1.1, "Logging into Fusion Middleware Control."

  2. In the left-hand Target Navigation Pane, after selecting Application Deployments, then Internal Applications, right-click Oracle RTD in the server in which Oracle RTD is deployed.

    Surrounding text describes em01.gif.

  3. Select System MBean Browser.

  4. To access the Oracle RTD MBeans, scroll down the System MBean Browser left-hand pane, and select Application Defined MBeans, then Oracle RTD, then the server where Oracle RTD is deployed.

The grouping categories for the Oracle RTD MBeans appear, as in the following example:

Surrounding text describes smb01.gif.

For more information about MBeans and MBean Browsers, see Oracle Fusion Middleware Administrator's Guide.

13.2 About JMX MBean Operations and Attributes

Oracle RTD MBeans can be accessed through tthe OracleRTD folder in the System MBean Browser. These MBeans can be used to manage various aspects of Oracle RTD. Each MBean consists of attributes and operations that can be used for informational and administration purposes. The attributes and operations described in this chapter are specific to Oracle RTD.

13.3 MBeans for Oracle Real-Time Decisions Cluster-Level Management

Management at the cluster level is for items that impact the entire cluster of servers. Note that if you have only one server, there is still cluster-level management.

This section contains the following topics:

13.3.1 About OracleRTD > SDManagement > SDClusterPropertyManager

The SDManagement > SDClusterPropertyManager MBean has the following attributes:

Attribute Description
Cluster Cluster configuration.
Deployment Configuration for deployment.
Misc Miscellaneous properties.

The SDManagement > SDClusterPropertyManager MBean has the following operation:

java.lang.Void restoreDefault()

Restores the default installation settings to Oracle RTD. If this command is run on a cluster, then values are restored to cluster defaults.

13.3.2 About OracleRTD > SDClusterPropertyManager > Misc

The SDClusterPropertyManager > Misc MBean has the following attributes:

Attribute Description
ArchivedModelCacheTimeToLive Maximum time in seconds an archived model is preserved in memory. The cache of archived models is used by Discovery Explorer.
ArchivedModelCatalogRefreshInterval Refresh interval in seconds for a catalog of archived models. The catalog is used by Discovery Explorer.
AutoFlushTimeout Interval in seconds controlling auto flush of database write buffers. Fractional values are supported.
DBOperationLogThresholdMilliSec All database operations that take longer than the specified threshold are logged.
DCOperationLogThresholdMilliSec All decision center requests that take longer than the specified threshold are logged.
DSManagesSessionAffinity Decision Service manages session affinity. When set to true, the decision service maintains a map of active session keys and, if necessary, will forward Integration Point requests to the cluster host owning the key's session. Should be disabled in single-host installations and in installations where session affinity is perfectly managed by the application server or external load balancer.
DSSessionIdleTimeoutMillisec Decision Service session idle timeout in milliseconds.
DSStrictSessionAffinityConcurrency How many concurrent DB requests can be outstanding, per Oracle RTD instance, to manage the strict Decision Service session directory. Keep this low for SQL Server to avoid DB-centric deadlocks. 0 means unlimited.
DatabaseComponentCloseTimeoutSeconds How long to wait for the database provider to close an individual component (for example, each BatchUpdater) before abandoning the effort.
DatabaseShutdownTimeoutSeconds How long to wait for the database provider to shutdown all its components before abandoning the effort.
DecisionServiceAddress Must be set for Decision Center to be able to test integration point requests from its Interactive Integration Map when Decision Center is not co-located with Decision Service.
DisableBatchDBOperations Boolean switch that controls batch database operations.
IntegrationPointGuaranteed RequestTimeout Guaranteed response time, in milliseconds, for Integration Point requests. (Service Level Guarantee). Zero means don't timeout Integration Point requests - suitable for debugging only.
ModelDSName The JNDI name of the datasource used by the Learning Service.
ModelSnapshotDSName The JNDI name of the datasource for the model snapshots.
ModelSnapshotMinAbsCorrelation Controls whether to snapshot all correlation rows or to set a minimum correlation value for snapshots. The default value of 0.000001 prevents snapshots of very small value correlation rows. Set the value to 0 to snapshot all correlation rows.
ModelSnapshotNumberOfBins Controls the number of bins for model snapshots. Numeric attribute values are automatically binned, or assigned to numeric ranges. The default number of bins is 5. To achieve greater resolution of your numeric data, increase the number of bins.

Note that for the same numeric attribute, Oracle RTD creates different bins in different time windows. Therefore, it is unlikely that you will be able to join numeric attribute values across time windows.
SystemDSName The JNDI name of the datasource.
WorkerThreadPoolSize The number of threads used for general purpose maintenance activities, not for normal Integration Point request processing. Maintenance activities include model maintenance, session timing, and timed-out request processing.

13.3.3 About OracleRTD > SDClusterPropertyManager > Cluster

The SDClusterPropertyManager > Cluster MBean has the following attributes:

Attribute Description
BatchManagerInitialWait The number of milliseconds to wait when the server first starts up before trying to start the Batch Manager Service.
BatchManagerRestartWait The number of milliseconds to wait after a computer fails or leaves the cluster before trying to restart the Batch Manager Service.
ChoiceHistoryCleanupChunkSize The chunk size to use when deleting old choice history records.
ChoiceHistoryCleanupThrottle A number between 0.1 and 1, inclusive. Higher throttle corresponds to higher speed.
GenerateDSCookies Generate Decision Server HTTP Cookies. Set to true to have Decision Server associate Integration Point requests with HTTP sessions, thus causing the Web container to generate container-specific session-affinity cookies.
GenerateDSSessionIdCookie Generate Decision Service Session-ID Cookies. Set to true to have Decision Service supply a cookie named ORTD_DS_SessionID identifying the DS session serving the current integration point request.
LearningDataStorageCleanupChunkSize The chunk size to use when deleting old learning data storage records.
LearningDataStorageCleanupThrottle A number between 0.1 and 1, inclusive. Higher throttle corresponds to higher speed.
LearningServiceInitialWait The number of milliseconds to wait when the server first starts up before trying to start the Learning Service.
LearningServiceRestartWait The number of milliseconds to wait after a computer fails or leaves the cluster before trying to restart the Learning Service.
OperationalDataCleanupPeriod The number of hours (fractions are allowed) between cleanup of the operational data (choice history, statistics, learning data storage) in the database.
RequireIntegrationPointAuthorization True if Integration Points can only be called by security principals granted the \\"decision_service\:normal\\" action on the containing Inline Service.
RestrictDSClients True if the hosts that can send Decision Service requests is restricted to a fixed list of trusted IP addressees.
StatisticsCleanupChunkSize The chunk size to use when deleting old statistic records.
StatisticsCleanupThrottle A number between 0.1 and 1, inclusive. Higher throttle corresponds to higher speed.
TrustedDSClients List of host IP addresses from which Decision Service requests will be accepted. Port is optional, separated from IP by ':'. Entries are separated by ';'.

Adding Trusted Decision Service Clients

If any host can send Decision Service requests, set RestrictDSClients to False (default value is True).

If you want to restrict the hosts allowed to send Decision Service requests hosts:

  • Set RestrictDSClients to True

  • For TrustedDSClients, enter a list of the host IP addresses of the client hosts allowed to send Decision Service requests

This change should be propagated to all existing instances, and should be effectively immediately.

13.3.4 About OracleRTD > SDClusterPropertyManager > Deployment

The SDClusterPropertyManager > Deployment MBean has the following attribute:

Attribute Description
AppPollingInterval How frequently, in seconds, the AppFactory polls the SDApps table to see if there are new apps.

13.3.5 About OracleRTD > SDCluster > SDManagement

Attributes and operations at the SDCluster level are meant to manage cluster-level features.

The SDCluster > SDManagement MBean has the following attributes:

Attribute Description
BatchManager Administers the Batch Manager Service attributes.
DeploymentStates Allows the setup and ordering of deployment states.
InlineServiceManager: Manages deployed Inline Services.
LearningService Administers the Learning Service attributes.
Members Members of the cluster. Each Member is listed through this attribute. Member is used to manage local server properties.
Properties Cluster properties configuration.
Security Security Manager.

13.4 MBeans for Oracle Real-Time Decisions Member-Level Management

This section provides information about management at the member level.

This section contains the following topics:

13.4.1 About OracleRTD > SDManagement > SDPropertyManager

The SDManagement > SDPropertyManager MBean has the following attributes:

Attribute Description
PerformanceMonitoring Performance counter properties.
Misc Miscellaneous properties.

13.4.2 About OracleRTD > SDPropertyManager > Performance Monitoring

The SDPropertyManager > Performance Monitoring MBean has the following attributes:

Attribute Description
DSPerfCounterAppend If true, performance data is appended to an existing file, if any. Otherwise, any existing file is overwritten when the server restarts.
DSPerfCounterEnabled Enables the writing of DS performance counters. This should not be enabled indefinitely, because the file grows without limit.
DSPerfCounterLogFile The tab-separated CSV file into which DS performance counts are periodically appended. If MS Excel is available, ds_perf.xls, supplied in the installation's etc directory, provides a convenient view.
DSPerfCounterLogInterval The update interval in milliseconds for DS performance counts.

For more information about using performance monitoring, see Chapter 11, "Performance Monitoring".

13.4.3 About OracleRTD > SDPropertyManager > Misc

The SDPropertyManager > Misc MBean has the following attributes:

Attribute Description
BatchAgentEnabled Whether or not Batch Agent should run in this instance.
BatchManagerEnabled Whether or not Batch Manager should run in this instance.
DecisionCenterEnabled Whether or not Decision Center should run in this instance.
DecisionServiceEnabled Whether or not Decision Service should run in this instance.
LearningServiceEnabled Whether or not Learning Service should run in this instance.
WorkerThreadPoolSize The number of threads used for general purpose maintenance activities, not for normal Integration Point request processing. Maintenance activities include model maintenance, session timing, and timed-out request processing.

13.4.4 About OracleRTD > Server > DecisionService

The Server > DecisionService MBean has the following read-only attributes:

Attribute Description
CurrentReceivedRequestsForwarded Number of requests that were forwarded from other servers to this server, and which have not yet been completely processed by this server.
CurrentRequestsForwarded Number of requests that have been forwarded from this server to other servers, and for which no acknowledgment has yet been received to indicate that the request has been processed by the forwarded-to server.
CurrentRequestsRunning Number of Integration Point requests that are currently being processed by Inline Services.
CurrentSessions Number of Decision Service sessions still open.
PeakReceivedRequestsForwarded Largest number of received requests forwarded.
PeakRequestsForwarded Largest number of requests forwarded.
RequestsForwarded Total number of requests forwarded to another server in the cluster.
TimedOutRequests Total number of requests that have timed out.
TotalReceivedRequestsForwarded Total number of received requests forwarded.
TotalRequests Total number of requests seen since the server started.
TotalRequestsForwarded Total number of requests forwarded.
TotalSessions Total number of Decision Service sessions created.

13.4.5 About OracleRTD > Server > SDManagement

The Server > SDManagement MBean has the following attributes:

Attribute Description
Properties Properties configuration.

13.4.6 About OracleRTD > Server > BatchAgent

The Server > BatchAgent MBean has the following attributes:

Attribute Description
ActiveBatches List of all batch jobs currently running on this batch agent, paused, or waiting to run. The list could be empty.
BatchNames List of batches registered with this batch agent.

13.4.7 About OracleRTD > Server > BatchManager

The Server > BatchManager MBean has the following attributes:

Attribute Description
ActiveBatches List of brief status information for all batch jobs currently running, paused, or waiting to run. The list could be empty.
BatchNames List of batches registered with the batch framework.

13.4.8 About OracleRTD > Server > BatchManager > Proxy > BatchManagerProxy

The Server > BatchManager > Proxy > BatchManagerProxy MBean has the following attribute:

Attribute Description
BatchManagerLocation The location where Batch Manager is running.

13.5 MBeans for Managing Inline Services

Use Inline Service Manager to manage the Inline Services deployed on the cluster.

This section contains the following topics:

13.5.1 About OracleRTD > SDManagement > InlineServiceManager

Each deployed Inline Service is displayed under the InlineServiceManager MBean.

The SDManagement > InlineServiceManager MBean has the following attribute:

Attribute Description
InlineServices List of deployed inline services.

The SDManagement > InlineServiceManager MBean has the following operation:

refreshMBeans()

Removes MBeans for applications no longer in the database, and creates MBeans for new ones.

removeAllServices()

Removes all Inline Services (loaded, loadable, failed).

13.5.2 About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]

InlineServiceManager MBeans can be viewed by choosing the name of an Inline Service and a Deployment State, for example: DC_Demo.Development.

The InlineServiceManager MBeans for a specific Inline Service have the following attributes:

Attribute Description
DeploymentState Development, QA, or Production.
LockStatus The lock status for the Inline Service.
ServiceId The service ID for the Inline Service.
Status Failed, Inactive, or Loadable.

The InlineServiceManager MBeans for a specific Inline Service have the following operations:

unlockService()

Unlocks this service.

removeService()

Stops an Inline Service in this server and removes the service from the database.

flushStatistics()

Flushes all of the statistics for this service to the database.

makeLoadable()

Does a test load on this server and, if successful, marks the service loadable.

deleteStatistics()

Flushes and deletes all of the statistics for this service from the database.

deleteChoiceHistory()

Deletes all of the choice history for this service from the database.

deleteAllOperationalData()

Deletes all of the operational data for this service from the database. This includes choice history, statistics, and the study.

Note:

The prediction model data remains in memory for a short time after a user runs the deleteAllOperationalData() operation.

deleteStudy()

Removes the study for this service.

13.5.3 Invoking Maintenance Operations

Maintenance Operations appear in a node under an Inline Service when both of the following conditions hold:

  • The Inline Service includes one or more Maintenance Operations

  • The Inline Service has the Status flag set to Loadable

Each Maintenance Operation appears in both of the BroadcastAsyncOperations and DirectBlockingOperations nodes, under Maintenance Operations.

Operations listed in the DirectBlockingOperations node are invoked on the local server only, and they return only after the operation has completed. The returned value will be displayed in a popup dialog. If the operation has return type "void," then "null" will appear. If the operation fails for any reason, a short error message will be displayed in a popup dialog, and a more detailed report can be found in the log of that server.

Operations listed in the BroadcastAsyncOperations node are invoked across every node of a cluster. The operation returns immediately with the number of cluster members who received the broadcast. If the cluster has just one node, the operation returns 1, and the invocation is still asynchronous.

The following run-time considerations apply for Maintenance Operations:

  • Oracle RTD does not guarantee that all cluster members are notified of Maintenance Operation invocations, although usually they will be.

    For example, if one member of a cluster is down when a Maintenance Operation is invoked, there is no notification to indicate that the cluster member should run the Maintenance Operation when it comes back up.

  • Ordering of Maintenance Operations is not guaranteed. For example, if two Maintenance Operations A and B are invoked in sequence, an Inline Service may run B before it runs A, or it may even run them simultaneously.

13.6 MBeans for Deployment States

By viewing the OracleRTD > DeploymentStates MBeans, you can see a list of deployment states that are available on the cluster.

This section contains the following topics:

13.6.1 About OracleRTD > SDManagement > DeploymentStates

The SDManagement > DeploymentStates MBean has the following attribute:

Attribute Description
StateObjectNames A listing of all deployment states available on the server.

13.6.2 About OracleRTD > Deployment States > [State]

MBeans for a particular Deployment State can be viewed by choosing OracleRTD > Deployment States, then choosing a Deployment State (for example, Development, QA, or Production).

Each Deployment States > [State] MBean has the following attributes:

Attribute Description
AllowHotSwapping Allow hot swapping of Inline Services with this deployment state in Decision Service.
Id ID of the deployment state.
Name Name of the deployment state.

Note:

If hot swapping is enabled for a deployment state, and an Inline Service is redeployed in the state, the existing Inline Service will remain active until all existing sessions close or timeout. New sessions will be created on the newly deployed Inline Service.

13.7 MBeans for Managing Learning Services

Managing Learning Services on the cluster allows you to check the status of the learning models and perform maintenance on them.

This section contains the following topics:

13.7.1 About OracleRTD > Server > LearningService

The Server > LearningService MBean has the following attribute:

Attribute Description
Studies A list of all Studies running on the Learning Server. The models of a Study are viewed by clicking on a Study.

13.7.2 About OracleRTD > Server > LearningService > Proxy > LearningServiceProxy

The Server > LearningService > Proxy > LearningServiceProxy MBean has the following attribute:

Attribute Description
LearningServiceLocation The location where Learning Service is running.

13.7.3 About OracleRTD > Learning Server > [Study]

The Learning Server > [Study] MBeans have the following attributes:

Attribute Description
Models Models belonging to this study.
Name The name of the study.

The Learning Server > [Study] MBeans have the following operation:

CompleteSnapshot()

Saves a snapshot of this study to the database (complete).

Delete()

Deletes this study.

DeleteSnapshot()

Removes this study's snapshot from the database.

IncrementalSnapshot()

Saves a snapshot of this study to the database (delta).

13.7.4 About OracleRTD > Study > [Model.Study]

The Study > [Model.Study] MBeans have the following attributes:

Attribute Description
Attributes Names of the model attributes. The names listed here match the attribute names in the session for your Inline Service.
Name The name of the model.
TimeWindows List of the ranges of time that have been learned about by this particular model.

The Study > [Study Name] > [Model] MBeans have the following operations:

Delete()

Deletes this model.

DeleteAttributeValue()

Erases model data collected for a value of an attribute. This operation accepts the following parameters:

AttributeName: The name of an attribute.

Value: The value to be deleted.

DeleteAttributeValueRange()

Erases model data collected for a range of values of an attribute. This operation accepts the following parameters:

AttributeName: The name of an attribute.

HighestValue: The highest value to be deleted.

LowestValue: The lowest value to be deleted.

StartNewTimeWindow()

Closes the current time window and starts a new one. Do not use this operation in a production environment, because it may impair future model learning.

13.8 Post-Deployment Management of Inline Services

The deployment of an Inline Service writes two types of data to the SDDB database, metadata and content.

Inline Service metadata describes the underlying elements and structure of the Inline Service, in effect, the framework of the Inline Service.

Inline Service content is data that changes at runtime, and consists of the following general types of data:

  • (A) Study (a collection of one or more models)

    Note:

    Each Inline Service's learnings are associated with a study name. If you want to redeploy an Inline Service and restart its learnings, deploy it with a new study name. Different study names can be used for Development, QA, and Production.

  • (B) Statistics

  • (C) Choice history

  • (D) Learning data

The following table shows the JMX MBeans and operations to use when you want to remove an Inline Service or to delete some or all of the content data of an Inline Service.

The Operation column shows the operation to select for the corresponding MBean-related Topic, which is described separately in this chapter. The Actions which relate to content data include one or more letter references to the content types in the preceding bulleted list.

Action Operation Topic
Removing an Inline Service

This deletes all the metadata of an Inline Service, but none of the content.

 removeService() About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]
Deleting Inline Service statistics (B) deleteStatistics() About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]
Deleting Inline Service choice history (C) deleteChoiceHistory() About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]
Deleting all Inline Service operational data (A)+(B)+(C)+(D)

This deletes all the content data of an Inline Service, but no metadata.

 deleteAllOperationalData() About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]
Deleting Inline Service study (A) deleteStudy() About OracleRTD > InlineServiceManager > [Inline Service.Deployment State]
(Alternate) Deleting Inline Service study (A) Delete() About OracleRTD > Learning Server > [Study]
Deleting models from an Inline Service study

This deletes a single model from a study (A).

 Delete() About OracleRTD > Study > [Model.Study]

Caution:

It is strongly advised that, prior to performing any remove or delete action, you backup the SDDB database.

13.9 System Properties

You can view and set the following system properties:

System Property Description
ModelDSName The JNDI name of the datasource used by the Learning Service. Default=SDDS.
ModelSnapshotDSName The JNDI name of the datasource used by the Learning Service to perform snapshots of its learning models. Default=SDDS.
RestrictClusterMembers True if the hosts that can be in the cluster is restricted to a fixed list of trusted IP addresses.
RestrictDSClients True if the hosts that can send Decision Service requests is restricted to a fixed list of trusted IP addresses.
SDGroupName This is the name of the Oracle RTD cluster, as recognized by Oracle RTD's cluster management.
SystemDSName This is the name of the Oracle RTD system datasource. Default=SDDS.
TrustedClusterMembers List of host IP addresses that can join the cluster. Port is optional, separated from IP by ':'. Entries are separated by ';'.
TrustedDSClients List of host IP addresses from which Decision Service requests will be accepted. Port is optional, separated from IP by ':'. Entries are separated by ';'.