Model Monitoring

Model monitoring allows you to monitor the quality of model predictions over time and helps you with insights on the causes of model quality issues.

The Model Monitor page allows you to create, run, and track model monitors and results. This page lists the model monitors. You can preview the model drift by selecting a single model monitor of interest, as shown in the screenshot below. In this screenshot, the monitor Power Consumption is selected. On the lower pane of the Model Monitors page, the Model Drift for the selected monitor is displayed. The X axis depicts the analysis period, and the Y axis depicts the model drift values. The horizontal red dotted line is the threshold value. The line depicts the drift value for each point in time for the analysis period. Hover your mouse over the line to view the drift values. Figure 16: Model Monitors page

Description of the illustration model_drift_mm.png

You can perform the following tasks on the Model Monitors page:

• Create: Create a model monitor.

• Edit: Select a model monitor and click Edit to edit the monitor.

• Duplicate: Select a model monitor and click Duplicate to create a copy of the monitor.

• Delete: Select a model monitor and click Delete to delete the model monitor.

• History: Select a model monitor and click History to view the runtime details of the model monitor. If there is a data monitor job associated with the model monitor, the data monitor runtime details are displayed on the lower pane. Click Back to Monitors to go back to the Model Monitoring page.

• Start: Start a model monitor.

• Stop: Stop a model monitor that is running.

• More: Select a model monitor, click More and then click:

  • Enable: Enable a model monitor schedule. By default, the model monitor is enabled. The status is displayed as SCHEDULED.

  • Disable: Disable a model monitor schedule. The status is displayed as DISABLED.

  • Show Managed Monitors: Click this option to view the models created and managed by OML Services REST API.

The Model Monitors page displays the following information about the model monitors:

• Name: This is the name of the model monitor.

• Data Monitor: The check mark in this column indicates that there is a data monitor job associated with the model monitor. Click on the three dots to view: Figure 17: View Results and Settings of Data Monitor

  

  Description of the illustration data_monitor_options.png

• Results: Displays the data monitoring results computed by the data monitor job. The data monitor name is auto-generated along with the prefix OML. The data drift and monitored features along with the computed statistics are displayed in view-only mode on a separate page. Click Monitors on the top left corner to return to the Model Monitors page. Click Details on the top right corner to view the settings of the data monitor. Figure 18: Data Monitor Results

  

  Description of the illustration results_dm_mm.png

• Settings: Displays the data monitor settings, additional settings, and the monitored features in view-only mode. Click Back on the top right corner of the page to return to the Model Monitors page. Figure 19: Data Monitor Settings

  

  Description of the illustration settings_dm_mm.png

• Baseline Data: This is a table or view that contains baseline data to monitor.

• New Data: This is a table or view with new data to be compared against the baseline data.

• Last Start Date: This is the date on which the model monitor was last started.

• Last Status: This is the latest status of the model monitor. The statuses are SUCCEEDED, FAILED, RUNNING, SCHEDULED.

• Next Run Date: This is the date on which the next run is scheduled.

• Status: This is the current status of the model monitor.

• Schedule: This is the frequency of the monitoring job, that is, how frequently the model monitor is set to run on the New Data.