Model Monitor

Model Monitoring refers to the process of closely tracking the performance of models in production and helps you to frequently monitor the distribution of the model and provides alerts in case of any exceptions.

In addition, it enables you to identify and eliminate bad quality predictions and poor technical performance of the models.

The model's robustness depends not only on the training of the feature-engineered data but also on how well the model is monitored after deployment. Typically a model's performance degrades over time, and it essential to detect the cause of the decrease in performance of the model. The main cause of the decline in performance can be drift in the independent or/and dependent features which may violate the model’s assumption and distribution about the data. When models are built, the model builder will create a snapshot of the dataset used for training the model and save it in a file which is later used for calculating drift with the current snapshot of the dataset.

To start the model monitoring, perform the following steps:

1. In the LHS menu, click Datasetto display the Dataset Summary window.
   This window displays the dataset records in a table.
2. Click Actionnext to corresponding Dataset and select View.
   The Profile Summary screen is displayed.
3. Navigate to Mode Monitorpage.
   All the models associated with the selected dataset will be displayed.

   Figure 8-22 Mode Monitor page

   
   

   

   
   
4. Click Start Model Monitoringor Monitor Settings icon.
   The Monitor Settings page is displayed.

   Figure 8-23 Monitor Settings page

   
   

   

   
   
5. Drag the slider and set the threshold settings. Threshold is the value in percentage, beyond which it is labelled as drifted.
6. Select the Schedule Type as required:
   • Daily: Select to run the schedule everyday.
   • Weekly: Select to run the schedule once in a week or the selected days in a week.
   • Monthly: Select to run the schedule once in a month or the selected days in a month.
   The monitoring for the model is started and Is Monitored option is enabled by default and you can disable it if you do not want to monitor.