5.2.4.4 Behavioral Model Monthly Model Validation

• This pre-seeded batch will be available in all workspaces (Production and Sandboxes).

  Note:

  This batch has to be executed in the Production workspace.
• This batch shows ongoing model drift and data quality with respect to new data every month (monthly).

Batch and Task Parameters

The batch contains a single task named Monthly_Model_Validation.

• Objective folder for Data Quality:

  Home / Modeling / Pipelines / ML4AML / Ongoing Model Validation / Monthly
  / Data Quality

• Objective folder for Model Drift :

  Home / Modeling / Pipelines / ML4AML / Ongoing Model Validation / Monthly
  / Model Drift

  Figure 5-37 Monthly Model Validation

  
  

  
  Description of "Figure 5-37 Monthly Model Validation"

  
  

  Note:

  • Do not change any batch/task parameter except Optional Parameters.
  • Optional Parameters can be edited from the Schedule Batch option.
• Optional Parameters:
  • model_group_name: Name of the Model Groups for which the model has been trained. Example: LOB1.
  • model_name: Name of the Model for which the model has been trained. Example: RMF.
  • focus: Name of the entity type or segment. Example: CUSTOMER.
  • model_id: User passes parameter as Deployed to use the deployed model. Example: Deployed.
  • FEATURE_INCLUDE: List of features to be included for data quality. The default None means which includes everything.
  • FEATURE_EXCLUDE: List of features to be excluded for data quality. The default None means which excludes nothing.

    Note:

    If both include and exclude actions are provided, then include takes precedence over exclude action.

    Example 1: feature_include="Feature1~Feature2"

    Example 2: feature_exclude="Feature3~Feature4~Feature5"

  • look_back_months: Number of periods to look back for getting drift history. The default value is 5.
  • Number_Of_Bins: Number of bins to be used in discretizing (scalar). The default value is 9.
  • Boot_Strap_Samples: Number of bootstrap samples on which to estimate thresholds. The default value is 5.
  • Standard_Deviation_Band_Sigma: Number of standard deviation bands (sigma band) for threshold setting to be used. The default value is 2 sigma.

    For example:

    model_group_name=LOB1,model_name=RMF,focus=CUSTOMER,Number_Of_Bins=9,Boot_Str ap_Samples=5,Standard_Deviation_Band_Sigma=2,look_back_ months=5,FEATURE_INCLUDE=None,FEATURE_EXCLUDE=None

    Figure 5-38 Define Task for Monthly Model Validation

    
    

    
    Description of "Figure 5-38 Define Task for Monthly Model Validation"

    
    

  Note:

  The Monthly Model Validation batch shows output metrics in the notebook only and it will not store in any of the data tables.