5.2.4.5 Behavioral Model Monthly Model Validation

  • This pre-seeded batch will be available in all workspaces (Production and Sandboxes).

    Note:

    This batch has to be executed in the Production workspace.
  • This batch shows ongoing model drift and data quality with respect to new data every month (monthly).
Batch and Task Parameters
The batch contains a single task named Monthly_Model_Validation.
  • Objective folder for Data Quality:
    Home / Modeling / Pipelines / ML4AML / Ongoing Model Validation / Monthly
/ Data Quality
  • Objective folder for Model Drift:
    Home / Modeling / Pipelines / ML4AML / Ongoing Model Validation / Monthly
/ Model Drift

    Figure 5-43 Monthly Model Validation


    Description of Figure 5-43 follows
    Description of "Figure 5-43 Monthly Model Validation"

  • Model: Retain the default settings.

    Note:

  • Optional Parameters:
    • model_group_name: Name of the Model Groups for which the model has been trained. Example: LOB1.
    • model_name: Name of the Model for which the model has been trained. Example: RMF.
    • focus: Name of the entity type or segment. Example: CUSTOMER.
    • model_id: User passes parameter as Deployed to use the deployed model. Example: Deployed.
    • FEATURE_INCLUDE: List of features to be included for data quality. The default None means which includes everything.
    • FEATURE_EXCLUDE: List of features to be excluded for data quality. The default None means which excludes nothing.

      Note:

      If both include and exclude actions are provided, then include takes precedence over exclude action.

      Example 1: feature_include="Feature1~Feature2"

      Example 2: feature_exclude="Feature3~Feature4~Feature5"

    • look_back_months: Number of periods to look back for getting drift history. The default value is 5.
    • Number_Of_Bins: Number of bins to be used in discretizing (scalar). The default value is 9.
    • Boot_Strap_Samples: Number of bootstrap samples on which to estimate thresholds. The default value is 5.
    • Standard_Deviation_Band_Sigma: Number of standard deviation bands (sigma band) for threshold setting to be used. The default value is 2 sigma
    • is_csi_grouped: Characteristic Stability Index (CSI) values for categories of a feature will be totaled and displayed if the parameter is set to True. By default, it is True.

      For example: model_group_name=LOB1,model_name=RMF,focus=CUSTOMER,Number_Of_Bins=9,Boot_Strap_Samples=5,Standard_Deviation_Band_Sigma=2,look_back_months=5,FEATURE_INCLUDE=None,FEATURE_EXCLUDE=None,is_csi_grouped=True

      Figure 5-44 Define Task for Monthly Model Validation


      Description of Figure 5-44 follows
      Description of "Figure 5-44 Define Task for Monthly Model Validation"

    Note:

    The Monthly Model Validation batch shows output metrics in the notebook only and it will not store in any of the data tables.