20.2.33 Purge Batch Details

Purge batches are used when your batch has not executed successfully to purge the data and execute the batch again.

If your batch has not executed successfully, has been explicitly interrupted or canceled, or was put on hold during the execution process, first try to restart the batch following the steps in Restart a Batch/Batch Group Re-start Batch/Batch Group If this is not successful, then you can purge the data and execute the batch again.

Note:

Purge batches are not a regular required activity. They should only be used when other methods to re-run the batches are not successful.

To purge the data which was partially processed during the interrupted or canceled batch execution, follow the steps described in the following table.

Note:

Purge Batches should be run in the reverse order of batch execution. For example, if batches are run in the following order:
  1. Ingestion
  2. TMScenario
  3. CMIngestion
  4. AMLtoCaseManagement
Then the purge batch order should be:
  1. AMLtoCaseManagement
  2. CMIngestion
  3. TMScenario
  4. Ingestion

Table 20-76 Executing Purge Batches

Failed Batch Name Purge Batch Steps to Purge Additional Considerations
AMLDataLoad PurgeStagingTables If the AMLDataLoad batch fails to execute, follow these steps:
  1. Run the PurgeStagingTables batch for the batch date and data origin on which the batch failed.
    The DELETEEXTFLAG parameter has been added to the PurgeStagingTables batch. By default, DELETEEXTFLAG is set to N.
    • Existing customers (provisioned prior to August 31 2024) should set DELETEEXTFLAG as N.
    • Customers Provisioned post August 31 2024 and still using AMLDATALOAD should set DELETEEXTFLAG as N to delete all Staging tables data and Stage Extension tables data.
    • Customers Provisioned post August 31 2024 and using StageDataLoad should set DELETEEXTFLAG as Y to delete Stage Extension tables data and not to use this batch for deleting stage tables data (other than Extension tables).
    Note: Customers running StageDataLoad does not need any purge batch. StageDataLoad has inbuilt feature of Truncate and load for that Date and Data Origin.
  2. Execute the AMLDataLoad batch for the batch date.
 _
Ingestion
  1. Run the PurgeTMTables and the PurgeAMIngestionTables batches by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the Ingestion batch for the batch date.
 _
AMLtoCaseManagement PurgeCMTables If the AMLtoCaseManagement batch fails to execute, follow these steps:
  1. Run the PurgeCMTables batch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the AMLtoCaseManagement batch for the batch date.
 _
KYCToCaseManagement PurgeCMTables If the KYCToCaseManagement batch fails to execute, follow these steps:
  1. Run the PurgeCMTables batch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the KYCToCaseManagement batch for the batch date.
 _
ScreeningToCaseManagement PurgeCMTables If the ScreeningToCaseManagement batch fails to execute, follow these steps:
  1. Run the PurgeCMTables batch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the ScreeningToCaseManagement batch for the batch date.
 _
AMLToCMEventData PurgeAMLToCMEventData If the AMLToCMEventData batch fails to execute, follow these steps:
  1. Run the PurgeAMLToCMEventData batch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the Curated CM batch for the batch date.
 _
KYCDeploymentInitiation PurgeKYCTables If the KYCDeploymentInitiation batch fails to execute, follow these steps:
  1. Run the PurgeKYCTablesbatch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
Customer Data Slice is purged as part of the PurgeKYCTables batch. It is not mandatory to run the PurgeCustomerSlicing batch separately after purging the KYCDeploymentInitiation batch.
KYCDaily PurgeKYCTables If the KYCDaily batch fails to execute, follow these steps:
  1. Run the PurgeKYCTables batch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the KYCDaily batch for the batch date.
 Customer Data Slice is purged as part of the PurgeKYCTables batch.
Customer Screening PurgeCSTables If any Customer Screening batch fails to execute, follow these steps:
  1. Run the PurgeCSTables batch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the appropriate Custoner Screening batch for the batch date.
  • After executing the purge batch, the metering records for that run will get deleted. To get the updated data for the metering in the Dashboard UI, you must run AMLMetrics from the Scheduler screen. Otherwise, you must wait until the next day for the latest data to reflect in the UI.
  • If you are purging a DI batch, you must enter the exact slice name label used in the failed DI batch as the slice name ($SLICENAME$) parameter in the PurgeCSTables batch.
CMIngestion PurgeCMIngestion If the CMIngestion batch fails to execute, follow these steps:
  1. Run the PurgeCMIngestion batch by providing the failed batchrunID as input parameter in the field $FCCBATCHRUNID$.
  2. Execute the CMIngestion batch for the batch date.
 The batch should not be executed for any past dates, but only for the batch date on which the batch failed.

PurgeStagingTables Batch Details

The following table provides the tasks that are configured for the PurgeStagingTables batch. These tasks must be executed in the following order:

Table 20-77 PurgeStagingTables Batch Details

Sequence Tasks for PurgeStagingTables Batch Jobs for PurgeStagingTables Batch Pipelines for PurgeStagingTables Batch
1 StartBatch Not Applicable Not Applicable
2 DeleteStagingTable Delete Staging Data Delete Staging Data
3 EndBatch Not Applicable Not Applicable

PurgeTMTables Batch Details

The following table provides the tasks that are configured for the PurgeTMTables batch. These tasks must be executed in the following order:

Table 20-78 PurgeTMTables Batch Details

Sequence Tasks for PurgeTMTables Batch Jobs for PurgeTMTables Batch Pipelines for PurgeTMTables Batch
1 StartBatch Not Applicable Not Applicable
2 DeleteTMAlertsTable Purge TM Alerts Tables Purge TM Alerts Tables
3 EndBatch Not Applicable Not Applicable

PurgeAMIngestionTables Batch Details

The following table provides the tasks that are configured for the PurgeAMIngestionTables batch. These tasks must be executed in the following order:

Table 20-79 PurgeAMIngestionTables Batch Details

Sequence Tasks for PurgeAMIngestionTables Batch Jobs for PurgeAMIngestionTables Batch Pipelines for PurgeAMIngestionTables Batch
1 StartBatch Not Applicable Not Applicable
2 DeleteAMTable Delete Business Data Delete Business Data
3 EndBatch Not Applicable Not Applicable

PurgeCMTables Batch Details

The following table provides the tasks that are configured for the PurgeCMTables batch. These tasks must be executed in the following order:

Table 20-80 PurgeCMTables Batch Details

Sequence Tasks for PurgeCMTables Batch Jobs for PurgeCMTables Batch Pipelines for PurgeCMTables Batch
1 StartBatch Not Applicable Not Applicable
2 DeleteCMTable Delete Case Management Data Delete Case Management Data
3 EndBatch Not Applicable Not Applicable

PurgeAMLToCMEventData Batch Details

The following table provides the tasks that are configured for the PurgeAMLToCMEventData batch. These tasks must be executed in the following order:

Table 20-81 PurgeAMLToCMEventData Batch Details

Sequence Tasks for PurgeAMLToCMEventData Batch Jobs for PurgeAMLToCMEventData Batch Pipelines for PurgeAMLToCMEventData Batch
1 StartBatch Not Applicable Not Applicable
2 CleanAMLCMEventData Clean AML to CM Event Data Clean AML to CM Event Data
3 EndBatch Not Applicable Not Applicable

PurgeKYCTables Batch Details

The following table provides the tasks that are configured for the PurgeKYCTables batch. These tasks must be executed in the following order:

Table 20-82 PurgeKYCTables Batch Details

Sequence Tasks for PurgeKYCTables Batch Jobs for PurgeKYCTables Batch Pipelines for PurgeKYCTables Batch
1 StartBatch Not Applicable Not Applicable
2 DeleteKYCTable Purge Know Your Customer Data Purge Know Your Customer Data
3 EndBatch Not Applicable Not Applicable

PurgeCSTables Batch Details

The following table provides the tasks that are configured for the PurgeCSTables batch. These tasks must be executed in the following order:

Table 20-83 PurgeCSTables Batch Details

Sequence Tasks for PurgeCSTables Batch Jobs for PurgeCSTables Batch Pipelines for PurgeCSTables Batch
1 StartBatch Not Applicable Not Applicable
2 DeleteCSTable Purge Customer Screening Data Purge Customer Screening Data
3 EndBatch Not Applicable Not Applicable

PurgeCMIngestion Batch Details

The following table provides the tasks that are configured for the PurgeCMIngestion batch. These tasks must be executed in the following order:

Table 20-84 PurgeCMIngestion Batch Details

Sequence Tasks for PurgeCMIngestion Batch Jobs for PurgeCMIngestion Batch Pipelines for PurgeCMIngestion Batch
1 StartBatch Not Applicable Not Applicable
2 DeleteCMTable Purge Case Management Ingestion Data Purge Case Management Ingestion Data
3 EndBatch Not Applicable Not Applicable

MonitorCSPurgeBatch Batch Details

The following table provides the tasks that are configured for the MonitorCSPurgeBatch batch. These tasks must be executed in the following order:

Note:

This batch will completely purge all data from the ADW database. It is recommended not to run it unless you intend to clean up all data from the reporting tables.

Table 20-85 MonitorCSPurgeBatch Batch Details

Sequence Tasks for MonitorCSPurgeBatch Batch Jobs for MonitorCSPurgeBatch Batch Pipelines for MonitorCSPurgeBatch Batch
1 StartBatchADW Not Applicable Not Applicable
2 FCCRSourceTabPurge FCCR SOURCE TABLE PURGE JOB FCCR SOURCE TABLE PURGE
3 FCCRSourceDeltaTabPurge FCCR SOURCE DELTA TABLE PURGE JOB FCCR SOURCE DELTA TABLE PURGE
4 FCCRTargetTabPurge FCCR TARGET TABLE PURGE JOB FCCR TARGET TABLE PURGE JOB
5 EndBatchADW Not Applicable Not Applicable