Data Survival

4.4.3 Data Survival

Note:

Ensure that only one pre-configured ruleset is enabled for Merging and Data Survival. See the Pre-configured Rulesets for Matching, Merging, and Data Survival section. The job will be failed with a unique constraint error if multiple rulesets are enabled.

Job

The ER_Run_Data_Survival_Engine.sh job performs the following:

ER_Merge_Engine: It triggers the merge engine, and records will be inserted in the mapping table based on the merge rules saved against the pipeline id argument.
ER_Data_Survival_Engine: It triggers the data survival engine, and final outputs will be stored in tables based on the dataset survival rule stored against pipeline id.

Configuration for Data Survival

FCC_ER_QUERIES_PROPERTIES: This table is to configure hints for SQL queries and hints for data survival (Job 3) rollback query is configured. This table is created in the Studio Schema on the Compliance Studio startup.

Steps

Navigate to <COMPLIANCE_STUDIO_INSTALLATION_PATH>/deployed/ficdb/bin directory.
Run the following command:
```
nohup ./ER_Run_Data_Survival_Engine.sh "<PIPELINE_ID>"
"<ER_SCHEMA_WALLET_ALIAS>" "<MATCH_TYPE>" "<BATCH_GROUP>" "<RUN_TYPE>" &
```
Note:
<BATCH_GROUP> refers to the FCC_PROCESSING_GROUP table in the Compliance Studio schema.

For example, you can use the following command for CSA_8128 pipeline.
FSDF 8128 version:
```
nohup ./ER_Run_Data_Survival_Engine.sh "CSA_8128"
"ER_SCHEMA_PP_ALIAS" "ER_SCHEMA_PP" "CSA_812" "FullLoad" "20151210"
"RUN" &
```
For more information about parameters, see the Parameters for Entity Resolution Job execution section.
Note:
- The user should not have Type “Distinct” and “All” together with other columns that return unique values in child tables.
- If the Batch, Backup, and Recovery processes fail when you execute the ER_Run_Data_Survival_Engine.sh, you need to re-run the same job again to ensure the Data is available in Archive only for the Mapping table (FCC_ER_MAPPING).
- To increase/decrease the execution efficiency according to the server size using ER_THREADS and ER_BATCH_SIZE parameters, perform the following:
  - Navigate to <COMPLIANCE_STUDIO_INSTALLATION_PATH>/ deployed/ficdb/bin
  - Open the ER_Run_Data_Survival_Engine.sh and set the following parameters:
    
    export ER_THREADS=<No of threads>
    
    export ER_BATCH_SIZE=<Batch Size>
Validate to ensure Global party IDs are generated for the Entities in the following Staging Output tables after executing the job:
- STG_PARTY_MASTER
- STG_PARTY_DETAILS
- STG_PARTY_EMAIL_MAP
- STG_PARTY_PHONE_MAP
- STG_ADDRESS_MASTER
- STG_PARTY_ADDRESS_MAP
- STG_CUSTOMER_IDENTIFCTN_DOC
Note:
Data Survival process expects the above STG tables to retain the snapshot of the previous FIC_MIS_DATE to complete the process successfully.

Cleanup Steps for Job termination

Execution of manual cleanup is required in case of any fatal user’s error. After contacting My Oracle Support, you can perform cleanup steps. For more information about cleanup steps, see the Cleanup Steps When the Data Survival Job Terminated Manually section.

For more information about parameters, see the Parameters for Entity Resolution Job execution section

Properties for Global Party ID Persistence

When global parties change (parties are added or removed), the system can be configured to either create a new global party id or to keep one of the existing ids depending on need to preserve global party in downstream systems.

The fcc_er_guid_persist_config table contains the configuration for Global Party ID Persistence.

The following table describes column/flag deatils in the FCC_ER_GUID_PERSIST_CONFIG.

Table 4-2 FCC_ER_GUID_PERSIST_CONFIG

Column Name/Flag	Description
V_ACTION	It represents the different actions that can be performed on the Global Party ID. The possible actions are: create rearrange add delete merge split and merge merge and add split
F_PERSIST_GUID	This flag represents whether the Global Party ID should be persisted or not whenever it undergoes change in an ER batch. The valid values are Y and N. The GUID is persisted if the flag is set to Y for the particular action. Note: This flag is not applicable for create and rearrange actions as these are manual actions.
F_MANUAL_APPROVAL	This flag represents manual approval is required when GUID undergoes change in an ER batch. The valid values are Y and N. If the flag is set to Y, then user gets the request to approve the changes in the UI. For more information, see the Pending - System Requests Tab section in the OFS Compliance Studio User Guide.
F_DEFAULT_VALUE	This flag represents the default value that will override the values present in the F_PERSIST_GUID and F_MANUAL_APPROVAL flags.
F_PERSIST_MANUAL_ACTI ON	This flag represents whether the Global Party ID can be persisted or not through manual action. The valid values are Y and N. The GUID is persisted if the flag is set to Y for the particular action. If the flag is set to N, then new Global Party ID will be created. Note: This flag is applicable only for Merge, Split, Create and Rearrange actions.

Note:

Only the flags in F_PERSIST_GUID and F_MANUAL_APPROVAL should be modified. F_DEFAULT_VALUE should not be modified for any action.
For add and delete actions, the GUID always persists irrespective of the user input in the F_PERSIST_GUID flag.
For delete action, manual approval is not required irrespective of the user input provided in the F_MANUAL_APPROVAL flag.
If F_PERSIST_GUID and F_MANUAL_APPROVAL flags for the split action are set to Y and Y respectively, then flags for split and merge action will also be considered as Y and Y regardless of the user input. Similarly, If F_PERSIST_GUID and F_MANUAL_APPROVAL flags for the split and merge action are set to Y and Y respectively, then flags for the split action will also be considered as Y and Y regardless of the user input.

The following image shows default configuration of the fcc_er_guid_persist_config table.

Figure 4-11 fcc_er_guid_persist_config table

Description of "Figure 4-11 fcc_er_guid_persist_config table"