Default Data in the tables

4.6.1 Default Data in the tables

The following tables store the table structure definition for Party Master:

FCC_M_ER_TABLES: This table contains information about different tables required by the product as part of an Entity Resolution process. The values in the column V_FSDF_VERSION differentiate FSDF versions to the tables belong to. This is used for creating Datasets and Data Surviving Rules.
FCC_M_ER_TABLES_TL: This table contains translative information for FCC_M_ER_TABLES, with multiple translations based on the Locale column.
FCC_M_ER_COLUMNS: This table contains information about columns a table has. It has mappings of columns and tables so that you can get the table's available columns information based on table Id. This is used for creating Datasets and Data Surviving Rules.
FCC_M_ER_ATTRIBUTE: This table contains information about columns. It has a column's information such as logical name and description. This is used for creating Datasets and Data Surviving Rules.
FCC_M_ER_ATTRIBUTE_COLUMN_MAP: This table contains mapping information of attributes and columns. It also stores information about the relationship between tables. This is used for creating Datasets and Data Surviving Rules.
FCC_M_ER_ATTRIBUTE_TL: This table contains translative information for table FCC_M_ER_ATTRIBUTE, which can have multiple translation information based on the Locale column.

The following tables store the Dataset definition:

FCC_M_ER_DATASET: This table contains information about Datasets. It has a master (parent) table information like STG_PARTY_MASTER_PRE (when resolving FSDF data), output table, and pipeline Id, and tables where the data will flow when the data survival job is run.
FCC_M_ER_DATASET_GROUP: This table contains information about a Group of other tables that are part input dataset. It has an input group table like STG_PARTY_ADDRESS_PRE and also stores the join condition with the Master table, STG_PARTY_MASTER_PRE.
FCC_M_ER_DATASET_MAP: This table contains information about the mapping table, which provides the relationship between the Master and Group tables. For example, STG_PARTY_ADDRESS_MAP_PRE stores the relationship between the STG_PARTY_MASTER_PRE and STG_PARTY_ADDRESS_PRE tables.
FCC_M_ER_DATASET_TL: This table contains translative information for table FCC_M_ER_DATASET, which can have multiple translations based on the Locale column.

The following tables store the Preconfigured Match and Merge Ruleset:

FCC_MATCH_RULESET: This table contains the information of the Rulesets created in Matching Rules UI. It gives information like the Pipeline ID, Ruleset Name, and Ruleset Description and contains ruleset details in JSON format.
FCC_MERGE_RULESET: This table contains the information of the Rulesets created in Merge Rules UI. It gives information like the Pipeline ID, Ruleset Name, and Ruleset Description and contains ruleset details in JSON format.

The following tables store the Dataset Survival Rule:

FCC_DATASURV_RULES: This table contains the information on the Rules created in Data Survival Rules UI. It gives information like the Pipeline ID, Ruleset Name, and Ruleset Description and contains ruleset details in JSON format. This table contains information only for the Master table.
FCC_DATASURV_GROUPS: This table contains data survival rules, such as rule id, UI JSON, and query JSON. UI JSON is used on the UI side, and query JSON is used as input JSON for the Data survival engine. This table contains information only for child tables.
FCC_DATASURV_TYPE: This table contains information about different Data Survival Algorithms, such as Longest, Latest, Most Common, etc. There is a Type drop-down on Data Survival UI to choose values (fetched from this table) for a particular column.
If users want to add custom Data Survival method, follow these steps:
1. Open the Compliance Studio schema.
2. In the FCC_DATASURV_TYPE table, add a new row and update the following:
  1. N_TYPE_ID: Provide the numerical value based on the existing sequence order. For example, this FCC_DATASURV_TYPE table having 10 ids already then you should provide N_TYPE_ID as 11 for the new custom method.
  2. V_TYPE_NAME: Provide the name for the custom method. This name will be displayed in the Data Survival’s Type drop-down list in the Compliance Studio UI.
  3. V_TYPE_CD: Provide logical name for the custom method.
    
    Note:
    For the logical name, special characters are not allowed except underscore (_) and should not contain any spaces.
  4. F_IS_CUSTOM_TYPE: Set the value as “T”.
  5. N_SEQ_ID: Provide the numerical value based on the existing sequence order. For example, this FCC_DATASURV_TYPE table having 10 sequences already and you should provide N_SEQ_ID as 11 for the new custom method.
3. Navigate to <COMPLIANCE_STUDIO_INSTALLATION_PATH>/deployed/entity-resolution/ extensions/data-survival directory.
4. Open the UserDefinedMethods.py file and add the custom function inside the class UserDefinedMethods.
  For example, if you are adding custom function as gender_criteria then update as follows:
```
class UserDefinedMethods:
           def method(self, entity_list, func_name, entity_constraint=None,
entity_index=0):
          func = getattr(self, func_name)
          return func(entity_list, entity_constraint, entity_index)

          def gender_criteria(self, entity_list, entity_constraint=None,
entity_index=0):          
          if all(elem == 'A' for elem in entity_list):
             return 'A'
          elif all(elem == 'B' for elem in entity_list):
          log.info("Inside B {}".format(entity_list))
          return 'B'
    else:
     return 'U'
```
  Note:
  
  The custom method should be added in the <COMPLIANCE_STUDIO_INSTALLATION_PATH>/entityresolution/ extensions/data-survival/ UserDefinedMethods.py file as well. In case of reinstallation; the custom method is preserved.
  
  The method name for the custom data survival type in the python file should be the same as the value given for V_TYPE_CD column in the FCC_DATASURV_TYPE table.
  Data survival rules of out-of-the-box ER pipeline survive the “Latest” data based on FIC_MIS_DATE. Since data for ER is always considered as a complete snapshot for the extraction date (FIC_MIS_DATE), the FIC_MIS_DATE will be standard across the entire snapshot. Hence ER internally considers the additionally maintained D_LAST_UPDATED_DATE column in H$ tables to find out the latest data for survival. This is achieved by an additional set of metadata maintained in the following tables:
  - FCC_M_ER_PROCESSING_COLUMNS: This table stores the table name, column name, and ER pipeline id.
  - FCC_DS_REF_COLUMN_MAPPING: This table stores the table name, reference column name (the standard column of the table, i.e., FIC_MIS_DATE), target column name (the actual column on which “Latest” should be considered, i.e., D_LAST_UPDATED_DATE), and ER pipeline id.
    For Example, the sample records for both tables are as follows:
    
    Figure 4-30 Sample Record for FCC_M_ER_PROCESSING_COLUMNS
    
    Description of "Figure 4-30 Sample Record for FCC_M_ER_PROCESSING_COLUMNS"
    
    Figure 4-31 Sample Record for FCC_DS_REF_COLUMN_MAPPING
    
    Description of "Figure 4-31 Sample Record for FCC_DS_REF_COLUMN_MAPPING"
    
    Note:
    These metadata tables should be seeded with appropriate values in any similar use cases.

The following table stores the flattening data query:

FCC_STUDIO_ER_QUERIES: This table contains queries to fattening data from input tables for each pipeline id. The information in this table can be amended via an API if additional attributes need to be brought into matching.

The following tables to populate fields in Match and Merge Ruleset UI:

FCC_ER_INDEX: This table contains the index name on the ruleset UI screen in Source Index Name and Target Index Name Field.
FCC_IDX_M_JSON_MAP: This table contains the mapping of each index populated on OpenSearch, making the initial candidate selection for records to be scored by the matching engine. This is required for Match and Merge Rulesets mapping screen. You need to add custom attributes for mapping manually. For more information on how to map, see the Steps section.
FCC_ER_ATTRIBUTES: This table contains attributes matched in ruleset UI in Source and target attribute for the respective index.

Note:
The Original ID is not masked but underlying all the attributes are hidden using the F_IS_MASKED column in the fcc_er_attributes table. This attribute is applicable only for Merge and Split Global Entities UI.
FCC_IDX_M_LOOKUP: This table contains the file name/index name of synonyms and Stopwords, which are used to improve the performance of Name/Address matching.
FCC_IDX_M_LOOKUP_VALUES: This table contains populated values for the above index names.
FCC_ER_M_BKP_CONFIG: This table contains the backup and failure recovery details.