Multiple Data Origin Support

The data-loading service supports multiple data origin files to load data into the stage tables with different batches having different Data Origins.

To provide data using multiple Data Origin files, use the following file formats:

• For Non-Split Format: YYYYMMDD_Tablename_DataOrigin.CSV

  For example: 20201218_STG_CASA_TXNS_MAN.csv

  20201218_STG_CASA_TXNS_UK.csv

• For Split Format: YYYYMMDD_Tablename_DataOrigin_#.CSV

  For example: 20201218_STG_CASA_TXNS_MAN_1.csv

  20201218_STG_CASA_TXNS_MAN_2.csv

  20230727_STG_CASA_TXNS_UK_1.csv

  20230727_STG_CASA_TXNS_UK_2.csv

To execute batches using multiple data origin, update Schedule Batch parameters as follows:

• $DATAORIGIN$ : This should be the Data Origin Name which is provided in the file name. Example: MAN / UK
• $F_DATAORIGIN$: This must be set as True
  • If the value of $F_DATAORIGIN$ is False then the multiple data origins will not be considered. It will pick the CSV files without having the Data Origin name in the file format.
  • If the value of $F_DATAORIGIN$ is True then the multiple data origins will be considered. It will pick the CSV files which are having the Data Origin name in the file format.

    Note:

    There are no changes for existing or single data origin customers.