Multiple Data Origin Support
The data-loading service supports multiple data origin files to load data into the stage tables with different batches having different Data Origins.
To provide data using multiple Data Origin files, use the following file
formats:
- For Non-Split Format:
YYYYMMDD_Tablename_DataOrigin.CSV
For example: 20201218_STG_CASA_TXNS_MAN.csv
20201218_STG_CASA_TXNS_UK.csv
- For Split Format:
YYYYMMDD_Tablename_DataOrigin_#.CSV
For example: 20201218_STG_CASA_TXNS_MAN_1.csv
20201218_STG_CASA_TXNS_MAN_2.csv
20230727_STG_CASA_TXNS_UK_1.csv
20230727_STG_CASA_TXNS_UK_2.csv
- $DATAORIGIN$ : This should be the Data Origin Name which is provided in the file name. Example: MAN / UK
- $F_DATAORIGIN$: This must be set as True
- If the value of $F_DATAORIGIN$ is False then the multiple data origins will not be considered. It will pick the CSV files without having the Data Origin name in the file format.
- If the value of $F_DATAORIGIN$ is
True then the multiple data origins will be
considered. It will pick the CSV files which are
having the Data Origin name in the file
format.
Note:
There are no changes for existing or single data origin customers.