1. Implementation Guide
  2. Data File Generation
  3. Files Types and Data Format
  4. Context Files

Context Files

Before creating and processing a data file on the platform, choose the fields that will be populated and instruct the platform to only look for data in those columns. This configuration is handled through the use of Context (CTX) Files that are uploaded alongside each base data file. For example, the context file for PRODUCT.csv will be PRODUCT.csv.ctx (appending the .ctx file descriptor to the end of the base filename).

Within each context file you must provide a single column containing:

  •  One or more parameters defining the behavior of the file load and the format of the file.

  •  The list of fields contained in the source file, in the order in which they appear in the file specification:

    • #TABLE#<Staging Table Name>#

    • #DELIMITER#<Input Value>#

    • #DATEFORMAT#<Input Value>#

    • #REJECTLIMIT#<Input Value>#

    • #RECORDDELIMITER#<Input Value>#

    • #IGNOREBLANKLINES#<Input Value>#

    • #SKIPHEADERS#<Input Value>#

    • #TRIMSPACES#<Input Value>#

    • #TRUNCATECOL#<Input Value>#

    • #COLUMNLIST#<Input Value>#

      <COL1>

      <COL2>

      <COL3>

The following is an example context file for the CALENDAR.csv data file:

File Name: CALENDAR.csv.ctx

File Contents:
#TABLE#W_MCAL_PERIOD_DTS#
#DELIMITER#,#
#DATEFORMAT#YYYY-MM-DD#
#REJECTLIMIT#1#
#RECORDDELIMITER#\n#
#IGNOREBLANKLINES#false#
#SKIPHEADERS#1#
#TRIMSPACES#rtrim#
#TRUNCATECOL#false#
#COLUMNLIST#
MCAL_CAL_ID
MCAL_PERIOD_TYPE
MCAL_PERIOD_NAME
MCAL_PERIOD
MCAL_PERIOD_ST_DT
MCAL_PERIOD_END_DT
MCAL_QTR
MCAL_YEAR
MCAL_QTR_START_DT
MCAL_QTR_END_DT
MCAL_YEAR_START_DT
MCAL_YEAR_END_DT

The file must be UNIX formatted and have an end-of-line character on every line, including the last one. As shown above, the final EOL may appear as a new line in a text editor. The #TABLE# field is required: it indicates the name of the database staging table updated by the file. The COLUMNLIST tag is also required: it determines the columns the customer uses in their .dat or .csv file. The column list must match the order of fields in the file from left to right, which must also align with the published file specifications. Include the list of columns after the #COLUMNLIST# tag. Most of the other parameters are optional and the rows can be excluded from the context file. However, this will set values to system defaults that may not align with your format.

Note:

Both RI and AI Foundation can use these context files to determine the format of incoming data.

The server maintains a copy of all the context files used, so you do not need to send a context file every time. If no context files are found, the Analytics and Planning uses the last known configuration.

For additional format options, the available values used are from the DBMS_CLOUD package options in ADW.

If you want to retrieve the latest copy of the context files, the RI_ZIP_UPLOAD_CTX_JOB job in process RI_ZIP_UPLOAD_CTX_ADHOC can be run from the AIF DATA standalone schedule in POM. This job will extract all the context files from the custom_ext_table_config directory, package them in a zip file, and upload that file to Object Storage. The zip file is named RAP_CTX.zip, and will use ris/outgoing as the prefix for File Transfer Services (FTS) to access it.

In addition to being able to obtain copies of the files, there is also a database table named C_DIS_ADW_EXT_TABLE_CONFIG that holds the context file information that was last uploaded to the database. Except for the COLUMN_LIST and FORMAT_OPTIONS columns, the data in the other columns on the table is editable using the Control & Tactical Center screen in AI Foundation, so you can provide override values. The table does not have any initial data; it is populated when a CTX file is processed by RI_UPDATE_TENANT_JOB. When a CTX file is provided and data is also present in the table, the priority is to use the CTX file. If a CTX file was not provided in the current batch run, then the data on this table will be used. After the batch run, this table will reflect the most recently used CTX file configurations.

A change in format data in the table will trigger an update to ADW only if the values are different from what was last sent. This is done by comparing the entries at the FORMAT_OPTIONS column. Modifying the COLUMN_LIST in this table will not trigger a request to ADW to update the options. COLUMN_LIST is not editable through the Control & Tactical Center screen, as it only serves as a view to show the contents of the last payload sent to ADW. Sending the updates through a CTX file is the preferred method for modifying the column list. If no CTX files are provided, the RI_UPDATE_TENANT_JOB will end immediately instead of pushing the same configurations to ADW again. If you notice slow performance on this job then you can stop providing CTX files when they are not changing, and the job will finish within 10-20 seconds.

There is an OVERRIDE_REJECTLIMIT_TO_DT column on the table that will determine whether a REJECTLIMIT value other than zero is used. If this date column is null or is already past the current date, then the REJECTLIMIT will be reset to 0 and will trigger an update to ADW. The REJECTLIMIT value provided in the table will be used until the date specified in this column.