1. Implementation Guide
  2. Data File Generation
  3. Files Types and Data Format
  4. Second-generation SaaS Architecture

Second-generation SaaS Architecture

Note:

If you are an existing RI or Science customer in our older architecture, or your environments were provisioned initially on version 19 or earlier, then this information does NOT apply. Refer to the section after this for Legacy context file handling.

Before creating and processing a data file on the platform, choose the fields that will be populated and instruct the platform to only look for data in those columns. This configuration is handled through the use of Context (CTX) Files that are uploaded alongside each base data file. For example, the context file for PRODUCT.csv will be PRODUCT.csv.ctx (appending the .ctx file descriptor to the end of the base filename).

Within each context file you must provide a single column containing:

  •  One or more parameters defining the behavior of the file load and the format of the file.

  •  The list of fields contained in the source file, in the order in which they appear in the file specification:

    • #TABLE#<Staging Table Name>

    • #DELIMITER#<Input Value>#

    • #DATEFORMAT#<Input Value>#

    • #REJECTLIMIT#<Input Value>#

    • #RECORDDELIMITER#<Input Value>#

    • #IGNOREBLANKLINES#<Input Value>#

    • #SKIPHEADERS#<Input Value>#

    • #TRIMSPACES#<Input Value>#

    • #TRUNCATECOL#<Input Value>#

    • #COLUMNLIST#<Input Value>#

      <COL1>

      <COL2>

      <COL3>

The following is an example context file for the CALENDAR.csv data file:

File Name: CALENDAR.csv.ctx

File Contents:
#TABLE#W_MCAL_PERIOD_DTS#
#DELIMITER#,#
#DATEFORMAT#YYYY-MM-DD#
#REJECTLIMIT#1#
#RECORDDELIMITER#\n#
#IGNOREBLANKLINES#false#
#SKIPHEADERS#1#
#TRIMSPACES#rtrim#
#TRUNCATECOL#false#
#COLUMNLIST#
MCAL_CAL_ID
MCAL_PERIOD_TYPE
MCAL_PERIOD_NAME
MCAL_PERIOD
MCAL_PERIOD_ST_DT
MCAL_PERIOD_END_DT
MCAL_QTR
MCAL_YEAR
MCAL_QTR_START_DT
MCAL_QTR_END_DT
MCAL_YEAR_START_DT
MCAL_YEAR_END_DT

The file must be UNIX formatted and have an end-of-line character on every line, including the last one. As shown above, the final EOL may appear as a new line in a text editor. The #TABLE# field is required: it indicates the name of the database staging table updated by the file. The COLUMNLIST tag is also required: it determines the columns the customer uses in their .dat or .csv file. The column list must match the order of fields in the file from left to right, which must also align with the published file specifications. Include the list of columns after the #COLUMNLIST# tag. Most of the other parameters are optional and the rows can be excluded from the context file. However, this will set values to system defaults that may not align with your format.

Note:

Both RI and Science can use these context files to determine the format of incoming data.

The server maintains a copy of all the context files used, so you do not need to send a context file every time. If no context files are found, the Analytics Platform uses the last known configuration.

For additional format options, the available values used are from the DBMS_CLOUD package options in ADW.