Setting up the Incremental File Adapter

To register an incremental file data source:

1. From the Data Integration home page, and then Actions, select Applications.

2. On the Applications page, click (Add icon).

3. On the Create Application page, and then Category, select Data Source.

   

4. From Type, select Incremental File.

5. From File, specify the initial source data file to use as the template.

   Click to search for a file from the File Browser page.

   When selecting a file, note the following:

   • The source data file must be a delimited data file.

   • Data files used must contain a one-line header, which describes the delimited columns.

   • Both numeric and non-numeric data can be loaded.

6. Optional: In Prefix, specify a prefix to make the application name unique.

   The prefix is concatenated with the application name to form a unique application name. For example, if you want to name an application with the same name as an existing one, you can assign your initials as the prefix.

7. Click OK.

8. From the Data Integration home page, click (Create), and then select Integration.

   The General page is displayed in Create Integration view.

9. Create the integration between the source and target application and then click Save and Continue.

   1. In Name and Description, enter a name and description for the new integration.

   2. In Location, enter a new location name, or pick an existing location to specify where to load data.

   3. From the Source () drop-down, select File.

   4. From the File Browser, navigate to and select the incremental file, and then click OK.

      When you select the incremental file, click File Options to preview and manage the format of the contents on the File Import page.

   5. From the Target (), select the target application to which to load the incremental data.

   6. From Cube, select the plan type of the target system.

   7. From Category, select the category mappings for categorizing and mapping source system data to a target Scenario dimension member.

      The categories listed are those that you created in the setup, such as "Actual." For more information, see Managing Category Mappings.

   8. Optional: Select any applicable location attributes for the integration. For more information, see Selecting Location Attributes.

   9. Click Save and Continue.

10. On the Map Dimensions page, map dimensions (build an import format) between the source and target.

    See Mapping Dimensions.

    1. From Type, select the delimited file type.

       For example, you might select Delimited - Numeric Data as the format of the file.

    2. From the File Delimiter drop-down, select a type of delimiter.

       Available delimiters:

       • comma (,)

       • exclamation (!)

       • semicolon (;)

       • colon (:)

       • pipe (|)

       • tab

       • tilde (~)

    3. In the mappings grid, map the source columns in the source data-load file to the dimensions in the target application.

       The dimensions from the target application are populated automatically.

       If the import format has already been defined for the file, then the source and target columns are mapped automatically.

       If you are adding a new import format or editing an existing import format, complete the following:

       • In Column, specify the field number from the file to import.

       • In Select Source Dimension, specify the name of the source dimension to assign to the target application.

         Multiple source columns of the same dimension can be mapped to target dimensions. For example, you can map four "Account" source columns.

       • Add a source or target expression: assign an expression that operates on values directly from the source or target.

         See Using Source Expressions and Using Target Expressions.

    4. Optional: For a comma delimited file, select an additional row to map in the import format by clicking to the right of a row and selecting the row to add from the drop-down.

       Available rows:

       • Source Period
         • Year
         • Period
         • Period Number
       • Currency
       • Attribute
       • Description
       • Dimension Row
         • Account
         • Version
         • Entity
         • View

       You can also skip a row.

       For more information, see Creating the Dimension Maps.

    5. Click Save and Continue.

11. From Map Members page, map any members from the source to target.

    For more information, see Mapping Members.

12. From the Options page, click the Filters tab.

    Note:

    You can also select the Source File, Incremental Processing Options, and Last Process ID options on the Filters tab on the Run Integration page.

13. From Source File, select the data file name that contains the data you are loading. It may be the same one from which you created the data source application, or another file that has data as well as the appropriate header.

    Select the file that contains your data, as before. It may have the same name as the original file, or it may have a new name. The differences in the file (i.e., the incremental load file) is created automatically between the two files loaded. So, if file A.txt has 100 rows and file B.txt has 300 rows where the first 100 are identical, your first load should select file A.txt when the ID is 0. The second load will be against file B.txt and the ID automatically points to the load ID that was assigned to A.

14. From Incremental Processing Options, select the method for sorting data in the source file.

    Available options:

    • Do not sort source file—Source file is compared as provided. This option assumes that the source file is generated in the same order each time. In this case, the system performs a file comparison, and then extracts the new and changed records. This option makes the incremental file load perform faster.
    • Sort source file—Source file is sorted before performing the file comparison for changes. In this option the source file is first sorted. The sorted file is then compared to the prior sorted version of this file. Sorting a large file consumes a lot system resources and performs slower.

    Note:

    If you have an integration that uses the Do Not Sort option and then switch to the Sort source file option, then the first load has invalid results since the files are in different order. Subsequent runs load data correctly.

15. The Last Process ID shows the last run ID for the original source data file.

    When the load is first run for the original data file, the Last Process ID shows the value of 0.

    When the load is run again, the Last Process ID shows the run number of the last load.

    If the newly created file comparison version and the original data file shows no differences, or the file is not found, the value of the Last Process ID is assigned to the last load ID that ran successfully.

    To reload all data, set the Last Process ID back to 0, and select a new source file to reset the baseline.

16. Click Save.

17. From the Data Integration home page, select the integration, and then click .

18. From the Run Integration page, select the Options tab

19. Complete the following options and click Run.

    • Import Mode
    • Export Mode
    • Start Period
    • End Period

    For more information on running an integration, see Running an Integration.