10.7.1 Populating Workspace

The workspace is populated with data from the datasets in External sources. When you are creating a workspace the table definitions are created. The Data movement from production to simulation occurs when you populate the screen.

To populate the Workspace, follow these steps:

  1. Navigate to the Workspace Summary page. The page displays Workspace records in a table.
  2. Click Action next to corresponding Workspace to launch Workspace and select Populate

    Workspace to populate the Workspace with data from a dataset data in the Populate Workspace window.

  3. For using Template for Populating Workspace, follow these steps:
    1. Click Use Template. Select KYCJurisdictionFilterTemplate.zip.
    2. Add the required SQL filters.
    3. Click Populate Workspace and then select either Create Batch or Create and Execute Batch.

      The following table provides field descriptions for the Oracle Populate Workspace window.

      Table 10-6 Populate Workspace

      Object Type Object Name
      Workspace Code The code of the Workspace.
      Purpose The description for the Workspace
      Creation Date The date on which the Workspace was created.
      Data Source Type The source of data. The value can be the OFSAA Data Schema or an external data source.
      Data Filter - Global Enter the data filter that needs to be applied on all the tables selected for data sourcing. For example: If MISDATE is equal to Today, then it is applied to all tables (wherever it is available) for selected Data Sources during population. If this field is not found (MISDATE) in the tables, it is not updated.
      Data Filter - Table level Provide the data filters individually on the tables here. NOTE: You can provide multiple table names for the same SQL filter. For example, there are two tables called Student and Employee in the target data source, and below filters are applied: l MISDATE as Today for Student and Employee tables l ID as 1 for Student table Then, Student table will be populated with MISDATE and ID filters and Employee table will be populated with only MISDATE filter. Global Filters will not be applicable for those tables on which filters have been applied individually. If the same table name is provided in more than one rows here, then filter condition is generated as a conjunction of all the provided filters.
      Fetch Size Enter the Fetch size for data upload.
      Batch Commit Size Enter the Batch Commit size for data upload.
      Write Mode You can either overwrite the existing data (truncate and insert) or to append to the existing data. You can choose to either overwrite the data or append to the existing data.
      Rejection Threshold

      The following two options are available:

      Custom Rejection Threshold: Enter the maximum of number of inserts that may fail for any of the selected tables. You can provide the maximum number of inserts that can fail while loading data to a given table from all the sources. In case of threshold breach, all the inserts into the particular target schema will be rolled back. However, it will continue with populating the next target schema.

      Unlimited: Here, all the errors will be ignored during the data population.
  4. Click Populate Workspace to start the process.

    Here, you can create the batch using Create Batch, or create and execute using the Create and Execute Batch option. On selecting either of these options, a workspace population task gets added to the batch.

    Note:

    You may require approval from an approver to populate the workspace.
    • When you select the Create and Execute Batch option, it allows you to create batch and triggers the batch as well.
    • When you select the Create Batch option, it allows you to prepare the batch and then execute or schedule the batch at a later time through Scheduler Service window. The Workspace population task execution can be tracked in the Monitor Batch window.
  5. Navigate to Scheduler Service and select Define Task.
  6. Enter the following parameters for workspace population.
    • Additional Parameters : Enter the Additional Parameters in following format: 
      {"fetch_size" :10, "batch_commit_size" :1000, "rejection_threshold"
:"UNLIMITED", "write_mode" :"OVERWRITE"}

      Global Filter provided input will be applied as a data filter on all the tables selected for data sourcing.

    • Table Filter : You can provide data filters individually on the tables here. You must provide multiple table names for the same SQL filter. Global Filters will not be applicable for those tables on which filters have been applied individually. In case the same table name is provided in more than one rows here, the filter condition will be generated as a conjunction of all the provided filters. Enter the Table Filters in following the format: 
      [{"id":1,"filter":"","tables":["TABLE1", "TABLE2"]},
{"id":2,"filter":"","tables":["TABLE2"]}]

      Note:

      You can run workspace population for a given workspace any number of times. New tables may be added to the definition. Any new table added to the definition, that is not present in the target schema will be physicalized on update of the workspace. Also, you can add new sources if required. Any table that is deselected from the data sourcing definition will not be dropped.
  7. To drop and create Sequence in the Workspace Schema:
    1. DROP SEQUENCE FCC_KYC_MODEL_SIMULATION_SEQ;
    2. Get the max value N_MODEL_SIM_SKEY from table FCC_KYC_MODEL_SIMULATION SELECT MAX(N_MODEL_SIM_SKEY) FROM FCC_KYC_MODEL_SIMULATION; -- value
    3. CREATE SEQUENCE FCC_KYC_MODEL_SIMULATION_SEQ MINVALUE 1 MAXVALUE 999999999999999999999999999 INCREMENT BY 1 START WITH ##value+1## CACHE 20 NOORDER NOCYCLE NOKEEP NOSCALE GLOBAL;