1. Extensibility Guide
  2. ETL Hooks
  3. Execute ETL Hooks

Execute ETL Hooks

The ETL Hooks can be executed at different staging levels.

  1. Before populating the reporting tables, execute CUSTOM ROUTINE BEFORE STAGE TABLES POPULATION, and/or CUSTOM ROUTINE BEFORE REPORTING TABLES POPULATION.

  2. Before populating the signal tables, execute ETL Hooks CUSTOM ROUTINE AFTER REPORTING TABLES POPULATION, CUSTOM ROUTINE BEFORE SIGNAL HELPER TABLES POPULATION, and/or CUSTOM ROUTINE AFTER SIGNAL HELPER TABLES POPULATION.

  3. After populating reporting and signal tables and before populating the results, execute CUSTOM ROUTINE AFTER ETL.

Figure 3-1 Sequence of ETL Hook Execution

Sequence of ETL Hook Execution

To execute an ETL Hook:

  1. On the Modify Argus Mart page, enter a procedure name in the text box relevant to the ETL Hook execution stage. The ETL type can be initial or incremental where the customized data will be fetched. The ETL Hook will look for the database object that matches the procedure name.

    A procedure name denotes the ETL Hook that comprises the custom procedure, or steps. Based on the stage (initial or incremental) where this procedure is being called, the data is modified before sending it to the destination tables.

  2. If the database object (or procedure) is found, the ETL is executed as follows:

    • If the ETL executes without any errors, then move to the next procedure.

    • If the ETL executes with errors, then log the error(s) and exit.

      When executing ETL, if there is any error while populating the staging schema tables, error(s) are logged in the table ETL_STAGE_LOG, whereas while populating the mart schema tables, error(s) are logged in the table ETL_MART_LOG.

  3. If the database object (or procedure) is not found, then log the error(s) and fail the ETL. In this case, the ETL may be executed if you have explicitly created an exception-handling for such cases to absorb any exceptions and move on to the next procedure.

    Optionally, to resolve this issue, create a procedure of that name, provide an existing procedure name, or remove the configuration.

WARNING:

  • You should not modify the existing data or names of the objects. Although additional objects can be created, all changes should first be tested on a test environment, before implementing them in a production environment.

  • No data commit should be done within ETL Hooks.

Parent topic: ETL Hooks