Introduction to Workflows

A workflow is made up of multiple data flows organized in a sequence in which they must be executed. Each data flow is executed as a step. You can also add workflows within a workflow.

When you execute a workflow, a data flow either succeeds or fails. Depending on whether the first data flow succeeds or fails, you can choose the next data flow that must be executed.

Here is an example of a workflow:

Description of introduction-workflows.png follows
Description of the illustration introduction-workflows.png

In this example, the workflow performs the following actions:

  1. Execute the "Weekly FDA feed" data flow.
  2. If the "Weekly FDA feed" data flow execution is successful, execute the "Weekly CDC alerts" data flow.
  3. If the "Weekly CDC alerts" data flow execution is successful, execute the "Load Medicare" data flow.
  4. If any of the above data flow fails, then execute the "Audit_error_log" data flow.

Create a New Workflow

To create a new workflow for your project:
  1. On the Home page, click the required Project title.

    You are navigated to the Project Details page.

  2. In the left pane, click Workflows.

    The Workflow page appears.

  3. On the Workflow page, click Create Workflow.

    The Create Workflow page appears.

  4. In the Name field provide a name for the new workflow and click Create.

    The new workflow is created and listed in the Workflow page with its Name and Folder details.

  5. Click the Workflow to configure the Workflow Details.
  6. From the left panel drag the data flows, workflows, or data loads that you want to run in the workflow. If you have connected to any other Data Transforms instance, you can also add data loads that you have created in that Data Transforms instance. See Creating a Data Transforms Connection for Remote Data Load and Running a Data Load for more information.
  7. Select either the ok (green arrow) icon, the not ok (red arrow) or the ok/not ok (black arrow) in the toolbar.

    This defines the mode that subsequent links drawn on the canvas will be set to.

    • ok (green) defines the success path.
    • not ok (red) defines the failure path.
    • ok/not ok (black) defines a path that is followed on success or failure.
    • Sleep (clock icon) defines the duration of delay between steps.
  8. Select the data flow and click the Connector icon (Connector icon) next to it to connect it with the next data flow.
  9. After defining all the required Workflow details,
    • Click save icon, to save the created/designed workflow.
    • Click auto layout icon, to align the nodes of the designed workflow.
    • Click execute icon, to execute the created workflow.
    • Select a single step in the canvas and click the Execute Step icon (Execute Step icon), to execute only the selected data flow or workflow.

      To check the status of the workflow, see the Status panel on the right below the Properties Panel. For details about the Status panel, see Viewing Status of Data Loads, Data Flows, and Workflows. This panel shows the link to the Job ID that you can click to monitor the execution status on the Jobs page. For more information about jobs see Managing Jobs.

    • Click Zoom-in and zoom-out icons, to maximize or minimize the created workflow diagram in the design canvas.

      The newly created workflows get listed in the Project Details page. The following details are displayed:

      • Name of the Workflow
      • Folder corresponding to the workflow - Default Folder

      To view the details of a workflow, click the name of the workflow and you are navigated to the Workflow Details page.

    In the left pane, you can search for the required Data Flow or Workflow using the Name filter. In the Name text box, enter the name of the required Data Flow or Workflow.

    Select a step in the canvas and check the Properties Panel available at the right side of the design canvas to know the following properties of the selected step in a created Data Flow or Workflow:

    • Name
    • Linked Object


      You cannot edit this field.
    • Step -
      • First Step - Select the First Step check-box, to execute the selected step as the first step for execution in a Data Flow or Workflow.


        You can select only a single step as a first step for execution in a Data Flow or Workflow.
    • Number of attempts on Failure
    • Time between attempts in seconds(s)
    • Log steps in journal - Select any of the following options from the drop-down box:
      • Always
      • Never
      • Error