Create an Integration Task to Configure and Run a Data Flow
In Data Integration, an integration task lets you to prepare your data flows for execution.
You can add parameters to a data flow, so that it can be used with different values each time you run the task. Default parameter values can be updated within an integration task. You can then validate the data flow with the updated parameters, before you publish the task to an Application.
In this tutorial, you:
- Create an integration task to configure runtime context for a data flow.
- Create an Application to contain and run published tasks.
- Publish a task to an Application.
- Create and monitor a task run.
Before You Begin
1. Creating an Integration Task
In the tab bar, click Open tab (plus icon), and then select Projects.
- On the Projects page, click DI_Lab.
- On the DI_Lab project details page, click Tasks in the left submenu.
On the Tasks page, click Create task and select Integration.
The Create integration task page opens in a new tab.
On the Create integration task page, change the Name to
Load Customers Lab.
The value in the Identifier field is auto-generated based on the value you enter for Name. You can change the auto-generated value, but once you save the task, you cannot update it again.
Entering a Description for the task is optional.
- In the Data Flow section, click Select.
- In the Select a data flow panel, select Load Customers and Revenue Data, and then click Select.
- Click Create and close.
2. Creating an Application
An Application is a container for published tasks, data flows, and their dependencies. You can run published tasks in an Application for testing, or roll them out into production.
- On the workspace Home page, in the Quick actions tile, click Create application.
- On the main Create application page, click Create blank application.
Lab Applicationfor the Name.
- Verify or change the Compartment in which this Application is to be created.
The Application details page for
Lab Applicationopens in a new tab.
3. Publishing a Task to an Application
- On the DI_Lab project details page, click Tasks in the submenu.
- In the Tasks list, click the Actions icon (three dots) for Load Customers Lab and then select Publish to application.
- In the Publish to application dialog, select Lab Application.
A notification message displays. When you publish a task, a patch is created in the Application.
You can still modify the task or edit its data flow without impacting the published task. This enables you to test a version of your data flow, while working on some new changes.
Go to the Lab Application details page, and click Patches on the left submenu to view details of the task patch.
A patch contains updates to a published task in an Application.When you publish a task, a publish patch is created. When you unpublish a task, an unpublish patch is created. You can find a list of patches and their status on the Application details page. Learn more about Patches.
Click Refresh to get the latest status updates.
When the status of the patch changes to Success, a published task entry is created on the Tasks list of the Application details page.
Click Tasks on the Lab Application details page.
The Tasks page shows the list of tasks you have published to the Application. You run published tasks from the Tasks page.
4. Creating and Monitoring a Task Run
- On the Lab Application details page, click Tasks in the submenu on the left.
In the Tasks list, click the Actions icon (three dots) for Load Customers Lab and then select Run.
A success message appears. Running a task creates a task run on the Runs page of the Application. You're automatically brought to the Runs page, where you can view all task runs and their status. The initial status of a run is
Click Refresh to get the latest task run status updates.
Learn more about task runs
When the run status turns to Success, you can see the values for Data read and Rows written.
Now that you've created an integration task to configure and run a data flow, you can Load Data into Autonomous Data Warehouse Using a Data Loader Task.