Using Data Integration Applications

In Data Integration, an Application is a container at the workspace level for runtime objects such as published tasks and task runs.

You can schedule and run published tasks, and monitor and compare task runs in an Application.

Each workspace in Data Integration is created with a Default Application that you can use right away. You might, however, want to create your own Applications. You can use Applications to manage the life cycle of your Data Integration projects from development to testing to deployment in production. Data Integration lets you create an Application from scratch, or as a copy of another Application. When creating a copy, only the published tasks and their dependencies from the source Application are copied to the new target Application. In the target Application, you can map to different data assets, connections, and schemas. You can also refresh or sync the target Application when you have changes in the source Application.

The following describes how you might use Applications to manage your runtime objects for different projects in a workspace. For example, if you have a project for Enterprise Resource Planning (ERP) data flows and tasks, and a project for Human Capital Management (HCM), you might create an ERP Application and an HCM Application. In addition, you might also have different Applications for development and testing. The testing Applications can be created using existing resources in the respective development Applications, as shown in the following illustration. When changes are made to the number and type of published tasks in a development Application, you can sync the testing Application with the changes.

Applications in a workspace with two projects

The existing (source) Application that you want to copy does not have to be in the same compartment or workspace as the new (target) Application. For example, if you have separate workspaces for development, testing, and production, you can create your testing Application in workspace 2 as a copy of the development Application that is in workspace 1, and sync changes when needed. Similarly, you can create your production Application in workspace 3 as a copy of the testing Application that is in workspace 2, and sync changes when needed.

Applications in three workspaces

Accessing the Applications

Data Integration organizes Applications in a workspace by compartment.

You access the list of Applications in a compartment in a workspace using any one of the following ways:

  • Click Open tab (plus icon) in the tab bar of the workspace home page, and then select Applications.
  • Click the Applications link under the tab bar on the workspace home page.

On the Applications page, use the Compartment menu to navigate to the compartment where you have created Applications. By default the list of Applications in a compartment is sorted by the Updated column in descending order. To locate an Application in a long list, you can change the sort order, sort by Application name, or filter the list by name.

To filter the list, in the search field enter the complete name of the Application to do a full text search.

You can also use the Favorite Filter menu to display only those Applications that are currently marked as favorite. Clicking Clear removes the filter.

On the Applications list, the Actions menu next to each Application has these options:

  • View details: Displays the Application details and the runtime objects created in the Application such as published tasks and runs.
  • Edit: Lets you edit the name and description for the Application.
  • Copy OCID: Copies to the clipboard the OCID of the Application, which you can use with APIs
  • Copy application key: Copies to the clipboard the key of the Application, which you can use with APIs.
  • Delete: Lets you delete the Application.
Note

You cannot edit or delete the Default Application.

See also Viewing Application Details and Monitoring Applications.

Creating Applications

Data Integration lets you create Applications in several ways.

You can:

Creating an Application (Blank)

You can create Applications in any compartment in your workspace.

To create a blank Application:

  1. On the workspace home page, select Create application from the Quick actions tile.

    Alternatively, navigate to the Applications page and click Create application.

  2. Enter a name and description (optional).
    Note

    Identifier is a system-generated value based on the Name you enter. You can change the value only at the time of creation. You cannot change the identifier after the Application is created.
  3. Verify or change the Compartment in which this Application is to be created.
  4. (Optional) In the Tags section, add tags to help you search for Data Integration resources within your tenancy.
  5. Click Create.

    A notification message displays. After a short while, the Tasks page of the new Application displays.

Creating an Application by Copying From Existing Resources

You can create an Application by making a copy of an existing Application that's in the same workspace, another workspace, or another compartment.

Only the tasks and the resource dependencies (data assets, connections, schemas) from the existing Application (source) are copied into the new Application (target) as a patch.

If the new Application (target) is in a different tenancy from the existing Application (source), you need to set up cross-tenancy policies. See Policy Examples to Set Up Cross-Tenancy Access.

After the new Application (target) is created, you can map the references, and run the tasks. References you can map are the data assets, connections, and schemas associated with the published tasks, and also the tasks associated with published pipelines. You cannot unpublish the tasks, and you cannot publish other tasks to the target Application. You can, however, sync the target Application with changes from the source Application.

To create an Application as a copy of an existing Application:

  1. Navigate to the workspace where you want to create a new application by copying from an existing application.
  2. On the workspace home page, select Create application from the Quick actions tile.

    You can also click Create application on the Applications page.

  3. Select the option to use existing resources to create the application.
  4. Enter a name and description (optional).
    Note

    Identifier is a system-generated value based on the Name you enter. You can change the value only at the time of creation. You cannot change the identifier after the Application is created.
  5. Verify or change the Compartment in which this Application is to be created.
  6. Specify the source Application details:
    1. Enter the OCID of the tenancy where the source Application is.

      By default, the current tenancy OCID is provided in the field.

    2. Select the Compartment and Workspace that has the existing application you want to make a copy of.
    3. Select the Application to copy.
  7. (Optional) In the Tags section, add tags to help you search for Data Integration resources within your tenancy.
  8. Click Create.

    A notification message displays, with a link to the Patches page of the Application (target). Data Integration creates a sync patch when an Application is created as a copy of another Application. After a while, the Tasks page of the Application (target) displays.

In the new Application, you can remap the data assets and connections associated with the published tasks. You can also remap the schemas under connections, and the tasks that are associated with published pipelines in the Application.

Viewing Application Details

On the Applications page, select the Application you want to view.

Alternatively, you can select View details from the Actions menu for the Application you want to view.

From the Application details page, you can access the published tasks, schedules, task schedules, task runs, and patches that the Application contains.

You can also access the Monitor application page where you can see basic task run information, and the Application's references. References are the data assets, connections, and schemas that are associated with the published tasks, and also the tasks associated with published pipelines in the Application.

If the Application was created as a copy of another Application, Source application shows the name of the application that this Application is a copy of. If the source name is truncated, click the tooltip icon to see the full name. To see more details, click View details next to the tooltip icon. You can see details such as the OCID of the source Application, and the source's workspace and compartment and their OCID values. To sync the Application with changes from the Source application, see Syncing Applications.

On the Application details page, you can access:

Editing Applications

You can edit an Application to change its name or description.

The Default Application, however, cannot be edited.

To edit an Application:

  1. In the tab bar, click Open tab (plus icon) and select Applications.
  2. On the Applications page, select Edit from the Actions menu for the Application you want to edit.

    Alternatively, navigate to the Application details page and click Edit.

  3. Make your changes, then click Save changes to apply your modifications.

Deleting Applications

You can delete an Application from the Applications page.

The Default Application, however, cannot be deleted.

Deleting an Application permanently deletes it and the resources it contains. The Application cannot be restored after you delete it.

To delete an Application:

  1. In the tab bar, click Open tab (plus icon) and select Applications.
  2. On the Applications page, select Delete from the Actions menu for the Application you want to delete.

    Alternatively, navigate to the Application details page and click Delete.

  3. In the Delete application dialog, enter delete to confirm that you want to delete the Application, and then click Delete.

Syncing Applications

If an Application was created as a copy of another Application (the source), you might have to sync the Application with changes from the source Application.

When you sync the Application (target) with the source Application, any new task changes in the source are applied to the target Application as a patch. For example, if integration task B was removed from the source Application since the target was created or last synced, that change is patched to the target Application.

On the Sync application page, you can review the number and type of task changes in the source Application before you initiate a sync. You can filter the list by a task type and a change type (Add, Update, Remove).

To sync an Application:

  1. In the tab bar, click Open tab (plus icon) and select Applications.
  2. From the Applications page, select the Application (target) you want to sync.
  3. On the Application details page, click Sync next to the source Application name.
  4. On the Sync application page, review the changes from the source Application that are applied to the current (target) Application when you initiate the sync.
  5. Click Sync to start the patch process.
    A notification message displays, with a View Patch Status link.

Copying an Application OCID

You use Application OCIDs  in REST APIs, SDKs, and CLIs.

To copy an Application OCID:

  1. In the tab bar, click Open tab (plus icon) and select Applications.
  2. On the Applications page, click the Actions menu for the Application whose OCID you want to copy, and then select Copy OCID.

    The Application's OCID is copied to the clipboard.

    You can also copy the Application's OCID from the Application details page.

Moving an Application to a Different Compartment

You can move Data Integration Applications from one compartment to another compartment within the same workspace.

When you move an Application to a different compartment, all Data Integration resources within that Application, such as tasks and schedules, are moved.

Before you move an Application, ensure that you add the following policy:

allow service dataintegration to inspect compartments in compartment <target-compartment-name>

The policy ensures that you can check that the target compartment is in a good state after the Application is moved.

  1. In the tab bar, click Open tab (plus icon) and select Applications.
  2. From the Applications page, for the Application you want to move, select Move resource from the Actions menu.
  3. In the Move resource to a different compartment dialog, select the new compartment to move the application to.
  4. Click Move resource.

Using the API

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use the following operations to manage Applications in Data Integration:

When you publish a task or a group of tasks to an Application, or when you unpublish a task, Data Integration creates a patch in the Application. To manage patches, see these APIs.