Oracle Cloud Infrastructure Documentation

Data Sources

In Data Integration data assets represent the data sources that you use as sources and targets for your integration tasks.

See Supported Data Sources for Data Assets for the data source types that you can use to create the data assets for reading from sources and writing to targets. Data assets can be databases, flat files, and so on.

You create and access data assets from the Data assets page in a workspace. From the workspace's home page, click Data assets to access the Data assets page. The Data assets page lists all the data assets you have created in the workspace.

On the Data assets page, click Create data asset to use a panel to create the data asset for the type of data source that you want to use. You can also access the panel from the workspace's home page by clicking Create data asset on the Quick actions tile.

When you create a data asset, you specify one default connection to that data source. A connection is a user and password combination that is used to connect to the data source. For most data source types, you can use secrets in Oracle Cloud Infrastructure Vault to store the password that you need to connect to the source using the default connection. By using OCI Vault, you provide the OCID of the secret when specifying the connection details, so you don't have to enter the actual password when you create the data asset. For sources with Oracle wallets, you can also use OCI Vault secrets for the wallet and passwords. See OCI Vault Secrets and Oracle Wallets.

After you create your data asset, you're brought to the details page where you can review or update the data asset name, description, and tags. You can see the connections, schemas, and data entities. For more information, see Viewing Data Asset Details.

You can create additional connections for a data asset as needed. Each connection should give you access to additional schemas and data entities  to integrate. For more information on connections, see Creating a Connection.

Creating a Data Asset

When you create a data asset, you select the data source type and then provide the data source details and the corresponding default connection details to that source. You can use the Console or the APIs to create data assets.

Each type of data asset has a different set of source and connection fields. See the appropriate section under Data Sources for the instructions to create the data asset that you want to use.

For most data assets, instead of providing the actual password for a connection, you can choose to use a secret in Oracle Cloud Infrastructure Vault that has the password for the connection user. Take note of the following before using OCI Vault secrets:

  • Ensure that you have set up the required policies for using OCI vault and secrets.
  • Create a secret for the database password.
  • If you use Oracle wallets, you must modify the wallet zip before you create a secret for the wallet contents. You also need a secret for the wallet password, and a secret for the database password.

See OCI Vault Secrets and Oracle Wallets.

If you intend to publish a task to an application in OCI Data Flow, note also the following when creating your data assets:

  • Use OCI Vault secrets that contain the passwords to connect to the data sources. This is required for passing credentials securely across OCI services.
  • Specify the Fully Qualified Domain Name (FQDN) for the database hosts. OCI Data Flow does not allow connections through host IP addresses.

See Considerations and Support Information for more information about what's supported and what you should consider when connecting to data sources.

To create a data asset, use the procedure of your choice.

  • The procedure described in this topic is a general one that is used across all types of data sources. The specific fields and properties for a data source type (such as Object Storage or Oracle Database) are available in Data Sources.

    1. On the workspace's home page, select Create data asset on the Quick actions tile.

      You can also click Data asset on the tab bar, and then click Create data asset on the Data assets page that displays.

    2. On the Create data asset page, enter a Name and Description (optional).

      The Identifier is a system-generated value based on the name. You can change the value, but after you create and save the data asset, you cannot update the identifier.

    3. From the Type menu, select the type of data source you need.
    4. For the data source type that you've selected, complete the properties and the default connection fields.
      See the appropriate section under Data Sources for the properties and connection field instructions to create the data asset.
    5. Click Create.

  • Use the oci data-integration data-asset create command and required parameters to create a data asset:

    oci data-integration data-asset create [OPTIONS]

    For a complete list of flags and variable options for CLI commands, see the Command Line Reference.

  • Run the CreateDataAsset operation to create a data asset.