Configuring a Private Network

You can configure your data catalog to access data sources hosted in private networks.

By configuring data catalog to access a private network, you can:

  • Harvest Oracle Cloud Infrastructure data sources that are only accessible privately.
  • Harvest on-premise data sources that are connected to an Oracle Cloud Infrastructure Virtual Cloud Network (VCN) using VPN Connect or FastConnect.
Note

You can access and harvest on premise or private data sources in Data Catalog using either their Fully Qualified Domain Name (FQDN) or private IP. The FQDN must have an A record in the configured DNS server and must not be an Oracle reserved public domain, such as oracle.com or adb.oracle.com. Valid FQDN examples: wxyz.adb.oracle.com and <hostname>.<subnet DNS label>.<VCN DNS label>.oraclevcn.com.

To allow your data catalog to access a private network, you must:

  1. Create a private endpoint for your data catalog.
  2. Attach the private endpoint to your data catalog.
  3. Use the private endpoint while creating a data asset.

Prerequisites

One of the ways Oracle Cloud Infrastructure lets you configure private access for your resources is using private endpoints.

Data Catalog uses private endpoints to access the private network where your data sources are hosted. You must have the required data catalog permissions to use the Data Catalog private endpoints.

Additionally, to create, update, or delete private endpoints in Oracle Cloud Infrastructure, you need to obtain certain permissions in Oracle Cloud Infrastructure Identity and Access Management (IAM) for the relevant compartments in your tenancy. The following table lists the required permissions for virtual networking resources in Oracle Cloud Infrastructure for the private endpoint operations.

Operation Required Access on Underlying Resources
Create a private endpoint

For the private endpoint compartment:

  • Create VNIC (VNIC_CREATE)
  • Delete VNIC (VNIC_DELETE)
  • Update members in a network security group (NETWORK_SECURITY_GROUP_UPDATE_MEMBERS)
  • Associate a network security group (VNIC_ASSOCIATE_NETWORK_SECURITY_GROUP)

For the subnet compartment:

  • Attach subnet (SUBNET_ATTACH)
  • Detach subnet (SUBNET_DETACH)
Update a private endpoint

For the private endpoint compartment:

  • Update VNIC (VNIC_UPDATE)
  • Update members in a network security group (NETWORK_SECURITY_GROUP_UPDATE_MEMBERS)
  • Associate a network security group (VNIC_ASSOCIATE_NETWORK_SECURITY_GROUP)
Delete a private endpoint

For the private endpoint compartment:

  • Delete VNIC (VNIC_DELETE)
  • Update members in a network security group (NETWORK_SECURITY_GROUP_UPDATE_MEMBERS)

For the subnet compartment:

  • Detach subnet (SUBNET_DETACH)
Note

If you are managing the data catalog private endpoints resource, we recommend that you also have the manage work requests permission. This ensures that you are able to view the logs and error messages that are encountered while working with private endpoints.

Creating a Private Endpoint

Oracle Cloud Infrastructure lets you create private endpoints within your service so that you can access resources that are only available using private IPs. In Data Catalog, you create a private endpoint to configure the private network where your data source is hosted.

Before you create a private endpoint in Data Catalog, you must have the following details:

  • The name of the Virtual Cloud Network (VCN) used to access your data source.
  • The name of the subnet in the VCN.
  • The list of DNS zones used to resolve the FQDNs or private IPs of the data sources that you want to harvest.
    Important

    If the data sources in your private network are accessed with FQDNs and private IPs, you only need the list of all the FQDNs when you create the private endpoint.

    If all the data sources in your private network are accessed with private IPs, then while creating a private endpoint you must specify at least one resolvable FQDN or DNS.

    For private autonomous databases, use the FQDN of the database as the DNS zone. For custom data sources running on Oracle Cloud Infrastructure compute Virtual Machines (VMs), you can specify the FQDN of the VM, or the domain name of the subnet in which the VM is provisioned, or the domain name of the VCN.

Here's how you create a private endpoint:

  1. Open the navigation menu and click Analytics and AI. Under Data Lake, click Data Catalog.
  2. From the Data Catalog service page, click Private Endpoints.
  3. Click Create Private Endpoint.
  4. Select the compartment where you want to create the private endpoint. You can create the private endpoint in a different compartment than the compartment where your data catalog is created.
  5. Enter a name to identify the private endpoint.
  6. Select the VCN that is created to provide private access to your data source.
  7. Select the Subnet that has the private endpoint to access your data source.
  8. Enter the DNS zones to resolve. The DNS zone can be a DNS domain name or a FQDN. You can enter upto 30 DNS zones.
    Example: Consider a database with FQDN dcat.dbsubnet.dbvcn.oraclevcn.com installed on a machine. The permitted domain name entries can be: dbvcn.oraclevcn.com (VCN DNS), dbsubnet.dbvcn.oraclevcn.com (subnet DNS), or dcat.dbsubnet.dbvcn.com (FQDN). The private IP of the machine can not be specified.
    Important

    If the data sources in your private network are accessed with FQDNs and private IPs, you only need to enter the list of all the FQDNs.

    If all the data sources in your private network are accessed with private IPs, then enter at least one resolvable FQDN.

  9. (Optional) Add tags to identify this private endpoint resource.
  10. Click Create.
The private endpoint is created. The create process can take a couple of minutes. When the private endpoint is created successfully, the private endpoint is in ACTIVE status.

Viewing Private Endpoints

All private endpoints created in Data Catalog are listed in the Private Endpoints page. To view details for a specific private endpoint, click the private endpoint name. The private endpoint details page displays. Alternatively, click the Actions icon (three dots) for the private endpoint and select View Details.

From the Private Endpoints list, you can also edit, move, or delete a private endpoint.

A private endpoint can be in one of the following statuses:

Creating
The private endpoint is being created.
Active
The private endpoint is successfully created and ready for use.
Updating
The private endpoint details, such as name, DNS zones, or compartment, are being updated.
Moving
The private endpoint is being moved to a new compartment.
Deleting
The private endpoint is being deleted.
Deleted
The private endpoint is successfully deleted.
Failed
The private endpoint was not created, updated, or deleted successfully.

Attaching a Private Endpoint

You attach a private endpoint to a data catalog to enable the data catalog data assets  to be harvested  using the configured private network. You can attach a private endpoint to only one data catalog and you can attach only one private endpoint to a data catalog.

Before you attach a private endpoint to a data catalog, you must create a private endpoint.

Here's how you attach a private endpoint to a data catalog.

  1. From the Data Catalogs list in the Console, click the Actions icon (three dots) for the data catalog where you want to attach a private endpoint and then select Attach Private Endpoint.
  2. From the Attach Private Endpoint dialog, select the private endpoint that you want to attach to the data catalog. If the private endpoint was created in a different compartment than the data catalog, then you have to change the compartment before selecting the private endpoint.
  3. Click Attach.

A notification displays indicating that the private endpoint is being attached to the data catalog. A notification is also displayed after the private endpoint is attached successfully to the data catalog.

Using a Private Endpoint

When you create a data asset in the data catalog service console, you can specify that you want to use the private endpoint you have created and attached to the data catalog.

Caution

Do not specify that you want to use a private endpoint without creating and attaching a private endpoint to the data catalog. If you do, you receive an error while creating the data asset.

Editing a Private Endpoint

Here's how you edit the private endpoint name and DNS zones:

  1. From the Data Catalog page in the Console, click Private Endpoints.
  2. From the Private Endpoints list, click the Actions icon (three dots) for the private endpoint you want to edit and select Edit. Alternatively, you can click the name of the private endpoint to open the private endpoint details page and then click Edit.
  3. Modify the private endpoint name and DNS zones.
  4. Click Save Changes.
A notification displays indicating that your changes are saved successfully.

Moving a Private Endpoint

You can move the private endpoint resource from the compartment you created it in to a different compartment.

Here's how you move a private endpoint to a different compartment:

  1. From the Data Catalog page in the Console, click Private Endpoints.
  2. From the Private Endpoints list, click the Actions icon (three dots) for the private endpoint you want to move and select Move Resource.
  3. Select the new compartment for the private endpoint resource.
  4. Click Move Resource.
A notification displays indicating that the private endpoint resource is moved to the new compartment successfully. You might notice the private endpoint status change to Moving. After the move is completed successfully, the private endpoint status changes back to Active.

Detaching a Private Endpoint

You detach a private endpoint from the data catalog it is attached to before you can delete the private endpoint. Also, if you want to attach a different private endpoint to a data catalog, you must detach any private endpoint already attached to the data catalog.

Here's how you detach a private endpoint from a data catalog.

  1. From the Data Catalogs page in the Console, click the Actions icon (three dots) for the data catalog where you want to detach a private endpoint and then select Detach Private Endpoint.
  2. From the Detach Private Endpoint dialog, click Yes, Detach.

A notification displays indicating that the private endpoint is being detached from the data catalog. A notification is also displayed after the private endpoint is detached successfully from the data catalog.

Important

The data catalog data assets that use the private endpoint that you detached from the data catalog can no longer be harvested again.

Deleting a Private Endpoint

You can delete a private endpoint only if it is not attached to any data catalog.

Caution

If you attempt to delete a private endpoint that is still attached to a data catalog, you receive a warning that the private endpoint can't be deleted. You must first detach the private endpoint from that data catalog and then delete the private endpoint.

Here's how you delete a private endpoint:

  1. From the Data Catalog page in the Console, click Private Endpoints.
  2. From the Private Endpoints list, click the Actions icon (three dots) for the private endpoint you want to delete and select Delete.
  3. In the Delete Private Endpoint dialog, type DELETE to confirm that you want to delete the private endpoint and then click Delete.
A notification displays indicating that the private endpoint is deleted successfully.
Important

The data catalog data assets that use the private endpoint that you deleted can no longer be harvested again.