Oracle Cloud Infrastructure Documentation

Configuring Hive to Access Data Catalog Metastore

Configure Hive to access Data Catalog metastore.

  1. To configure Data Catalog metastore libs and associated Object Storage jars, sign in to the un0 node in your cluster and run following dcli command. 
    $ sudo dcli bash /usr/lib/oci-dcat-metastore-client/bin/configure-metastore-jars.sh
    Note

    To undo and restore the original jars, run $ sudo dcli bash /usr/lib/oci-dcat-metastore-client/bin/configure-metastore-jars.sh undo.
  2. Access Apache Ambari.
  3. From the side toolbar, under Services select Hive.
  4. Select the Configs tab, and then select Advanced.
  5. Expand the Custom hive-site section.
  6. Add or update the oci.metastore.uris key with the Data Catalog endpoint. Example: https://datacatalog.<region-identifier>.oci.oraclecloud.com:443.
  7. Add or update the oracle.dcat.metastore.id key with the OCID for the metastore that you want to use. Example: ocid1.datacatalogmetastore.oc1.<unique_ID>.
  8. Add or update the oracle.dcat.metastore.client.custom.authentication_provider key with the value com.oracle.pic.dcat.metastore.commons.auth.provider.UserPrincipalsCustomAuthenticationDetailsProvider.
  9. Add or update the hive.metastore.warehouse.dir key with the Object Storage path for the managed table. Example: oci://bucket-name@tenancy-name-of-bucket/path/to/managed/table/directory.
  10. Add or update the hive.metastore.warehouse.external.dir key with the Object Storage path for the external table. Example: oci://bucket-name@tenancy-name-of-bucket/path/to/external/table/directory.
  11. Add or update the hive.metastore.uris key with the Data Catalog endpoint. Example: https://datacatalog.<region-identifier>.oci.oraclecloud.com:443.
  12. Add or update the OCI_TENANT_METADATA key with the OCID of the client tenancy. Example: ocid1.tenancy.oc1.<unique_ID>.
  13. Add or update the OCI_USER_METADATA key with the OCID of the user calling the Metadata API. Example: ocid1.user.oc1.<unique_ID>.
  14. Add or update the OCI_FINGERPRINT_METADATA key with the fingerprint for the key pair being used. Example: 54:53:cd:d0:f1:ee:b7:c9:b5:52:d5:27:2c:58:36:13.
  15. Add or update the OCI_PVT_KEY_FILE_PATH key with the full path and file name of the private key used for authentication. We recommend the file be on the local file system. Example: /etc/hive/key.pem.
    Note

    Ensure that the key file is placed on all nodes and readable by Hive and Tez users.
  16. Add or update the OCI_PASSPHRASE_METADATA key with a passphase, if any.
  17. Add or update the OCI_REGION_METADATA key with the region. Example: us-ashburn-1.
  18. Select Restart to restart the Hive service in the Big Data Service cluster.