Note:

Use OCI Object Storage Amazon S3 Compatibility API with AWS SDK for C++

Introduction

Oracle Cloud Infrastructure (OCI) Object Storage enables customers to securely store any type of data in its native format. With built-in redundancy, OCI Object Storage is ideal for building modern applications that require scale and flexibility, as it can be used to consolidate multiple data sources for analytics, backup, or archive purposes.

OCI Object Storage is foundational to cloud workloads, it’s elasticity, scalability, reliability, and cost efficiency has made it the primary storage for unstructured data in the cloud. As a result of its popularity there is a vast ecosystem of tools to work with OCI Object Storage.

OCI Object Storage provides an Amazon S3 Compatibility API, customers can continue to use their existing Amazon S3 tools and make minimal changes to their applications to work with OCI Object Storage. The Amazon S3 Compatibility API and OCI Object Storage datasets are congruent. If data is written to OCI Object Storage using the Amazon S3 Compatibility API, the data can be read back using the native OCI Object Storage API and conversely. Customers who use the AWS SDK for C++ may find the authentication aspect challenging when using the Amazon S3 Compatibility API, this tutorial aims to provide two simple examples to get started on using the AWS SDK for C++.

Note: These are example scripts using the AWS SDK for C++, they are not official Oracle applications and are not maintained or supported by Oracle. Since they are examples, you may want to review, update, and customize them for your use case in your own software repository. Oracle does not certify or support third-party SDKs.

Objectives

Prerequisites

Task 1: Understand Path and Virtual-Hosted Style URLs

At the time of writing this tutorial, the OCI Object Storage Amazon S3 Compatibility API supports path-style URLs. This means the bucket will be in the URL path.

For example:

https://MyNamespace.compat.objectstorage.us-ashburn-1.oraclecloud.com/MyBucketName

OR

https://MyNamespace.compat.objectstorage.us-ashburn-1.oraclecloud.com/MyBucketName/MyObjectName

In virtual-hosted style URLs, the OCI Object Storage bucket name is included in the subdomain (or in the beginning) of the URL.

For example:

https://MyBucketName.MyNamespace.compat.objectstorage.us-ashburn-1.oraclecloud.com

OR

https://MyBucketName.MyNamespace.compat.objectstorage.us-ashburn-1.oraclecloud.com/MyObjectName

By default, the AWS SDK for C++ uses virtual-hosted style URLs. When following the code examples produced by AWS, and substituting values from your OCI tenancy, an error will be displayed similar to the following output. For more information, see Creating, listing, and deleting buckets.

curlCode: 60, SSL peer certificate or SSH remote key was not OK

Because the code is using a virtual-hosted style URL the code will not reference the OCI Object Storage bucket nor a valid certificate.

The S3Client setting will need to be changed to use path-style URLs. The config should be set using the following command.

Aws::Client::ClientConfiguration config; 

S3Client s3Client(config, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, false);

Task 2: Determine your Tenancy Namespace and S3 API Compartment

  1. Log in to your Linux system.

    Note: All commands will be run from your Linux system where you have the necessary prerequisites installed and configured.

  2. Run the oci os ns get-metadata OCI CLI command.

    $ oci os ns get-metadata
{
  "data": {
    "default-s3-compartment-id": "ocid1.tenancy.oc1..bbbbcccdabcc0x0aaaaaa00aaaaaaaaaaa0b0b00ccccccccbbbbbbbb00aa",
    "default-swift-compartment-id": "ocid1.tenancy.oc1..bbbbcccdabcc0x0aaaaaa00aaaaaaaaaaa0b0b00ccccccccbbbbbbbb00aa",
    "namespace": "MyNamespace"
  }
}

  3. If the S3 compartment OCID contains the word tenancy it is the root compartment. Get the name using the following command.

    $ oci iam compartment get --compartment-id <compartment_ocid>

    Or, for the tenancy root compartment run the following command.

    $ oci iam tenancy get --tenancy-id <tenancy_ocid>

    Alternatively, you can use the OCI Console by navigating to your profile in the upper right corner, and selecting tenancy (your_tenancy_name) to determine the namespace and S3 API compartment.

    OCI Tenancy Screen

    The S3 compartment will be used again in Task 4.

Task 3: Create a Customer Secret Key to Access the S3 Compatible API

Generate secret key using OCI CLI commands.

  1. Run the oci iam customer-secret-key create command.

    oci iam customer-secret-key create --display-name display-name --user-id ocid1.user.oc1..aaaaaaaa-user-ocid-sdd6ahdouq
{
"data": {
     "display-name": "display-name",
     "id": "7aaaa3462aa34271a276002015f30674a5325aaa",
     "inactive-status": null,
     "key": "1aaa577aaaa/aa3aa92aa7aa7aaaaaaAa0aaaAa8AAa=",
     "lifecycle-state": "ACTIVE",
     "time-created": "2024-07-19T19:58:03.794000+00:00",
     "time-expires": null,
     "user-id": "ocid1.user.oc1..aaaaaaaa-user-ocid-sdd6ahdouq"
},
"etag": "e63038c73fc24fa087a2a4c3339ef709"
}

  2. Copy the ID (AWS_ACCESS_KEY_ID) and the key (AWS_SECRET_ACCESS_KEY) to somewhere secure to be used later to set environment variables.

    Note: Your user OCID can be found in your OCI CLI config file, usually stored in $HOME/.oci/config.

Or

A secret key can also be generated using the OCI Console and navigating to your profile in the upper right corner, then selecting your displayed username. Click the Customer secret keys link, Generate secret key and set a display name. Click copy and store the secret key in a secure location and click Close.

Navigating to "Customer secret keys" from profile

After closing the Generate secret key window, look for the display name in the list of Customer secret keys, hover your mouse over the Access key and copy the access key.

Copying the access key

Task 4: Collect Bucket Information for Setup

  1. Go to the OCI Console, navigate to Storage and click Buckets.

  2. Click the bucket name from your prerequisites and note down the following information.

    • The bucket Name.
    • The Location of objects, this will be the prefix.
    • The Region name. Get the region code from here: Regions and Availability Domains. For example, US East(Ashburn) would be represented by us-ashburn-1.

    Collecting bucket and object information for set-up

Task 5: Set Environment Variables

Set the environment variables based on information from the previous tasks using the following command.

export AWS_ACCESS_KEY_ID="<your access key ID from Task 3>"
export AWS_SECRET_ACCESS_KEY="<your secret key ID from Task 3>"
export OCI_REGION="bukcet region from Task 4>"
export OCI_NAMESPACE="<namespace name from Task 1>"
export OCI_BUCKET="<your bucket name from Task 4>"
export OCI_PREFIX="<object prefix name from Task 4>"

For example:

export AWS_ACCESS_KEY_ID="da34baaa4ab029f51c34c1cee83d40f0dEXAMPLE" 
export AWS_SECRET_ACCESS_KEY="7w3uMS6kYiYkUpziSlLFcBimBsYDJfojwCWKEXAMPLE=" 
export OCI_REGION="us-ashburn-1" 
export OCI_NAMESPACE="MyNamespace"
export OCI_BUCKET="Images" 
export OCI_PREFIX="2024/12/18/Camera"

Task 6: Build the C++ Examples

  1. Run the following command to clone the example repository in git.

    $ git clone https://github.com/tonymarkel/OCI_AWS_CPP_SDK_S3_Examples.git

  2. Run the following commands to change to the directory containing the C++ examples, make a build directory, change to that directory, then run build and make.

    $ OCI_AWS_CPP_SDK_S3_Examples 
$ mkdir build 
$ cd build 
$ cmake .. 
$ make

Note: These are example scripts using the AWS SDK for C++, they are not official Oracle applications and are not maintained or supported by Oracle. Since they are examples, you may want to review, update, and customize them for your use case in your own software repository. Oracle does not certify or support third-party SDKs.

Task 7: Run the Examples

  1. Run the following command to get the list of buckets.

    $ ./listBuckets $OCI_NAMESPACE $OCI_REGION $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY

    For example:

    $ ./listBuckets $OCI_NAMESPACE $OCI_REGION $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY 
S3 Endpoint is: https://MyNamespace.compat.objectstorage.us-ashburn-1.oraclecloud.com/ 
Found 3 buckets 
Demo 
Images 
Logs

  2. Run the following command to get the list of objects.

    For example:

    $ ./listObjects $OCI_NAMESPACE $OCI_REGION $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY $OCI_BUCKET $OCI_PREFIX 
S3 Object Path is: https://MyNamespace.compat.objectstorage.us-ashburn-1.oraclecloud.com 
Bucket is: Images 
Prefix is: 2024/12/18/Camera 
LISTING OBJECTS 
Found 3 objects 
2024/12/18/Camera/Image1.jpeg 
2024/12/18/Camera/Image2.jpeg 
2024/12/18/Camera/DefinitelyNotACat.jpeg

Note: If you have issues, go back to Task 5 and make sure your environment variables are set correctly.

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.