Object Storage Overview

Object Storage Resources

The Object Storage service uses these components to organize the data stored in object storage:

Objects

Any type of data, regardless of content type, is stored as an object.

An object is composed of the object itself and metadata about the object. Each object is stored in a bucket. The object is processed as a single entity.

Buckets

Buckets are logical containers for storing objects.

Users or systems create buckets as needed. A bucket is associated with a single compartment that has policies that determine what actions a user can perform on a bucket and on all the objects in the bucket. Buckets cannot be nested.

You determine the bucket names, but each bucket name must be unique within a namespace.

Namespace

An Object Storage namespace serves as the top-level container for all buckets and objects.

When your tenancy is provisioned, the tenancy is assigned one unique system-generated and immutable Object Storage namespace name. The namespace spans all compartments within the tenant.

Within a namespace, buckets and objects exist in flat hierarchy, but you can simulate a directory structure to help navigate a large set of objects.

Compartment

A compartment is the primary building block used to organize your cloud resources.

When your tenancy is provisioned, a root compartment is created for you. You can then create compartments under your root compartment to organize your resources. You control access by creating policies that specify what actions groups of users can take on the resources in those compartments. An Object Storage bucket can only exist in one compartment.

Object Naming Prefixes and Hierarchies

Within an Object Storage namespace, buckets and objects exist in a flat structure. However, you can simulate a directory structure by adding a prefix string that includes one or more forward slashes (/) to an object name. Doing so lets you list one directory at a time, which is helpful when navigating a large set of objects.

For example:

marathon/finish_line.jpg
marathon/participants/p_21.jpg

If you add prefixes to object names, you can:

Use the OCI CLI or Compute API to perform bulk downloads and bulk deletes of all objects at a specified level of the hierarchy.
Use the Compute Web UI to display a hierarchical view of your objects in virtual folders. In the previous example, marathon would be displayed as a folder containing an object named finish_line.jpg and participants would be a subfolder of marathon, containing an object named p_21.jpg. You can bulk upload objects to any level of the hierarchy and perform bulk deletes of all the objects in a bucket or folder.

Bulk operations at a specified level of the hierarchy do not affect objects in any level above.

When naming objects, you can also use prefix strings without a delimiter. No delimiters would allow search operations in the Compute Web UI and certain bulk operations in the OCI CLI orCompute API to match on the prefix portion of the object name. For example, in the object names below, the string gloves_27_ can serve as a prefix for matching purposes when performing bulk operations:

gloves_27_dark_green.jpg
gloves_27_light_blue.jpg

When you perform bulk uploads with any of the UIs, you can add a prefix string to the names of the files you are uploading.

Object Names

Unlike other resources, objects do not have Oracle Cloud Identifiers (OCIDs). Instead, users define an object name when they upload an object.

Use the following guidelines when naming an object:

The maximum length for an object and bucket name is 255 characters.
Valid characters are letters (upper or lower case), numbers, and characters other than line feed, carriage return, and NULL.
Bucket names and object names are case-sensitive.
Use only Unicode characters for which the UTF-8 encoding does not exceed 1024 bytes. Clients are responsible for URL-encoding characters.
Make the name unique within the bucket.
Object names can include one or more forward slash (/) characters in the name. See Object Naming Prefixes and Hierarchies.

Optional Response Headers and Metadata

When you upload objects, you can provide optional response headers and user-defined metadata. Response headers are HTTP headers sent from Object Storage to Object Storage clients when objects are downloaded.

User-defined metadata are name-value pairs stored with an object.

You can use the Compute Web UI, Compute API, or OCI CLI to provide these optional attributes.

Important:

No validation is performed on the response headers or metadata you provide.

You can specify values for the following response headers:

Content-Disposition

Defines presentation only information for the object. Specifying values for this header has no effect on Object Storage behavior. Programs that read the object determine what to do based on the value provided. For example, you could use this header to let users download objects with custom file names in a browser. For Example:
```
attachment; filename="fname.ext"
```
Cache-Control

Defines the caching behavior for the object. Specifying values for this header has no effect on Object Storage behavior. Programs that read the object determine what to do based on the value provided. For example, you could use this header to identify objects that require caching restrictions. For Example:
```
no-cache, no-store
```

Metadata

You specify user-defined metadata in the form of name-value pairs. User-defined metadata names are stored and returned to Object Storage clients with the mandatory prefix of opc-meta-.

Multipart Uploads

The Object Storage service supports multipart uploads for more efficient and resilient uploads, especially for large objects.

You can perform multipart uploads using the Compute API or CLI. The Compute Web UI uses multipart uploads to upload objects larger than 64 MiB.

With multipart uploads, individual parts of an object can be uploaded in parallel to reduce the amount of time you spend uploading. Multipart uploads performed through the API can also minimize the impact of network failures by letting you retry a failed part upload instead of requiring you to retry an entire object upload.

Multipart uploads can accommodate objects that are too large for a single upload operation. For large uploads performed through the API, you have the flexibility of pausing between the uploads of individual parts, and resuming the upload when your schedule and resources allow.

Object Parts

With multipart upload, you split the object you want to upload into individual parts. Individual parts can be as large as 50 GiB. The maximum size for an uploaded object is 10 TiB.

Decide what part number you want to use for each part. Part numbers can range from 1 to 10,000. You do not need to assign contiguous numbers, but Object Storage constructs the object by ordering part numbers in ascending order.

Multipart Upload API

Before you use the multipart upload Compute API, you are responsible for creating the parts to upload. Object Storage provides API operations for the remaining steps.

A multipart upload performed using the API consists of the following steps:

Initiate an upload.
Upload object parts.
Commit the upload.

The service also provides API operations for listing in-progress multipart uploads, listing the object parts in an in-progress multipart upload, and aborting in-progress multipart uploads initiated through the API.

Multipart Upload CLI

When you perform a multipart upload using the OCI CLI, you do not need to split the object into parts as you are required to do by the Compute API. Instead, you specify the part size of your choice, and Object Storage splits the object into parts and performs the upload of all parts automatically. You can choose to set the maximum number of parts that can be uploaded in parallel. By default, the CLI limits the number of parts that can be uploaded in parallel to three. When using the CLI, you do not have to perform a commit when the upload is complete.

You can also use the CLI to list in-progress multipart uploads, and to abort multipart uploads initiated through the API.

Pre-Authenticated Requests

Pre-authenticated requests provide a way to let users access a bucket or an object without having their own credentials, as long as the request creator has permissions to access those objects.

For example, you can create a request that lets an operations support user upload backups to a bucket without owning API keys. Or, you can create a request that lets a business partner update shared data in a bucket without owning API keys.

When you create a pre-authenticated request, a unique URL is generated. Anyone you provide this URL to can access the Object Storage resources identified in the pre-authenticated request, using standard HTTP tools like curl and wget.

Important:

Assess the business requirements and the security ramifications of pre-authenticated access to a bucket or objects.

A pre-authenticated request URL gives anyone who has the URL access to the targets identified in the request. Carefully manage the distribution of the URL.

Required Permissions

To Create a Pre-Authenticated Request

You need the PAR_MANAGE permission to the target bucket or object.

You must also have the appropriate permissions for the access type that you are granting. For example:

If you are creating a pre-authenticated request for uploading objects to a bucket, you need the OBJECT_CREATE and OBJECT_OVERWRITE permissions.
If you are creating a pre-authenticated request for read/write access to objects in a bucket, you need the OBJECT_READ, OBJECT_CREATE, and OBJECT_OVERWRITE permissions.

Important:

If the creator of a pre-authenticated request is deleted or loses the required permissions after they created the request, the request will no longer work.

To Use a Pre-Authenticated Request

Permissions of the pre-authenticated request creator are checked each time you use a pre-authenticated request.

The pre-authenticated request no longer works when any of the following occurs:

Permissions of the pre-authenticated request creator have changed.
The user who created the pre-authenticated request is deleted.
A Federated user who created the pre-authenticated request has lost the user capabilities that they had when they created the request.
Pre-authenticated request has expired.

Types of Pre-Authentication Requests

When creating a pre-authenticated request, you have the following options:

You can specify the name of a bucket that a pre-authenticated request user has write access to and can upload one or more objects to.
You can specify the name of an object that a pre-authenticated request user can read from, write to, or read from and write to.

Scope and Constraints

Understand the following scope and constraints regarding pre-authenticated requests:

Users can't list bucket contents.
You can create an unlimited number of pre-authenticated requests.
There is no time limit to the expiration date that you can set.
You can't edit a pre-authenticated request. If you want to change user access options in response to changing requirements, you must create a new pre-authenticated request.
The target and actions for a pre-authenticated request are based on the creator's permissions. The request is not, however, bound to the creator's account login credentials. If the creator's login credentials change, a pre-authenticated request is not affected.
You cannot delete a bucket that has a pre-authenticated request associated with that bucket or with an object in that bucket.

Important:

The unique URL provided by the system when you create a pre-authenticated request is the only way a user can access the bucket or object specified as the request target. Copy the URL to durable storage. The URL is displayed only at the time of creation and cannot be retrieved later.

Retention Rules

Retention rules provide immutable, WORM-compliant storage options for data written to Object Storage for data governance, regulatory compliance, and legal hold requirements.

Retention rules can also protect your data from accidental or malicious update, overwrite, or deletion. Retention rules can be locked to prevent rule modification and data deletion or modification even by administrators.

Retention rules are configured at the bucket level and are applied to all individual objects in the bucket.

Object Storage provides a flexible approach to data retention that supports the following use cases.

Regulatory Compliance

Your industry might require you to retain a certain class of data for a defined length of time. Your data retention regulations might also require that you lock the retention settings. If you lock the settings, the only change you can make is to increase the retention duration.

For Object Storage regulatory compliance, you create a time-bound retention rule and specify a duration. Object modification and deletion are prevented for the duration specified. Duration is applied to each object individually, and is based on the object's Last Modified timestamp. Lock the rule as required.
Data Governance

You might need to protect certain data sets as a part of internal business process requirements. While retaining the data for a defined length of time is necessary, that time period could change.

Create a time-bound retention rule and specify a duration. Object modification and deletion are prevented for the duration specified. Duration is applied to each object individually, and is based on the object's Last Modified timestamp. To be able to delete the rule and allow changes to the duration as required, do not lock the rule.
Legal Hold

You might need to preserve certain business data in response to potential or on-going lawsuits. A legal hold does not have a defined retention period and remains in effect until removed.

For Object Storage legal holds, you create an indefinite retention rule. Object modification and deletion are prevented until you delete the rule. You cannot lock an indefinite retention rule because the rule has no duration.

It's important to understand retention duration for time-bound rules. Even though you are creating retention rules for a bucket, the duration of a rule is applied to each object in the bucket individually, and is based on the object's Last Modified timestamp. Let's say you have two objects in the bucket, ObjectX and ObjectY. ObjectX was last modified 14 months ago and ObjectY was last modified 3 months ago. You create a retention rule with a duration of 1 year. This rule prevents the modification or deletion of ObjectY for the next 9 months. The rule allows the modification or deletion of ObjectX because the retention rule duration (1 year) is less that the object's Last Modified timestamp (14 months). If ObjectX is overwritten some time in the coming year, modification and deletion would be prevented for the rule duration time remaining.

Locking a retention rule is an irreversible operation. Not even a tenancy administrator can delete a locked rule. There is a mandatory 14-day delay before a rule is locked. This delay lets you thoroughly test, modify, or delete the rule or the rule lock before the rule is permanently locked. A rule is active at the time of creation. The lock only controls whether the rule itself can be modified. After a rule is locked, only increases in the duration are allowed. Object modification is prevented and the rule can only be deleted by deleting the bucket. A bucket must be empty before it can be deleted.

Scope and Constraints

Retention rules can be applied to a bucket in the Object Storage.
The actions that you can perform on a bucket with active retention rules are limited. You cannot update, overwrite, or delete objects or object metadata until the retention rule is deleted (indefinite rule) or for the duration specified (time-bound rules). The duration for time-bound rules is applied to each object individually, and is based on the object's Last Modified timestamp.
You can create multiple retention rules for a bucket. Indefinite retention rule is applied before any time-bound rule is considered.
When a retention rule is locked, the rule can only be deleted by deleting the bucket. A bucket must be empty before it can be deleted.

Interaction Between Retention and Other Object Storage Features

Carefully review the policies and rules that you have in place for the other Object Storage features that you are using. Some of these policies and rules might not make sense with retention rules. This section describes some key things you need to know about the interaction between retention rules and other Object Storage features.

Multipart Uploads

Uncommitted (unfinished or failed) multipart uploads are not protected by retention rules and can be deleted at any time.

Versioning

You cannot add retention rules to a bucket that has versioning enabled.
You cannot enable versioning on a bucket with active retention rules.
You can add retention rules to bucket that has versioning suspended. However, you cannot resume versioning with active retention rules.

Troubleshooting Retention Rules

Unable to Create a Retention Rule

If creating a retention rule fails, the most likely cause is missing or incomplete IAM permissions. Rule creation requires:

User permissions that let you access the bucket and manage the objects in those buckets.
Minimally, BUCKET_UPDATE and RETENTION_RULE_MANAGE permissions.
Minimally, BUCKET_UPDATE and RETENTION_RULE_MANAGE permissions.

Unable to Lock a Retention Rule

If locking a retention rule fails, the most likely cause is missing or incomplete IAM permissions. Minimally, BUCKET_UPDATE, RETENTION_RULE_MANAGE, and RETENTION_RULE_LOCK permissions are required to lock retention rules.

Unable to Delete a Retention Rule

You cannot delete a time-bound retention rule that is locked. When a retention rule is locked, the rule can only be deleted by deleting the bucket. A bucket must be empty before it can be deleted.

If deleting an indefinite retention rule fails, the most likely cause is missing or incomplete IAM permissions. Rule deletion requires:

User permissions that let you access the bucket and manage the objects in those buckets.
Minimally, BUCKET_UPDATE and RETENTION_RULE_MANAGE permissions.

Object Versioning

Object versioning provides data protection against accidental or malicious object update, overwrite, or deletion.

Object versioning is enabled at the bucket level. Versioning directs Object Storage to automatically create an object version each time one of these actions takes place:

A new object is uploaded.
An existing object is overwritten.
When an object is deleted.

You can enable object versioning at bucket creation time or later.

A bucket that is versioning-enabled can have many versions of an object. There is always one latest version of the object and zero or more previous versions.

Object Version Deletion

No object is physically deleted from a bucket that has versioning enabled until you take explicit action to do so.

When you delete an object without targeting a specific version, the latest object version becomes a previous object version and a special delete marker is created that marks the deletion point. A delete marker contains only minimal metadata. If you delete a folder, a delete marker is created for each object in the folder. You can simply delete the delete marker to make that deleted version become the latest object version.

When you upload an object with the same name as the delete marker, the uploaded object becomes the latest version of the object. The delete marker remains. There can be multiple delete markers for an object and you can recover any of the previous object versions.

Object version deletion is different. When you delete an object version, the version is permanently deleted. Permanent deletion also happens if you explicitly delete the latest version by version ID. All delete operations that target a specific object version ID permanently deletes the data.

Scope and Constraints

Versioning can be enabled on a bucket in Object Storage.
You can rename the latest version of an object, but you cannot rename a previous object version. Renaming an object creates a new object.

Interaction Between Versioning and Other Object Storage Features

This section describes some key things you need to know about the interaction between object versioning and other Object Storage features.

Copying Objects

If you copy the latest version of an object to a different bucket, only the object is copied. None of the object's previous versions are copied. You can copy a previous version of an object to another bucket, but that action creates either the latest version of a new object or a new object version in the destination bucket.

Retention Rules

You cannot add retention rules to a bucket that has versioning enabled.
You cannot enable versioning on a bucket with active retention rules.
You can add retention rules to bucket that has versioning suspended. However, you cannot resume versioning with active retention rules.

Troubleshooting Versioning

Unable to Enable Versioning

If enabling versioning fails, the most likely cause is missing or incomplete IAM permissions. Enabling versioning requires:

User permissions that let you use the bucket and manage the objects in that bucket.
Minimally, BUCKET_UPDATE permissions.

Unable to Delete a Bucket

If deleting a bucket fails, the most likely cause is that the bucket is not empty.

You can permanently delete an empty bucket. You cannot delete a bucket that contains any of the following:

Any objects
Previous versions of an object
A multipart upload in progress
A pre-authenticated request

Tip:

When you delete an object in a version-enabled bucket, a previous version of that object is created. Select Show Deleted Objects to display the object versions that might prevent you from deleting the bucket.

Unable to Delete a Previous Version

If deleting a previous object version fails, the most likely cause is missing or incomplete IAM permissions. Object version deletion requires:

User permissions that let you use the bucket and manage the objects in that bucket.
Minimally, OBJECT_VERSION_DELETE permissions.