The 5800 System Data Model

The 5800 system stores two types of data: arbitrary object data and structured metadata records. Every metadata record is associated with exactly one data object. Every data object has at least one metadata record. A unique object identifier (OID) is returned when a metadata record is stored. This OID can later be used to retrieve the metadata record or its data object. In addition, metadata records can be retrieved by a query:

OID ↔ Metadata Record -> Underlying Object Data

There are two types of metadata, system metadata and user metadata. You cannot override the names and types of system metadata.

Each object in the 5800 system archive consists of some arbitrary bytes of data together with associated metadata that describes the data. Once an object is stored, it is immutable. The 5800 system programming model does not allow the data or the metadata associated with an object to be changed once the object has been stored, in other words the system is a Write-Once Read-Multiple (WORM) archive. Each object corresponds to a single stream of data and a single set of metadata; there are no “grouped objects” or “compound objects” other than by application convention.

Each object corresponds to a single stream of data and a single set of metadata. There are no “grouped objects” or “compound objects” other than by application convention. Similarly, there are no “links” or “associations“ from one object to another. The customer application is shielded from all details of how or where the object is stored. Internally, the actual location of an object might change over time, or several objects might even share the same underlying storage. The customer application can retrieve the object without knowing these details.

A stream of data is stored in the object archive using storeObject. Once stored, each such object is associated with an object identifier or objectid (OID). The storeObject operation takes both a stream of data and an optional set of user metadata information and returns an OID. The OID can be remembered outside of the 5800 system and may later be used to retrieve the data associated with that object using the retrieveObject operation.

Every object has metadata whether or not user metadata was supplied at the time of the store. For example each object has system metadata that is system assigned and can never be modified by the user. The OID is associated with the metadata record that represents this object as a whole; the metadata record is then associated with the underlying data:

OID ↔ Metadata Record -> Underlying Object Data

The retrieveObject operation takes an OID as input and returns a stream of bytes as output that are identical to the bytes stored during the storeObject operation. Both the storeObject and retrieveObject operations handle the data in a streaming manner. Not all of the data need be present in client memory or in server memory at the same time, which is a crucial point for working with large objects.

For the 5800 system Release 1.1, data sizes up to 400 GBytes are tested and supported. Using sizes even smaller than this may be appropriate as a best practice. For more information, see Chapter 5, Programming Considerations and Best Practices.

From within a customer application, the storing of an object into the archive is an all-or-nothing event. Either the object is stored or it is not; there are no partial stores. If a store operation is interrupted, the entire storeObject call fails. Once an OID is returned to the customer application, the object is known to be durable. In the event of an outage that causes some data loss, the system should be no more likely to lose a newly stored object than any other object. There is no way to tie together two different store operations so that both either succeed together or fail together.

A stored object may or may not immediately be queryable. For more information, see The 5800 System Query Integrity Model.