4.12 Setting up Swift

Swift provides object-based persistent storage for virtual machines. It can also be used as the storage for the Cinder backup service.

To deploy Swift object storage, you need to:

4.12.1 Preparing the Storage Devices

The storage devices used for Swift are automatically detected when you deploy the Swift service to a storage node. To detect the disks, the disks must be partitioned with a fixed partition label. For performance reasons, it is recommended that you do not enable RAID on any of the storage devices.

To prepare the disks, create a GUID Partition Table (GPT) partition on each disk and label each partition with the same partition label. By default, the partition label is KOLLA_SWIFT_DATA (the label is configurable). For example, to create a partition on the /dev/sdb disk with the default partition label, use the following command:

# parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_SWIFT_DATA 1 -1
Caution

When you run this command, all the data on the disk will be lost.

After creating the partition, create an XFS file system in the partition. The following command creates an XFS file system in the /dev/sdb1 partition:

# mkfs.xfs -f /dev/sdb1

The following is an example script that partitions three disks and creates the XFS file systems:

#!/usr/bin/env bash
for d in sdb sdc sdd; do
    parted /dev/${d} -s -- mklabel gpt mkpart KOLLA_SWIFT_DATA 1 -1
    mkfs.xfs -f /dev/${d}1
done

The Swift storage devices are automatically mounted under /srv/node. Create this mount point manually on each storage node:

# mkdir -p /srv/node

If the mlocate package is installed on the storage nodes, it is recommended that you add /srv/node to the PRUNEPATHS entry in /etc/updatedb.conf to prevent updatedb from indexing directories below /srv/node.

More information about preparing disks and file systems for Swift, see the Swift Deployment Guide at http://docs.openstack.org/developer/swift/deployment_guide.html.

4.12.2 Building the Swift Rings

You need to create the Swift rings so that each storage node knows where to store the objects. The path to an object stored in Swift consists of an account (tenant or project), a container, and the object itself. Accounts and containers are the means by which objects are organized and replicated in the storage. Swift rings are databases, as follows:

  • The Swift account servers use the account ring to manage the list of containers for each account.

  • The Swift container servers use the container ring to manage the list of objects in each container.

  • The Swift object servers use the object ring to manage the list of where objects are located on the storage devices.

Swift storage is organized into regions (for geographical separation), which contain zones (for availability, such as a building, a data center, or a server rack), which contain storage servers, and disks (inside or attached to the storage servers).

For more information about the Swift rings and how storage is organized, see the OpenStack Swift documentation:

Before you deploy Swift services, you build the initial Swift rings manually on the master node, using the swift-ring-builder utility. To avoid any issues with software versions, it is best to run the swift-ring-builder utility in a Docker container using the ol-openstack-swift-base image that is included with the Oracle OpenStack for Oracle Linux images. To do this:

For ease of management, it is recommended that you use a script to create the Swift rings. An example script is included in this section. The user that runs the script must be a member of the Docker group (so that they can run Docker commands). It is best that this user is also a member of the kolla group.

If you are using the Oracle Container Registry, you might need to sign in at https://container-registry.oracle.com before you can run the swift-ring-builder utility. After you accept the terms, you also need to register your login credentials for the Oracle Container Registry with Docker so that you can pull (download) images:

$ docker login -u user-name container-registry.oracle.com
Password:
Login Succeeded

The Swift rings are created as custom configuration files in the /etc/kolla/config/swift directory on the master node. When you deploy the Swift service, the rings are copied to the storage nodes. You have to manually create the /etc/kolla/config/swift directory on the master node:

# mkdir -p /etc/kolla/config/swift

The example script creates the following files:

  • account.builder and account.ring.gz for the account ring.

  • container.builder and container.ring.gz for the container ring.

  • object.builder and object.ring.gz for the object ring.

You must use these file names in your own scripts.

Caution

Make sure you keep a backup of the .builder files, as you need these files if you want to make changes to the rings.

Example 4.3 Example Swift Ring Configuration Script

The following is an example configuration script for setting up the Swift rings by running the swift-ring-builder utility in a Docker container. Note the following:

  • The REGISTRY variable contains the host name and port of the Docker registry that contains the Oracle OpenStack for Oracle Linux images. If you are using the Oracle Container Registry, set this variable to container-registry.oracle.com.

  • The NAMESPACE variable contains the namespace used to store the Oracle OpenStack for Oracle Linux images in the Docker registry. If you are using the Oracle Container Registry, set this variable to openstack. If you are using Docker Hub, set this variable to oracle.

  • The TAG variable contains the tag to use when pulling the ol-openstack-swift-base image. Usually this is the Oracle OpenStack for Oracle Linux release number such as 3.0.1.

  • The script uses the STORAGE array to store the IP addresses of the storage nodes (three storage nodes in this example). Replace node_IP with the IP addresses of your nodes.

  • The .builder files are created using a 10 3 1 specification, which configures 1024 (2^10) maximum partitions, 3 replicas of each object, and 1 hour minimum time between moving a partition more than once. A partition is a directory on a storage device rather than a hard disk division. It is recommended to have a minimum of three replicas.

  • The script does not specify any regions. When you run the script, you see warnings that no regions are specified, and that the default region 1 will be used.

  • The script places each storage node in its own zone. You may want to organize your storage differently.

  • The script assumes the default Swift ports 6000 (object server), 6001 (account server) and 6002 (container server) are used. The ports are configurable.

  • The script assumes there is an sdb1 partition on each storage node.

  • The script assigns the same weight (1) to balance the distribution of partitions on drives across the cluster. This assumes all the disks are the same size.

#!/usr/bin/env bash

REGISTRY=registry_hostname:registry_port
NAMESPACE=namespace
TAG=openstack_release

STORAGE[1]=node_IP
STORAGE[2]=node_IP
STORAGE[3]=node_IP

# Create the object ring
docker run --rm \
  -v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
  ${REGISTRY}/${NAMESPACE}/ol-openstack-swift-base:${TAG} \
  swift-ring-builder /etc/kolla/config/swift/object.builder create 10 3 1

for NODE in 1 2 3; do
  echo "object.builder: Adding ${STORAGE[$NODE]} to zone z${NODE}"
  docker run --rm \
    -v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
    ${REGISTRY}/${NAMESPACE}/ol-openstack-swift-base:${TAG} swift-ring-builder \
    /etc/kolla/config/swift/object.builder add z${NODE}-${STORAGE[$NODE]}:6000/sdb1 1
done

# Create the account ring
docker run --rm \
  -v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
  ${REGISTRY}/${NAMESPACE}/ol-openstack-swift-base:${TAG} \
  swift-ring-builder /etc/kolla/config/swift/account.builder create 10 3 1

for NODE in 1 2 3; do
  echo "account.builder: Adding ${STORAGE[$NODE]} to zone z${NODE}"
  docker run --rm \
    -v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
    ${REGISTRY}/oracle/ol-openstack-swift-base:${TAG} swift-ring-builder \
    /etc/kolla/config/swift/account.builder add z${NODE}-${STORAGE[$NODE]}:6001/sdb1 1
done

# Create the container ring
docker run --rm \
  -v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
  ${REGISTRY}/${NAMESPACE}/ol-openstack-swift-base:${TAG} \
  swift-ring-builder /etc/kolla/config/swift/container.builder create 10 3 1

for NODE in 1 2 3; do
  echo "container.builder: Adding ${STORAGE[$NODE]} to zone z${NODE}"
  docker run --rm \
    -v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
    ${REGISTRY}/${NAMESPACE}/ol-openstack-swift-base:${TAG} swift-ring-builder \
    /etc/kolla/config/swift/container.builder add z${NODE}-${STORAGE[$NODE]}:6002/sdb1 1
done

# Rebalance the rings
for ring in object account container; do
  docker run --rm \
    -v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
    ${REGISTRY}/${NAMESPACE}/ol-openstack-swift-base:${TAG} swift-ring-builder \
    /etc/kolla/config/swift/${ring}.builder rebalance
done

4.12.3 Enabling and Configuring Swift

Perform the following steps on the master node.

  1. Enable the Swift service.

    $ kollacli property set enable_swift yes
  2. Set the Swift hash path prefix and suffix.

    The hash path prefix and suffix are random numbers that are used in Swift's hashing process when storing objects.

    To set the prefix and suffix, use the following commands

    $ kollacli password set swift_hash_path_prefix
    $ kollacli password set swift_hash_path_suffix

    The kollacli password set command prompts you for the password (the prefix or suffix), and to confirm the password. The password value is not displayed on screen. The values for the prefix and suffix must be different. Use a cryptographically secure random number generator to generate the number, for example:

    # openssl rand –hex 10

    Alternatively, you can use the kollacli password init command to set secure random numbers for the prefix and suffix. However, this command also configures all the passwords that have not been set for individual services, which may not be want you want.

    Caution

    For security reasons, keep these values secret. The prefix and suffix values are stored in the /etc/kolla/passwords.yml file. Only the kolla user and members of the kolla group have read-write access to this file.

    Do not lose these values. Ensure you have a secure backup of the passwords.yml file.

    Once you have deployed Swift, you must not change these values.

  3. Configure the Swift storage devices to use.

    The default property settings (shown below) are sufficient. Use the swift_devices_name property to specify the partition label if you used a different label.

    $ kollacli property set swift_devices_match_mode strict
    $ kollacli property set swift_devices_name KOLLA_SWIFT_DATA
    Caution

    Do not change the swift_devices_mount_point property from the default value of /srv/node.