Oracle Cloud Infrastructure Documentation

Configuring Attachments to Ultra High Performance Volumes

When you attach a volume configured for the Ultra High Performance level, to achieve the optimal performance, the volume attachment must be multipath-enabled.

The Block Volume service attempts to enable the attachment for multipath when the volume is being attached. If not all of the prerequisites have been addressed, the volume attachment will not be multipath-enabled.

To determine if the volume attachment is multipath-enabled, see Checking If a Volume Attachment is Multipath-Enabled.

This topic describes the prerequisites and the steps you can take to ensure the volume attachment is multipath-capable.

For iSCSI attachments, see Multipath-Enabled iSCSI Attachments. After you have confirmed that the iSCSI attachment is multipath-enabled, see Working with Multipath-Enabled iSCSI-Attached Volumes for steps you can use for connecting to and working with the volume.

For paravirtualized attachments, see Multipath-Enabled Paravirtualized Attachments.

Multipath-Enabled iSCSI Attachments

The section describes the prerequisites and steps required to configure multipath-enabled iSCSI volume attachments.

Prerequisites

Following is a list of prerequisites and requirements for multipath-enabled iSCSI attachments.

  1. The instance must be based on a supported shape. See Supported Compute Shapes for Multipath-Enabled iSCSI Attachments for more information.

  2. The instance must be running a supported image, see Supported Images for Multipath-Enabled iSCSI Attachments for more information.

  3. The maximum number of ultra high performance block volume attachments is 32.

    • Multiple ultra high performance volumes are supported on instances with Oracle Cloud Agent version 1.39 or later. Check the version with one of the following commands:

      Oracle Linux: yum info oracle-cloud-agent

      Ubuntu: snap info oracle-cloud-agent

    • If an ultra high performance volume is already attached before using the multiple ultra high performance volumes feature, then we recommend that you reattach the volumes to achieve best performance. If you don't reattach the volumes, then you will not achieve best performance.
    • If an ultra high performance volume is attached to the instance, and if Oracle Cloud Agent is upgraded to version 1.39, or later, for the first time, then we recommend that you reboot the instance to achieve best performance. If you don't reboot the instance, then you will not achieve the best maximum performance per instance.

  4. The Block Volume Management plugin must be enabled for the instance. See Enabling the Block Volume Management Plugin for more information.

  5. The compute instance must have either a public IP address or a service gateway for the Block Volume Management plugin to be able to connect to Oracle services, see Service gateways or public IP addresses.
  6. Permissions must be configured to allow the Block Volume Management plugin to report the iSCSI setup results for multipath-enabled iSCSI attachments, see Configure Permissions.

  7. The volume attachment must be configured to use a consistent device path. See Connecting to Volumes With Consistent Device Paths for more information.

Important

You can use oci-utils or oci-iscsi-config when configuring and working with iSCSI attached volumes, however you should ensure that you don't use both at the same time. If you've started using one of these tools to work with a volume, you should continue to use the same tool with that volume, and not switch to using the other tool. If you use both, you could encounter the behavior described in Volume Attachment Incorrectly Reported as Multipath-Enabled.

Supported Images for Multipath-Enabled iSCSI Attachments

For multipath-enabled attachments, the attached instance must be running a Linux-based image.

Note

Multipath-enabled attachments are not supported for Windows instances or Oracle Autonomous Linux instances.

Required Custom Image Configuration to Support Multipath-Enabled iSCSI Attachments

For a custom image to support multipath-enabled iSCSI attachments, the Unbreakable Enterprise Kernel (UEK) version must be UEK6U1 or higher. The UEK6U1 UEK is associated with the kernel major release version 5.4.17-2036, released in November, 2020.

After you have verified that the custom image UEK version is one that supports multipath-enabled attachments, you need to update the Storage.Iscsi.MultipathDeviceSupported property for the image to true. For more information, see Configuring Image Capabilities for Custom Images.

Supported Compute Shapes for Multipath-Enabled iSCSI Attachments

This section identifies the Compute shapes that support multipath-enabled iSCSI attachments. For more details, such as performance characteristics and maximum number of attachments, see Performance Details for Shapes.

VM Shapes

Current VM shapes configured for 16 cores or more support multipath-enabled attachments. See VM Shapes for iSCSI and Paravirtualized Attached Volumes for performance characteristics of volumes attached to VMs with iSCSI attachments. To verify that a shape supports multipath-enabled attachments look for the value Yes in the Supports Ultra High Performance (UHP) column in the VM Shapes table.

Bare Metal Shapes

All current bare metal shapes support multipath-enabled iSCSI attachments. See for more information Bare Metal Shapes for performance characteristics of block volumes attached to bare metal instances.

Multipath-Enabled Paravirtualized Attachments

The section describes the prerequisites and steps required to configure multipath-enabled paravirtualized volume attachments.

Prerequisites

Following is a list of prerequisites and requirements for multipath-enabled paravirtualized attachments.

  1. The instance must be based on a supported shape. See Supported Compute Shapes for Multipath-Enabled Paravirtualized Attachments for more information.

  2. The instance must be running a supported image, see Supported Images for Multipath-Enabled Paravirtualized Attachments for more information.

  3. Maximum number of ultra high performance block volume attachments is 32, with some exceptions as noted in Block Volume Capabilities and Limits.

    If an ultra high performance volume is already attached, then we recommend that you reattach the volumes to achieve best performance. If you do not reattach the volumes, then you will not achieve best maximum performance per instance.

Supported Images for Multipath-Enabled Paravirtualized Attachments

For multipath-enabled attachments, the attached instance must be running one of the following images or a custom image based on one of these images:

  • Oracle Linux
  • Ubuntu
  • CentOS
  • Windows
Note

Multipath-enabled attachments are not supported for Oracle Autonomous Linux instances.

Supported Compute Shapes for Multipath-Enabled Paravirtualized Attachments

Current VM shapes configured for 16 cores or more support multipath-enabled attachments. See VM Shapes for iSCSI and Paravirtualized Attached Volumes for performance characteristics of volumes attached to VMs with paravirtualized attachments. To verify that a shape supports multipath-enabled attachments look for the value Yes in the Supports Ultra High Performance (UHP) column in the VM Shapes table.

Note

Multipath-enabled attachments to bare metal instances only support iSCSI attachments.

For more details, such as performance characteristics and maximum number of attachments, see Performance Details for Shapes.

Checking If a Volume Attachment is Multipath-Enabled

Learn how to check if a resource attached to a volume is multipath-enabled.

    1. Open the navigation menu  and select Storage. Under Block Storage, select Block Volumes.
    2. Under List Scope, in the Compartment list, select the name of the compartment where the block volume was created.
    3. In the Block Volumes list, select the name of the volume.
    You can check whether a volume attachment is multipath-enabled in the Console from the Volume Details page or the Instance Details page.

    From the Volume Details Page

    1. Open the navigation menu  and select Storage. Under Block Storage, select Block Volumes.

    2. Click the block volume that you want to check the volume attachment for.

    3. Click Attached Instances in the Resources section.

    4. Check the value displayed in the Multipath column.

      • Yes: The volume is configured for the Ultra High Performance level and the volume attachment is multipath-enabled. No further action is required.

      • No: The volume is not configured for the Ultra High Performance level, the volume does not need to be multipath-enabled. No further action is required.

      • No with a warning icon: The volume is configured for the Ultra High Performance level, but the volume attachment is not multipath-enabled. To achieve optimal performance, you need to ensure the volume is attached to a supported instance shape, and that the required prerequisites are configured.

    The following image shows the multipath column in the Console.

    Multipath column values in the Console.

    From the Instance Details Page

    1. Open the navigation menu  and select Compute. Under Compute, select Instances.

    2. Click the instance that you want to check the volume attachment for.

    3. Click Attached Block Volumes in the Resources section.

    4. Check the value displayed in the Multipath column.

      • Yes: The volume is configured for the Ultra High Performance level and the volume attachment is multipath-enabled. No further action is required.
      • No: The volume is not configured for the Ultra High Performance level, the volume does not need to be multipath-enabled. No further action is required.

      • No with a warning icon: The volume is configured for the Ultra High Performance level, but the volume attachment is not multipath-enabled. To achieve optimal performance, you need to ensure the volume is attached to a supported instance shape, and that the required prerequisites are configured.

    The following image shows the multipath column in the Console.

    Multipath column values in the Console.

  • Use the oci compute volume-attachment get command and specify the --volume-attachment-id parameter to check the multipath enablement of a volume:

    oci compute volume-attachment get --volume-attachment-id <volume_group_id>

    For example:

    oci compute volume-attachment get --volume-attachment-id ocid1.volumeattachment.oc1.phx.<unique_ID>

    The is-multipath property will be true for multipath-enabled attachments and false for attachments that are not multipath-enabled.

    For a complete list of parameters and values for CLI commands, see the CLI Command Reference.

  • Run the GetVolumeAttachment operation and specify the volumeAttachmentId to get information for a volume attachment.

    The is-multipath property will be true for multipath-enabled attachments and false for attachments that are not multipath-enabled.