Using Exascale Block Storage with iSCSI

5.2 Using Exascale Block Storage with iSCSI

This topic outlines the procedure for using Exascale block storage in conjunction with iSCSI.

The Exascale block store provides capabilities to create and manage arbitrary-sized raw block volumes based on Exascale storage.

You can use standard iSCSI protocols and tools to utilize an Exascale volume as an iSCSI target. The iSCSI target is implemented on an Exadata compute node and supports iSCSI initiators that may be external to Exadata.

iSCSI is recommended for use-cases where a client wants to use the volume outside the Exadata RDMA Network Fabric or requires a bootable volume.

Before you can use Exascale block storage with iSCSI, note the following:

The block store manager (BSM) service must be running in the Exascale cluster.
The block store virtual IP addresses (VIPs) must be configured by the Exascale administrator.
The block store worker (BSW) service must be running on each Exadata compute node that you want act as an iSCSI initiator.

To begin, you can create an Exascale block volume and iSCSI volume attachment.

For example:

@>mkvault myvault
Vault @myvault created.

@>mkvolume 2g --attributes name=ivol1 --vault myvault
Created volume with id 2:390535212ea7448299bcc7bc7f7653b2

@>lsvolume 2:390535212ea7448299bcc7bc7f7653b2 --detail
id                                   2:390535212ea7448299bcc7bc7f7653b2
name                                 ivol1
blockSizeIOPS                        8192
burstDurationSec                     120
burstIOPS                            2000
burstIOPSCreditPerSec                0
burstInitialDurationSec              60
burstInitialMBPS                     0
burstMBPS                            16
compartmentId
contentType                          DATA
creationTime                         2021-08-29 20:53:57 PDT
filePath                             @myvault/vol.2ccedfbe32404fcca1859b4421aeb58a
mediaType                            HC
numAttachments                       0
iopsProvisioned                      10
bandwidthProvisioned                 1
redundancyType                       normal
size                                 2G
state                                AVAILABLE
user                                 slc15ziq
vault                                @myvault
vipId                                897
burstInitialIOPS                     2000


@>mkvolumeattachment 2:390535212ea7448299bcc7bc7f7653b2 iqn.1988-12.com.example.exa01 --protocol iscsi --attributes chapUserId=X1234567-user-1,chapPassword=X1234567-pass-1,chapMutualUserId=X1234567-user-2,chapMutualPassword=X1234567-pass-2
Created attachment with id 2-2:9a7bced09e514bd9bb701d0556020c1c

@>lsvolumeattachment --protocol iscsi 2-2:9a7bced09e514bd9bb701d0556020c1c --detail
id                                   2-2:9a7bced09e514bd9bb701d0556020c1c
attachTime                           2021-08-29 21:00:13 PDT
initiatorIQN                         iqn.1988-12.com.example.exa01
state                                AVAILABLE
targetIQN                            iqn.2017-01.com.example.exa01.x8664:sn.dae3f3e1-ef8d-47f9-ae5b-3707b15b2072
targetPortal                         192.168.184.190:3260
volume                               2:390535212ea7448299bcc7bc7f7653b2
user                                 exa01

@>

An iSCSI attachment creates a association between the Exascale block volume (iSCSI target) and an iSCSI initiator.

In the example, the volume ID is 2:390535212ea7448299bcc7bc7f7653b2, which corresponds to the Exascale block volume that was created at the beginning of the example. The iSCSI initiator ID is iqn.1988-12.com.example.exa01. This value is an attribute of the iSCSI initiator.

After attachment, the iSCSI initiator can use standard iSCSI protocols and commands to interact with the target.

The following example shows the use of iSCSI client tools on Linux to access the attachment from the previous example. In the example, note the use of the attribute values that are derived directly from the volume attachment details in the lsvolumeattachment output.

# # Set variables for required attributes. Values come from Exascale volume attachment attributes.
# export ip=192.168.184.190
# export port=3260
# export target=iqn.2017-01.com.example.exa01.x8664:sn.dae3f3e1-ef8d-47f9-ae5b-3707b15b2072
# export username=X1234567-user-1
# export password=X1234567-pass-1
# export mutual_username=X1234567-user-2
# export mutual_password=X1234567-pass-2
# # Discover the iSCSI target.
# iscsiadm --mode discoverydb --type sendtargets --portal $ip:$port --discover
192.0.2.190:3260,1 iqn.2017-01.com.example.exa01.x8664:sn.dae3f3e1-ef8d-47f9-ae5b-3707b15b2072
# # Configure CHAP.
# iscsiadm --mode node --portal $ip:$port --op=update --name node.session.auth.authmethod --value=CHAP
# iscsiadm --mode node --portal $ip:$port --op=update --name node.session.auth.username --value=$username
# iscsiadm --mode node --portal $ip:$port --op=update --name node.session.auth.password --value=$password
# iscsiadm --mode node --portal $ip:$port --op=update --name node.session.auth.username_in --value=$mutual_username
# iscsiadm --mode node --portal $ip:$port --op=update --name node.session.auth.password_in --value=$mutual_password
# # Connect to the iSCSI target.
# iscsiadm --mode node -T $target --portal $ip:$port --login
Logging in to [iface: default, target: iqn.2017-01.com.example.exa01.x8664:sn.dae3f3e1-ef8d-47f9-ae5b-3707b15b2072, portal: 192.168.184.190,3260] (multiple)
Login to [iface: default, target: iqn.2017-01.com.example.exa01.x8664:sn.dae3f3e1-ef8d-47f9-ae5b-3707b15b2072, portal: 192.168.184.190,3260] successful.
# # The Exascale volume is now attached using iSCSI as /dev/sda.
# lsscsi
[2:0:0:0]    disk    ORACLE   ExascaleVolume   1.0   /dev/sda
# # Confirmation that the unit serial number for /dev/sda maps back to Exascale volume ID 2:390535212ea7448299bcc7bc7f7653b2.
# sg_inq -p 0x80 /dev/sda
VPD INQUIRY: Unit serial number page
  Unit serial number: 2:390535212ea7448299bcc7bc7f7653b2
# # Now use the device to create and mount a file system.
# mkfs -t ext4 /dev/sda
mke2fs 1.42.9 (28-Dec-2013)
/dev/sda is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=16 blocks, Stripe width=16 blocks
131072 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=536870912
16 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

# mkdir /mnt/ivol1
# mount /dev/sda /mnt/ivol1
# df /mnt/ivol1
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/sda         1998672  6144   1871288   1% /mnt/ivol1
#

Note:

Depending on your use case, you may need to modify the permissions on the iSCSI device files before using them. For example, to use the iSCSI device files as Oracle ASM grid disks, you may need to configure them using udev rules similar to the following:

# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sda", OWNER="oracle", GROUP="asmdba", MODE="0660"

Parent topic: Using Exascale Block Storage