How to Configure iSCSI Storage Using COMSTAR and Multiple Paths

Perform this procedure to configure OpenSolaris Common Multiprotocol SCSI TARget (COMSTAR) on locally attached storage, to share access among multiple cluster nodes. This procedure uses multiple paths between iSCSI initiators and iSCSI targets and also configures a mirrored ZFS storage pool to provide high availability. This procedure optionally includes configuring the I/O multipathing feature (MPxIO).

Note –

If you use single paths between iSCSI initiators and iSCSI targets, go instead to How to Configure iSCSI Storage Using COMSTAR and Single Paths.

Before You Begin

Ensure that the storage configuration meets Open HA Cluster 2009.06 requirements. See iSCSI Storage.

(Optional) If you intend to use I/O multipathing (MPxIO), on each node ensure that the I/O multipathing feature is enabled for iSCSI.

The feature is enabled when the mpxio-disable property is set to no.
phys-schost# cat /kernel/drv/iscsi.conf … mpxio-disable="no";
For more information about I/O multipathing, see Solaris Fibre Channel Storage Configuration and Multipathing Support Guide.

Determine the IP address of each adapter that is used for the private interconnect.

You will specify these addresses later when you create iSCSI target ports. Output is similar to the following:

phys-schost# /usr/cluster/bin/clinterconnect status
=== Cluster Transport Paths ===

Endpoint1               Endpoint2               Status
---------               ---------               ------
phys-schost-1:adapter1    phys-schost-2:adapter1    Path online
phys-schost-1:adapter2    phys-schost-2:adapter2    Path online

phys-schost# ifconfig adapter1
nge1: flags=201008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4,CoS> mtu
1500 index 3
        inet 172.16.1.1 netmask ffffff80 broadcast 172.16.1.127
        ether 0:14:4f:8d:9b:3 
phys-schost# ifconfig adapter2
e1000g1: flags=201008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4,CoS>
mtu 1500 index 4
        inet 172.16.0.129 netmask ffffff80 broadcast 172.16.0.255
        ether 0:15:17:35:9b:a1

On each node, perform the procedures that are listed in Configuring an iSCSI Storage Array With COMSTAR (Task Map).

Observe the following additional instructions when you configure a COMSTAR iSCSI target in an Open HA Cluster 2009.06 configuration:

Task

Documentation

Special Instructions

1. Perform basic setup.

Getting Started with COMSTAR

To create the SCSI logical unit, perform the procedure How to Create a Disk Partition SCSI Logical Unit.

If you specify a whole disk instead of a slice to the sbdadm create-lu command, run the cldevice clear command afterwards to clear the DID namespace.

2. Configure iSCSI target ports.

How to Configure iSCSI Target Ports

Create a target for each private-network adapter on each node.

3. Configure the iSCSI target.

How to Configure an iSCSI Target for Discovery

Use either static discovery or SendTargets. Do not use dynamic discovery.

4. Make a logical unit available.

How to Make Logical Units Available for iSCSI and iSER

5. Configure an initiator system to access target storage.

How to Configure an iSCSI Initiator

Specify the node's clprivnet IP address as the target system. To determine the IP address of the clprivnet interface, run the following command. Output is similar to the following:

phys-schost# ifconfig clprivnet0
    clprivnet0:
    flags=1009843<UP,BROADCAST,RUNNING,MULTICAST,\
    MULTI_BCAST,PRIVATE,IPv4>
    mtu 1500 index 5
    inet 172.16.4.1 netmask fffffe00 broadcast \
    172.16.5.255
    ether 0:0:0:0:0:1

When completed, on each node update and populate the global-devices namespace.
phys-schost# scdidadm -r phys-schost# cldevice populate

Disable fencing for each of the newly created devices.

phys-schost# /usr/cluster/bin/cldevice set -p default_fencing=nofencing-noscrub device

From one node, create a mirrored ZFS storage pool from the DID devices that you created on each node.
phys-schost# zpool create pool mirror /dev/did/dsk/dNsX /dev/did/dsk/dYsX

From one node, configure the mirrored ZFS storage pool as an HAStoragePlus resource.

phys-schost# /usr/cluster/bin/clresourcegroup resourcegroup
phys-schost# /usr/cluster/bin/clresourcetype register HAStoragePlus
phys-schost# /usr/cluster/bin/clresource create -g resourcegroup -t HASToragePlus  \
-p Zpools=pool resource
phys-schost# /usr/cluster/bin/clresourcegroup manage resourcegroup
phys-schost# /usr/cluster/bin/clresourcegroup online resourcegroup

Next Steps

If you want to use IP Security Architecture (IPsec) to provide secure TCP/IP communication on the cluster interconnect, go to How to Configure IP Security Architecture (IPsec) on the Cluster Private Interconnect.

Otherwise, configure the data services that you want to run on your cluster. Go to Configuring Data Services.