JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle ZFS Storage Appliance Administration Guide
search filter icon
search icon

Document Information

Preface

1.  Introduction

2.  Status

3.  Configuration

Configuration

Introduction

Initial

Initial Configuration

Prerequisites

Summary

BUI

Configuring Management Port

CLI

Performing Initial Configuration with the CLI

Network

Network Configuration

Devices

Datalinks

Interfaces

IP MultiPathing (IPMP)

Performance and Availability

Routing

Routing Entries

Routing Properties

BUI

Configuration

Addresses

Routing

CLI

Tasks

BUI

CLI

Storage

Introduction

Configure

Configuration Rules and Guidelines

Verification

Allocation on SAS-2 Systems

Profile Configuration

Import

Add

Unconfig

Scrub

Tasks

BUI

SAN

SAN

Terminology

Targets and Initiators

Target and Initiator Groups

BUI

CLI

Terms

SAN Terminology

FC

Fibre Channel

Target Configuration

Clustering Considerations

Initiator Configuration

Clustering Considerations

Performance Considerations

Troubleshooting

Queue Overruns

Link-level Issues

BUI

Changing modes of FC ports

Viewing discovered FC ports

Creating FC Initiator Groups

Associating a LUN with an FC initiator group

CLI

Changing modes of FC ports

Viewing discovered FC ports

Creating FC Initiator Groups

Associating a LUN with an FC initiator group

Scripting Aliases for Initiators and Initiator Groups

FCMPxIO

Configuring FC Client Multipathing

Tasks

FCMPxIO Tasks

Configuring Solaris Initiators

Configuring Windows Initiators

Windows Tunables - Microsoft DSM Details

Configuring Linux Initiators

Configuring VMware ESX Initiators

Troubleshooting

See Also

iSCSI

Introduction

Target Configuration

Clustering Considerations

Initiator Configuration

Planning Client Configuration

Solaris iSCSI/iSER and MPxIO Considerations

Troubleshooting

Observing Performance

Tasks

iSCSI Tasks

CLI

Adding an iSCSI target with an auto-generated IQN

Adding an iSCSI target with a specific IQN and RADIUS authentication

Adding an iSCSI initiator which uses CHAP authentication

Adding an iSCSI target group

Adding an iSCSI initiator group

SRP

Introduction

Target configuration

Clustering Considerations

Initiator configuration

Observing Performance

Multipathing Considerations

VMWare 4.0

Path Selection Plugin (psp)

Storage Array Type Plugin (satp)

VMWare ESX 4.0 Issues

Tasks

SRP Tasks

CLI

Users

Introduction

Roles

Authorizations

Properties

Users

Roles

BUI

CLI

Tasks

BUI

CLI

Generic

Preferences

Introduction

BUI

CLI

SSH Public Keys

Alerts

Introduction

Actions

Send Email

Send SNMP trap

Send Syslog Message

Resume/Suspend Dataset

Resume/Suspend Worksheet

Execute Workflow

Threshold Alerts

BUI

CLI

Tasks

BUI

Cluster

Clustering

Features and Benefits

Drawbacks

Terminology

Subsystem Design

Cluster Interconnect I/O

Resource Management Concepts

Takeover and Failback

Configuration Changes in a Clustered Environment

Clustering Considerations for Storage

Clustering Considerations for Networking

Clustering Considerations for Infiniband

Redundant Path Scenarios

Preventing 'Split-Brain' Conditions

Estimating and Reducing Takeover Impact

Tasks

Cluster Tasks

Setup Procedure

Shutting Down a Clustered Configuration

Node Cabling

JBOD Cabling

BUI

Tasks

Cluster Tasks

Unconfiguring Clustering

4.  Services

5.  Shares

6.  Integration

Glossary

SRP

Introduction

SCSI RDMA Protocol, is a protocol supported by the appliance for sharing SCSI based storage over a network that provides RDMA services (i.e. InfiniBand).

Target configuration

SRP ports are shared with other IB port services such as IPoIB and RDMA. The SRP service may only operate in target mode. SRP targets have the following configurable properties.

Property
Description
Target EUI
The Extended Unique Identifier (EUI) for this target. The EUI is automatically assigned by the system and is equal to the HCA GUID over which the SRP port service is running.
Alias
A human-readable nickname for this target.

In addition to those properties, the BUI indicates whether a target is online or offline:

icon
description
image:Status: On
Target is online
image:Status: Off
Target is offline
Clustering Considerations

On clustered platforms, peer targets should be configured into the same target group for highly available (multi-pathed) configurations. SRP multipathed I/O is an initiator-side configuration option.

Initiator configuration

SRP initiators have the following configurable properties.

Property
Description
Initiator EUI
The EUI for this initiator.
Alias
A human-readable nickname for this initiator.

Observing Performance

SRP performance can be observed via analytics, whereby one can breakdown operations or throughput by initiator or target. {{Server}}/wiki/images/cfg_san_srp.png

Multipathing Considerations

The following sections provide a guide for setting up host clients.

VMWare 4.0

The VMware Native MultiPath Plugin (nmp) has two components that can be changed on a device by device, path by path, or array basis.

Path Selection Plugin (psp)

Controls which physical path is used for I/O

     # esxcli nmp psp list
     Name           Description                        
     VMW_PSP_MRU    Most Recently Used Path Selection  
     VMW_PSP_RR     Round Robin Path Selection         
     VMW_PSP_FIXED  Fixed Path Selection  
Storage Array Type Plugin (satp)

Controls how failover works

The SATP has to be configured to recognize the array vendor or model string in order to change the basic failover mode from a default Active/Active type array to ALUA.

By default the Sun Storage 7000 cluster was coming up as a Active/Active array only.

To manually change this you need to use the ESX CLI to add rules to have the ALUA plugin claim the 7000 luns.

   # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7120" -c "tpgs_on"
   # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7320" -c "tpgs_on"
   # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7420" -c "tpgs_on"
    options are:
    -s VMW_SATP_ALUA - for the ALUA SATP
    -e description of the rule
    -V Vendor 
    -M Model
    -c claim option for Target Portal Group (7000 seems to support implicit)

If no luns have been scanned/discovered, you can simply rescan the adapter to find new luns, they should be claimed by the ALUA plugin. If luns are already present, reboot the ESX host.

After the reboot, we should see the luns being listed under the VMW_SATP_ALUE array type.

   # esxcli nmp device list
   naa.600144f096bb823800004b707f2d0001
   Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config:
      {implicit_support=on;explicit_support=off;explicit_allow=on;
       alua_followover=on; {TPG_id=0,TPG_state=AO}{TPG_id=1,TPG_state=STBY}}
   Path Selection Policy: VMW_PSP_MRU
   Path Selection Policy Device Config: Current Path=vmhba_mlx4_1.1.1:C0:T1:L0
   Working Paths: vmhba_mlx4_1.1.1:C0:T1:L0

Relevant lun path lists should show a Active and a Standby path

   # esxcli nmp path list
   gsan.80fe53553e0100282100-gsan.80fe8f583e0100282100
   -naa.600144f096bb823800004b707f2d0001
   Runtime Name: vmhba_mlx4_1.1.1:C0:T2:L0
   Device: naa.600144f096bb823800004b707f2d0001
   Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001)
   Group State: standby
   Storage Array Type Path Config: 
   {TPG_id=1,TPG_state=STBY,RTP_id=256,RTP_health=UP}
   Path Selection Policy Path Config: {non-current path}
   gsan.80fe53553e0100282100-gsan.80fe73583e0100282100
   -naa.600144f096bb823800004b707f2d0001
   Runtime Name: vmhba_mlx4_1.1.1:C0:T1:L0
   Device: naa.600144f096bb823800004b707f2d0001
   Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001)
   Group State: active
   Storage Array Type Path Config: 
   {TPG_id=0,TPG_state=AO,RTP_id=2,RTP_health=UP}
   Path Selection Policy Path Config: {current path}
VMWare ESX 4.0 Issues

The esxcl nmp path list command should report an active and a standby path one each for the SRP targets in a cluster configuration.

[root@ib-client-5 vmware]# esxcli nmp path list
gsan.80fe53553e0100282100-gsan.80fe8f583e0100282100-  
naa.600144f096bb823800004b707f2d0001
   Runtime Name: vmhba_mlx4_1.1.1:C0:T2:L0
   Device: naa.600144f096bb823800004b707f2d0001
   Device Display Name: Local SUN Disk 
(naa.600144f096bb823800004b707f2d0001)
   Group State: standby
   Storage Array Type Path Config: 
{TPG_id=1,TPG_state=STBY,RTP_id=256,RTP_health=UP}
   Path Selection Policy Path Config: {non-current path}
gsan.80fe53553e0100282100-gsan.80fe73583e0100282100-
   naa.600144f096bb823800004b707f2d0001
   Runtime Name: vmhba_mlx4_1.1.1:C0:T1:L0
   Device: naa.600144f096bb823800004b707f2d0001
   Device Display Name: Local SUN Disk
(naa.600144f096bb823800004b707f2d0001)
   Group State: active
   Storage Array Type Path Config:
{TPG_id=0,TPG_state=AO,RTP_id=2,RTP_health=UP}
   Path Selection Policy Path Config: {current path}

When this problem occurs, the active or standby path may not be shown in the output of esxcli nmp path list.

Workaround: None

When this problem happens, the Linux guest system log will report in its /var/log/messages log:

Feb 10 16:10:00 ib-client-5 vmkernel: 1:21:41:36.385 cpu3:4421)<3>ib_srp:
Send tsk_mgmt target[vmhba_mlx4_1.1.1:2] out of TX_IU head 769313 tail 769313 lim 0

Workaround: Reboot guest VM

Tasks

This section describes instructions for using the BUI to configure iSER and SRP targets and initiators.

SRP Tasks

iSER Target Configuration

In the BUI, iSER targets are managed as iSCSI targets on the Configuration > SAN screen.

  1. To configure ibd interfaces, select the ibd interface (or ipmp), and drag it to the Datalinks list to create the datalink on the Configuration > Network screen.
  2. Drag the Datalink to the Interfaces list to create a new interface. image:Image
  3. To create an iSER target, got to the Configuration > SAN screen.
  4. Click the iSCSI Targets link and then click the image:Add item add icon to add a new iSER target with an alias.
  5. To create a target group, drag the target you just created to the iSCSI Target Group list. image:Image
  6. To create an initiator, click the Initiator link and then click the iSCSI initiators link.
  7. Click the image:Add item add icon to add a new initiator.
  8. Enter the Initiator IQN and an alias and click OK. While creating an initiator group is optional, if you don't create a group, the LUN associated with the target will be available to all initiators.
  9. To create a group, drag the initiator to the iSCSI Initiator Groups list. image:Image
  10. To create a LUN, go to the Shares screen and click the LUN link.
  11. Click the image:Add item add icon and associate the new LUN with target or initiator groups you created already using the Target Group and Initiator Groups menu. image:Image

SRP Target Configuration

This procedure describes the steps for configuring SRP targets.

  1. Connect HCA ports to IB interfaces.
  2. : The targets are automatically discovered by the appliance.
  3. To create the target group, go to the Configuration > SAN screen.
  4. Click the Target link and then click SRP targets.
  5. :The SRP targets page appears.
  6. To create the target group, use the image:Move move icon to drag a target to the Target Groups list.
  7. (Optional) To create an initiator and initiator group on the Initiator screen, click the image:Add item icon, collect GUID from initiator, assign it a name, and drag it to initiator group.
  8. To create a LUN and associate it with the SRP target and initiators you created in the previous steps, go to the Shares screen.
  9. Click the LUN link and then click the LUN image:Add item icon. Use the Target Group and Initiator Group menus on the Create LUN dialog to select the SRP groups to associate with the LUN.

CLI

The following example demonstrates how to create an SRP target group named targetSRPgroup using the CLI configuration san targets srp groups context:

swallower:configuration san targets srp groups> create
swallower:configuration san targets srp group (uncommitted)> set name=targetSRPgroup
                         name = targetSRPgroup (uncommitted)
swallower:configuration san targets srp group (uncommitted)> 
set targets=eui.0002C903000489A4
                      targets = eui.0002C903000489A4 (uncommitted)
swallower:configuration san targets srp group (uncommitted)> commit
swallower:configuration san targets srp groups> list
GROUP     NAME
group-000 targetSRPgroup
         |
         +-> TARGETS
             eui.0002C903000489A4

The following example demonstrates how to create a LUN and associate it with the targetSRPgroup using the CLI shares CLI context:

swallower:shares default> lun mylun
swallower:shares default/mylun (uncommitted)> set targetgroup=targetSRPgroup
                  targetgroup = targetSRPgroup (uncommitted)
swallower:shares default/mylun (uncommitted)> set volsize=10
                      volsize = 10 (uncommitted)
swallower:shares default/mylun (uncommitted)> commit
swallower:shares default> list
Filesystems:
NAME             SIZE    MOUNTPOINT
test             38K     /export/test
LUNs:
NAME              SIZE    GUID
mylun             10G     600144F0E9D19FFB00004B82DF490001