JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun ZFS Storage 7000 System Administration Guide
search filter icon
search icon

Document Information

Preface

1.  Introduction

2.  Status

3.  Configuration

Configuration

Introduction

Initial

Initial Configuration

Prerequisites

Summary

BUI

Configuring Management Port

CLI

Performing Initial Configuration with the CLI

Network

Network Configuration

Devices

Datalinks

Interfaces

IP MultiPathing (IPMP)

Performance and Availability

Routing

Routing Entries

Routing Properties

BUI

Configuration

Addresses

Routing

CLI

Tasks

BUI

CLI

Infiniband Upgrade Procedures for Q3.2010

Storage

Introduction

Configure

Configuration Rules and Guidelines

Verification

Allocation on SAS-1 Systems

Allocation on SAS-2 Systems

Profile Configuration

Import

Add

Unconfig

Scrub

Tasks

BUI

SAN

SAN

Terminology

Targets and Initiators

Target and Initiator Groups

BUI

CLI

Terms

SAN Terminology

FC

Fibre Channel

Target Configuration

Clustering Considerations

Initiator Configuration

Switch Considerations

Clustering Considerations

Performance Considerations

Troubleshooting

Queue Overruns

Link-level Issues

BUI

Changing modes of FC ports

Viewing discovered FC ports

Creating FC Initiator Groups

Associating a LUN with an FC initiator group

CLI

Changing modes of FC ports

Viewing discovered FC ports

Creating FC Initiator Groups

Associating a LUN with an FC initiator group

Scripting Aliases for Initiators and Initiator Groups

Protocol Support

IP based Protocol Support

Initiator Support

Introduction

Supported InfiniBand Client-Side Data Protocols

Supported Fiber Channel Initiators

FCMPxIO

Configuring FC Client Multipathing

Configuring Solaris Initiators

Configuring Windows Initiators

Windows Tunables - Microsoft DSM Details

Configuring Linux Initiators

Configuring VMware ESX Initiators

Troubleshooting

See Also

iSCSI

Introduction

Target Configuration

Clustering Considerations

Initiator Configuration

Planning Client Configuration

Solaris iSCSI/iSER and MPxIO Considerations

Troubleshooting

Observing Performance

BUI

Creating an Analytics Worksheet

CLI

Adding an iSCSI target with an auto-generated IQN

Adding an iSCSI target with a specific IQN and RADIUS authentication

Adding an iSCSI initiator which uses CHAP authentication

Adding an iSCSI target group

Adding an iSCSI initiator group

SRP

Introduction

Target configuration

Clustering Considerations

Initiator configuration

Observing Performance

Multipathing Considerations

Linux with OFED SRP Initiator

OFED 1.5 Issues

VMWare 4.0

Path Selection Plugin (psp)

Storage Array Type Plugin (satp)

VMWare ESX 4.0 Issues

BUI

iSER Target Configuration

SRP Target Configuration

CLI

Users

Introduction

Roles

Authorizations

Properties

Users

Roles

BUI

CLI

Tasks

BUI

CLI

Generic

Preferences

Introduction

BUI

CLI

SSH Public Keys

Alerts

Introduction

Actions

Send Email

Send SNMP trap

Send Syslog Message

Resume/Suspend Dataset

Resume/Suspend Worksheet

Execute Workflow

Threshold Alerts

BUI

CLI

Tasks

BUI

Cluster

Clustering

Features and Benefits

Drawbacks

Terminology

Subsystem Design

Cluster Interconnect I/O

Resource Management Concepts

Takeover and Failback

Configuration Changes in a Clustered Environment

Clustering Considerations for Storage

Clustering Considerations for Networking

Clustering Considerations for Infiniband

Redundant Path Scenarios

Preventing 'Split-Brain' Conditions

Estimating and Reducing Takeover Impact

Setup Procedure

Node Cabling

JBOD Cabling

BUI

Unconfiguring Clustering

4.  Services

5.  Shares

6.  Analytics

7.  Application Integration

Glossary

Index

FCMPxIO

Configuring FC Client Multipathing

The Sun ZFS Storage 7000 series uses Asymmetric Logical Unit Access (ALUA) to provide FC target multipathing support. Please refer to SCSI Primary Commands (SPC) definition on t10 at http://www.t10.org if you need more information.

The following instructions provide a guide for setting up the FC host clients that are connected to a FC target enabled clustered appliance.

Configuring Solaris Initiators

See the FC initiator qualification table for the set of Solaris versions that are currently qualified for ALUA. MPxIO is enabled on Solaris x86 platforms but disabled on SPARC by default. The mpathadm show LU command shows the path state changing from active to standby or standby to active. Alternately, you can also use luxadm display to show path state.

The stmsboot utility enables and disables MPxIO, for example:

  1. To enable MPxIO, run stmsboot -D fp -e

  2. To disable MPxIO, run stmsboot -D fp -d

  3. To verify the state, run mpathadm show LU

Configuring Windows Initiators

ALUA multipathing is supported by native Windows 2008/R2 MPIO only. R2 is required to avoid issues during cluster failover and takeover. SP2 is not supported.

  1. Verify that the FC HBA Windows driver is installed and the HBA is operational.

  2. Install or verify installation of the Windows Server 2008 MPIO Optional Component. Configure multipath support for the SS7000 by issuing the mpclaim.exe -r -i -a ""command at a Windows Command Prompt. This will force a system reboot and is necessary to complete MPIO setup and ensure proper path/LUN discovery.

  3. Once the client has rebooted, verify that Windows Client can discover and access SS7000 LUN(s) and the correct number of paths and path states are displayed. This can be verified using the Windows Disk Management utility. For each LUN on the SS7000 there should be only one corresponding disk available in the Disk Management GUI.

  4. In the event of a SS7000 node failure, the default Microsoft DSM timer counters may be insufficient to ensure I/O continues uninterrupted. To alleviate this, we recommend setting the following Timer Counter values in the DSM details section of a disks Multi-Path Disk Device properties.

Windows Tunables - Microsoft DSM Details
Windows Tunable
Description
Default Value
Recommended Value
PathVerifyEnabled
Enables path verification by MPIO on all paths every “N” seconds . N depends on the value set in PathVerificationPeriod.
Disabled
Enabled
PathVerificationPeriod
Used to indicate the periodicity (in seconds) with which MPIO has been requested to perform path verification. This field is only used if PathVerifyEnabled = TRUE.
30 seconds
5 seconds
RetryInterval
Specifies the interval of time after which a failed request is retried (after the DSM has decided so, and assuming that the IO has been retried less number of times than RetryCount).
1 second
5 seconds
RetryCount
Specifies the number of times a failed IO occurs before the DSM determines that a failing request must be retried.
3
300
PDORemovePeriod
Controls the amount of time (in seconds) that the multipath pseudo-LUN will continue to remain in system memory, even after losing all paths to the device.
20 seconds
1500 seconds

Errata:

Configuring Linux Initiators

The following instructions cover:

Note that only certain kernel and multipath driver revisions are supported; see supported initiators table for details.

1. Ensure the correct device-mappers are installed.

2. Stop the multipathd service.

# service multipathd stop
Stopping multipathd daemon:                                [  OK  ]

3. Add the following lines to /etc/multipath.conf:

NOTE: replace <i>NNNN</i> with the appropriate model number -- e.g., 7420.
 device 
 { 
    vendor                     "SUN" 
    product                    "Sun Storage <i>NNNN</i>" 
    getuid_callout             "/sbin/scsi_id -g -u -s /block/%n" 
    prio_callout               "/sbin/mpath_prio_alua /dev/%n" 
    hardware_handler           "0" 
    path_grouping_policy       group_by_prio 
    failback                   immediate 
    no_path_retry              queue 
    rr_min_io                  100 
    path_checker               tur 
    rr_weight                  uniform 
 } 
 defaults {  
   find_multipaths             yes  
   user_friendly_names         yes  
 }  
 
 devices {  
   device {  
     vendor                    "SUN"  
     product                   "Sun Storage <i>NNNN</i>"  
     getuid_callout            "/sbin/scsi_id -g -u -s /block/%n"  
     prio                      alua  
     hardware_handler          "1 alua"  
     path_grouping_policy      group_by_prio  
     failback                  immediate  
     no_path_retry             600  
     rr_min_io                 100  
     path_checker              tur  
     rr_weight                 uniform features "0"  
   }  
 }  
 defaults { 
   multipath_tool              "/sbin/multipath -v0" 
   udev_dir                    /dev 
   polling_interval            100 
   default_selector            "round-robin 0" 
   default_path_grouping_policy  group_by_prio 
   default_getuid_callout      "/lib/udev/scsi_id -g -u -d /dev/%n" 
   default_prio_callout        "/bin/true" 
   prio                        "alua" 
   default_features            "0" 
   rr_min_io                   100 
   failback                    immediate 
   user_friendly_names         yes 
   path_checker                tur 
   no_path_retry               1000 
 } 

 device { 
   vendor                      "SUN" 
   product                     "Sun Storage <i>NNNN</i>" 
 } 

4. Enable multipath and verify by starting the multipathd service.

#service multipathd start
Starting multipathd daemon:                                [  OK  ]

5. Run the multipath command after the SCSI bus rescan is finished to verify multipath I/O is enabled. Note that standby paths will be shown as due to a known Linux bug. For this reason, it is recommended that users verify the paths are actually operational before putting the system into production. For more details, refer to the Troubleshooting section below.

#multipath –ll
sdd: checker msg is "tur checker reports path is down"
mpath1 (3600144f094f0bd0300004b31c88f0001) dm-2 SUN,Sun Storage 7410 (or 7310)
[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=50][active]
\_ 2:0:0:0 sdb 8:16  [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 2:0:1:0 sdd 8:48  [failed][faulty]

Configuring VMware ESX Initiators

VMware ESX HCL

http://www.vmware.com/resources/compatibility/search.php?action=search&deviceCategory=san&key=Oracle

For ESX 4.1 servers:

No addition rules need to be added the default driver is ALUA aware.

For ESX 4.0 servers:

1. Verify the current SATP plugin that is in use by issuing the esx nmp device list command

# esxcli nmp device list
naa.600144f0ed81720500004bb3c1f60002
   Device Display Name: SUN Fibre Channel Disk (naa.600144f0ed81720500004bb3c1f60002)
   Storage Array Type: VMW_SATP_DEFAULT_AA
   Storage Array Type Device Config:
   Path Selection Policy: VMW_PSP_FIXED
   Path Selection Policy Device Config: {preferred=vmhba0:C0:T1:L0;current=vmhba0:C0:T1:L0}
   Working Paths: vmhba0:C0:T1:L0

VMW_SATP_DEFAULT_AA is the default plugin. This plugin is not ALUA-capable.

2. Add rules to enable the ALUA plugin for the ZFS Storage 7000 arrays by using the esxcli nmp satp addrule command.

# esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7110" -c "tpgs_on"
# esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7210" -c "tpgs_on"
# esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7310" -c "tpgs_on"
# esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7410" -c "tpgs_on"
# esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7120" -c "tpgs_on"
# esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7320" -c "tpgs_on"
# esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7420" -c "tpgs_on"

3. Verify the rule was correctly added.

# esxcli nmp satp listrules | grep SUN                     

VMW_SATP_ALUA SUN Sun Storage 7110 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7310 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7410 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7210 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN ZFS Storage 7120 tpgs_on ZFS Storage 7000 VMW_SATP_ALUA SUN ZFS Storage 7320 tpgs_on ZFS Storage 7000 VMW_SATP_ALUA SUN ZFS Storage 7420 tpgs_on ZFS Storage 7000

4. Reboot the VMware ESX server. When server has rebooted, check to ensure the correct plugin is now in effect with the esxcli nmp device list command.

# esxcli nmp device list
naa.600144f0ed81720500004bb3c1f60002
   Device Display Name: SUN Fibre Channel Disk (naa.600144f0ed81720500004bb3c1f60002)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on;explicit_support=off;
   explicit_allow=on;alua_followover=on;{TPG_id=0,TPG_state=STBY}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_MRU
   Path Selection Policy Device Config: Current Path=vmhba1:C0:T1:L0
   Working Paths: vmhba1:C0:T1:L0

Troubleshooting

This section describes troubleshooting known issues.

Multipath-tools version 0.4.7 bundled in OEL 5.4 is unable to recognize paths in ALUA standby access state

In SCSI spec, a target port which is in standby state does not respond to Test Unit Ready command, so standby paths are shown as in multipath command output.

The fix for this problem is committed into the multipath-tool source tree on 2009-04-21 (which is later than its 0.4.8 official release). Users have to obtain the latest version of the multipath-tool source code from: http://christophe.varoqui.free.fr/

Users should get the latest source code from its git repository. The multipath-tools-0.4.8.tar.bz2 tarball does not contain the fix.

Finally, the status shown in multipath command output does not impact functionality like I/O and failover/failback, so updating the package is not mandatory.

See Also

http://docs.sun.com/app/docs/doc/816-5166/fcinfo-1m?l=en&a=view&q=fcinfo

http://docs.sun.com/source/819-0139-12/

http://www.microsoft.com/downloads/details.aspx?FamilyID=CBD27A84-23A1-4E88-B198-6233623582F3&displaylang=en

http://www.redhat.com/docs/manuals/csgfs/browse/4.6/DM_Multipath/index.html