Skip Navigation Links | |
Exit Print View | |
Oracle Solaris Administration: Devices and File Systems Oracle Solaris 11 Information Library |
1. Managing Removable Media (Overview)
2. Managing Removable Media (Tasks)
3. Accessing Removable Media (Tasks)
4. Writing CDs and DVDs (Tasks)
5. Managing Devices (Overview/Tasks)
6. Dynamically Configuring Devices (Tasks)
7. Using USB Devices (Overview)
9. Using InfiniBand Devices (Overview/Tasks)
11. Administering Disks (Tasks)
12. SPARC: Setting Up Disks (Tasks)
13. x86: Setting Up Disks (Tasks)
14. Configuring Storage Devices With COMSTAR
COMSTAR and iSCSI Technology (Overview)
COMSTAR Software and Hardware Requirements
Configuring COMSTAR (Task Map)
Configuring Dynamic or Static Target Discovery
Configuring iSCSI Devices With COMSTAR
How to Enable the STMF Service
How to Back Up and Restore a COMSTAR Configuration
How to Create the iSCSI Target
How to Configure an IB HCA for iSER
How to Configure an iSCSI Initiator
How to Remove Discovered iSCSI Targets
Creating iSCSI Target Portal Groups
How to Create a Target Portal Group for iSCSI Targets
Making SCSI Logical Units Available
Configuring Fibre Channel Devices With COMSTAR
Configuring Fibre Channel Ports For COMSTAR
How to Display Existing FC Port Bindings
How to Set All FC Ports to a Specific Mode
How to Set Selected FC Ports to Initiator or Target Mode
Making Logical Units Available for FC and FCoE
How to Make Logical Units Available for FC and FCoE
Configuring FCoE Devices With COMSTAR
Enabling 802.3x PAUSE and Jumbo Frames on the Ethernet Interface
How to Create FCoE Target Ports
How to Verify That an FCoE Target Port Is Working
How to Delete FCoE Target Ports
Configuring SRP Devices With COMSTAR
How to Enable the SRP Target Service
How to Verify SRP Target Status
Configuring Authentication in Your iSCSI-Based Storage Network
How to Configure CHAP Authentication for Your iSCSI Initiator
How to Configure CHAP Authentication for Your iSCSI Target
Using a Third-Party RADIUS Server to Simplify CHAP Management in Your iSCSI Configuration
How to Configure a RADIUS Server for Your iSCSI Target
How to Configure a RADIUS Server for Your iSCSI Initiator
Oracle Solaris iSCSI and RADIUS Server Error Messages
Setting Up iSCSI Multipathed Devices in Oracle Solaris
How to Enable Multiple iSCSI Sessions for a Target
Monitoring Your iSCSI Configuration
How to Display iSCSI Configuration Information
Modifying iSCSI Initiator and Target Parameters
How to Modify iSCSI Initiator and Target Parameters
Troubleshooting iSCSI Configuration Problems
No Connections to the iSCSI Target From the Local System
How to Troubleshoot iSCSI Connection Problems
iSCSI Device or Disk Is Not Available on the Local System
How to Troubleshoot iSCSI Device or Disk Unavailability
Use LUN Masking When Using the iSNS Discovery Method
15. Configuring and Managing the Oracle Solaris Internet Storage Name Service (iSNS)
16. The format Utility (Reference)
17. Managing File Systems (Overview)
18. Creating and Mounting File Systems (Tasks)
19. Configuring Additional Swap Space (Tasks)
20. Copying Files and File Systems (Tasks)
You can set up and configure a COMSTAR Internet SCSI (iSCSI) target and make it available over the network. The iSCSI features can work over a normal Internet connection (such as Ethernet) using the standard iSCSI protocol. The iSCSI protocol also provides naming and discovery services, authentication services using CHAP and RADIUS, and centralized management through iSNS.
If the system has at least one InfiniBand (IB) Host Channel Adapter (HCA) and connectivity exists between the initiator and the target, the iSCSI connection uses iSCSI Extensions for RDMA (iSER) for enhanced data throughput. iSER used with an IB HCA provides high bandwidth, low CPU utilization, and a single network connection that multiple protocols can share.
The iSER target and initiator both use a component called iSCSI Data Mover to provide iSCSI connection services. Use of iSER is automatic whenever both the initiator and the target are configured to use IP addresses that correspond to IB-capable devices.
COMSTAR uses SMF to store its current, persistent configuration, such as logical unit mapping, host group definitions, and target group definitions. When the service is enabled during boot or when using the svcadm command, it clears any stale configuration data inside the kernel framework, and then reloads the configuration from the SMF repository into the driver. After the configuration is loaded, any changes that are made to the configuration are automatically updated inside the driver database, as well as inside the SMF repository. For example, any changes made through the stmfadm command are automatically updated in both areas.
The COMSTAR target mode framework runs as the stmf service. By default, the service is disabled. You must enable the service to use COMSTAR functionality. You can identify the service with the svcs command. If you have not rebooted the server since installing the group/feature/storage-server package, the service might not be enabled correctly.
target# pkg install group/feature/storage-server Packages to install: 75 Create boot environment: No Services to restart: 7 DOWNLOAD PKGS FILES XFER (MB) Completed 75/75 9555/9555 105.7/105.7 PHASE ACTIONS Install Phase 13347/13347 PHASE ITEMS Package State Update Phase 75/75 Image State Update Phase 2/2 Loading smf(5) service descriptions: 17/17 Loading smf(5) service descriptions: 3/3 PHASE ITEMS Reading Existing Index 8/8 Indexing Packages 75/75 Indexing Packages 75/75 Optimizing Index... PHASE ITEMS Indexing Packages 573/573
target# svcadm enable stmf # svcs stmf STATE STIME FMRI online 09:42:32 svc:/system/stmf:default
After you complete your COMSTAR configuration, make a copy that can be restored, if needed.
# svccfg export -a stmf > COMSTAR.backup
# svccfg import COMSTAR.backup
The logical unit provider for creating disk-type LUNs is called sbd. However, you must initialize the storage for the logical unit before you can share a disk-type LUN.
The disk volume provided by the server is referred to as the target. When the LUN is associated with an iSCSI target, it can be accessed by an iSCSI initiator.
The process for creating SCSI LUNs is as follows:
Initialize the storage for the LUN, also known as the backing store.
Create a SCSI LUN by using the backing store.
When a LUN is created, it is assigned a global unique identifier (GUID), for example, 600144F0B5418B0000004DDAC7C10001. The GUID is used to refer to the LUN in subsequent tasks, such as mapping a LUN to select hosts.
The following steps are completed on the system that is providing the storage device.
target# zpool create sanpool mirror c2t3d0 c2t4d0
target# zfs create -V 2g sanpool/vol1
target# stmfadm create-lu /dev/zvol/rdsk/sanpool/vol1 Logical unit created: 600144F0B5418B0000004DDAC7C10001
You can find the device path for the ZFS volume in the /dev/zvol/rdsk/pool-name/ directory.
target# stmfadm list-lu LU Name: 600144F0B5418B0000004DDAC7C10001
This command makes the LUN accessible to all systems.
target# stmfadm add-view 600144F0B5418B0000004DDAC7C10001
If you want to restrict the LUN view to specific systems, see How to Restrict LUN Access to Selected Systems.
target# stmfadm list-view -l 600144F0B5418B0000004DDAC7C10001 View Entry: 0 Host group : All Target group : All LUN : 0
This procedure assumes that you are logged in to the local system will contains the iSCSI target.
target# svcadm enable -r svc:/network/iscsi/target:default
Confirm that the service is enabled.
target# svcs -l iscsi/target fmri svc:/network/iscsi/target:default name iscsi target enabled true state online next_state none state_time Mon May 23 14:48:59 2011 logfile /var/svc/log/network-iscsi-target:default.log restarter svc:/system/svc/restarter:default dependency require_any/error svc:/milestone/network (online) dependency require_all/none svc:/system/stmf:default (online)
target# itadm create-target Target iqn.1986-03.com.sun:02:73d12edc-9bb9-cb44-efc4-c3b36c039405 successfully created
target# itadm list-target -v TARGET NAME STATE SESSIONS iqn.1986-03.com.sun:02:73d12edc-9bb9-cb44-efc4-c3b36c039405 online 0 alias: - auth: none (defaults) targetchapuser: - targetchapsecret: unset tpg-tags: default
An InfiniBand (IB) Host Channel Adapter (HCA) is required to take full advantage of the iSCSI Extensions for RDMA (iSER) capabilities. To use iSER, you must configure the HCA on both the target and the initiator.
See the vendor documentation for details.
The target and the initiator must be on the same subnet. This example uses ibd0 as the driver.
# ipadm create-addr ibd0
# ipadm create-addr -T static -a local=10.1.190.141/24 ibd0/v4addr
# ipadm show-addr ADDROBJ TYPE STATE ADDR lo0/v4 static ok 127.0.0.1/8 e1000g0/_b dhcp ok 10.1.190.141/24 lo0/v6 static ok ::1/128 e1000g0/_a addrconf ok fe80::214:4fff:fe27:360c/10
target# ping initiator-ip initiator# ping target-ip
Part of the initiator configuration process is to identify the iSCSI target discovery method, which presents an initiator with a list of available targets. You can configure iSCSI targets for static, SendTargets, or iSNS dynamic discovery. Dynamic discovery using the SendTargets option is the optimum configuration for an iSCSI initiator that accesses a large number of targets, such over an iSCSI to Fibre Channel bridge. SendTargets dynamic discovery requires the IP address and port combination of the iSCSI target for the iSCSI initiator to perform the target discovery. The most common discovery method is SendTargets.
When configuring the target discovery method, you must provide the following information, depending on which method you choose:
SendTargets – Target IP address
iSNS – iSNS server address
Static – Target IP address and target name
For more information about configuring target discovery methods, see Configuring Dynamic or Static Target Discovery.
initiator# svcadm enable network/iscsi/initiator
target# ipadm show-addr ADDROBJ TYPE STATE ADDR lo0/v4 static ok 127.0.0.1/8 e1000g0/_b dhcp ok 10.80.227.189/24 lo0/v6 static ok ::1/128 e1000g0/_a addrconf ok fe80::214:4fff:fe27:360c/10 target# itadm list-target -v TARGET NAME STATE SESSIONS iqn.1986-03.com.sun:02:73d12edc-9bb9-cb44-efc4-c3b36c039405 online 0 alias: - auth: none (defaults) targetchapuser: - targetchapsecret: unset tpg-tags: default
initiator# iscsiadm add static-config iqn.1986-03.com.sun:02:73d12edc-9bb9-cb44-efc4-c3b36c039405, 10.80.227.189
initiator# iscsiadm list static-config Static Configuration Target: iqn.1986-03.com.sun:02:73d12edc-9bb9-cb44-efc4-c3b36c039405, 10.80.227.189:3260
The iSCSI connection is not initiated until the discovery method is enabled. See the next step.
If you have configured a dynamically discovered (SendTargets) target, configure the SendTargets discovery method.
initiator# iscsiadm add discovery-address 10.80.227.189
If you have configured a dynamically discovered (iSNS) target, configure the iSNS discovery method.
initiator# iscsiadm add isns-server 10.80.227.189
If you have configured a dynamically discovered (SendTargets) target, enable the SendTargets discovery method.
initiator# iscsiadm modify discovery --sendtargets enable
If you have configured a dynamically discovered (iSNS) target, enable the iSNS discovery method.
initiator# iscsiadm modify discovery --iSNS enable
If you have configured static targets, enable the static target discovery method.
initiator# iscsiadm modify discovery --static enable
initiator# devfsadm -i iscsi
The associated targets are logged out after you perform any of the following actions:
Remove a discovery address
Remove an iSNS server
Remove a static configuration
Disable a discovery method
If these associated targets are still in use, for example, they have mounted file systems, the logout of these devices will fail, and they will remain on the active target list.
This optional procedure assumes that you are logged in to the local system where access to an iSCSI target device has already been configured.
If you need to disable the SendTargets discovery method, use the following command:
initiator# iscsiadm modify discovery --sendtargets disable
If you need to disable the iSNS discovery method, use the following command:
initiator# iscsiadm modify discovery --iSNS disable
If you need to disable the static target discovery method, use the following command:
initiator# iscsiadm modify discovery --static disable
Remove an iSCSI SendTargets discovery entry.
For example:
initiator# iscsiadm remove discovery-address 10.0.0.1:3260
Remove an iSCSI iSNS discovery entry.
For example:
# iscsiadm remove isns-server 10.0.0.1:3205
Remove a static iSCSI discovery entry.
For example:
initiator# iscsiadm remove static-config eui.5000ABCD78945E2B,10.0.0.1
Note - If you attempt to disable or remove a discovery entry that has an associated logical unit (LUN) in use, the disable or remove operation fails with the following message:
logical unit in use
If this errors occurs, stop all associated I/O on the LUN, unmount the file systems, and so on. Then, repeat the disable or remove operation.
For example:
target# itadm delete-target target-IQN
This command might error if the target is still online or busy:
The target is online or busy. Use the -f (force) option, or 'stmfadm offline-target iqn.1986-03.com.sun:02:99619b8a-a4dc-4cfb-93f0-ee3debe7b0c8' itadm delete-target failed with error 16
You can create a target portal group (TPG) to manage the discovery of multiple iSCSI and iSER targets. A TPG is a list of IP addresses to determine upon which interfaces a specific iSCSI target will listen.
A TPG contains IP addresses and TCP port numbers. To use this capability, you need to do the following:
Create a TPG as a list of ip-address:port specifiers by using the itadm create-tpg command.
Bind a specific iSCSI target to a TPG by using the itadm modify-target -t command.
When an iSCSI target is made active, an iSCSI listener is created for each IP address and port belonging to a TPG associated with that target.
A TPG is an efficient way to control which targets are discovered through specific ports. For example, you could restrict your iSCSI target so that it is available only through one specific IP address or only through a set of iSER-capable IP addresses.
Note - Do not confuse target portal groups with target groups. A target group is a list of SCSI target ports that are all treated the same when creating views. Creating a view can help you facilitate LUN mapping. Each view entry specifies a target group, as host group, and a LUN. For more information on Target Groups and LUN mapping, see Making SCSI Logical Units Available and stmfadm(1M).
To learn about static and iSNS target discovery, see Configuring Dynamic or Static Target Discovery. The iSCSI initiator uses the iscsiadm command to discover TPGs. For more information, see iscsiadm(1M) and itadm(1M).
When you use the SendTargets discovery and iSER at the same time, a common convention is to use a TPG to associate a specific iSCSI target port with only iSER-capable IP addresses. For example, if a target system has four IP addresses, A, B, C, and D, and only addresses B and C are iSER-capable, then addresses B and C could be added to a TPG, and assigned to a target T.
An iSCSI initiator with both Ethernet and InfiniBand (IB) interfaces could use the SendTargets discovery method to discover the possible storage targets. Without the use of TPGs, the initiator might always prefer the use of the Ethernet interfaces over the IB interfaces. By associating target T only with the IB interfaces, the initiator correctly prefers using its IB-capable interface when connecting to target T.
You can create a target portal group (TPG) by providing a unique name, and a TPG Tag (ranging from 2–65535) is automatically generated. TPG Tag 1 is reserved for the default TPG that is used when you do not explicitly set a TPG on the target. The portal for the default TPG matches requests from all network interfaces on port 3260.
The following steps shows how to create two TPGs, TPGA and TPGB, that use port 8000 for the IP addresses in TPGB.
target# itadm create-tpg TPGA 192.168.0.1 192.168.0.2 target# itadm create-tpg TPGB 192.168.0.2:8000 192.168.0.2:8000
Note - IPv4 portals are specified in dotted address notation (for example, 192.168.0.1). IPv6 portal addresses must be enclosed in square brackets.
# itadm modify-target -t TPGA,TPGB eui.20387ab8943ef7548
# itadm list-tpg -v
You can remove a TPG with the itadm delete-tpg command.
After the devices have been discovered by the Oracle Solaris iSCSI initiator, the login negotiation occurs automatically. The Oracle Solaris iSCSI driver determines the number of available LUNs and creates the device nodes. Then, the iSCSI devices can be treated as any other SCSI device.
You can create a ZFS storage pool on the LUN and then create a ZFS file system.
You can view the iSCSI disks on the local system by using the format utility.
initiator# format 0. c0t600144F0B5418B0000004DDAC7C10001d0 <SUN-COMSTAR-1.0 cyl 1022 alt 2 hd 128 sec 32> /scsi_vhci/disk@g600144f0b5418b0000004ddac7c10001 1. c8t0d0 <Sun-STK RAID INT-V1.0 cyl 17830 alt 2 hd 255 sec 63> /pci@0,0/pci10de,375@f/pci108e,286@0/disk@0,0 2. c8t1d0 <Sun-STK RAID INT-V1.0-136.61GB> /pci@0,0/pci10de,375@f/pci108e,286@0/disk@1,0 3. c8t2d0 <Sun-STK RAID INT-V1.0-136.61GB> /pci@0,0/pci10de,375@f/pci108e,286@0/disk@2,0 4. c8t3d0 <Sun-STK RAID INT-V1.0 cyl 17830 alt 2 hd 255 sec 63> /pci@0,0/pci10de,375@f/pci108e,286@0/disk@3,0 Specify disk (enter its number): 0 selecting c0t600144F0B5418B0000004DDAC7C10001d0 [disk formatted]
In the above output, disk 0 is an iSCSI LUN under MPxIO control. Disks 1-4 are local disks.
initiator# zpool create pool-name c0t600144F0B5418B0000004DDAC7C10001d0 initiator# zfs create pool-name/fs-name
The ZFS file system is automatically mounted when created and is remounted at boot time.
Simply registering a logical unit (LUN) with the STMF framework does not make it available to hosts (initiators) on the network. This section describes how to make LUNs visible to initiator hosts for the following configurations.
For iSCSI, Fibre Channel, and FCoE configurations, a LUN must be mapped before it can be accessed. You can choose one of the following methods, both of which use the stmfadm command:
Simple mapping – Exposes the LUN to all initiators through all the ports, using one command. Making LUNs available to all hosts uses this method.
Selective mapping– Enables you to specify the hosts that can access the LUN. Making LUNs available to selected hosts uses this method. This process includes the following steps:
Defining host groups – A host group is a name given to a set of hosts (initiators) that are allowed to access the same LUNs. This step is not needed if the same set of LUNs is visible to all the hosts, as in simple mapping.
Defining target groups – A target group is a name given to a set of target ports that export the same set of LUNs to the same set of host groups. This step is not needed if the same set of LUNs is visible to all the hosts, as in simple mapping.
Adding one or more views for each logical unit – Adding a view creates a bridge between the LUN and the host initiator. When an initiator from the host group logs in to a target port contained in the target group, the LUN is visible.
Note - Don't confuse a target group with a target portal group (TPG). A TPG is a list of IP addresses that an iSCSI target listens to. A TPG can help you restrict an iSCSI target so that it is available only through one specific IP address. For more information on target groups, see stmfadm(1M).
A view entry consists of four components: host group, target group, logical unit (LUN), and LUN identifier. Of these four components, only the LUN identifier is required. If the other components are omitted, the following default values are assumed:
If the host group is omitted, the all initiators value is assumed.
If the target group is omitted, the all targets value is assumed.
If the LUN is omitted, the system chooses a suitable LUN for the entry.
This procedure makes a LUN available to all initiator hosts on a storage network.
# stmfadm list-lu -v
3 # stmfadm add-view GUID-number
Use this procedure to restrict LUNs to selected hosts on a storage network. If you are using Fibre Channel ports, first identify the World Wide Names (WWN). Then, selectively map a logical unit number (LUN) to the ports on host-a, for example. A target group (targets-0) is also defined for a given set of target ports that export the same set of LUNs to the same host group.
For information about configuring a target group, see stmfadm(1M).
initiator# fcinfo hba-port HBA Port WWN: 210000e08b195dae Port Mode: Initiator Port ID: 0 OS Device Name: /dev/cfg/c8 Manufacturer: QLogic Corp. Model: 375-3108-xx Firmware Version: 03.03.28 FCode/BIOS Version: fcode: 1.13; Serial Number: not available Driver Name: qlc Driver Version: 20100408-3.01 Type: unknown State: offline Supported Speeds: 1Gb 2Gb Current Speed: not established Node WWN: 200000e08b195dae NPIV Not Supported . . .
target# stmfadm create-hg host-a 210000e08b195dae 210100e08b395dae
target# stmfadm add-hg-member -g host-a
target# stmfadm create-tg targets-0
Each SCSI target can be a member of only one target group.
target# stmfadm add-tg-member -g targets-0 wwn.guid-number
target# stmfadm list-lu -v
target# stmfadm add-view -h host-a -t targets-0 -n 1 guid-number