Using Fibre Channel SR-IOV Virtual Functions

Starting with the Oracle VM Server for SPARC 3.1.1 release provides support for SR-IOV Fibre Channel. An SR-IOV Fibre Channel host bus adapter (HBA) might have one or more ports each of which appears as SR-IOV physical function. You can identify Fibre Channel physical functions by the IOVFC string in its device name.

Each Fibre Channel physical function has unique port and node world-wide name (WWN) values that are provided by the card manufacturer. When you create virtual functions from a Fibre Channel physical function, the virtual functions behave like a Fibre Channel HBA device. Each virtual function must have a unique identity that is specified by the port WWN and node WWN of the SAN fabric. You can use the Logical Domains Manager to automatically or manually assign the port and node WWNs. By assigning your own values, you can fully control the identity of any virtual function.

The Fibre Channel HBA virtual functions use the N_Port ID Virtualization (NPIV) method to log in to the SAN fabric. Because of this NPIV requirement, you must connect the Fibre Channel HBA port to an NPIV-capable Fibre Channel switch. The virtual functions are managed entirely by the hardware or the firmware of the SR-IOV card. Other than these exceptions, Fibre Channel virtual functions work and behave the same way as a non-SR-IOV Fibre Channel HBA device. The SR-IOV virtual functions have the same capabilities as the non-SR-IOV devices, so all types of SAN storage devices are supported in either configuration.

The virtual functions' unique port and node WWN values enable a SAN administrator to assign storage to the virtual functions in the same way as he would for any non-SR-IOV Fibre Channel HBA port. This mananagement includes zoning, LUN masking, and quality of service (QoS). You can configure the storage so that it is accessible exclusively to a specific logical domain without being visible to the physical function in the root domain.

You can use both the static and dynamic SR-IOV methods to manage Fibre Channel SR-IOV devices.

Fibre Channel SR-IOV Hardware Requirements

For information about the required PCIe Fibre Channel SR-IOV hardware, see PCIe SR-IOV Hardware and Software Requirements in Oracle VM Server for SPARC 3.1.1.1, 3.1.1, and 3.1 Release Notes .

For Fibre Channel SR-IOV support, the root domain must be running at least the Oracle Solaris 11.1.17.0.0 OS. The I/O domains must be running at least the Oracle Solaris 11.1.17.0.0 OS.

Fibre Channel SR-IOV Requirements and Limitations

The Fibre Channel SR-IOV feature has the following recommendations and limitations:

• The SR-IOV card must run the latest version of firmware that supports the SR-IOV feature.

• The Fibre Channel PCIe card must be connected to a Fibre Channel switch that supports NPIV and that is compatible with the PCIe card.

• The Logical Domains Manager properly autogenerates unique port-wwn and node-wwn property values by connecting the control domains of all systems to the same SAN fabric and by being part of the same multicast domain.

  If you cannot configure this environment, you must manually provide the node-wwn and port-wwn values when you create the virtual function. This behavior ensures that there are no naming conflicts. See World-Wide Name Allocation for Fibre Channel Virtual Functions.

Fibre Channel Device Class-Specific Properties

You can use the ldm create-vf or the ldm set-io commands to set the following Fibre Channel virtual function properties:

bw-percent

Specifies the percentage of the bandwidth to be allocated to the Fibre Channel virtual function. Valid values are from 0 to 100. The total bandwidth value assigned to a Fibre Channel physical function's virtual functions cannot exceed 100. The default value is 0 so that the virtual function gets a fair share of the bandwidth that is not already reserved by other virtual functions that share the same physical function.

node-wwn

Specifies the node world-wide name (WWN) for the Fibre Channel virtual function. Valid values are non-zero. By default, this value is allocated automatically. If you manually specify this value, you must also specify a value for the port-wwn property. For more information, see World-Wide Name Allocation for Fibre Channel Virtual Functions.

port-wwn

Specifies the port WWN for the Fibre Channel virtual function. Valid values are non-zero. By default, this value is allocated automatically. If you manually specify this value, you must also specify a value for the node-wwn property. For more information, see World-Wide Name Allocation for Fibre Channel Virtual Functions.

You cannot modify the node-wwn or the port-wwn property values while the Fibre Channel virtual function is in use. However, you can modify the bw-percent property value dynamically even when the Fibre Channel virtual function is in use.

World-Wide Name Allocation for Fibre Channel Virtual Functions

The Logical Domains Manager supports both the automatic allocation and the manual assignment of world-wide names for the Fibre Channel virtual functions.

Automatic World-Wide Name Allocation

Logical Domains Manager allocates a unique MAC address from its automatic MAC address allocation pool and creates IEEE format node-wwn and port-wwn property values.

port-wwn = 10:00:XX:XX:XX:XX:XX:XX
node-wwn = 20:00:XX:XX:XX:XX:XX:XX

XX:XX:XX:XX:XX:XX is the automatically allocated MAC address.

This automatic allocation method produces unique WWNs when the control domains of all systems that are connected to the same Fibre Channel fabric are also connected by Ethernet and are part of the same multicast domain. If you cannot meet this requirement, you must manually assign unique WWNs, which is required on the SAN.

Manual World-Wide Name Allocation

You can construct unique WWNs by using any method. This section describes how to to create WWNs from the Logical Domains Manager manual MAC address allocation pool. You must guarantee the uniqueness of the WWNs you allocate.

Logical Domains Manager has a pool of 256,000 MAC addresses that are available for manual allocation in the 00:14:4F:FC:00:00 - 00:14:4F:FF:FF:FF range.

The following example shows the port-wwn and node-wwn property values based on the 00:14:4F:FC:00:01 MAC address:

port-wwn = 10:00:00:14:4F:FC:00:01
node-wwn = 20:00:00:14:4F:FC:00:01

00:14:4F:FC:00:01 is the manually allocated MAC address. For information about automatic MAC address allocation, see Assigning MAC Addresses Automatically or Manually.

---

Note - It is best to manually assign the WWNs to ensure predictable configuration of the SAN storage.

You must use the manual WWN allocation method when all systems are not connected to the same multicast domain by Ethernet. You can also use this method to guarantee that the same WWNs are used when Fibre Channel virtual functions are destroyed and re-created.

---

Creating Fibre Channel SR-IOV Virtual Functions

This section describes how to dynamically create and destroy virtual functions. If you cannot use the dynamic methods to perform these actions, initiate a delayed reconfiguration on the root domain before you create or destroy virtual functions.

How to Create a Fibre Channel SR-IOV Virtual Function

If you cannot use this dynamic method, use the static method instead. See Static SR-IOV.

1. Identify the physical function device.

   primary# ldm list-io

   Note that the name of the physical function includes the location information for the PCIe SR-IOV card or on-board device.

2. If I/O virtualization for the bus that has the physical function is not enabled already, enable it.

   Perform this step only if I/O virtualization is not enabled already for the bus that has the physical function.

   See How to Enable I/O Virtualization for a PCIe Bus.

3. Create a single virtual function or multiple virtual functions from a physical function either dynamically or statically.

   After you create one or more virtual functions, you can assign them to a guest domain.

   • Dynamic method:
     • To create multiple virtual functions from a physical function all at the same time, use the following command:

       primary# ldm create-vf -n number | max pf-name

       Use the ldm create-vf -n max command to create all the virtual functions for that physical function at one time. This command automatically allocates the port and node WWNs for each virtual function and sets the bw-percent property to the default value, which is 0. This value specifies that fair share bandwidth is allocated to all virtual functions.

       ---

       Tip  - Create all virtual functions for the physical function at once. If you want to manually assign WWNs, first create all of the virtual functions and then use the ldm set-io command to manually assign your WWN values for each virtual function. This technique minimizes the number of state transitions when creating virtual functions from a physical function.

       ---

       You can use either the path name or the pseudonym name to specify virtual functions. However, the recommended practice is to use the pseudonym name.

     • To create one virtual function from a physical function, use the following command:

       primary# ldm create-vf [bw-percent=value] [port-wwn=value node-wwn=value] pf-name

       You can also manually specify Fibre Channel class-specific property values.

   • Static method:
     1. Initiate a delayed reconfiguration.

        primary# ldm start-reconf root-domain-name

     2. Create a single virtual function or multiple virtual functions from a physical function.

        Use the same commands as shown previously to dynamically create the virtual functions.

     3. Reboot the root domain.
        • To reboot the non-primary root domain:

          primary# ldm stop-domain -r root-domain

        • To reboot the primary root domain:

          primary# shutdown -i6 -g0 -y

Example 6-19  Displaying Information About the Fibre Channel Physical Function

This example shows information about the /SYS/MB/PCIE7/IOVFC.PF0 physical function:

• This physical function is from a board in a PCIe slot, PCIE7.

• The IOVFC string indicates that the physical function is a Fibre Channel SR-IOV device.

primary# ldm list-io
NAME                                      TYPE   BUS      DOMAIN   STATUS
----                                      ----   ---      ------   ------
pci_0                                     BUS    pci_0    primary  IOV
pci_1                                     BUS    pci_1    rootdom1 IOV
niu_0                                     NIU    niu_0    primary
niu_1                                     NIU    niu_1    primary
/SYS/MB/PCIE0                             PCIE   pci_0    primary  OCC
/SYS/MB/PCIE2                             PCIE   pci_0    primary  OCC
/SYS/MB/PCIE4                             PCIE   pci_0    primary  OCC
/SYS/MB/PCIE6                             PCIE   pci_0    primary  EMP
/SYS/MB/PCIE8                             PCIE   pci_0    primary  EMP
/SYS/MB/SASHBA                            PCIE   pci_0    primary  OCC
/SYS/MB/NET0                              PCIE   pci_0    primary  OCC
/SYS/MB/PCIE1                             PCIE   pci_1    rootdom1 OCC
/SYS/MB/PCIE3                             PCIE   pci_1    rootdom1 OCC
/SYS/MB/PCIE5                             PCIE   pci_1    rootdom1 OCC
/SYS/MB/PCIE7                             PCIE   pci_1    rootdom1 OCC
/SYS/MB/PCIE9                             PCIE   pci_1    rootdom1 OCC
/SYS/MB/NET2                              PCIE   pci_1    rootdom1 OCC
/SYS/MB/NET0/IOVNET.PF0                   PF     pci_0    primary
/SYS/MB/NET0/IOVNET.PF1                   PF     pci_0    primary
/SYS/MB/PCIE5/IOVNET.PF0                  PF     pci_1    rootdom1
/SYS/MB/PCIE5/IOVNET.PF1                  PF     pci_1    rootdom1          
/SYS/MB/PCIE7/IOVFC.PF0                   PF     pci_1    rootdom1
/SYS/MB/PCIE7/IOVFC.PF1                   PF     pci_1    rootdom1
/SYS/MB/NET2/IOVNET.PF0                   PF     pci_1    rootdom1
/SYS/MB/NET2/IOVNET.PF1                   PF     pci_1    rootdom1

The following command shows more details about the specified physical function. The maxvfs value indicates the maximum number of virtual functions that is supported by the device.

primary# ldm list-io -l /SYS/MB/PCIE7/IOVFC.PF0
NAME                                      TYPE   BUS      DOMAIN   STATUS   
----                                      ----   ---      ------   ------   
/SYS/MB/PCIE7/IOVNET.PF0                  PF     pci_0    rootdom1           
[pci@400/pci@1/pci@0/pci@6/SUNW,fcdev@0]
    maxvfs = 8

Example 6-20  Dynamically Creating a Fibre Channel Virtual Function Without Setting Optional Properties

This example dynamically creates a virtual function without setting any optional properties. In this case, the ldm create-vf command automatically allocates the default bandwidth percentage, port world-wide name (WWN), and node WWN values.

Ensure that I/O virtualization is enabled on the pci_1 PCIe bus. See How to Enable I/O Virtualization for a PCIe Bus.

You can use the ldm create-vf command to create all the virtual functions from the /SYS/MB/PCIE7/IOVFC.PF0 physical function.

primary# ldm create-vf -n max /SYS/MB/PCIE7/IOVFC.PF0
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF0
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF1
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF2
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF3
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF4
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF5
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF6
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF7

Example 6-21  Dynamically Creating a Fibre Channel Virtual Function and Setting Properties

This example dynamically creates a virtual function while setting the bw-percent property value to 25 and specifies the port and node WWNs.

primary# ldm create-vf port-wwn=10:00:00:14:4F:FC:00:01 \
node-wwn=20:00:00:14:4F:FC:00:01 bw-percent=25 /SYS/MB/PCIE7/IOVFC.PF0

Example 6-22  Statically Creating a Fibre Channel Virtual Function Without Setting Optional Properties

This example statically creates a virtual function without setting any optional properties. In this case, the ldm create-vf command automatically allocates the default bandwidth percentage, port world-wide name (WWN), and node WWN values.

First you initiate a delayed reconfiguration on the rootdom1 domain. Then, enable I/O virtualization on the pci_1 PCIe bus. Because the pci_1 bus has already been assigned to the rootdom1 root domain, use the ldm set-io command to enable I/O virtualization.

primary# ldm start-reconf rootdom1
Initiating a delayed reconfiguration operation on the rootdom1 domain.
All configuration changes for other domains are disabled until the rootdom1
domain reboots, at which time the new configuration for the rootdom1 domain
will also take effect.

primary# ldm set-io iov=on pci_1

Now, you can use the ldm create-vf command to create all the virtual functions from the /SYS/MB/PCIE7/IOVFC.PF0 physical function.

primary# ldm create-vf -n max /SYS/MB/PCIE7/IOVFC.PF0

------------------------------------------------------------------------------
Notice: The rootdom1 domain is in the process of a delayed reconfiguration.
Any changes made to the rootdom1 domain will only take effect after it reboots.
------------------------------------------------------------------------------

Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF0
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF1
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF2
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF3
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF4
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF5
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF6
Created new vf: /SYS/MB/PCIE7/IOVFC.PF0.VF7

Finally, reboot the rootdom1 root domain to make the changes take effect in one of the following ways:

• rootdom1 is a non-primary root domain

  primary# ldm stop-domain -r rootdom1

• rootdom1 is the primary domain

  primary# shutdown -i6 -g0 -y

Destroying Fibre Channel SR-IOV Virtual Functions

A virtual function can be destroyed if it is not currently assigned to a domain. A virtual function can be destroyed only in the reverse sequential order of creation, so only the last virtual function that was created can be destroyed. The resulting configuration is validated by the physical function driver.

How to Destroy a Fibre Channel SR-IOV Virtual Function

If you cannot use this dynamic method, use the static method instead. See Static SR-IOV.

1. Identify the physical function device.

   primary# ldm list-io

2. Destroy a single virtual function or multiple virtual functions either dynamically or statically.
   • Dynamic method:
     • To destroy all of the virtual functions from a physical function at one time, use the following command:

       primary# ldm destroy-vf -n number | max pf-name

       You can use either the path name or the pseudonym name to specify virtual functions. However, the recommended practice is to use the pseudonym name.

       Use the ldm destroy-vf -n max command to destroy all the virtual functions for that physical function at one time.

       If you specify number as an argument to the –n option, the last number of virtual functions are destroyed. Use this method as it performs this operation with only one physical function device driver state transition.

     • To destroy a specified virtual function:

       primary# ldm destroy-vf vf-name

   • Static method:
     1. Initiate a delayed reconfiguration.

        primary# ldm start-reconf root-domain-name

     2. Destroy either a single virtual function or multiple virtual functions.
        • To destroy all of the virtual functions from the specified physical function at the same time, use the following command:

          primary# ldm destroy-vf -n number | max pf-name

          You can use either the path name or the pseudonym name to specify virtual functions. However, the recommended practice is to use the pseudonym name.

        • To destroy a specified virtual function:

          primary# ldm destroy-vf vf-name

     3. Reboot the root domain.
        • To reboot the non-primary root domain:

          primary# ldm stop-domain -r root-domain

        • To reboot the primary root domain:

          primary# shutdown -i6 -g0 -y

Example 6-23  Dynamically Destroying Multiple Fibre Channel SR-IOV Virtual Functions

This example shows the results of destroying all the virtual functions from the /SYS/MB/PCIE5/IOVFC.PF1 physical function. The ldm list-io output shows that the physical function has eight virtual functions. The ldm destroy-vf -n max command destroys all the virtual functions, and the final ldm list-io output shows that none of the virtual functions remain.

primary# ldm list-io
...
/SYS/MB/PCIE5/IOVFC.PF1                   PF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF0               VF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF1               VF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF2               VF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF3               VF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF4               VF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF5               VF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF6               VF     pci_1                      
/SYS/MB/PCIE5/IOVFC.PF1.VF7               VF     pci_1                      
primary# ldm destroy-vf -n max /SYS/MB/PCIE5/IOVFC.PF1
primary# ldm list-io
...
/SYS/MB/PCIE5/IOVFC.PF1                   PF     pci_1

Example 6-24  Destroying a Fibre Channel Virtual Function

This example shows how to statically destroy the virtual functions from the /SYS/MB/PCIE7/IOVFC.PF0 physical function.

primary# ldm start-reconf rootdom1
Initiating a delayed reconfiguration operation on the rootdom1 domain.
All configuration changes for other domains are disabled until the rootdom1
domain reboots, at which time the new configuration for the rootdom1 domain
will also take effect.

primary# ldm destroy-vf -n max /SYS/MB/PCIE7/IOVFC.PF0
primary# ldm stop-domain -r rootdom1

Modifying Fibre Channel SR-IOV Virtual Functions

The ldm set-io command modifies the current configuration of a virtual function by changing the property values or by setting new properties.

If you cannot use this dynamic method, use the static method instead. See Static SR-IOV.

You can use the ldm set-io command to modify the bw-percent, port-wwn, and node-wwn properties.

You can dynamically change only the bw-percent propery while the virtual functions are assigned to a domain.

How to Modify a Fibre Channel SR-IOV Virtual Function

1. Identify the physical function device.

   primary# ldm list-io

   Note that the name of the physical function includes the location information for the PCIe SR-IOV card or on-board device.

2. Modify a virtual function.

   primary# ldm set-io [bw-percent=value] [port-wwn=value node-wwn=value] pf-name

   Unlike the bw-percent property value, which you can dynamically change at any time, you can dynamically modify the port-wwn and node-wwn property values only when the virtual function is not assigned to a domain.

Example 6-25  Modifying a Fibre Channel SR-IOV Virtual Function

This example modifies the specified virtual function, /SYS/MB/PCIE7/IOVFC.PF0.VF0, to specify the bandwidth percentage and the port and node WWN values.

primary# ldm set-io port-wwn=10:00:00:14:4f:fb:f4:7c \
node-wwn=20:00:00:14:4f:fb:f4:7c bw-percent=25 /SYS/MB/PCIE7/IOVFC.PF0.VF0

Adding and Removing Fibre Channel SR-IOV Virtual Functions on I/O Domains

How to Add a Fibre Channel SR-IOV Virtual Function to an I/O Domain

If you cannot dynamically remove the virtual function, use the static method. See Static SR-IOV.

1. Identify the virtual function that you want to add to an I/O domain.

   primary# ldm list-io

2. Add a virtual function either dynamically or statically.
   • To dynamically add a virtual function:

     primary# ldm add-io vf-name domain-name

     vf-name is the pseudonym name or the path name of the virtual function. The recommended practice is to use the pseudonym name. domain-name specifies the name of the domain to which you add the virtual function.

     The device path name for the virtual function in the domain is the path shown in the list-io -l output.

   • To statically add a virtual function:
     1. Stop the domain and then add the virtual function.

        primary# ldm stop-domain domain-name
        primary# ldm add-io vf-name domain-name

        vf-name is the pseudonym name or the path name of the virtual function. The recommended practice is to use the pseudonym name. domain-name specifies the name of the domain to which you add the virtual function. The specified guest must be in the inactive or bound state.

        The device path name for the virtual function in the domain is the path shown in the list-io -l output.

     2. Restart the domain.

        primary# ldm start-domain domain-name

Example 6-26  Adding a Fibre Channel Virtual Function

This example shows how to dynamically add the /SYS/MB/PCIE7/IOVFC.PF0.VF0 virtual function to the ldg2 domain.

primary# ldm add-io /SYS/MB/PCIE7/IOVFC.PF0.VF0 ldg2

If you cannot add the virtual function dynamically, use the static method:

primary# ldm stop-domain ldg2
primary# ldm add-io /SYS/MB/PCIE7/IOVFC.PF0.VF0 ldg2
primary# ldm start-domain ldg2

How to Remove a Fibre Channel SR-IOV Virtual Function From an I/O Domain

If you cannot use this dynamic method, use the static method instead. See Static SR-IOV.

1. Identify the virtual function that you want to remove from an I/O domain.

   primary# ldm list-io

2. Remove a virtual function either dynamically or statically.
   • To dynamically remove a virtual function:

     primary# ldm rm-io vf-name domain-name

     vf-name is the pseudonym name or the path name of the virtual function. The recommended practice is to use the device pseudonym. domain-name specifies the name of the domain from which you remove the virtual function.

   • To statically remove a virtual function:
     1. Stop the I/O domain.

        primary# ldm stop-domain domain-name

     2. Remove the virtual function.

        primary# ldm rm-io vf-name domain-name

        vf-name is the pseudonym name or the path name of the virtual function. The recommended practice is to use the device pseudonym. domain-name specifies the name of the domain from which you remove the virtual function. The specified guest must be in the inactive or bound state.

     3. Start the I/O domain.

        primary# ldm start-domain domain-name

Example 6-27  Dynamically Removing a Fibre Channel Virtual Function

This example shows how to dynamically remove the /SYS/MB/PCIE7/IOVFC.PF0.VF0 virtual function from the ldg2 domain.

primary# ldm remove-io /SYS/MB/PCIE7/IOVFC.PF0.VF0 ldg2

If the command succeeds, the virtual function is removed from the ldg2 domain. When ldg2 is restarted, the specified virtual function no longer appears in that domain.

If you cannot remove the virtual function dynamically, use the static method:

primary# ldm stop-domain ldg2
primary# ldm remove-io /SYS/MB/PCIE7/IOVFC.PF0.VF0 ldg2
primary# ldm start-domain ldg2

Advanced SR-IOV Topics: Fibre Channel SR-IOV

This section describes some advanced topics related to using Fibre Channel SR-IOV virtual functions.

Accessing a Fibre Channel Virtual Function in a Guest Domain

The ldg2 console log shows the operations of the assigned Fibre Channel virtual function device. Use the fcadm command to view and access the Fibre Channel virtual function device.

ldg2# fcadm hba-port
HBA Port WWN: 100000144ffb8a99
       Port Mode: Initiator
       Port ID: 13d02
       OS Device Name: /dev/cfg/c3
       Manufacturer: Emulex
       Model: 7101684
       Firmware Version: 7101684 1.1.60.1
       FCode/BIOS Version: Boot:1.1.60.1 Fcode:4.03a4
       Serial Number: 4925382+133400002R
       Driver Name: emlxs
       Driver Version: 2.90.15.0 (2014.01.22.14.50)
       Type: N-port
       State: online
       Supported Speeds: 4Gb 8Gb 16Gb
       Current Speed: 16Gb
       Node WWN: 200000144ffb8a99
       NPIV Not Supported

The output shows that the Fibre Channel port is connected to a Fibre Channel switch. Use the format command to show the visible LUNs.

ldg2# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
   0. c2d0 <Unknown-Unknown-0001-25.00GB>
      /virtual-devices@100/channel-devices@200/disk@0
   1. c3t21000024FF4C4BF8d0 <SUN-COMSTAR-1.0-10.00GB>
      /pci@340/pci@1/pci@0/pci@6/SUNW,emlxs@0,2/fp@0,0/ssd@w21000024ff4c4bf8,0
Specify disk (enter its number): ^D
ldg2#