Go to main content

Oracle® VM Server for SPARC 3.6 Administration Guide

Exit Print View

Updated: September 2019
 
 

Configuring Resilient I/O Domains

How to Configure a Resilient I/O Domain

Before You Begin

Use only the PCIe cards that support the I/O domain resiliency feature. See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=REFERENCE&id=1325454.1.

Ensure that the I/O domain, root domain, service domain, and primary domain are all running at least the Oracle Solaris 11.4 SRU 13 OS.

  1. On the root domain, set the failure-policy property to ignore.
    primary# ldm set-domain failure-policy=ignore root-domain-name

    Note - If you add any devices to the I/O domain that are not supported for resiliency, that domain is no longer resilient. So, reset the failure-policy property value to stop, reset, or panic.

    For information about domain dependencies, see Configuring Domain Dependencies.

  2. On the I/O domain, set the master property to the name of the root domain.
    primary# ldm set-domain master=root-domain-name I/O-domain-name
  3. Configure multipathing across the paths.
Example 31  Using IPMP to Configure Multipathing With Ethernet SR-IOV Functions

This example shows how to use IPMP to configure network virtual-function devices for a resilient I/O domain. For more information, see Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle Solaris 11.4.

  1. Identify two Ethernet SR-IOV physical functions that are assigned to different root domains.

    In this example, the root-1 and root-2 root domains have Ethernet SR-IOV physical functions.

    primary# ldm list-io | grep root-1 | grep PF
    /SYS/PCI-EM8/IOVNET.PF0                    PF     pci_1    root-1
    primary# ldm list-io | grep root-2 | grep PF
    /SYS/RIO/NET2/IOVNET.PF0                   PF     pci_2    root-2
  2. Create two Ethernet virtual functions on each of the specified physical functions.

    primary# ldm create-vf /SYS/MB/NET0/IOVNET.PF0
    Created new vf: /SYS/PCI-EM8/IOVNET.PF0.VF0
    primary# ldm create-vf /SYS/RIO/NET2/IOVNET.PF0
    Created new vf: /SYS/RIO/NET2/IOVNET.PF0.VF0
  3. Assign the Ethernet virtual functions to the io-1 I/O domain.

    primary# ldm add-io /SYS/PCI-EM8/IOVNET.PF0.VF0 io-1
    primary# ldm add-io /SYS/RIO/NET2/IOVNET.PF0.VF0 io-1
  4. Configure the Ethernet virtual functions into an IPMP group on the I/O domain.

    1. Identify the newly added network devices, net1 and net2, on the I/O domain.

      i0-1# dladm show-phys
      LINK        MEDIA        STATE    SPPED    DUPLEX    DEVICE
      net0        Ethernet     up       1000     full      vnet0
      net1        Ethernet     up       1000     full      igbvf0
      net2        Ethernet     up       1000     full      igbvf1
    2. Create IP interfaces for the newly added network devices.

      io-1# ipadm create-ip net1
      io-1# ipadm create-ip net2
    3. Create the ipmp0 IPMP group for the two network interfaces.

      io-1# ipadm create-ipmp -i net1 -i net2 ipmp0
    4. Assign an IP address to the IPMP group.

      This example configures the DHCP option.

      io-1# ipadm create-addr -T dhcp ipmp0/v4
    5. Check the status of the IPMP group interface.

      io-1# ipmpstat -g
Example 32  Using Oracle Solaris I/O Multipathing to Configure Multipathing With Fibre Channel SR-IOV Functions

This example shows how to use Oracle Solaris I/O multipathing to configure Fibre Channel virtual-function devices for a resilient I/O domain. For more information, see Managing SAN Devices and I/O Multipathing in Oracle Solaris 11.4.

  1. Identify two Fibre Channel SR-IOV physical functions that are assigned to different root domains.

    In this example, the root-1 and root-2 root domains have Fibre Channel SR-IOV physical functions.

    primary# ldm list-io | grep root-1 | grep PF
    /SYS/PCI-EM4/IOVFC.PF0                    PF     pci_1    root-1
    primary# ldm list-io | grep root-2 | grep PF
    /SYS/PCI-EM15/IOVFC.PF0                   PF     pci_2    root-2
  2. Create two virtual functions on each of the specified physical functions.

    For more information, see How to Create a Fibre Channel SR-IOV Virtual Function.

    primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:60:00 \
    node-wwn=20:00:00:14:4f:fc:60:00 /SYS/PCI-EM4/IOVFC.PF0
    Created new vf: /SYS/PCI-EM4/IOVFC.PF0.VF0
    primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:70:00 \
    node-wwn=20:00:00:14:4f:fc:70:00 /SYS/PCI-EM15/IOVFC.PF0
    Created new vf: /SYS/PCI-EM15/IOVFC.PF0.VF0
  3. Add the newly created virtual functions to the io-1 I/O domain.

    primary# ldm add-io /SYS/PCI-EM4/IOVFC.PF0.VF0 io-1
    primary# ldm add-io /SYS/PCI-EM15/IOVFC.PF0.VF0 io-1
  4. Determine whether Oracle Solaris I/O multipathing is enabled on the I/O domain by using the prtconf -v command.

    If the output for the fp device includes the following device property setting, Oracle Solaris I/O multipathing is enabled:

    mpxio-disable="no"

    If the mpxio-disable property is set to yes, update the property value to no in the /etc/driver/drv/fp.conf file and then reboot the I/O domain.

    If the mpxio-disable device property does not appear in the prtconf -v output, add the mpxio-disable="no" entry to the /etc/driver/drv/fp.conf file and then reboot the I/O domain.

  5. Check the status of Oracle Solaris I/O multipathing group.

    io-1# mpathadm show LU
    
    Logical Unit:  /dev/rdsk/c0t600A0B80002A384600003D6B544EECD0d0s2
         mpath-support:  libmpscsi_vhci.so
         Vendor:  SUN
         Product:  CSM200_R
         Revision:  0660
         Name Type:  unknown type
         Name:  600a0b80002a384600003d6b544eecd0
         Asymmetric:  yes
         Current Load Balance:  round-robin
         Logical Unit Group ID:  NA
         Auto Failback:  on
         Auto Probing:  NA
    
         Paths:
                 Initiator Port Name:  100000144ffc6000
                 Target Port Name:  201700a0b82a3846
                 Override Path:  NA
                 Path State:  OK
                 Disabled:  no
    
                 Initiator Port Name:  100000144ffc7000
                 Target Port Name:  201700a0b82a3846
                 Override Path:  NA
                 Path State:  OK
                 Disabled:  no
    
         Target Port Groups:
                 ID:  1
                 Explicit Failover:  yes
                 Access State:  active
                 Target Ports:
                         Name:  201700a0b82a3846
                         Relative ID:  0