Before You Begin
Use only the PCIe cards that support the I/O domain resiliency feature. See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=REFERENCE&id=1325454.1.
Ensure that the I/O domain, root domain, service domain, and primary domain run at least the Oracle Solaris 11.2 SRU 8 OS and the Logical Domains Manager 3.2 software.
primary# ldm set-domain failure-policy=ignore root-domain-name
For information about domain dependencies, see Configuring Domain Dependencies.
primary# ldm set-domain master=root-domain-name I/O-domain-name
For information about using IPMP to configure multipathing, see Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle Solaris 11.3.
For information about using Oracle Solaris I/O multipathing to configure multipathing, see Managing SAN Devices and Multipathing in Oracle Solaris 11.3.
This example shows how to use IPMP to configure network virtual-function devices for a resilient I/O domain. For more information, see Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle Solaris 11.3.
Identify two Ethernet SR-IOV physical functions that are assigned to different root domains.
In this example, the root-1 and root-2 root domains have Ethernet SR-IOV physical functions.
primary# ldm list-io | grep root-1 | grep PF /SYS/PCI-EM8/IOVNET.PF0 PF pci_1 root-1 primary# ldm list-io | grep root-2 | grep PF /SYS/RIO/NET2/IOVNET.PF0 PF pci_2 root-2
Create two Ethernet virtual functions on each of the specified physical functions.
primary# ldm create-vf /SYS/MB/NET0/IOVNET.PF0 Created new vf: /SYS/PCI-EM8/IOVNET.PF0.VF0 primary# ldm create-vf /SYS/RIO/NET2/IOVNET.PF0 Created new vf: /SYS/RIO/NET2/IOVNET.PF0.VF0
Assign the Ethernet virtual functions to the io-1 I/O domain.
primary# ldm add-io /SYS/PCI-EM8/IOVNET.PF0.VF0 io-1 primary# ldm add-io /SYS/RIO/NET2/IOVNET.PF0.VF0 io-1
Configure the Ethernet virtual functions into an IPMP group on the I/O domain.
Identify the newly added network devices, net1 and net2, on the I/O domain.
i0-1# dladm show-phys LINK MEDIA STATE SPPED DUPLEX DEVICE net0 Ethernet up 0 unknown vnet0 net1 Ethernet up 1000 full igbvf0 net2 Ethernet up 1000 full igbvf1
Create IP interfaces for the newly added network devices.
io-1# ipadm create-ip net1 io-1# ipadm create-ip net2
Create the ipmp0 IPMP group for the two network interfaces.
io-1# ipadm create-ipmp -i net1 -i net2 ipmp0
Assign an IP address to the IPMP group.
This example configures the DHCP option.
io-1# ipadm create-addr -T dhcp ipmp0/v4
Check the status of the IPMP group interface.
io-1# ipmpstat -g
This example shows how to use Oracle Solaris I/O multipathing to configure Fibre Channel virtual-function devices for a resilient I/O domain. For more information, see Managing SAN Devices and Multipathing in Oracle Solaris 11.3.
Identify two Fibre Channel SR-IOV physical functions that are assigned to different root domains.
In this example, the root-1 and root-2 root domains have Fibre Channel SR-IOV physical functions.
primary# ldm list-io | grep root-1 | grep PF /SYS/PCI-EM4/IOVFC.PF0 PF pci_1 root-1 primary# ldm list-io | grep root-2 | grep PF /SYS/PCI-EM15/IOVFC.PF0 PF pci_2 root-2
Create two virtual functions on each of the specified physical functions.
For more information, see How to Create a Fibre Channel SR-IOV Virtual Function.
primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:60:00 \ node-wwn=20:00:00:14:4f:fc:60:00 /SYS/PCI-EM4/IOVFC.PF0 Created new vf: /SYS/PCI-EM4/IOVFC.PF0.VF0 primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:70:00 \ node-wwn=20:00:00:14:4f:fc:70:00 /SYS/PCI-EM15/IOVFC.PF0 Created new vf: /SYS/PCI-EM15/IOVFC.PF0.VF0
Add the newly created virtual functions to the io-1 I/O domain.
primary# ldm add-io /SYS/PCI-EM4/IOVFC.PF0.VF0 io-1 primary# ldm add-io /SYS/PCI-EM15/IOVFC.PF0.VF0 io-1
Determine whether Oracle Solaris I/O multipathing is enabled on the I/O domain by using the prtconf -v command.
If the output for the fp device includes the following device property setting, Oracle Solaris I/O multipathing is enabled:
mpxio-disable="no"
If the mpxio-disable property is set to yes, update the property value to no in the /etc/driver/drv/fp.conf file and then reboot the I/O domain.
If the mpxio-disable device property does not appear in the prtconf -v output, add the mpxio-disable="no" entry to the /etc/driver/drv/fp.conf file and then reboot the I/O domain.
Check the status of Oracle Solaris I/O multipathing group.
io-1# mpathadm show LU
Logical Unit: /dev/rdsk/c0t600A0B80002A384600003D6B544EECD0d0s2
mpath-support: libmpscsi_vhci.so
Vendor: SUN
Product: CSM200_R
Revision: 0660
Name Type: unknown type
Name: 600a0b80002a384600003d6b544eecd0
Asymmetric: yes
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA
Paths:
Initiator Port Name: 100000144ffc6000
Target Port Name: 201700a0b82a3846
Override Path: NA
Path State: OK
Disabled: no
Initiator Port Name: 100000144ffc7000
Target Port Name: 201700a0b82a3846
Override Path: NA
Path State: OK
Disabled: no
Target Port Groups:
ID: 1
Explicit Failover: yes
Access State: active
Target Ports:
Name: 201700a0b82a3846
Relative ID: 0