Before You Begin
Use only the PCIe cards that support the I/O domain resiliency feature. See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=REFERENCE&id=1325454.1.
Ensure that the I/O domain, root domain, service domain, and primary domain run at least the Oracle Solaris 11.2 SRU 8 OS and the Logical Domains Manager 3.2 software.
primary# ldm set-domain failure-policy=ignore root-domain-name
For information about domain dependencies, see Configuring Domain Dependencies.
primary# ldm set-domain master=root-domain-name I/O-domain-name
For information about using IPMP to configure multipathing, see Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle Solaris 11.3.
For information about using Oracle Solaris I/O multipathing to configure multipathing, see Managing SAN Devices and Multipathing in Oracle Solaris 11.3.
This example shows how to use IPMP to configure network virtual-function devices for a resilient I/O domain. For more information, see Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle Solaris 11.3.
Identify two Ethernet SR-IOV physical functions that are assigned to different root domains.
In this example, the root-1 and root-2 root domains have Ethernet SR-IOV physical functions.
primary# ldm list-io | grep root-1 | grep PF /SYS/PCI-EM8/IOVNET.PF0 PF pci_1 root-1 primary# ldm list-io | grep root-2 | grep PF /SYS/RIO/NET2/IOVNET.PF0 PF pci_2 root-2
Create two Ethernet virtual functions on each of the specified physical functions.
primary# ldm create-vf /SYS/MB/NET0/IOVNET.PF0 Created new vf: /SYS/PCI-EM8/IOVNET.PF0.VF0 primary# ldm create-vf /SYS/RIO/NET2/IOVNET.PF0 Created new vf: /SYS/RIO/NET2/IOVNET.PF0.VF0
Assign the Ethernet virtual functions to the io-1 I/O domain.
primary# ldm add-io /SYS/PCI-EM8/IOVNET.PF0.VF0 io-1 primary# ldm add-io /SYS/RIO/NET2/IOVNET.PF0.VF0 io-1
Configure the Ethernet virtual functions into an IPMP group on the I/O domain.
Identify the newly added network devices, net1 and net2, on the I/O domain.
i0-1# dladm show-phys LINK MEDIA STATE SPPED DUPLEX DEVICE net0 Ethernet up 1000 full vnet0 net1 Ethernet up 1000 full igbvf0 net2 Ethernet up 1000 full igbvf1
Create IP interfaces for the newly added network devices.
io-1# ipadm create-ip net1 io-1# ipadm create-ip net2
Create the ipmp0 IPMP group for the two network interfaces.
io-1# ipadm create-ipmp -i net1 -i net2 ipmp0
Assign an IP address to the IPMP group.
This example configures the DHCP option.
io-1# ipadm create-addr -T dhcp ipmp0/v4
Check the status of the IPMP group interface.
io-1# ipmpstat -g
This example shows how to use Oracle Solaris I/O multipathing to configure Fibre Channel virtual-function devices for a resilient I/O domain. For more information, see Managing SAN Devices and Multipathing in Oracle Solaris 11.3.
Identify two Fibre Channel SR-IOV physical functions that are assigned to different root domains.
In this example, the root-1 and root-2 root domains have Fibre Channel SR-IOV physical functions.
primary# ldm list-io | grep root-1 | grep PF /SYS/PCI-EM4/IOVFC.PF0 PF pci_1 root-1 primary# ldm list-io | grep root-2 | grep PF /SYS/PCI-EM15/IOVFC.PF0 PF pci_2 root-2
Create two virtual functions on each of the specified physical functions.
For more information, see How to Create a Fibre Channel SR-IOV Virtual Function.
primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:60:00 \ node-wwn=20:00:00:14:4f:fc:60:00 /SYS/PCI-EM4/IOVFC.PF0 Created new vf: /SYS/PCI-EM4/IOVFC.PF0.VF0 primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:70:00 \ node-wwn=20:00:00:14:4f:fc:70:00 /SYS/PCI-EM15/IOVFC.PF0 Created new vf: /SYS/PCI-EM15/IOVFC.PF0.VF0
Add the newly created virtual functions to the io-1 I/O domain.
primary# ldm add-io /SYS/PCI-EM4/IOVFC.PF0.VF0 io-1 primary# ldm add-io /SYS/PCI-EM15/IOVFC.PF0.VF0 io-1
Determine whether Oracle Solaris I/O multipathing is enabled on the I/O domain by using the prtconf -v command.
If the output for the fp device includes the following device property setting, Oracle Solaris I/O multipathing is enabled:
mpxio-disable="no"
If the mpxio-disable property is set to yes, update the property value to no in the /etc/driver/drv/fp.conf file and then reboot the I/O domain.
If the mpxio-disable device property does not appear in the prtconf -v output, add the mpxio-disable="no" entry to the /etc/driver/drv/fp.conf file and then reboot the I/O domain.
Check the status of Oracle Solaris I/O multipathing group.
io-1# mpathadm show LU
Logical Unit: /dev/rdsk/c0t600A0B80002A384600003D6B544EECD0d0s2
mpath-support: libmpscsi_vhci.so
Vendor: SUN
Product: CSM200_R
Revision: 0660
Name Type: unknown type
Name: 600a0b80002a384600003d6b544eecd0
Asymmetric: yes
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA
Paths:
Initiator Port Name: 100000144ffc6000
Target Port Name: 201700a0b82a3846
Override Path: NA
Path State: OK
Disabled: no
Initiator Port Name: 100000144ffc7000
Target Port Name: 201700a0b82a3846
Override Path: NA
Path State: OK
Disabled: no
Target Port Groups:
ID: 1
Explicit Failover: yes
Access State: active
Target Ports:
Name: 201700a0b82a3846
Relative ID: 0