Before You Begin
Use only the PCIe cards that support the I/O domain resiliency feature. See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=REFERENCE&id=1325454.1.
Ensure that the I/O domain, root domain, service domain, and primary domain run at least the Oracle Solaris 11.2 SRU 8 OS and the Logical Domains Manager 3.2 software.
primary# ldm set-domain failure-policy=ignore root-domain-name
For information about domain dependencies, see Configuring Domain Dependencies.
primary# ldm set-domain master=root-domain-name I/O-domain-name
For information about using IPMP to configure multipathing, see Administering TCP/IP Networks, IPMI, and IP Tunnels in Oracle Solaris 11.2.
For information about using MPxIO to configure multipathing, see Managing SAN Devices and Multipathing Oracle Solaris 11.2.
This example shows how to use IPMP to configure network virtual-function devices for a resilient I/O domain. For more information, see Administering TCP/IP Networks, IPMI, and IP Tunnels in Oracle Solaris 11.2.
Identify two Ethernet SR-IOV physical functions that are assigned to different root domains.
In this example, the root-1 and root-2 root domains have Ethernet SR-IOV physical functions.
primary# ldm list-io | grep root-1 | grep PF /SYS/PCI-EM8/IOVNET.PF0 PF pci_1 root-1 primary# ldm list-io | grep root-2 | grep PF /SYS/RIO/NET2/IOVNET.PF0 PF pci_2 root-2
Create two Ethernet virtual functions on each of the specified physical functions.
primary# ldm create-vf /SYS/MB/NET0/IOVNET.PF0 Created new vf: /SYS/PCI-EM8/IOVNET.PF0.VF0 primary# ldm create-vf /SYS/RIO/NET2/IOVNET.PF0 Created new vf: /SYS/RIO/NET2/IOVNET.PF0.VF0
Assign the Ethernet virtual functions to the io-1 I/O domain.
primary# ldm add-io /SYS/PCI-EM8/IOVNET.PF0.VF0 io-1 primary# ldm add-io /SYS/RIO/NET2/IOVNET.PF0.VF0 io-1
Configure the Ethernet virtual functions into an IPMP group on the I/O domain.
Identify the newly added network devices, net1 and net2, on the I/O domain.
i0-1# dladm show-phys LINK MEDIA STATE SPPED DUPLEX DEVICE net0 Ethernet up 0 unknown vnet0 net1 Ethernet up 1000 full igbvf0 net2 Ethernet up 1000 full igbvf1
Create IP interfaces for the newly added network devices.
io-1# ipadm create-ip net1 io-1# ipadm create-ip net2
Create the ipmp0 IPMP group for the two network interfaces.
io-1# ipadm create-ipmp -i net1 -i net2 ipmp0
Assign an IP address to the IPMP group.
This example configures the DHCP option.
io-1# ipadm create-addr -T dhcp ipmp0/v4
Check the status of the IPMP group interface.
io-1# ipmpstat -g
This example shows how to use MPxIO to configure Fibre Channel virtual-function devices for a resilient I/O domain. For more information, see Managing SAN Devices and Multipathing Oracle Solaris 11.2.
Identify two Fibre Channel SR-IOV physical functions that are assigned to different root domains.
In this example, the root-1 and root-2 root domains have Fibre Channel SR-IOV physical functions.
primary# ldm list-io | grep root-1 | grep PF /SYS/PCI-EM4/IOVFC.PF0 PF pci_1 root-1 primary# ldm list-io | grep root-2 | grep PF /SYS/PCI-EM15/IOVFC.PF0 PF pci_2 root-2
Create two virtual functions on each of the specified physical functions.
For more information, see How to Create a Fibre Channel SR-IOV Virtual Function.
primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:60:00 \ node-wwn=20:00:00:14:4f:fc:60:00 /SYS/PCI-EM4/IOVFC.PF0 Created new vf: /SYS/PCI-EM4/IOVFC.PF0.VF0 primary# ldm create-vf port-wwn=10:00:00:14:4f:fc:70:00 \ node-wwn=20:00:00:14:4f:fc:70:00 /SYS/PCI-EM15/IOVFC.PF0 Created new vf: /SYS/PCI-EM15/IOVFC.PF0.VF0
Add the newly created virtual functions to the io-1 I/O domain.
primary# ldm add-io /SYS/PCI-EM4/IOVFC.PF0.VF0 io-1 primary# ldm add-io /SYS/PCI-EM15/IOVFC.PF0.VF0 io-1
Determine whether MPxIO is enabled on the I/O domain by using the prtconf -v command.
If the output for the fp device includes the following device property setting, MPxIO is enabled:
mpxio-disable="no"
If the mpxio-disable property is set to yes, update the property value to no in the /etc/driver/drv/fp.conf file and then reboot the I/O domain.
If the mpxio-disable device property does not appear in the prtconf -v output, add the mpxio-disable="no" entry to the /etc/driver/drv/fp.conf file and then reboot the I/O domain.
Check the status of MPxIO group.
io-1# mpathadm show LU Logical Unit: /dev/rdsk/c0t600A0B80002A384600003D6B544EECD0d0s2 mpath-support: libmpscsi_vhci.so Vendor: SUN Product: CSM200_R Revision: 0660 Name Type: unknown type Name: 600a0b80002a384600003d6b544eecd0 Asymmetric: yes Current Load Balance: round-robin Logical Unit Group ID: NA Auto Failback: on Auto Probing: NA Paths: Initiator Port Name: 100000144ffc6000 Target Port Name: 201700a0b82a3846 Override Path: NA Path State: OK Disabled: no Initiator Port Name: 100000144ffc7000 Target Port Name: 201700a0b82a3846 Override Path: NA Path State: OK Disabled: no Target Port Groups: ID: 1 Explicit Failover: yes Access State: active Target Ports: Name: 201700a0b82a3846 Relative ID: 0