The following procedure explains how to create an I/O domain that includes PCIe SR-IOV virtual functions.
Plan ahead to minimize the number of reboots of the root domain, which minimizes downtime.
Before You Begin
Before you begin, ensure that you have enabled I/O virtualization for the PCIe bus that is the parent of the physical function from which you create virtual functions. See How to Enable I/O Virtualization for a PCIe Bus.
primary# ldm list-io
primary# ldm create-vf pf-name
You can run this command for each virtual function that you want to create. You can also use the –n option to create more than one virtual function from the same physical function in a single command. See Example 10, Creating Multiple SR-IOV Ethernet Virtual Functions and the ldm(1M) man page.
primary# ldm list-io
primary# ldm add-io vf-name domain-name
The following Oracle Solaris 11 command shows the availability of the virtual function:
guest# dladm show-phys
The following dynamic example shows how to create a virtual function, /SYS/MB/NET0/IOVNET.PF0.VF0, for a physical function, /SYS/MB/NET0/IOVNET.PF0, and assign the virtual function to the ldg1 I/O domain.
This example assumes that the following circumstances are true:
The OS on the primary domain supports dynamic SR-IOV operations
The pci_0 bus is assigned to the primary domain and has been initialized for I/O virtualization operations
The /SYS/MB/NET0/IOVNET.PF0 physical function belongs to the pci_0 bus
The /SYS/MB/NET0/IOVNET.PF0 physical function does not have any existing virtual functions assigned to domains
The ldg1 domain is active and booted and its OS supports dynamic SR-IOV operations
Create the virtual function from the /SYS/MB/NET0/IOVNET.PF0 physical function.
primary# ldm create-vf /SYS/MB/NET0/IOVNET.PF0 Created new vf: /SYS/MB/NET0/IOVNET.PF0.VF0
Add the /SYS/MB/NET0/IOVNET.PF0.VF0 virtual function to the ldg1 domain.
primary# ldm add-io /SYS/MB/NET0/IOVNET.PF0.VF0 ldg1
The following command shows that the virtual function has been added to the ldg1 domain.
primary# ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ /SYS/PM0/CMP0/NIU_CORE NIU niu_0 primary /SYS/PM0/CMP1/NIU_CORE NIU niu_1 primary /SYS/PM0/CMP0/PEX BUS pci_0 primary IOV /SYS/PM0/CMP1/PEX BUS pci_1 /SYS/MB/PCIE0 PCIE pci_0 primary OCC /SYS/MB/PCIE2 PCIE pci_0 primary OCC /SYS/MB/PCIE4 PCIE pci_0 primary OCC /SYS/MB/PCIE6 PCIE pci_0 primary EMP /SYS/MB/PCIE8 PCIE pci_0 primary EMP /SYS/MB/SASHBA PCIE pci_0 primary OCC /SYS/MB/NET0 PCIE pci_0 primary OCC /SYS/MB/PCIE1 PCIE pci_1 primary OCC /SYS/MB/PCIE3 PCIE pci_1 primary OCC /SYS/MB/PCIE5 PCIE pci_1 primary OCC /SYS/MB/PCIE7 PCIE pci_1 primary EMP /SYS/MB/PCIE9 PCIE pci_1 primary EMP /SYS/MB/NET2 PCIE pci_1 primary OCC /SYS/MB/NET0/IOVNET.PF0 PF pci_0 primary /SYS/MB/NET0/IOVNET.PF1 PF pci_0 primary /SYS/MB/PCIE5/IOVNET.PF0 PF pci_1 primary /SYS/MB/PCIE5/IOVNET.PF1 PF pci_1 primary /SYS/MB/NET2/IOVNET.PF0 PF pci_1 primary /SYS/MB/NET2/IOVNET.PF1 PF pci_1 primary /SYS/MB/NET0/IOVNET.PF0.VF0 VF pci_0 ldg1Example 17 Statically Creating an I/O Domain by Assigning an SR-IOV Virtual Function to It
The following static example shows how to create a virtual function, /SYS/MB/NET0/IOVNET.PF0.VF0, for a physical function, /SYS/MB/NET0/IOVNET.PF0, and assign the virtual function to the ldg1 I/O domain.
This example assumes that the following circumstances are true:
The OS on the primary domain does not support dynamic SR-IOV operations
The pci_0 bus is assigned to the primary domain and has not been initialized for I/O virtualization operations
The /SYS/MB/NET0/IOVNET.PF0 physical function belongs to the pci_0 bus
The /SYS/MB/NET0/IOVNET.PF0 physical function does not have any existing virtual functions assigned to domains
The ldg1 domain is active and booted and its OS does not support dynamic SR-IOV operations
The ldg1 domain has the auto-boot? property set to true so that the domain boots automatically when the domain is started
First, initiate a delayed reconfiguration on the primary domain, enable I/O virtualization, and create the virtual function from the /SYS/MB/NET0/IOVNET.PF0 physical function.
primary# ldm start-reconf primary Initiating a delayed reconfiguration operation on the primary domain. All configuration changes for other domains are disabled until the primary domain reboots, at which time the new configuration for the primary domain will also take effect. primary# ldm set-io iov=on pci_0 primary# ldm create-vf /SYS/MB/NET0/IOVNET.PF0 ------------------------------------------------------------------------------ Notice: The primary domain is in the process of a delayed reconfiguration. Any changes made to the primary domain will only take effect after it reboots. ------------------------------------------------------------------------------ Created new vf: /SYS/MB/NET0/IOVNET.PF0.VF0
Next, shut down the primary domain.
primary# shutdown -i6 -g0 -y
Stop the ldg1 domain, add the virtual function, and start the domain.
primary# ldm stop ldg1 primary# ldm add-io /SYS/MB/NET0/IOVNET.PF0.VF0 ldg1 primary# ldm start-domain ldg1
The following command shows that the virtual function has been added to the ldg1 domain.
primary# ldm list-io NAME TYPE BUS DOMAIN STATUS ---- ---- --- ------ ------ /SYS/PM0/CMP0/NIU_CORE NIU niu_0 primary /SYS/PM0/CMP1/NIU_CORE NIU niu_1 primary /SYS/PM0/CMP0/PEX BUS pci_0 primary IOV /SYS/PM0/CMP1/PEX BUS pci_1 /SYS/MB/PCIE0 PCIE pci_0 primary OCC /SYS/MB/PCIE2 PCIE pci_0 primary OCC /SYS/MB/PCIE4 PCIE pci_0 primary OCC /SYS/MB/PCIE6 PCIE pci_0 primary EMP /SYS/MB/PCIE8 PCIE pci_0 primary EMP /SYS/MB/SASHBA PCIE pci_0 primary OCC /SYS/MB/NET0 PCIE pci_0 primary OCC /SYS/MB/PCIE1 PCIE pci_1 primary OCC /SYS/MB/PCIE3 PCIE pci_1 primary OCC /SYS/MB/PCIE5 PCIE pci_1 primary OCC /SYS/MB/PCIE7 PCIE pci_1 primary EMP /SYS/MB/PCIE9 PCIE pci_1 primary EMP /SYS/MB/NET2 PCIE pci_1 primary OCC /SYS/MB/NET0/IOVNET.PF0 PF pci_0 primary /SYS/MB/NET0/IOVNET.PF1 PF pci_0 primary /SYS/MB/PCIE5/IOVNET.PF0 PF pci_1 primary /SYS/MB/PCIE5/IOVNET.PF1 PF pci_1 primary /SYS/MB/NET2/IOVNET.PF0 PF pci_1 primary /SYS/MB/NET2/IOVNET.PF1 PF pci_1 primary /SYS/MB/NET0/IOVNET.PF0.VF0 VF pci_0 ldg1