How to Add an InfiniBand Virtual Function to an I/O Domain

This procedure describes how to add an InfiniBand SR-IOV virtual function to an I/O domain.

  1. Stop the I/O domain.
    primary# ldm stop-domain domain-name
  2. Add one or more virtual functions to the I/O domain.

    vf-name is the pseudonym name or the path name of the virtual function. The recommended practice is to use the pseudonym name. domain-name specifies the name of the domain to which you add the virtual function. The specified I/O domain must be in the inactive or bound state.

    primary# ldm add-io vf-name domain-name
  3. Start the I/O domain.
    primary# ldm start-domain domain-name

Example 8-16 Adding an InfiniBand Virtual Function

The following example shows how to add the /SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF2 virtual function to the iodom1 I/O domain.

First, identify the virtual function that you want to assign.

primary# ldm list-io
NAME                                      TYPE   BUS      DOMAIN  STATUS
----                                      ----   ---      ------  ------
/SYS/PM0/CMP0/PEX                         BUS    pci_0    primary IOV
...
/SYS/MB/RISER1/PCIE4/IOVIB.PF0            PF     pci_0    primary
...
/SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF0        VF     pci_0
/SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF1        VF     pci_0
/SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF2        VF     pci_0
/SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF3        VF     pci_0

To add a virtual function to an I/O domain, it must be unassigned. The DOMAIN column indicates the name of the domain to which the virtual function is assigned. In this case, the /SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF2 is not assigned to a domain.

To add a virtual function to a domain, the domain must be in the inactive or bound state.

primary# ldm list-domain
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    32    64G      0.2%  0.2%  56m
iodom1           active     -n----  5000    8     8G        33%   33%  25m

The ldm list-domain output shows that the iodom1 I/O domain is active, so it must be stopped.

primary# ldm stop iodom1
LDom iodom1 stopped
primary# ldm list-domain
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    32    64G      0.0%  0.0%  57m
iodom1           bound      ------  5000    8     8G

Now you can add the virtual function to the I/O domain.

primary# ldm add-io /SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF2 iodom1
primary# ldm list-io
...
/SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF2        VF     pci_0    iodom1

Note that you can add more than one virtual function while an I/O domain is stopped. For example, you might add other unassigned virtual functions such as /SYS/MB/RISER1/PCIE4/IOVIB.PF0.VF3 to iodom1. After you add the virtual functions, you can restart the I/O domain.

primary# ldm start-domain iodom1
LDom iodom1 started
primary# ldm list-domain
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    32    64G      1.0%  1.0%  1h 18m
iodom1           active     -n----  5000    8     8G        36%   36%  1m