Go to main content

Oracle® VM Server for SPARC 3.6 Administration Guide

Exit Print View

Updated: September 2019
 
 

SR-IOV Overview


Note - Because root domains cannot have dependencies on other root domains, a root domain that owns a PCIe bus cannot have its PCIe endpoint devices or SR-IOV virtual functions assigned to another root domain. However, you can assign a PCIe endpoint device or virtual function from a PCIe bus to the root domain that owns that bus.

The Peripheral Component Interconnect Express (PCIe) single root I/O virtualization (SR-IOV) implementation is based on version 1.1 of the standard as defined by the PCI-SIG. The SR-IOV standard enables the efficient sharing of PCIe devices among virtual machines and is implemented in the hardware to achieve I/O performance that is comparable to native performance. The SR-IOV specification defines a new standard wherein new devices that are created enable the virtual machine to be directly connected to the I/O device.

A single I/O resource, which is known as a physical function, can be shared by many virtual machines. The shared devices provide dedicated resources and also use shared common resources. In this way, each virtual machine has access to unique resources. Therefore, a PCIe device, such as an Ethernet port, that is SR-IOV-enabled with appropriate hardware and OS support can appear as multiple, separate physical devices, each with its own PCIe configuration space.

For more information about SR-IOV, see the PCI-SIG web site (http://www.pcisig.com/).

The following figure shows the relationship between virtual functions and a physical function in an I/O domain.

Figure 3  Using Virtual Functions and a Physical Function in an I/O Domain

image:Shows how to use virtual functions and physical functions in an I/O domain.

    SR-IOV has the following function types:

  • Physical function – A PCI function that supports the SR-IOV capabilities as defined by the SR-IOV specification. A physical function contains the SR-IOV capability structure and manages the SR-IOV functionality. Physical functions are fully featured PCIe functions that can be discovered, managed, and manipulated like any other PCIe device. Physical functions can be used to configure and control a PCIe device.

  • Virtual function – A PCI function that is associated with a physical function. A virtual function is a lightweight PCIe function that shares one or more physical resources with the physical function and with virtual functions that are associated with that physical function. Unlike a physical function, a virtual function can only configure its own behavior.

Each SR-IOV device can have a physical function and each physical function can have up to 256 virtual functions associated with it. This number is dependent on the particular SR-IOV device. The virtual functions are created by the physical function.

After SR-IOV is enabled in the physical function, the PCI configuration space of each virtual function can be accessed by the bus, device, and function number of the physical function. Each virtual function has a PCI memory space, which is used to map its register set. The virtual function device drivers operate on the register set to enable its functionality and the virtual function appears as an actual PCI device. After creation, you can directly assign a virtual function to an I/O domain. This capability enables the virtual function to share the physical device and to perform I/O without CPU and hypervisor software overhead.

    You might want to use the SR-IOV feature in your environment to reap the following benefits:

  • Higher performance and reduced latency – Direct access to hardware from a virtual machines environment

  • Cost reduction – Capital and operational expenditure savings, which include:

    • Power savings

    • Reduced adapter count

    • Less cabling

    • Fewer switch ports

The Oracle VM Server for SPARC SR-IOV implementation includes both static and dynamic configuration methods. For more information, see Static SR-IOV and Dynamic SR-IOV.

    The Oracle VM Server for SPARC SR-IOV feature enables you to perform the following operations:

  • Creating a virtual function on a specified physical function

  • Destroying a specified virtual function on a physical function

  • Assigning a virtual function to a domain

  • Removing a virtual function from a domain

To create and destroy virtual functions in the SR-IOV physical function devices, you must first enable I/O virtualization on that PCIe bus. You can use the ldm set-io or ldm add-io command to set the iov property to on. You can also use the ldm add-domain or ldm set-domain command to set the rc-add-policy property to iov. See the ldm(8) man page.


Note - By default, PCIe buses are enabled for I/O virtualization on systems starting with the SPARC M7, SPARC T7, and SPARC S7 series server and the Fujitsu M10 server.

Assigning a SR-IOV virtual function to a domain creates an implicit dependency on the domain providing the SR-IOV physical function service. You can view these dependencies or view domains that depend on this SR-IOV physical function by using the ldm list-dependencies command. See Listing Domain I/O Dependencies.