Writing Device Drivers for Oracle^® Solaris 11.3

Overview of SR-IOV Device Driver

The SR-IOV functionality is comprised of the Physical Function (PF) driver and the Virtual Function (VF) driver. The following sections describe the PF and VF drivers and the details of the required device configuration.

Physical Function (PF) Driver

The PF driver of a SR-IOV device is used to manage the Physical Function (PF) of an SR-IOV capable device. A PCI Function that supports the SR-IOV capabilities is defined in the SR-IOV specification. A PF contains the SR-IOV capability structure and is used to manage the SR-IOV functionality. PFs are fully featured PCIe functions which can be discovered, managed, and manipulated like any other PCIe device. PFs have full configuration resources, and can be used to configure or control the PCIe device. A PF driver exhibits the following characteristics:

• Visible in Root Domain only

• May or may not possess data movement capabilities. The PF driver should function even in SR-IOV mode.

• Controls the enabling and disabling of the SR-IOV capability through APIs provided by the Oracle Solaris IOV Framework.

• The number of VFs to be configured for a given PF is determined by the system administrator. This number defined either in Machine Descriptors (MD) on the SPARC OVM platforms or a configuration file in the bare metal environment.

• The PF driver enables the VFs during the attachment phase by calling the Oracle Solaris IOV framework through the DDI interface. If the PF driver does not enable VFs during attachment, the Oracle Solaris IOV framework will attempt to configure VFs after the attachment as long as the driver callback flags indicates that the driver has IOV capability is supported.

• PFs can enable and disable each associated VF individually through a device specific mechanism

Virtual Function (VF) Driver

A function that is associated with a Physical Function. A VF is a lightweight PCIe function that shares one or more physical resources with the Physical Function and with other VFs that are associated with the same PF. A VF driver exhibits the following characteristics:

• Visible in both the root domain and I/O domain

• Can initiate communication with its PF either through the HW mailbox or an OS–provided interface

• Not visible in the root domain until the following conditions are satisfied:

  • The root domain has booted up

  • The PF driver attaches and invokes the configuration of the VF

  • The VFs are enabled by the root domain's Oracle Solaris IOV framework

  • The system firmware allocates resources to the VFs

• Not visible in the I/O Domain until the following conditions are satisfied:

  • VFs are already enabled and are visible in the root domain

  • VFs are assigned to the I/O Domain

  • VFs are probed in the I/O Domain by the Oracle Solaris firmware (OBP)

Device Configuration Parameters

The PF driver must support the configuration parameters listed in the following table. These parameters may be exported to the SPARC OVM Manager. Configuration is complete only when all the parameters are configured.

pci.conf File

The PCI configuration information file, /etc/pci.conf enables the system to save PCI configurations such as the number of VFs of a particular PF. The pci.conf file provides the following:

• To persist the PCI configuration so that VFs can be created automatically upon booting of the system.

• Since the configuration file is part of the boot_archive, VFs can be used during the booting of the system.

See pci.conf File for more information.

Setting Device Configuration Parameters

• SPARC: The parameters can be set through the ldm command. See the ldm (1M) man page for details.

• x86: The class-specific parameters can be specified by the pci.conf file. The following example shows the parameters set in a pci.conf file.

[System_Configuration]
[[path=/pci@0,0/pci8086,3a40@1c/pci108e,4848@0,1]]
num-vf=2

[Device_Configuration]
[[path=/pci@0,0/pci8086,3a40@1c/pci108e,4848@0,1]]
VF[0] = {
		primary-mac-addr = 0xaabbccddeeff
		alt-mac-addrs = 0x102233445556, 0x102233445557
		vlan-id = 20, 30
}

VF[1] = {
		primary-mac-addr = 0xaabbccddeef1
		alt-mac-addrs = 0x102233445568
		vlan-id = 20, 30, 40, 50
} 

SR-IOV Configuration on SPARC OVM Platform

The SPARC OVM Manager is responsible for the SR-IOV configuration on all SPARC OVM platforms. The SPARC OVM Manager is responsible for the following operations:

• Obtain a list of PFs that have SR-IOV capable drivers

• Obtain the device specific parameters supported by the drivers

• Validate a specific device configuration

• Update the Machine Descriptor (MD) file with all valid configuration details along with the assignment and removal of VFs

The following figure shows a high-level view of the SPARC OVM configuration.

Figure 27  High-Level View of SPARC OVM Configuration

SR-IOV Configuration on Bare Metal Platforms

At the time of Oracle Solaris 11 release, no configuration tool configures SR-IOV on bare metal platforms, including x86.