You can assign an individual PCIe endpoint (or direct I/O-assignable) device to a domain. This use of PCIe endpoint devices increases the granularity of the device assignment to I/O domains. This capability is delivered by means of the direct I/O (DIO) feature.
The DIO feature enables you to create more I/O domains than the number of PCIe buses in a system. The possible number of I/O domains is now limited only by the number of PCIe endpoint devices.
A PCIe endpoint device can be one of the following:
A PCIe card in a slot
An on-board PCIe device that is identified by the platform
The following diagram shows that the PCIe endpoint device, PCIE3, is assigned to an I/O domain. Both bus pci_0 and the switch in the I/O domain are virtual. The PCIE3 endpoint device is no longer accessible in the primary domain.
In the I/O domain, the pci_0 block and the switch are a virtual root complex and a virtual PCIe switch, respectively. This block and switch are similar to the pci_0 block and the switch in the primary domain. In the primary domain, the devices in slot PCIE3 are a “shadow” form of the original devices and are identified as SUNW,assigned.
Caution - You cannot use Oracle Solaris hot-plug operations to hot-remove a PCIe endpoint device after that device is removed from the primary domain by using the ldm rm-io command. For information about replacing or removing a PCIe endpoint device, see Making PCIe Hardware Changes.
Figure 6-2 Assigning a PCIe Endpoint Device to an I/O Domain
Use the ldm list-io command to list the PCIe endpoint devices.
Though the DIO feature permits any PCIe card in a slot to be assigned to an I/O domain, only certain PCIe cards are supported. See Direct I/O Hardware and Software Requirements in Oracle VM Server for SPARC 18.104.22.168, 3.1.1, and 3.1 Release Notes .
Caution - PCIe cards that have a bridge are not supported. PCIe function-level assignment is also not supported. Assigning an unsupported PCIe card to an I/O domain might result in unpredictable behavior.
The following items describe important details about the DIO feature:
This feature is enabled only when all the software requirements are met. See Direct I/O Hardware and Software Requirements in Oracle VM Server for SPARC 22.214.171.124, 3.1.1, and 3.1 Release Notes .
Only PCIe endpoints that are connected to a PCIe bus assigned to a root domain can be assigned to another domain with the DIO feature.
I/O domains that use DIO have access to the PCIe endpoint devices only when the root domain is running.
Rebooting the root domain affects I/O domains that have PCIe endpoint devices. See Rebooting the Root Domain. The root domain also performs the following tasks:
Initializes and manages the PCIe bus.
Handles all bus errors that are triggered by the PCIe endpoint devices that are assigned to I/O domains. Note that only the primary root domain receives all PCIe bus-related errors.