Creating an I/O Domain by Assigning PCIe Endpoint Devices
You can assign an individual PCIe endpoint (or direct I/O-assignable) device to a domain. This use of PCIe endpoint devices increases the granularity of the device assignment to I/O domains. This capability is delivered by means of the direct I/O (DIO) feature.
The DIO feature enables you to create more I/O domains than the number of PCIe buses in a system. The possible number of I/O domains is now limited only by the number of PCIe endpoint devices.
A PCIe card in a slot
An on-board PCIe device that is identified by the platform
A PCIe endpoint device can be one of the following:
Note - Because root domains cannot have dependencies on other root domains, a root domain that owns a PCIe bus cannot have its PCIe endpoint devices or SR-IOV virtual functions assigned to another root domain. However, you can assign a PCIe endpoint device or virtual function from a PCIe bus to the root domain that owns that bus.
The following diagram shows that the PCIe endpoint device, PCIE3, is assigned to an I/O domain. Both bus pci_0 and the switch in the I/O domain are virtual. The PCIE3 endpoint device is no longer accessible in the primary domain.
In the I/O domain, the pci_0 block and the switch are a virtual root complex and a virtual PCIe switch, respectively. This block and switch are similar to the pci_0 block and the switch in the primary domain. In the primary domain, the devices in slot PCIE3 are a “shadow” form of the original devices and are identified as SUNW,assigned.
 | Caution - You cannot use Oracle Solaris hot-plug operations to hot-remove a PCIe endpoint device after that device is removed from the primary domain by using the ldm remove-io command. For information about replacing or removing a PCIe endpoint device, see Making PCIe Hardware Changes. |
Figure 5 Assigning a PCIe Endpoint Device to an I/O Domain
Use the ldm list-io command to list the PCIe endpoint devices.
Though the DIO feature permits any PCIe card in a slot to be assigned to an I/O domain, only certain PCIe cards are supported. See Direct I/O Hardware and Software Requirements.
| Caution - PCIe cards that have a bridge are not supported. PCIe function-level assignment is also not supported. Assigning an unsupported PCIe card to an I/O domain might result in unpredictable behavior. |
This feature is enabled only when all the software requirements are met. See Direct I/O Hardware and Software Requirements.
Only PCIe endpoints that are connected to a PCIe bus assigned to a root domain can be assigned to another domain with the DIO feature.
I/O domains that use DIO have access to the PCIe endpoint devices only when the root domain is running.
Rebooting the root domain affects I/O domains that have PCIe endpoint devices. See Rebooting the Root Domain With PCIe Endpoints Configured. The root domain also performs the following tasks:
Initializes and manages the PCIe bus.
Handles all bus errors that are triggered by the PCIe endpoint devices that are assigned to I/O domains. Note that only the primary domain receives all PCIe bus-related errors.
The following items describe important details about the DIO feature: