A root domain has a PCIe root complex assigned to it. This domain owns the PCIe fabric and provides all fabric-related services, such as fabric error handling. A root domain is also an I/O domain, as it owns and has direct access to physical I/O devices. The primary domain is the default root domain.
You can perform direct I/O and SR-IOV operations on PCIe buses that are assigned to any root domain. You can now perform the following operations for all root domains, including non-primary root domains:
Show the status of PCIe slots
Show the SR-IOV physical functions that are present
Assign a PCIe slot to an I/O domain or a root domain
Remove a PCIe slot from an I/O domain or a root domain
Create a virtual function from its physical function
Destroy a virtual function
Assign a virtual function to another domain
Remove a virtual function from another domain
The Logical Domains Manager obtains the PCIe endpoint devices and SR-IOV physical function devices from the Logical Domains agents that run in the non-primary root domains. This information is cached while the root domain is down after it is first discovered but only until the root domain is booted.
You can perform direct I/O and SR-IOV operations only when the root domain is active. Logical Domains Manager operates on the actual devices that are present at that time. The cached data might be refreshed when the following operations occur:
The Logical Domains agent is restarted in the specified root domain
A hardware change, such as a hot-plug operation, is performed in the specified root domain
Use the ldm list-io command to view the PCIe endpoint device status. The output also shows the sub-devices and physical function devices from the root complexes that are owned by each non-primary root domain.
You can use apply the following commands to any root domain:
Delayed reconfiguration support has been extended to include non-primary root domains. However, it can be used only to run the ldm add-io, ldm remove-io, ldm set-io, ldm create-vf and ldm destroy-vf commands. The delayed reconfiguration can be used for any operation that cannot be completed by using dynamic operations such as the following:
Performing direct I/O operations
Creating and destroying virtual functions from a physical function that does not meet the dynamic SR-IOV configuration requirements.
Caution - Plan ahead to minimize the number of reboots of the root domain, which minimizes downtime.