A PCI device can be loaned out to a guest domain by means of SDIO. If during normal use some problem or fault with the loaned-out device is detected, the Oracle Solaris primary domain retires that device without notifying the Hypervisor.
Because the device is retired, the Oracle Solaris primary domain never instructs the Hypervisor to enable PCI access for that device. The state of the PCI device becomes indeterminate.
When the guest domain associated with the device is subsequently restarted, the guest domain hangs during the OpenBoot probe of PCI devices. The hang is identified by this message being displayed on the guest domain's console:
NOTICE: Probing PCI devices
Workaround: Remove the device that transitioned into an indeterminate state from that guest domain. Perform these steps on the Oracle Solaris primary domain.
Stop the affected guest domain.
Remove the device in an indeterminate state from the guest domain using the ldm rm-io command.
Restart the guest domain.