Oracle VM Server for SPARC uses logical domain channels (LDCs) to implement all communications such as console, virtual I/O, and control traffic. An LDC is the method used to enable communications between two endpoints. Although typically each endpoint is in a different domain, the endpoints can be in the same domain to enable loopback communications.
The Oracle VM Server for SPARC 3.2 software and system firmware provide a large pool of LDC endpoints that you can use for the control domain and guest domains. This LDC endpoint pool is only available for the SPARC T4, SPARC T5, SPARC M5, SPARC M6, and Fujitsu M10 platforms. The number of LDCs in the pool is based on the platform type as follows:
SPARC T4 – 1984 LDC endpoints per guest domain, 98304 LDC endpoints total
SPARC T5 – 1984 LDC endpoints per guest domain, 98304 LDC endpoints total
SPARC M5 – 1984 LDC endpoints per guest domain, 98304 LDC endpoints per physical domain
SPARC M6 – 1984 LDC endpoints per guest domain, 98304 LDC endpoints per physical domain
Fujitsu M10 – 1984 LDC endpoints per guest domain, 196608 LDC endpoints total
The required system firmware to support the LDC endpoint pool is 8.5.2 for SPARC T4 and 9.2.1 for SPARC T5, SPARC M5, and SPARC M6, and XCP2240 for Fujitsu M10.
The following LDC endpoint limits still apply if you run an older version of the system firmware on a supported platform or on an UltraSPARC T2, UltraSPARC T2 Plus, or SPARC T3 platform:
UltraSPARC T2 system – 512 LDC endpoints
UltraSPARC T2 Plus, SPARC T3, SPARC T4, SPARC T5, SPARC M5, SPARC M6, and Fujitsu M10 platforms – 768 LDC endpoints
This limitation might be an issue on the control domain because of the potentially large number of LDC endpoints that are used for both virtual I/O data communications and the Logical Domains Manager control of the other domains.
If you attempt to add a service or bind a domain so that the number of LDC endpoints exceeds the limit on any single domain, the operation fails with an error message similar to the following:
13 additional LDCs are required on guest primary to meet this request, but only 9 LDCs are available
The following guidelines enable you to plan properly for using LDC endpoints and explain why you might experience an overflow of the LDC capabilities of the control domain:
The control domain uses approximately 15 LDC endpoints for various communication purposes with the hypervisor, Fault Management Architecture (FMA), and the system processor (SP), independent of the number of other domains configured. The number of LDC endpoints used by the control domain depends on the platform and on the version of the software that is used.
The Logical Domains Manager allocates an LDC endpoint to the control domain for every domain, including itself, for control traffic.
Each virtual I/O service on the control domain uses one LDC endpoint for every connected client of that service. Each domain needs at least a virtual network, a virtual disk, and a virtual console.
The following equation incorporates these guidelines to determine the number of LDC endpoints that are required by the control domain:
15 + number-of-domains + (number-of-domains x number-of-virtual-services) = total-LDC-endpoints
number-of-domains is the total number of domains including the control domain and number-of-virtual-services is the total number of virtual I/O devices that are serviced by this domain.
The following example shows how to use the equation to determine the number of LDC endpoints when there is a control domain and eight additional domains:
15 + 9 + (8 x 3) = 48 LDC endpoints
The following example has 45 guest domains and each domain includes five virtual disks, two virtual networks, and a virtual console. The calculation yields the following result:
15 + 46 + 45 x 8 = 421 LDC endpoints
Depending upon the number of supported LDC endpoints of your platform, the Logical Domains Manager will either accept or reject the configuration.
If you run out of LDC endpoints on the control domain, consider creating service domains or I/O domains to provide virtual I/O services to the guest domains. This action enables the LDC endpoints to be created on the I/O domains and the service domains instead of on the control domain.
A guest domain can also run out of LDC endpoints. This situation might be caused by the inter-vnet-link property being set to on, which assigns additional LDC endpoints to guest domains to connect directly to each other.
The following equation determines the number of LDC endpoints that are required by a guest domain when inter-vnet-link=off:
2 + number-of-vnets + number-of-vdisks = total-LDC-endpoints
2 represents the virtual console and control traffic, number-of-vnets is the total number of virtual network devices assigned to the guest domain, and number-of-vdisks is the total number of virtual disks assigned to the guest domain.
The following example shows how to use the equation to determine the number of LDC endpoints per guest domain when inter-vnet-link=off and you have two virtual disks and two virtual networks:
2 + 2 + 2 = 6 LDC endpoints
The following equation determines the number of LDC endpoints that are required by a guest domain when inter-vnet-link=on:
2 + [[(number-of-vnets-from-vswX x number-of-vnets-in-vswX)] ...] + number-of-vdisks = total-LDC-endpoints
2 represents the virtual console and control traffic, number-of-vnets-from-vswX is the total number of virtual network devices assigned to the guest domain from the vswX virtual switch, number-of-vnets-in-vswX is the total number of virtual network devices on the vswX virtual switch, and number-of-virtual-disks is the total number of virtual disks assigned to the guest domain.
The following example shows how to use the equation to determine the number of LDC endpoints per guest domain when inter-vnet-link=on and you have two virtual disks and two virtual switches. The first virtual switch has eight virtual networks and assigns four of them to the domain. The second virtual switch assigns all eight of its virtual networks to the domain:
2 + (4 x 8) + (8 x 8) + 2 = 100 LDC endpoints
To resolve the issue of running out of LDC endpoints on a guest domain, consider using the ldm add-vsw or ldm set-vsw command to set the inter-vnet-link property to off. This action reduces the number of LDC endpoints in the domain or domains that have the virtual network devices. However, the off property value does not affect the service domain that has the virtual switch because the service domain still requires an LDC connection to each virtual network device. When this property is set to off, LDC channels are not used for inter-vnet communications. Instead, an LDC channel is assigned only for communication between virtual network devices and virtual switch devices. See the ldm(1M) man page.