Oracle VM Server for SPARC uses logical domain channels (LDCs) to implement all communications such as console, virtual I/O, and control traffic. An LDC is the method used to enable communications between two endpoints. Although typically each endpoint is in a different domain, the endpoints can be in the same domain to enable loopback communications.
This software and system firmware provide a large pool of LDC endpoints that you can use for the control domain and guest domains. This LDC endpoint pool is available on servers starting with the SPARC T4, SPARC M5, and SPARC S7 series server and the Fujitsu M10 server. The number of LDCs in the pool is based on the platform type as follows:
SPARC T4 Server – 1984 LDC endpoints per logical domain
SPARC T5 Server – 4080 LDC endpoints per logical domain
SPARC T7 Series Server – 4080 LDC endpoints per logical domain
SPARC T8 Series Server – 4080 LDC endpoints per logical domain
SPARC M5 Server – 4080 LDC endpoints per logical domain
SPARC M6 Server – 4080 LDC endpoints per logical domain
SPARC M7 Series Server – 4080 LDC endpoints per logical domain
SPARC M8 Series Server – 4080 LDC endpoints per logical domain
Fujitsu M10 Server – 4080 LDC endpoints per logical domain
Fujitsu SPARC M12 Server – 4080 LDC endpoints per logical domain
SPARC S7 Series Server – 4080 LDC endpoints per logical domain
The required system firmware to support the LDC endpoint pool is as follows:
8.5.2 for SPARC T4 servers
9.2.1 for SPARC T5, SPARC M5, and SPARC M6 servers
Any released version for SPARC T7 and SPARC M7 series servers
Any released version for SPARC S7 series servers
Any released version for SPARC T8 and SPARC M8 series servers
XCP2240 for Fujitsu M10 servers
Any released version of the system firmware for Fujitsu SPARC M12 servers
The following LDC endpoint limits still apply if you run an older version of the system firmware on a supported platform or on an UltraSPARC T2, UltraSPARC T2 Plus, or SPARC T3 platform:
UltraSPARC T2 Server – 512 LDC endpoints per logical domain
UltraSPARC T2 Plus Server – 768 LDC endpoints per logical domain
SPARC T3 Server – 768 LDC endpoints per logical domain
SPARC T4 Server – 768 LDC endpoints per logical domain
SPARC T5 Server – 768 LDC endpoints per logical domain
SPARC M5 Server – 768 LDC endpoints per logical domain
SPARC M6 Server – 768 LDC endpoints per logical domain
Fujitsu M10 Server – 768 LDC endpoints per logical domain
If you attempt to add a service or bind a domain so that the number of LDC endpoints exceeds the limit on any single domain, the operation fails with an error message similar to the following:
13 additional LDCs are required on guest primary to meet this request, but only 9 LDCs are available
The following guidelines enable you to plan properly for using LDC endpoints and explain why you might experience an overflow of the LDC capabilities of the control domain:
The control domain uses approximately 15 LDC endpoints for various communication purposes with the hypervisor, Fault Management Architecture (FMA), and the system processor (SP), independent of the number of other domains configured. The number of LDC endpoints used by the control domain depends on the platform and on the version of the software that is used.
The Logical Domains Manager allocates an LDC endpoint to the control domain for every domain, including itself, for control traffic.
Each virtual I/O service on the control domain uses one LDC endpoint for every connected client of that service. Each domain needs at least a virtual network, a virtual disk, and a virtual console.
The following equation incorporates these guidelines to determine the number of LDC endpoints that are required by the control domain:
15 + number-of-domains + (number-of-domains x number-of-virtual-services) = total-LDC-endpoints
number-of-domains is the total number of domains including the control domain and number-of-virtual-services is the total number of virtual I/O devices that are serviced by this domain.
The following example shows how to use the equation to determine the number of LDC endpoints when there is a control domain and eight additional domains:
15 + 9 + (8 x 3) = 48 LDC endpoints
The following example has 45 guest domains and each domain includes five virtual disks, two virtual networks, and a virtual console. The calculation yields the following result:
15 + 46 + 45 x 8 = 421 LDC endpoints
Depending upon the number of supported LDC endpoints of your platform, the Logical Domains Manager will either accept or reject the configuration.
If you run out of LDC endpoints on the control domain, consider creating service domains or I/O domains to provide virtual I/O services to the guest domains. This action enables the LDC endpoints to be created on the I/O domains and the service domains instead of on the control domain.
A guest domain can also run out of LDC endpoints. This situation might be caused by the inter-vnet-link property being set to on, which assigns additional LDC endpoints to guest domains to connect directly to each other.
The following equation determines the number of LDC endpoints that are required by a guest domain when inter-vnet-link=off:
2 + number-of-vnets + number-of-vdisks = total-LDC-endpoints
2 represents the virtual console and control traffic, number-of-vnets is the total number of virtual network devices assigned to the guest domain, and number-of-vdisks is the total number of virtual disks assigned to the guest domain.
The following example shows how to use the equation to determine the number of LDC endpoints per guest domain when inter-vnet-link=off and you have two virtual disks and two virtual networks:
2 + 2 + 2 = 6 LDC endpoints
The following equation determines the number of LDC endpoints that are required by a guest domain when inter-vnet-link=on:
2 + [[(number-of-vnets-from-vswX x number-of-vnets-in-vswX)] ...] + number-of-vdisks = total-LDC-endpoints
2 represents the virtual console and control traffic, number-of-vnets-from-vswX is the total number of virtual network devices assigned to the guest domain from the vswX virtual switch, number-of-vnets-in-vswX is the total number of virtual network devices on the vswX virtual switch, and number-of-virtual-disks is the total number of virtual disks assigned to the guest domain.
The following example shows how to use the equation to determine the number of LDC endpoints per guest domain when inter-vnet-link=on and you have two virtual disks and two virtual switches. The first virtual switch has eight virtual networks and assigns four of them to the domain. The second virtual switch assigns all eight of its virtual networks to the domain:
2 + (4 x 8) + (8 x 8) + 2 = 100 LDC endpoints
Virtual network devices that you create by using at least the Oracle VM Server for SPARC 3.4 software have inter-vnet-link=auto by default. This feature automatically turns off inter-vnet-links when the number exceeds the threshold. However, any virtual network devices that you created with inter-vnet-link=on must be explicitly modified to change inter-vnet-link=off to reduce the number of LDC channels. For more information, see Inter-Vnet LDC Channels.
You can still set inter-vnet-link=off to reduce the number of LDC endpoints in the domain or domains that have the virtual network devices. However, the off property value does not affect the service domain that has the virtual switch because the service domain still requires an LDC connection to each virtual network device. When this property is set to off, LDC channels are not used for inter-vnet communications. Instead, an LDC channel is assigned only for communication between virtual network devices and virtual switch devices. See the ldm(1M) man page.