Physical Resource Allocation

When users deploy compute instances – or virtual machines – these consume physical resources provided by the hardware layer. The hypervisor manages the allocation of those physical resources based on algorithms that enable the best performance possible for a given configuration.

The compute nodes in Private Cloud Appliance have a Non-Uniform Memory Access (NUMA) architecture, meaning each CPU has access not only to its own local memory but also the memory of the other CPUs. Each CPU socket and its associated local memory banks are called a NUMA node. Local memory access, within the same NUMA node, always provides higher bandwidth and lower latency.

In general, the memory sharing design of NUMA helps with the scaling of multiprocessor workloads, but it may also adversely affect virtual machine performance if its resources are distributed across multiple NUMA nodes. Hypervisor policies ensure that a virtual machine's CPU and memory reside on the same NUMA node whenever possible. Using specific CPU pinning techniques, each virtual machine is pinned to one or multiple CPU cores so that memory is accessed locally on the NUMA node where the virtual machine is running. Cross-node memory transports are avoided or kept at a minimum, so the users of compute instances benefit from optimal performance across the entire system.

If a virtual machine can fit into a single NUMA node on a hypervisor host then this NUMA node is used for pinning, referred to as strict pinning. If a virtual machine cannot fit into single NUMA node on a hypervisor host, but can fit into multiple NUMA nodes, then multiple NUMA nodes are used for pinning, referred to as loose pinning.

Caution:

The CPU pinning applied through the hypervisor to optimize CPU and memory allocation to compute instances is not configurable by an appliance administrator, tenancy administrator or instance owner.

Private Cloud Appliance detects the NUMA topology of compute nodes during their provisioning (or any upgrade path from one release to another) and stores this information for use during the deployment of compute instances. The NUMA details of each compute instance are stored in the instance configuration. The NUMA settings are preserved when compute instances are migrated to another host compute node, but they might be overridden and dynamically adjusted if the target compute node is unable to accommodate that particular configuration.

Compute instances running on a Private Cloud Appliance that is not yet NUMA-aware, can take advantage of the optimization policies as soon as the system has been patched or upgraded. There is no action required from the administrator to align instances with NUMA topology; the existing instance configurations are made compatible as part of the process.