1. Administrator's Guide
  2. Deployment Optimization

4 Deployment Optimization

You can configure Oracle Linux Virtualization Manager so that your cluster is optimized and your hosts and virtual machine are highly available. You can also enable or disable devices (hot plug) while a virtual machine is running.

Optimizing Clusters, Hosts and Virtual Machines

Whether you have a new cluster, host, or virtual machine or existing ones, you can optimize resources such as CPU and memory and configure hosts and virtual machines for high availability.

Configuring Memory and CPUs

Using the Optimization tab when creating or editing a cluster, you can select the memory page sharing threshold for the cluster, and optionally enable CPU thread handling and memory ballooning on the hosts in the cluster. Some of the benefits are:

  • Virtual machines run on hosts up to the specified overcommit threshold. Higher values conserve memory at the expense of great CPU usage.

  • Hosts to run virtual machines with a total number of CPU cores greater than the number of cores in the host.

  • Memory overcommitment on virtual machines running on the hosts in the cluster.

  • Memory Overcommitment Manager (MoM) runs Kernel Same-page Merging (KSM) when it can yield a memory saving benefit.

Note:

If a virtual machine is running Oracle products, such as Oracle Database or other Oracle applications, that require dedicated memory, configuring memory overcommitment is not an available option.

Using the Resource Allocation tab when creating or editing a virtual machine, you can:

  • set the maximum amount of processing capability a virtual machine can access on its host.

  • pin a virtual CPU to a specific physical CPU.

  • guarantee an amount of memory for the virtual machine.

  • enable the memory balloon device for the virtual machine. Enable Memory Balloon Optimization must also be selected for the cluster.

  • improve the speed of disks that have a VirtIO interface by pinning them to a thread separate from the virtual machine's other functions.

For more information, refer to High Availability and Optimization in the Oracle Linux Virtualization Manager: Architecture and Planning Guide.

Configuring Cluster Memory and CPUs
Use the Administration Portal to optimize the usage of memory and CPUs at the cluster level:

  1. Select the Optimization tab of the New Cluster or Edit Cluster window.

  2. Choose a setting for Memory Optimization:

    • None - Disable memory overcommit

      Disables memory page sharing, which allows you to commit 100% of the physical memory to virtual machines.

    • For Server Load - Allow scheduling of 150% of physical memory

      Sets memory page sharing threshold to 150% of the system memory on each host.

    • For Desktop Load - Allow scheduling of 200% of physical memory

      Sets memory page sharing threshold to 200% of the system memory on each host.

  3. Under CPU Threads, check Count Threads As Cores to allow guests to use host threads as virtual CPU cores.

    Allowing hosts to run virtual machines with the total number of processing cores greater than the number of cores in the host may be useful for less CPU-intensive workloads.

  4. Under Memory Balloon, check Enable Memory Balloon Optimization to enable memory overcommitment on virtual machines running on hosts in this cluster.

    The MoM starts ballooning where and when possible. It is only limited by the guaranteed memory size of every virtual machine. Each virtual machine in the cluster needs to have a balloon device with relevant drivers, which is included unless you specifically remove it. Every host in the cluster receives a balloon policy update when its Status changes to Up.

    Note:

    Enable ballooning on virtual machines that have applications and loads that slowly consume memory, occasionally release memory, or stay dormant for long periods of time, such as virtual desktops.

  5. Under KSM Control, check Enable KSM to enable MoM to run KSM when necessary and when it can yield a memory saving benefit that outweighs its CPU cost.

  6. Click OK to save your changes.

Changing Memory Overcommit Manually

The memory overcommit settings in the Administration Portal allow you to disable overcommit or set it to 150% or 200%. If you require a different value for your cluster, you can change the setting manually.

  1. From a a command line, log into the Engine.

  2. Check the current memory overcommit settings:

    # engine-config -a | grep -i maxvdsmem
MaxVdsMemOverCommit: 200 version: general
MaxVdsMemOverCommitForServers: 150 version: general

  3. Change the memory overcommit settings:

    # engine-config -s MaxVdsMemOverCommitForServers=percentage
    # engine-config -s MaxVdsMemOverCommit=percentage
Configuring Virtual Machine Memory and CPUs

To optimize the usage of memory and CPUs for a virtual machine:

  1. Select the Resource Allocation tab of the New VM or Edit VM window.

  2. Under CPU Allocation, for the CPU Shares drop-down list select the level of CPU resources a virtual machine can demand relative to other virtual machines in the cluster.

    • Low=512

    • Medium=1024

    • High=2048

    • Custom=Enter a number in the field next to the drop-down list

  3. Under Memory Allocation, for Physical Memory Guaranteed enter an amount of memory.

    The amount of physical memory guaranteed for a virtual machine should be any number between 0 and its defined memory.

  4. Check Memory Balloon Device Enabled to enable the device for the virtual machine and allow memory overcommitment.

    Important:

    Since this check box is selected by default, make sure you have enabled memory ballooning for the cluster where the virtual machine's host resides.

  5. Under I/O Threads, check I/O Threads Enabled to improve the speed of disks that have a VirtIO interface by pinning them to a thread separate from the virtual machine's other functions.

    This check box is selected by default.

  6. Under Queues, check Multi Queues Enabled to create up to four queues per vNIC, depending on how many vCPUs are available.

    This check box is selected by default.

    To define a different number of queues per vNIC, you can create a custom property:
    # engine-config -s "CustomDeviceProperties={type=interface;prop={other-nic-properties;queues=[1-9][0-9]\*}}"

    where other-nic-properties is a list of pre-existing NIC custom properties separated by semicolons.

  7. Under Queues, check VirtIO-SCSI Enabled to enable or disable the use of VirtIO-SCSI on the virtual machine.

    This check box is selected by default.

  8. Click OK to save your changes.

Configuring a Highly Available Host

If you want the hosts in a cluster to be responsive and available when unexpected failures happen, you should use fencing. Fencing allows a cluster to react to unexpected host failures and enforce power saving, load balancing, and virtual machine availability policies.

A non-operational host is different from a non-responsive host. A non-operational host can communicate with the Manager, but isn't configured correctly, for example a missing logical network. A non-responsive host cannot communicate with the Manager. In a fencing operation, if a host becomes non-responsive, it is rebooted. If after a prescribed period of time the host remains non-responsive, manual intervention needs to be taken.

The Manager can perform management operations after it reboots, by a proxy host, or manually in the Administration Portal. All the virtual machines running on the non-responsive host are stopped, and highly available virtual machines are restarted on a different host. At least two hosts are required for power management operations.

Important:

If a host runs virtual machines that are highly available, power management must be enabled and configured.

For more information, refer to High Availability and Optimization in the Oracle Linux Virtualization Manager: Architecture and Planning Guide.

Configuring Power Management and Fencing on a Host

The Manager uses a proxy to send power management commands to a host power management device because the engine does not communicate directly with fence agents. The host agent (VDSM) executes power management device actions and another host in the environment is used as a fencing proxy. This means that you must have at least two hosts for power management operations.

When you configure a fencing proxy host, make sure the host is in:

  • the same cluster as the host requiring fencing.

  • the same data center as the host requiring fencing.

  • UP or Maintenance status to remain viable.

Power management operations can be performed in three ways:

  • by the Manager after it reboots

  • by a proxy host

  • manually in the Administration Portal

To configure power management and fencing on a host:

  1. Click Compute and select Hosts.

  2. Select a host and click Edit.

  3. Click the Power Management tab.

  4. Check Enable Power Management to enable the rest of the fields.

  5. Check Kdump integration to prevent the host from fencing while performing a kernel crash dump. Kdump integration is enabled by default.

    Important:

    If you enable or disable Kdump integration on an existing host, you must reinstall the host.

  6. (Optional) Check Disable policy control of power management if you do not want your host’s power management to be controlled by the scheduling policy of the host's cluster.

  7. To configure a fence agent, click the plus sign (+) next to Add Fence Agent.

    The Edit fence agent pane opens.

  8. Enter the Address (IP Address or FQDN) to access the host's power management device.

  9. Enter the User Name and Password of the of the account used to access the power management device.

  10. Select the power management device Type from the drop-down list.

  11. Enter the Port (SSH) number used by the power management device to communicate with the host.

  12. Enter the Slot number used to identify the blade of the power management device.

  13. Enter the Options for the power management device. Use a comma-separated list of key-value pairs.

    • If you leave the Options field blank, you are able to use both IPv4 and IPv6 addresses

    • To use only IPv4 addresses, enter inet4_only=1

    • To use only IPv6 addresses, enter inet6_only=1

  14. Check Secure to enable the power management device to connect securely to the host.

    You can use ssh, ssl, or any other authentication protocol your power management device supports.

  15. Click Test to ensure the settings are correct and then click OK.

    Test Succeeded, Host Status is: on displays if successful.

    Attention:

    Power management parameters (userid, password, options, etc.) are tested by the Manager only during setup and manually after that. If you choose to ignore alerts about incorrect parameters, or if the parameters are changed on the power management hardware without changing in the Manager as well, fencing is likely to fail when most needed.

  16. Fence agents are sequential by default. To change the sequence in which the fence agents are used:

    1. Review your fence agent order in the Agents by Sequential Order field.

    2. To make two fence agents concurrent, next to one fence agent click the Concurrent with drop-down list and select the other fence agent.

      You can add additional fence agents to this concurrent fence agent group.

  17. Expand the Advanced Parameters and use the up and down buttons to specify the order in which the Manager searches the host’s cluster and dc (data center) for a power management proxy.

  18. To add an additional power management proxy:

    1. Click the plus sign (+) next to Add Power Management Proxy.

      The Select fence proxy preference type to add pane opens.

    2. Select a power management proxy from the drop-down list and then click OK.

      Your new proxy displays in the Power Management Proxy Preference list.

    Note:

    By default, the Manager searches for a fencing proxy within the same cluster as the host. If The Manager cannot find a fencing proxy within the cluster, it searches the data center.

  19. Click OK.

From the list of hosts, the exclamation mark next to the host’s name disappeared, signifying that you have successfully configured power management and fencing.

Preventing Host Fencing During Boot

After you configure power management and fencing, when you start the Manager it automatically attempts to fence non-responsive hosts that have power management enabled after the quiet time (5 minutes by default) has elapsed. You can opt to extend the quiet time to prevent, for example, a scenario where the Manager attempts to fence hosts while they boot up. This can happen after a data center outage because a host’s boot process is normally longer than the Manager boot process.

You can configure quiet time using the engine-config command option DisableFenceAtStartupInSec: 

# engine-config -s DisableFenceAtStartupInSec=number
Checking Fencing Parameters

To automatically check the fencing parameters, you can configure the PMHealthCheckEnabled (false by default) and PMHealthCheckIntervalInSec (3600 sec by default) engine-config options. 

# engine-config -s PMHealthCheckEnabled=True
# engine-config -s PMHealthCheckIntervalInSec=number

When set to true, PMHealthCheckEnabled checks all host agents at the interval specified by PMHealthCheckIntervalInSec and raises warnings if it detects issues.

Configuring a Highly Available Virtual Machine

If you have virtual machines that run critical workloads, you might consider configuring these virtual machines for high availability. Only a highly available virtual machine automatically restarts on its original host or migrates to another host in the cluster if its original host:

  • has a hardware failure and becomes non-operational.

  • has scheduled downtime and is put in maintenance mode.

  • loses communication with external storage and becomes unavailable.

If a virtual machine's host is manually shut down, the virtual machine does not automatically migrate to another host. Further, virtual machines that share the same storage domain can live migrate between hosts that belong to the same cluster. For more information, see Migrating Virtual Machines between Hosts in the Oracle Linux Virtualization Manager: Administration Guide.

Note:

A highly available virtual machine does not restart if you shut it down cleanly from within the virtual machine or the Manager or if you shut down a host without first putting it into maintenance mode.

To enable a virtual machine to migrate to another available host in the cluster:

  • Configure power management and fencing for the host running the highly available virtual machine

  • Ensure the highly available virtual machine's host is part of a cluster of two or more available hosts

  • Check that the destination host is operational

  • Ensure the source and destination hosts can access the data domain where the virtual machine resides

  • Ensure the source and destination hosts can access the same virtual networks and VLANs

  • Check that the destination host has enough RAM and CPUs that are not in use to support the virtual machine's requirements

Virtual machines can also be restarted on another host even if the original host loses power if you have configured it to acquire a lease on a special volume on the storage domain. Acquiring a lease prevents the virtual machine from being started on two different hosts, which could result in virtual machine disk corruption.

If you configure high availability:

  • there is minimal service interruption because virtual machines are restarted within seconds and with no user intervention.

  • your resources are balanced by restarting virtual machines on a host with low current resource utilization.

  • you are ensured that there is sufficient capacity to restart virtual machines at all times.

You must configure high availability for each virtual machine using the following steps:

  1. Click Compute and then Virtual Machines.

  2. In the list of virtual machines, click to highlight a virtual machine and then click Edit.

  3. In the Edit Virtual Machine window, click the High Availability tab.

  4. Check Highly Available to enable high availability for the virtual machine.

  5. From the Target Storage Domain for VM Lease drop-down list, select No VM Lease (default) to disable the functionality or select a storage domain to hold the virtual machine lease.

    Virtual machines are able to acquire a lease on a special volume on the storage domain. This enables a virtual machine to start on another host even if the original host loses power. For more information, see Storage Leases in the Oracle Linux Virtualization Manager: Architecture and Planning Guide.

  6. From the Resume Behavior drop-down list, select AUTO_RESUME, LEAVE_PAUSED, OR KILL. If you defined a VM lease, KILL is the only option available.

  7. From the Priority list, select Low, Medium, or High.

    When virtual machine migration is triggered, a queue is created in which the high priority virtual machines are migrated first. If a cluster is running low on resources, only the high-priority virtual machines are migrated.

  8. Click OK.

Optimizing Virtual Machine Performance

You can optimize a virtual machine for high performance, so that it runs with performance metrics as close to bare metal as possible. When you choose high performance optimization, the virtual machine is configured with a set of automatic, and recommended manual, settings for maximum efficiency.

The high performance option is only accessible in the Administration Portal, by selecting High Performance from the Optimized for dropdown list in the Edit or New virtual machine, template, or pool window. This option is not available in the VM Portal.

If you change the optimization mode of a running virtual machine to high performance, some configuration changes require restarting the virtual machine. To change the optimization mode of a new or existing virtual machine to high performance, you may need to make manual changes to the cluster and to the pinned host configuration first.

A high performance virtual machine has certain limitations, because enhanced performance has a trade-off in decreased flexibility:

  • If pinning is set for CPU threads, IO threads, emulator threads, or NUMA nodes, according to the recommended settings, only a subset of cluster hosts can be assigned to the high performance virtual machine.

  • Many devices are automatically disabled, which limits the virtual machine’s usability.

Configuring a High Performance Virtual

To configure a high performance virtual machine:

  1. In the New or Edit window, select High Performance from the Optimized for drop-down menu.

    Selecting this option automatically performs certain configuration changes to this virtual machine.

  2. Click OK.

    If you have not set any manual configurations, the High Performance Virtual Machine/Pool Settings screen describing the recommended manual configurations appears.

    If you have set some of the manual configurations, the High Performance Virtual Machine/Pool Settings screen displays the settings you have not made.

    If you have set all the recommended manual configurations, the High Performance Virtual Machine/Pool Settings screen does not appear.

  3. If the High Performance Virtual Machine/Pool Settings screen appears, click Cancel to return to the New or Edit window to perform the manual configurations. For details, see Configuring the Recommended Manual Settings in oVirt Documentation.

    Alternatively, click OK to ignore the recommendations. The result may be a drop in the level of performance.

  4. Click OK.

    You can view the optimization type in the General tab of the details view of the virtual machine, pool, or template.

Configuring Huge Pages

You can configure a virtual machine for high performance, so that it runs with performance metrics as close to bare metal as possible. When you choose high performance optimization, the virtual machine is configured with a set of automatic and recommended manual settings for maximum efficiency. By using huge pages, you increase the page size which reduces the page table, reduces the pressure on the Translation Lookaside Buffer cache, and improves performance.

Huge pages are pre-allocated when a virtual machine starts to run (dynamic allocation is disabled by default).

Note:

If you configure huge pages for a virtual machine, you cannot hotplug or hot uplug memory.

To configure huge pages:

  1. In the Custom Properties tab, select hugepages from the custom properties list, which displays Please select a key…​ by default.

  2. Enter the huge page size in KB.

    You should set the huge page size to the largest size supported by the pinned host. The recommended size for x86_64 is 1 GB.

    The huge page size has the following requirements:

    • The virtual machine’s huge page size must be the same size as the pinned host’s huge page size.

    • The virtual machine’s memory size must fit into the selected size of the pinned host’s free huge pages.

    • The NUMA node size must be a multiple of the huge page’s selected size.

To enable dynamic allocation of huge pages:

  1. Disable the HugePages filter in the scheduler.

  2. In the [performance] section in /etc/vdsm/vdsm.conf set the following:

    use_dynamic_hugepages = true

Hot Plugging Devices on Virtual Machines

You can enable or disable devices while a virtual machine is running.

Hot Plugging vCPUs

Hot plugging vCPUs means enabling or disabling devices while a virtual machine is running.

Note:

Hot unplugging a vCPU is only supported if the vCPU was previously hot plugged. A virtual machine’s vCPUs cannot be hot unplugged to less vCPUs than it was originally created with.

Before you can hot plug vCPUs, you must meet the following prerequisites:

  • The virtual machine's operating system must be explicitly set and must support CPU hot plug. For details, see oVirt Documentation.

  • The virtual machine must have at least four vCPUs

  • Windows virtual machines must have the guest agents installed. See Windows Virtual Machines Lose Functionality Due To Deprecated Guest Agent in the Known Issues section of the Oracle Linux Virtualization Manager: Release Notes for more information.

Create vm with 4 cpus Hotplug 2 more (cpu count 6) Hot unplug cpus that you added (cpu count 4) Note that only previously hot plugged CPUs can be hot unplugged

To hot plug a vCPU:

  1. Click Compute and then select Virtual Machines.

  2. Select a virtual machine that is running and click Edit.

  3. Click the System tab.

  4. Change the value of Virtual Sockets as required.

  5. Click OK.

Hot Plugging Virtual Memory

Hot plugging memory means enabling or disabling devices while a virtual machine is running. Each time you hot plug memory, it appears as a new memory device under Vm Devices on the virtual machine's details page, up to a maximum of 16.

When you shut down and restart a virtual machine, these devices are cleared from Vm Devices without reducing the virtual machine's memory, allowing you to hot plug more memory devices.

Note:

This feature is only available for the self-hosted engine Engine virtual machine, which is currently a technology preview feature.

To hot plug virtual memory:

  1. Click Compute and then select Virtual Machines.

  2. Select a virtual machine that is running and click Edit.

  3. Click the System tab.

  4. Enter a new number for Memory Size. You can add memory in multiples of 256 MB. By default, the maximum memory allowed for the virtual machine is set to 4x the memory size specified.

    Note:

    Hot plugging virtual memory is only possible if the Max Memory Size is greater than the currently configured memory size.

  5. Click OK.

    The Pending Virtual Machine changes window opens.

  6. Click OK for the changes to take place immediately or check Apply later and then OK to wait for the next virtual machine restart.

  7. Click OK.

    You can see the virtual machine's updated memory in the Defined Memory field of the virtual machine's details page and you can see the added memory under Vm Devices.

You can also hot unplug virtual memory, but consider:

  • Only memory added with hot plugging can be hot unplugged.

  • The virtual machine's operating system must support memory hot unplugging.

  • The virtual machine must not have a memory balloon device enabled.

To hot unplug virtual memory:

  1. Click Compute and then select Virtual Machines.

  2. Click on the name of a virtual machine that is running.

    The virtual machine's details page opens.

  3. Click Vm Devices.

  4. In the Hot Unplug column, click Hot Unplug beside any memory device you want to remove.

    The Memory Hot Unplug windows opens with a warning.

  5. Click OK.

    Under General on the virtual machine details page, the Physical Memory Guaranteed value for the virtual machine is decremented automatically.