Prepare the KVM Host

Set BIOS and Firmware Parameters

The exact BIOS menu names vary by server vendor and model. Use the server hardware documentation to locate equivalent settings.

Table 8-5 Recommended BIOS Settings

Parameter	Function	Default	Recommended	Required
VT-x	Enables CPU virtualization extensions for VMs.	Enabled	Enabled	Mandatory
VT-d	Enables IOMMU for PCI passthrough (required for SR-IOV and DPDK in VMs).	Disabled	Enabled	Mandatory
SR-IOV	Allows NICs to create VFs for guest VMs.	Disabled	Enabled	Mandatory
CPU Profile	Controls CPU performance and power policy.	Balanced	Performance	Optional
Turbo	Allows CPU frequency boost under load.	Enabled	Enabled	Optional
C-states	Allows deep idle states that can add wake-up latency and jitter.	Enabled	Disabled	Optional

In the BIOS, set the following required parameters. (The location may be different depending on your BIOS.)

In the Advanced tab, set Intel Virtualization Technology to Enabled.
In the IO Virtualization tab:
- Set VT-d to Enabled
- Set SR-IOV to Enabled.
Under Advanced and then CPU Configuration or Processor Settings:
- Set the CPU profile to Performance.
- Set Turbo Mode to Enabled.
- Disable CPU C-states, C3, C6, or set the package C-state limit to C0/C1.
Save the configuration and reboot the host.

Verify the Host Operating System

Verify that Oracle Linux 9.7 or later is installed on the host.
```
cat /etc/oracle-release
```
If Oracle Linux is not installed, install Oracle Linux 9 before proceeding.
Update the system packages.
```
sudo dnf update -y
```

Verify Virtualization and IOMMU Support

Verify CPU virtualization support.
```
egrep -i 'vmx' /proc/cpuinfo
```
For Intel CPUs, the output must include vmx.
Verify IOMMU support after booting with the IOMMU kernel arguments.
```
sudo dmesg | grep -e DMAR -e IOMMU
```
If IOMMU is enabled, you'll see IOMMU enabled in the output.

Verify SR-IOV Support

SR-IOV (Single Root I/O Virtualization) extends PCI passthrough by partitioning a physical device into multiple Virtual Functions (VFs) that can be assigned to different virtual machines (VMs). This approach improves device utilization while maintaining near-native performance.

For SR-IOV to work properly, it must be supported by the physical interface, the device driver, and the operating system.

Step 1: Verify SR-IOV Support on the Physical Interface

The network interface cards (NICs) of your KVM server must support SR-IOV.

Identify your NICs.
```
sudo lspci | grep -i ethernet
```
Check your NIC's manual to verify whether SR-IOV is supported.

Step 2: Verify SR-IOV Support by the Device Driver

Identify the driver used for a specific interface.

 ethtool -i <interface> | grep driver

For example:

 ethtool -i eno1 | grep driver

The following NICs support SR-IOV.

Table 8-6 NICs and Drivers

NICs	PF kernel driver	VF kernel driver	PF user space driver	VF user space driver	Notes
Intel X540 / X550 (500 series)	ixgbe	ixgbevf	Keep on kernel (ixgbe)	vfio-pci	Enable IOMMU for vfio-pci. Bind VFs to vfio-pci for DPDK ixgbevf PMD.
Intel X710 / XL710 (700 series)	i40e	iavf	Keep on kernel (ice)	vfio-pci	The i40evf driver is deprecated; use iavf + vfio-pci for VFs.
Intel E810 (800 series)	ice	iavf	Keep on kernel (i40e)	vfio-pci	Enable IOMMU for vfio-pci. Bind VFs to vfio-pci for DPDK iavf PMD.
NVIDIA Mellanox ConnectX-5 / ConnectX-6	mix5_core	mix5_core	Keep on kernel (do not use vfio-pci)	Keep on kernel (do not use vfio-pci)	mlx5 PMD works with kernel mlx5_core + rdma-core/DevX. Binding to vfio-pci breaks mlx5 PMD.

The Intel NICs listed should use vfio-pci for a VF user space driver, but for PF user space drivers they should use the PF kernel driver. The NVIDIA NIC listed should not use vfio-pci but use the kernel driver for both PF and VF.

Note:

DPDK interfaces must be bound only to vfio-pci, which is a secure and production-safe driver. For production deployments, never use UIO-based drivers like uio_pci_generic, which provide less protection and do not support creating VFs. For further details on Linux drivers and their scope and limitations, see DPDK's Linux Drivers.

Step 3: Verify SR-IOV Support by the Operating System

Identify the BDF value of the interface.

ethtool -i <interface> | awk '/bus-info/ {split($2,a,":"); print a[2]":"a[3]}'

In the example below, 3a:00.0 is the BDF value:

ethtool -i eno1 | awk '/bus-info/ {split($2,a,":"); print a[2]":"a[3]}'
3a:00.0

Verify the device supports SR-IOV.

sudo lspci -v -s 3a:00.0 | grep -i sr-iov
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)

Verify the system supports SR-IOV.
```
cat /sys/class/net/<interface>/device/sriov_totalvfs
```
The value 0 means the system does not support SR-IOV. A value higher than 0 means the system supports SR-IOV.

Example output:
```
63
```

If the any level fails to support SR-IOV, you cannot use SR-IOV on that interface. Select a different interface and verify SR-IOV is supported.

Configure NUMA Nodes

For optimal RTP Proxy and DPDK performance, maintain NUMA locality between NICs, CPUs, and memory.

When selecting PFs for the network, Oracle recommends selecting two PFs from the same NUMA node. While not absolutely required, choosing two PFs for RTP Proxy traffic on the same NUMA node improves performance. When VFs are created from these PFs and assigned to the RTP Proxy VM, the traffic path inherits the NUMA affinity of the underlying NIC hardware.

If the PFs, CPUs, and memory are all aligned to the same NUMA node, packet reception, processing, and transmission remain NUMA-local. Maintaining NUMA locality helps reduce inter-socket traffic, remote memory accesses, and remote NIC queue access. This can lower latency and jitter while improving packet-processing efficiency and overall throughput.

In contrast, if one PF is selected from NUMA node 0 and another PF from NUMA node 1, RTP Proxy forwarding between the two datapath ports becomes a cross-NUMA datapath. In such a deployment, one side of the forwarding path will involve remote access to NIC queues, descriptor rings, or memory associated with the other NUMA node. This can lead to higher CPU overhead, increased latency/jitter, and reduced datapath performance, especially at higher traffic rates.

Maintaining end-to-end NUMA alignment keeps the RTP Proxy datapath as local as possible and delivers the most efficient and predictable performance.

Step 1: Create Virtual Functions from Physical Functions

Select both Physical Functions (PFs) from the same NUMA node on the host. Create virtual functions (VFs) from physical functions (PFs) that belong to the same host NUMA node whenever possible. This approach helps maintain NUMA locality for RTP Proxy datapath traffic and reduces cross-NUMA forwarding overhead. After you verify that SR-IOV is supported, create the required VFs. Before you create VFs, verify whether VFs already exist on the selected interface.

Verify whether any VFs exist for a specific interface.
```
ip link show <interface> | grep vf
```
If grep finds no matches, no VFs exist
If no VFs exist, create them.
In the following example, the number 4 specifies the number of VFs to create for each PF.

Note:
Each RTP Proxy VM requires at least 2 VFs.
```
echo 4 | sudo tee /sys/class/net/<PF_INTERFACE_1>/device/sriov_numvfs
echo 4 | sudo tee /sys/class/net/<PF_INTERFACE_2>/device/sriov_numvfs
```

To make these VFs persistent, append them to the rc.local file.

sudo bash -c 'cat >> /etc/rc.d/rc.local << "EOF"

# Create VFs
echo 4 > /sys/class/net/<PF_INTERFACE_1>/device/sriov_numvfs
echo 4 > /sys/class/net/<PF_INTERFACE_2>/device/sriov_numvfs

EOF'

Make the rc.local file executable.
```
sudo chmod +x /etc/rc.d/rc.local
```

Step 2: Reserve and Allocate Hugepages

For RTP Proxy deployments, Oracle recommends using hugepages because they help reduce CPU overhead, improve throughput, and provide lower and more predictable latency for high-speed packet-processing workloads.

You reserve hugepages from the same NUMA node as the PFs used for RTP Proxy traffic. If you deploy two RTP Proxy VMs and configure each VM with 16 GB of memory, reserve at least 32 GB of hugepage memory on the host. Then each VM receives its entire 16 GB memory allocation from hugepage-backed host memory.

Note:

When creating the guest VM, use the <memoryBacking> parameter in KVM to define the memory.

Note:

This example shows reserving hugepages on NUMA node 0. To reserve hugepages on NUMA node 1, change the node0 in the examples to node1.

Reserve the hugepages.
Because each hugepage is 2MB, this example reserves 32GB (16384 x 2 MB = 32768 MB = 32GB).
```
echo 16384 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
```

Verify the system reserved the hugepages.

grep -H . /sys/devices/system/node/node0/hugepages/hugepages-2048kB/{nr_hugepages,free_hugepages}
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages:16384
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/free_hugepages:16384

The contents of nr_hugepages show how many hugepages have been reserved, and the free_hugepages file shows how many hugepages are currently free and unallocated.

Make the hugepage allocation persistent across reboots with the following command.

sudo bash -c 'cat >> /etc/rc.d/rc.local << "EOF"

echo 16384 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

EOF'

Isolate CPUs for RTP Proxies

Although RTP Proxy workloads execute within guest VMs, the guest virtual CPUs (vCPUs) are scheduled on the host's physical CPUs. By default, the Linux scheduler can assign host operating system processes, kernel threads, interrupt handling, background services, and other VM workloads to the same physical CPUs used by RTP Proxy. Sharing CPUs in this manner can introduce resource contention, scheduling jitter, and latency variations that negatively affect RTP media packet processing performance.

To reduce these effects, CPU isolation is configured on the host by using kernel parameters such as:

isolcpus
nohz_full
rcu_nocbs

These parameters designate a subset of host CPUs for dedicated RTP Proxy workloads and minimize host operating system activity on those CPUs.

Housekeeping CPUs are the host CPUs that remain available for operating system and hypervisor functions. They are created by excluding selected CPUs from the isolcpus, nohz_full, and rcu_nocbs configurations.

Housekeeping CPUs are used for:

Host operating system processes
Interrupt handling
KVM/QEMU emulator threads
Memory management activities
Background services and system maintenance tasks
Virtualization management overhead

There is no dedicated kernel parameter for reserving housekeeping CPUs. Instead, CPUs become housekeeping CPUs by remaining outside the CPU isolation configuration. When determining the number of CPUs to isolate, reserve sufficient housekeeping CPUs to support host operating system activity and virtualization overhead. CPUs designated as housekeeping CPUs must not be assigned to RTP Proxy guest VMs.

As the number of RTP Proxy guest VMs increases, additional housekeeping CPU capacity may be required to accommodate increased hypervisor and management workloads. Systems handling higher traffic volumes or operating at higher VM densities may also require a larger housekeeping CPU allocation. The appropriate housekeeping CPU reservation should be validated based on the host CPU topology, interrupt load, virtualization overhead, and overall VM density.

When planning your CPU isolation strategy, consider the following best practices:

Select CPUs from the same NUMA node.
Never isolate all the host CPUs.
The CPU range must be based on your actual server topology. Use lscpu to identify your CPU range.
On hyper-threaded systems, use sibling CPUs.

If your KVM host already has multiple NUMA nodes configured, choose which node you will use. For example, if your host has the following NUMA configuration, you might select node0 to run the RTP Proxies.

NUMA node(s): 2
NUMA node0 CPU(s): 0-21,44-65
NUMA node1 CPU(s): 22-43,66-87

Here NUMA node0 contains:

CPUs 0 - 21: the primary sibling set
CPUs 44 - 65: the hyper-threaded sibling set

Below is a sample CPU configuration for 2 RTP Proxy VMs.

Note:

The CPU lists in these examples are samples only. Customize isolated and housekeeping CPUs for your server CPU topology and NUMA layout.

Table 8-7 Example CPU Configuration

RTP Proxy	Housekeeping CPUs	Isolated CPUs	NUMA
VM1	0, 44	6-13, 50-57	0
VM2	1, 45	14-21, 58-65	0

To set aside these CPUs for RTP Proxy, you update the host kernel parameters to isolate the CPU range from 6 to 21, and from 50 to 65. See the grubby example below for sample commands. These CPUs are then reserved for the RTP Proxy guest VM and excluded from normal host OS scheduling activity.

Note:

Use the command lscpu | grep NUMA to identify which CPUs are assigned to which NUMA node.

Use the following table to understand what each kernel parameter does.

Table 8-8 Kernel Parameters

Parameter	Purpose	Category	Required
isolcpus	Marks these CPUs as isolated from the scheduler’s general load balancing	Performance	Optional
nohz_full	Reduces interruptions on selected CPUs for smoother, faster workloads	Performance	Optional
rcu_nocbs	Offloads RCU callbacks from those CPUs to housekeeping CPUs i.e Moves background kernel tasks away from important CPUs	Performance	Optional
intel_iommu	Enables Intel VT-d IOMMU for DMA isolation and for vfio-pci use intel_iommu=on enables PCI passthrough	Functionality and Performance	Mandatory
iommu	Enables pass-through mode for IOMMU on non‑vfio devices to reduce overhead iommu=pt reduces IOMMU overhead to improve NIC performance.	Performance	Mandatory
intel_pstate	Enables or disables the Intel P-state driver which controls how CPU frequency scales	Performance	Optional

Check the existing kernel parameters.
```
cat /etc/default/grub
```

Add required parameters to isolate a subset of CPUs for the RTP Proxy VMs.

sudo grubby --update-kernel=ALL \
  --args="isolcpus=6-21,50-65 nohz_full=6-21,50-65 rcu_nocbs=6-21,50-65"

For Intel CPUs, add kernel parameters that enable IOMMU.

sudo grubby --update-kernel=ALL \
  --args="intel_iommu=on iommu=pt intel_pstate=disable"

Reboot the host.
```
sudo reboot
```
Verify the kernel parameters were loaded.
```
cat /proc/cmdline
```

Enable RSS

Enable Receive Side Scaling (RSS) on the interface selected for SR-IOV. Before you enable RSS, verify that the network interface card (NIC) supports RSS.

Verify RSS support.

ethtool -k <interface> | grep receive-hashing

For example:

ethtool -k eno1 | grep receive-hashing
receive-hashing: on

Verify the number of RX/TX queues.

ethtool -l <interface>

For example:

ethtool -l eno1
Channel parameters for eno1:
Pre-set maximums:
RX:             n/a
TX:             n/a
Other:          1
Combined:       40
Current hardware settings:
RX:             n/a
TX:             n/a
Other:          1
Combined:       40

Review the Combined value under Current hardware settings:

If the value is greater than 1, RSS is enabled.
If the value is 1, the NIC supports RSS, but RSS isn't enabled.

Set your Combined value to a number between 2 and the Combined value under Pre-set maximums.
```
ethtool -L <interface> combined <value>
```

Install KVM

Install virtualization packages.

sudo dnf groupinstall -y "Virtualization Host"
sudo dnf install -y virt-install virt-manager virt-viewer

Start libvirtd.
```
sudo systemctl enable --now libvirtd
```
Verify that libvirtd is running.
```
sudo systemctl status libvirtd
```