Prepare the KVM Guest

Create a VM

For the resource requirements for the RTP Proxy VM, see Table 8-4.

Use virt-manager to create a virtual machine with the following properties:

In Virtual Machine Manager, select New.
Select Local install media and then Forward.
Select Browse, then browse to your Oracle Linux 9 ISO, and then select Choose Volume.
If your system does not automatically detect Oracle Linux, unselect "Automatically detect from the installation media / source" checkbox and enter "Oracle Linux 9.7". Select Forward.
Set the memory to at least 16GB (or the recommended 32GB) and set the CPUs to at least 12 (or the recommended 16). Select Forward.
Set the storage size to at least 40GB and select Forward.
Enter a name for this virtual machine.
Select "Customize configuration before install" and select Finish.

Configure Chipset, Disk, and Primary NIC

Under Overview, and then Hypervisor Details, set the Chipset parameter to Q35. Click Apply.
Under IDE Disk 1, set the Disk bus to SATA. Click Apply.
Under IDE CDROM 1, set the Disk bus to SATA. Click Apply.
Under NIC, set the network source so the RTP Proxy can communicate with the Converged Application Server. This will typically be a Bridge device on the same network as the Converged Application Server.

Configure PCI Host Devices

Attach SR-IOV VFs to the virtual machine. Every VM must be assigned two VFs. Each VF comes from a different PF. Before assigning PCI hardware (SR-IOV VFs) to the VM, you must identify the correct PF interface and the available VFs under it to ensure proper network connectivity.

On your KVM host, identify which PF is used to communicate with Session Border Controllers, and which PF is used to communicate with Media Function Engines.

Run the following script on the KVM host to identify which VFs are used and which are available.

Replace PF_1 and PF_2 at the beginning of the script with the names of your physical interfaces.

PFS="<PF_1> <PF_2>"; used_vfs=$(for vm in $(virsh list --all --name); do [ -n "$vm" ] && virsh dumpxml "$vm"; done | sed -n "s/.*<address domain='0x\([0-9a-fA-F]*\)' bus='0x\([0-9a-fA-F]*\)' slot='0x\([0-9a-fA-F]*\)' function='0x\([0-9a-fA-F]*\)'.*/\1:\2:\3.\4/p" | awk -F'[:.]' '{printf "%04x:%02x:%02x.%x\n", strtonum("0x"$1), strtonum("0x"$2), strtonum("0x"$3), strtonum("0x"$4)}' | sort -u); for pf in $PFS; do all_vfs=$(for vf in /sys/class/net/$pf/device/virtfn*; do [ -e "$vf" ] && readlink -f "$vf" | xargs basename; done | sort -u); echo "===== Used VFs under $pf ====="; used_under_pf=$(comm -12 <(echo "$all_vfs") <(echo "$used_vfs")); [ -n "$used_under_pf" ] && echo "$used_under_pf" || echo "No used VFs under $pf"; echo; echo "===== Available VFs under $pf ====="; available_vfs=$(comm -23 <(echo "$all_vfs") <(echo "$used_vfs")); [ -n "$available_vfs" ] && echo "$available_vfs" || echo "No available VFs under $pf"; echo; done

For example:

# PFS="eno2 eno3"; used_vfs=$(for vm in $(virsh list --all --name); do [ -n "$vm" ] && virsh dumpxml "$vm"; done | sed -n "s/.*<address domain='0x\([0-9a-fA-F]*\)' bus='0x\([0-9a-fA-F]*\)' slot='0x\([0-9a-fA-F]*\)' function='0x\([0-9a-fA-F]*\)'.*/\1:\2:\3.\4/p" | awk -F'[:.]' '{printf "%04x:%02x:%02x.%x\n", strtonum("0x"$1), strtonum("0x"$2), strtonum("0x"$3), strtonum("0x"$4)}' | sort -u); for pf in $PFS; do all_vfs=$(for vf in /sys/class/net/$pf/device/virtfn*; do [ -e "$vf" ] && readlink -f "$vf" | xargs basename; done | sort -u); echo "===== Used VFs under $pf ====="; used_under_pf=$(comm -12 <(echo "$all_vfs") <(echo "$used_vfs")); [ -n "$used_under_pf" ] && echo "$used_under_pf" || echo "No used VFs under $pf"; echo; echo "===== Available VFs under $pf ====="; available_vfs=$(comm -23 <(echo "$all_vfs") <(echo "$used_vfs")); [ -n "$available_vfs" ] && echo "$available_vfs" || echo "No available VFs under $pf"; echo; done
===== Used VFs under eno2 =====
0000:3a:10.1
0000:3a:10.3
0000:3a:10.5
0000:3a:10.7
0000:3a:11.1
0000:3a:11.3
0000:3a:11.5

===== Available VFs under eno2 =====
0000:3a:11.7

===== Used VFs under eno3 =====
0000:82:10.0
0000:82:10.2
0000:82:10.4
0000:82:10.6

===== Available VFs under eno3 =====
0000:82:11.0
0000:82:11.2
0000:82:11.4
0000:82:11.6

Select Add Hardware and then PCI Host Device.
Select the PCI Host Device that matches an available VF BDF number that corresponds to the physical interface where the RTP Proxy will communicate with the Session Border Controller.
Select Add Hardware and then PCI Host Device.
Select the PCI Host Device that matches an available VF BDF number that corresponds to the physical interface where the RTP Proxy will communicate with the Media Flow Engines.

Install the Operating System

Select Begin Installation to start the installation of Oracle Linux.
During installation, assign the IPs to both the VF interfaces such that they are L2 adjacent to the external SBC and MFE sides respectively.
Once the installation has completed, apply all updates.
```
sudo dnf update -y
```
Shut down the VM.

Align VM Resources by NUMA Node

The guest VM configuration must remain aligned with the host NUMA node selected during PF selection. For example, if the RTP Proxy VFs are created from PFs on host NUMA node 0:

Pin the guest vCPUs to host CPUs on NUMA node 0.
Pin the guest memory to NUMA node 0 by using <numatune>.
Allocate hugepage-backed guest memory from NUMA node 0 by using <memoryBacking>.

This end-to-end NUMA alignment keeps NIC access, guest CPU execution, and guest memory allocation local to the same NUMA node, reducing cross-node memory access and improving performance.

Use the command lscpu | grep NUMA to identify which CPUs to use for this VM.

Assume the following NUMA layout:

NUMA node(s):                            2
NUMA node0 CPU(s):                       0-21,44-65
NUMA node1 CPU(s):                       22-43,66-87

Use the command virsh edit <guest-vm-name> to map the VM's virtual CPUs to the host's CPUs.

Below is an sample. Adjust the CPU ranges based on your system topology.

  <vcpu placement='static'>16</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='7'/>
    <vcpupin vcpu='2' cpuset='8'/>
    <vcpupin vcpu='3' cpuset='9'/>
    <vcpupin vcpu='4' cpuset='10'/>
    <vcpupin vcpu='5' cpuset='11'/>
    <vcpupin vcpu='6' cpuset='12'/>
    <vcpupin vcpu='7' cpuset='13'/>
    <vcpupin vcpu='8' cpuset='50'/>
    <vcpupin vcpu='9' cpuset='51'/>
    <vcpupin vcpu='10' cpuset='52'/>
    <vcpupin vcpu='11' cpuset='53'/>
    <vcpupin vcpu='12' cpuset='54'/>
    <vcpupin vcpu='13' cpuset='55'/>v
    <vcpupin vcpu='14' cpuset='56'/>
    <vcpupin vcpu='15' cpuset='57'/>
    <emulatorpin cpuset='6-13,50-57'/>
  </cputune>

Restrict your VM memory to a single NUMA node.
For example, if this VM should be restricted to node 0, use the following <numatune> configuration
```
<numatune>
    <memory mode='strict' nodeset='0' />
</numatune>
```

Enable hugepage-backed memory on the VM.

<memoryBacking>
  <hugepages>
    <page size='2048' unit='KiB'/>
  </hugepages>
</memoryBacking>

Enable vIOMMU for the guest.

<iommu model='intel'>
  <driver intremap='on' caching_mode='on'/>
</iommu>

Start the Virtual Machine

Start the virtual machine.
```
virsh start <guest-vm-name>
```
On the guest VM, enable console access from the KVM host.
```
sudo systemctl enable --now serial-getty@ttyS0
```
From the KVM host, connect to the virtual machine.
```
virsh console <guest-vm-name>
```

Prepare the Guest OS

Enable IOMMU in the guest's kernel parameters.
The following command reserves 14 of the guest's CPUs for the RTP Proxy and leaves 2 for housekeeping.
```
sudo grubby --update-kernel=ALL --args="isolcpus=2-15 nohz_full=2-15 rcu_nocbs=2-15 intel_iommu=on iommu=pt"
```
Reboot the VM.
Confirm IOMMU is enabled.
The output of the following command should list several devices.
```
find /sys/kernel/iommu_groups/ -type l
```
If no devices are listed, IOMMU is not enabled within the VM.
Set up hugepages within the VM.

Note:
The entire guest VM memory should be backed by the hugepage of the host machine. If the guest VM is configured with 16GB memory, the entire 16GB of memory must be reserved as hugepages in the host machine. Further, for optimal performance, Oracle recommends reserving around 25% of the total RAM for hugepages within the VM. For example, if the guest VM has 16GB RAM, Oracle recommends reserving at least 4GB of memory for hugepages within the VM.
The following example creates 2048 2 MB hugepages and persists the setting across reboots.
1. Reserve 2048 hugepages (2 MB each = 4 GB total).
```
echo 2048 | sudo tee /proc/sys/vm/nr_hugepages
```
2. Make the hugepages configuration persistent across reboots.
```
echo "vm.nr_hugepages = 2048" | sudo tee /etc/sysctl.d/99-hugepages.conf
sudo sysctl --system
```
3. Create mount point for hugepages filesystem.
```
sudo mkdir -p /dev/hugepages
```
4. Configure hugetlbfs mount.
```
echo "nodev /dev/hugepages hugetlbfs defaults 0 0" | sudo tee -a /etc/fstab
```
5. Mount the hugepages filesystem.
```
sudo mount -a
```
6. Reload systemd daemon.
```
sudo systemctl daemon-reload
```
7. Verify the hugepages are configured.
```
grep -i huge /proc/meminfo
```
  If the hugepages are configured, the output of grep should be similar to the following:
```
HugePages_Total:    2048
HugePages_Free:     2048
```

Apply the network-latency tuned profile.

sudo dnf install -y tuned-profiles-cpu-partitioning
sudo tuned-adm profile network-latency

Install required packages.

sudo dnf install -y libibverbs rdma-core librdmacm pciutils ethtool

Disable IRQ balance.
```
sudo systemctl disable --now irqbalance
```