Mellanox ConnectX adapter not detected at boot

On systems that are using the Mellanox ConnectX adapters, the driver does not load the InfiniBand and RMDA modules at boot time, which results in a failure to detect the adapter when using RDMA and InfiniBand-related tools, such as the ibstat command.

Errors similar to the following are typically displayed:

ibpanic: [26013] main: stat of IB device 'mthca0' failed: No such file or directory

This issue occurs because although the mlx4_core and mlx5_core drivers are included in the initramfs to facilitate a PXE boot, the InfiniBand and RDMA modules are not included. If you need the driver for a PXE boot, you can reload it manually after booting, which will trigger the RDMA hotplug sequence, for example:

# modprobe mlx5_core
               

If you do not require the mlx4_core or mlx5_core driver for a PXE boot, you can remove these drivers from the initramfs, as they are loaded after boot. Then, the RDMA hotplug sequence is triggered normally.

To remove the drivers from the initramfs, create the /etc/dracut.conf.d/10-mlx_dracut-denylist.conf file and then add the following line:

omit_drivers+=" mlx4_* mlx5_* mlxfw "

After you have updated the file, rebuild the initramfs by running the following command:

# dracut -f
               

Reboot the system for the changes to take effect.

(Bug ID 31353413)