Working With the drgn Kernel Debugging Utility

9 Working With the drgn Kernel Debugging Utility

Drgn is a tool and a programming library that can be used to extract debug information from both the live kernel of the running machine, and memory crash dumps from halted systems (vmcore).

To configure an Oracle Linux system to generate vmcore crash dumps, follow the instructions in Working With Kernel Dumps.

Drgn can be used as part of a root cause analysis to provide extra metrics that aren't already exposed through existing dashboards and interfaces.

For more information, see https://drgn.readthedocs.io/.

(Optional) Installing DebugInfo Packages

You can optionally install *-debuginfo packages to add extra debugging symbols in generated core dumps. They're intended for development purposes only, so we recommend that you only install them in development environments.

Enable the Oracle Linux 9 debuginfo repository by creating the /etc/yum.repos.d/debuginfo.repo file with root privileges and the following contents:

[debuginfo]
name=Oracle Linux 9 Debuginfo Packages
baseurl=https://oss.oracle.com/ol9/debuginfo/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

Use the package manager to search for DebugInfo packages to install:
```
dnf search *-debuginfo
```

Installing drgn

If you're running Oracle Linux with the Unbreakable Enterprise Kernel (UEK), install the kernel-uek-debuginfo package by using the dnf command:
```
sudo dnf install -y kernel-uek-debuginfo-$(uname -r)
```
If you're running Oracle Linux with the Red Hat Compatible Kernel (RHCK), install the kernel-debuginfo package instead:
```
sudo dnf install -y kernel-debuginfo-$(uname -r)
```
Enable the ol9_addons repository:
```
sudo dnf config-manager --enable ol9_addons
```
For more information, see Oracle Linux: Managing Software on Oracle Linux.
Install the drgn package:
```
sudo dnf install -y drgn
```

Using the drgn Command

To debug the running kernel, use the drgn command to analyze the contents of the /proc/kcore dump file:

sudo drgn

To debug a running kernel or vmcore crash dump, specify the dump file by using the -c option. Optionally, specify the vmlinux and module symbols by also using the -s option:

sudo drgn -c path/to/dumpfile -s path/to/vmlinux

For example, to debug /proc/kcore for a live kernel and specify kernel drivers, run the following command:

sudo drgn  -c /proc/kcore -s /usr/lib/debug/lib/modules/$(uname -r)/vmlinux \
   -s /lib/modules/$(uname -r)/kernel/net/netfilter/xt_comment.ko.xz

To perform the same operation on a vmcore crash dump file:

sudo drgn -c /var/crash/127.0.0.1-2023-06-02-09:33:07/vmcore \
 -s /usr/lib/debug/lib/modules/5.15.0-101.103.2.1.el9uek.x86_64/vmlinux \
 -s /lib/modules/5.15.0-101.103.2.1.el9uek.x86_64/kernel/net/netfilter/xt_comment.ko.xz

For more information about how to use the drgn command, use the -h option:

sudo drgn -h

Using the drgn Library With Python

Unlike the crash utility, Drgn wasn't originally designed to be a standalone kernel debugging tool. Drgn is a Python programming library that exposes debugging information for scripting and review purposes.

Before you can start using drgn with Python scripts, ensure that Python is correctly installed on the system. For more information, see Oracle Linux 9: Installing and Managing Python.

The prog array variable contains the information about the kernel that you're debugging. For example, to return the data collected for slab_caches, run the following statements in the drgn shell:

prog["slab_caches"]

(struct list_head){
        .next = (struct list_head *)0xffff8b831d972260,
        .prev = (struct list_head *)0xffff8b8007c02060,
}

Standard python structures can also be used to iterate through debug information:

slab_caches = prog["slab_caches"]

slab_caches.next

*(struct list_head *)0xffff8b831d972260 = {
        .next = (struct list_head *)0xffff8b831d972460,
        .prev = (struct list_head *)slab_caches+0x0 = 0xffffffff836e3da0,
}

For more information about the drgn API and script syntax, see https://drgn.readthedocs.io/.