Forcing a Crash Dump
The next step is to make sure crash dumps are properly configured. First, confirm that dumpadm
is configured to save kernel crash dumps and that savecore
is enabled. See the
dumpadm
(8) man page for more information about crash dump parameters.
# dumpadm Dump content : kernel with ZFS metadata Dump device : /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash Savecore enabled : yes Save compressed : on
Starting with the Oracle Solaris 11.2 release, dump content is organized into sections. By default, dump content includes following two sections:
-
Core kernel pages
-
ZFS metadata pages
Note:
The line labeledDump content
in the above dumpadm
output shows kernel with ZFS metadata
.
You can optionally disable dumping of ZFS metadata pages by running the dumpadm
command as shown in the following example:
# dumpadm -c kernel-zfs Dump content : kernel without ZFS metadata Dump device : /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash Savecore enabled : yes Save compressed : on
See the
dumpadm
(8) man page for further details about other optional components of dump content that can be configured. In rest of the example, we assume default dump content configuration, that is, dump content includes both core kernel pages and ZFS metadata pages.
Next, reboot the system using the -d
flag to
reboot
(8), which forces the kernel to panic and save a crash dump.
# reboot -d Oct 15 12:54:30 testsystem reboot: initiated by jack on /dev/console updating /platform/sun4v/boot_archive panic[cpu101]/thread=4c078b08f80: forced crash dump initiated at user request 000002a10a3b7930 genunix:kadmin+600 (fc, 0, 10, 4, 5, 1) %l0-3: 00000000012ec6f8 00000000012ec400 0000000000000004 0000000000000004 %l4-7: 00000000000005cc 0000000000000010 0000000000000004 0000000000000004 000002a10a3b7a00 genunix:uadmin+1d0 (1, 4c07a1b5088, 0, 6d7000, ff00, 5) %l0-3: 0000040000923280 000000000003787c 0000000000000004 000000000003787c %l4-7: 000000000003787b 0000000000000000 0000000000000000 0000000000000000 syncing file systems... done dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel sections: zfs 0:10 96% done (kernel) 0:11 100% done (zfs) 100% done: 404632 (kernel) + 14302 (zfs) pages dumped, dump succeeded rebooting... Resetting...
When the system reboots, savecore
runs automatically to preserve each section of the crash dump in a separate file. In this example, we have two sections in the dump: core kernel and ZFS metadata. Therefore, two compressed dump files are produced when savecore
finishes. The two files are vmdump.
n and vmdump-zfs.
n.
When finished, a message similar to the following is displayed on the system console:
Oct 15 12:57:42 testsystem savecore: Decompress all crash dump files with '(cd /var/crash/data/ cbc9822c-2f13-63c6-d440-d2f118516775 && savecore -v 0)' or individual files with 'savecore -vf /var/crash/data/ cbc9822c-2f13-63c6-d440-d2f118516775/vmdump{,-<secname>}.0'
If the message does not appear immediately, check if savecore
is still running.
# pgrep savecore 864 # cd /var/crash/ # ls 0 bounds data # ls -l 0 lrwxrwxrwx 1 root root 41 Oct 15 12:57 0 -> data/cbc9822c-2f13-63c6-d440-d2f118516775 # ls data/cbc9822c-2f13-63c6-d440-d2f118516775 vmdump-zfs.0 vmdump.0
savecore
performs the following two tasks:
-
Creates a sub-directory with name
data
/uuid
under configured save directory and produces dump files in that sub-directory. In the above example,cbc9822c-2f13-63c6-d440-d2f118516775
is theuuid
of the operating system image for which crash dump is generated. -
Creates a symbolic link with numerical suffix as its name to
data
/uuid
directory. In above example0
is the name of symbolic link created bysavecore
.
Note:
If your dump directory contains no dump files, that partition might be out of space. You can free up space and runsavecore
(8) manually as root to save the dump.
If your dump directory contains multiple crash dumps, the ones you just created will be in one of the following file formats with the most recent modification time:
-
vmcore.
n -
vmcore-
<section>.
n -
vmdump.
n -
vmdump-
<section>n