The next step is to make sure crash dumps are properly configured. First, confirm that dumpadm is configured to save kernel crash dumps and that savecore is enabled. See the dumpadm (1M) man page for more information on crash dump parameters.
# dumpadm Dump content : kernel with ZFS metadata Dump device : /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash Savecore enabled : yes Save compressed : on
Starting with the Oracle Solaris 11.2 release, dump content is organized into sections. By default, dump content includes following two sections:
Core kernel pages
ZFS metadata pages
You can optionally disable dumping of ZFS metadata pages by running the dumpadm command as shown in the following example:
# dumpadm -c kernel-zfs Dump content : kernel without ZFS metadata Dump device : /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash Savecore enabled : yes Save compressed : on
See the dumpadm (1M) man page for further details about other optional components of dump content that can be configured. In rest of the example, we assume default dump content configuration, that is, dump content includes both core kernel pages and ZFS metadata pages.
Next, reboot the system using the –d flag to reboot (1M) , which forces the kernel to panic and save a crash dump.
# reboot -d Oct 15 12:54:30 testsystem reboot: initiated by jack on /dev/console updating /platform/sun4v/boot_archive panic[cpu101]/thread=4c078b08f80: forced crash dump initiated at user request 000002a10a3b7930 genunix:kadmin+600 (fc, 0, 10, 4, 5, 1) %l0-3: 00000000012ec6f8 00000000012ec400 0000000000000004 0000000000000004 %l4-7: 00000000000005cc 0000000000000010 0000000000000004 0000000000000004 000002a10a3b7a00 genunix:uadmin+1d0 (1, 4c07a1b5088, 0, 6d7000, ff00, 5) %l0-3: 0000040000923280 000000000003787c 0000000000000004 000000000003787c %l4-7: 000000000003787b 0000000000000000 0000000000000000 0000000000000000 syncing file systems... done dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel sections: zfs 0:10 96% done (kernel) 0:11 100% done (zfs) 100% done: 404632 (kernel) + 14302 (zfs) pages dumped, dump succeeded rebooting... Resetting...
When the system reboots, savecore runs automatically to preserve each section of the crash dump in a separate file. In this example, we have two sections in the dump: core kernel and ZFS metadata. Therefore, two compressed dump files are produced when savecore finishes. The two files are vmdump.n and vmdump-zfs.n.
When finished, a message similar to the following is displayed on the system console:
Oct 15 12:57:42 testsystem savecore: Decompress all crash dump files with '(cd /var/crash/data/ cbc9822c-2f13-63c6-d440-d2f118516775 && savecore -v 0)' or individual files with 'savecore -vf /var/crash/data/ cbc9822c-2f13-63c6-d440-d2f118516775/vmdump{,-<secname>}.0'
If the message does not appear immediately, check if savecore(1M) is still running.
# pgrep savecore 864 # cd /var/crash/ # ls 0 bounds data #ls -l 0 lrwxrwxrwx 1 root root 41 Oct 15 12:57 0 -> data/cbc9822c-2f13-63c6-d440-d2f118516775 # ls data/cbc9822c-2f13-63c6-d440-d2f118516775 vmdump-zfs.0 vmdump.0 #
savecore performs the following two tasks:
Creates a sub-directory with name data/uuid under configured save directory and produces dump files in that sub-directory. In the above example, cbc9822c-2f13-63c6-d440-d2f118516775 is the uuid of the operating system image for which crash dump is generated.
Creates a symbolic link with numerical suffix as its name to data/uuid directory. In above example 0 is the name of symbolic link created by savecore.
If your dump directory contains no dump files, that partition might be out of space. You can free up space and run savecore (1M) manually as root to save the dump.
If your dump directory contains multiple crash dumps, the ones you just created will be in one of the following file formats with the most recent modification time:
vmcore.n
vmcore-<section>.n
vmdump.n
vmdump-<section>n