Forcing a Crash Dump

The next step is to make sure crash dumps are properly configured. First, confirm that dumpadm is configured to save kernel crash dumps and that savecore is enabled. See the dumpadm (1M) man page for more information on crash dump parameters.

# dumpadm
Dump content      : kernel with ZFS metadata
Dump device       : /dev/zvol/dsk/rpool/dump (dedicated)
Savecore directory: /var/crash
Savecore enabled  : yes
Save compressed   : on

Starting with the Oracle Solaris 11.2 release, dump content is organized into sections. By default, dump content includes following two sections:

• Core kernel pages

• ZFS metadata pages

You can optionally disable dumping of ZFS metadata pages by running the dumpadm command as shown in the following example:

# dumpadm -c kernel-zfs
Dump content      : kernel without ZFS metadata
Dump device       : /dev/zvol/dsk/rpool/dump (dedicated)
Savecore directory: /var/crash
Savecore enabled  : yes
Save compressed   : on

See the dumpadm (1M) man page for further details about other optional components of dump content that can be configured. In rest of the example, we assume default dump content configuration, that is, dump content includes both core kernel pages and ZFS metadata pages.

Next, reboot the system using the –d flag to reboot (1M) , which forces the kernel to panic and save a crash dump.

# reboot -d
Oct 15 12:54:30 testsystem reboot: initiated by jack on /dev/console
updating /platform/sun4v/boot_archive

panic[cpu101]/thread=4c078b08f80: forced crash dump initiated at user request

000002a10a3b7930 genunix:kadmin+600 (fc, 0, 10, 4, 5, 1)
  %l0-3: 00000000012ec6f8 00000000012ec400 0000000000000004 0000000000000004
  %l4-7: 00000000000005cc 0000000000000010 0000000000000004 0000000000000004
000002a10a3b7a00 genunix:uadmin+1d0 (1, 4c07a1b5088, 0, 6d7000, ff00, 5)
  %l0-3: 0000040000923280 000000000003787c 0000000000000004 000000000003787c
  %l4-7: 000000000003787b 0000000000000000 0000000000000000 0000000000000000

syncing file systems... done
dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel 
sections: zfs
0:10  96% done (kernel)
0:11 100% done (zfs)
100% done: 404632 (kernel) + 14302 (zfs) pages dumped, dump succeeded
rebooting...
Resetting...

When the system reboots, savecore runs automatically to preserve each section of the crash dump in a separate file. In this example, we have two sections in the dump: core kernel and ZFS metadata. Therefore, two compressed dump files are produced when savecore finishes. The two files are vmdump.n and vmdump-zfs.n.

When finished, a message similar to the following is displayed on the system console:

Oct 15 12:57:42 testsystem savecore: Decompress all crash dump files with
'(cd /var/crash/data/
cbc9822c-2f13-63c6-d440-d2f118516775 && savecore -v 0)' 
or individual files with
'savecore -vf /var/crash/data/
cbc9822c-2f13-63c6-d440-d2f118516775/vmdump{,-<secname>}.0'

If the message does not appear immediately, check if savecore(1M) is still running.

# pgrep savecore
 864
# cd /var/crash/
# ls
  0 bounds  data
#ls -l 0
lrwxrwxrwx 1 root root 41 Oct 15 12:57 0 -> data/cbc9822c-2f13-63c6-d440-d2f118516775
# ls data/cbc9822c-2f13-63c6-d440-d2f118516775
  vmdump-zfs.0  vmdump.0
#

savecore performs the following two tasks:

• Creates a sub-directory with name data/uuid under configured save directory and produces dump files in that sub-directory. In the above example, cbc9822c-2f13-63c6-d440-d2f118516775 is the uuid of the operating system image for which crash dump is generated.

• Creates a symbolic link with numerical suffix as its name to data/uuid directory. In above example 0 is the name of symbolic link created by savecore.