Oracle Cloud Infrastructure Documentation

Collecting Debugging Information with sos

The sos command collects information about a system such as hardware configuration, software configuration, and operational state. You can also use the sos report command to enable diagnostics and analytical functions on the current system.

  • System configuration files
  • Hardware information
  • Running processes and services
  • Kernel details
  • Log files
  • Output from specific system commands

The generated report is useful in cases where you're being helped by Oracle Support in troubleshooting a problem in the system. The support representative can use the report to obtain an exact picture of the system, its resources, and all the applications and processes that exist in the system, and all other data that can help find the causes of the issues you're experiencing.

The sos utility requires the installation of the sos package. In Oracle Linux platform images, the sos utility is installed by default.

If you're using another Linux image, verify whether the sos utility is installed.

  1. Check to see if the sos utility is installed:
    rpm -q sos

    If the sos utility is installed, the command returns the version installed. For example:

    sos-4.7.2-2.0.2.el8_10.noarch
  2. If the sos utility isn't present, install it through the package manager. For example:
    sudo dnf install sos

sos Command Reference

This table provides information about the sos command.

Action Command Description
Create the sos report. sos report

Collects all diagnostic and configuration information from the system and its installed applications.
Hide sensitive information from the sos report. sos clean

Obfuscates information in an existing report before it's supplied to Oracle Support.

To obtain a list of options and arguments that you can use with the sos utility, run the following command:

sos report -h
optional arguments:
  -h, --help            show this help message and exit
                
Global Options:
  --batch               Do not prompt interactively
  --config-file CONFIG_FILE
                        specify alternate configuration file
...
Note

For more information, type man sos or see the sos user documentation.

Creating the sos Report

To collect diagnostic and configuration information from the system and its installed applications, generate a report by running the sos report command as root. If you're generating the report as related to a specific troubleshooting case, press Enter to continue and enter the case ID.

sudo sos report

sos report (version 4.8.1)

This command will collect diagnostic and configuration information from
this Oracle Linux system and installed applications.

An archive containing the collected information will be generated in
/var/tmp/sos.s4zah5s1 and may be provided to a Oracle America support
representative.

Any information provided to Oracle America will be treated in accordance
with the published support policies at:

        Distribution Website : https://support.oracle.com/
        Commercial Support   : https://support.oracle.com/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.
No changes will be made to system configuration.
Press ENTER to continue, or CTRL-C to quit.
Optionally, please enter the case id that you are generating this report for []:

The report is generated as an xa-compressed tar file in the /var/tmp directory. In the report's file name, the ID is dynamically created by the utility.

Important

When you engage Oracle Support to diagnose and troubleshoot issues that you have observed in the system, be aware that the report contains sensitive information specific to your company. Ensure that you review the contents of the report and identify sensitive information before sending the report to any third-party. See Hiding Sensitive Information in an sos Report.

Depending on the packages installed and your system's load, the report can take a long time to complete.

  • If the system has a lot of packages installed, use the following command to disable rpm database verification:
    sos report -k rpm.rpmva=off
  • If the system is heavily loaded, use the following command to disable the kernel.sysrq option:
    sos report -k kernel.sysrq=off

After you have provided information as prompted, the command proceeds to generate the report, which can take a considerable time to complete. At the end of the process, the screen displays a message similar to the following: 

Your sosreport has been generated and saved in:
/var/tmp/sosreport-hostname-case#-datestamp-ID.tar.xz

 Size   20.62MiB
 Owner  root
 sha256 428f7b4118acd2d349bb022946877d853aa0eefbb4d340af3839810dc634b8b7

Please send this file to your support representative.
...
Running plugins. Please wait ...
  Finishing plugins              [Running: unpackaged]
  Finished running plugins
Creating compressed archive...
Your sosreport has been generated and saved in:
        /var/tmp/sosreport-<user_name>-<os>-<date>.tar.xz

 Size   15.72MiB
 Owner  root
 sha256 0fcfef63887d76ebd5b831be5d93d7953b0b9a431d81c9665459fbc8ce4a5f16

Please send this file to your support representative.

Extra Usages of the sos Command

Customize the output of sos reports by using extra sos command options.

The sos report command can also be used with other options. For example, to only list available plugins and plugin options in the report. You can also obtain only information specific to a problem area and specify options to tailor the report that's generated.

  • To only list available plugins and plugin options in the report:
    sudo sos report -l

    The plugins that are displayed by the command are grouped according to the following sections:

    • All enabled plugins
    • All disabled plugins
    • Available options for all the plugins
    • Available plugin options

    See the sos user documentation about how to enable or disable plugins and how to set values for plugin options.

  • To record only information about Apache and Tomcat and to gather all the Apache logs:
    sudo sos report -o apache,tomcat -k apache.log=on
  • To enable all the Boolean options for all the loaded plugins (excluding the rpm.rpmva plugin) and verify all packages:
    sudo sos report -a -k rpm.rpmva=off

Hiding Sensitive Information in an sos Report

Obfuscate information in an sos report before supplying it to Oracle Support.

To secure sensitive information before sending the report externally, you can use the clean functionality of the sos utility. This functionality tries to obfuscate any information in the report that's considered to be sensitive, such as the following information:

  • IPv4 addresses and networks (network topologies are retained)

  • MAC addresses

  • Host names

  • Usernames

  • Any words or phrases that you specify with the --keyword option

To use the sos clean utility on a generated report, type the following command and follow the prompts that are displayed:

sudo sos clean /var/tmp/sosreport-hostname-case#-datestamp-ID.tar.xz
...
Users should review any resulting data and/or archives generated or processed by
this utility for remaining sensitive content before being passed to a third
party.

Press ENTER to continue, or CTRL-C to quit.

At the end of the process, the screen displays a message similar to the following:

Successfully obfuscated 1 report(s)

A mapping of obfuscated elements is available at
	/var/tmp/sosreport-<host0-2022-08-08-qxbegcn>-private_map

The obfuscated archive is available at
	/var/tmp/sosreport-<host0-2022-08-08-qxbegcn-obfuscated>.tar.xz

	Size	3.62MiB
	Owner	root

Please send the obfuscated archive to your support representative and keep the mapping file private

The resulting report that has been scrubbed of sensitive information is also stored in /var/tmp. However, the file name itself is revised. The hostname is generic, and importantly, obfuscated is added to the file name so you can identify the clean version of the report.

Caution

Consider the following about the sos clean utility:

  • The clean functionality is a best-effort method to identify and then mask sensitive information. However, sos clean doesn't guarantee that the coverage of the masking process is complete in a specific system.

  • Reports that are processed with the sos clean command obfuscate certain details which a third-party such as a support representative might need to provide better help when troubleshooting problems.

  • You must always audit archives and reports that are generated by the sos utility before sending any of these files externally.

To automatically clean any sos report that you create, use the following command syntax when generating a report:

sudo sos report --clean

For more information, type man sos or see the sos user documentation.

Reviewing sosreport Results

Configure and review the collection of debugging information on Oracle Linux.

The sos command is automatically configured to collect hardware information, system configuration files, and log data. You can enable and disable modules to match data protection requirements.

Note

The module information that's provided in this table relates to sos 4.8. To verify the modules you have installed, run the sos report command. The output includes the version of the sos utility that you're running.

Disabling modules prevents the sos command from collecting certain details that might be needed for advanced troubleshooting, such as networking information.

Module

Information Type

Included Files

anaconda

Installation log files

  • /root/install.log

  • /root/install.log.syslog

  • /var/log/anaconda

  • /var/log/anaconda.*

auditd

Audit log files

  • /etc/audit/auditd.conf

  • /etc/audit/audit.rules

  • /var/log/audit/*

boot

System boot process details

  • /etc/milo.conf

  • /etc/silo.conf

  • /boot/efi/efi/redhat/elilo.conf

  • /etc/yaboot.conf

  • /boot/yaboot.conf

cron

Root user cron commands

  • /etc/cron*

  • /etc/crontab

  • /var/log/cron

  • /var/spool/cron

cups

Printer log files

  • /etc/cups/*.conf

  • /etc/cups/*.types

  • /etc/cups/lpoptions

  • /etc/cups/ppd/*.ppd

  • /var/log/cups/*

date

Context data

  • /etc/localtime

devicemapper

Hardware details

filesys

List of all files in use

  • /proc/fs/*

  • /proc/mounts

  • /proc/filesystems

  • /proc/self/mounts

  • /proc/self/mountinfo

  • /proc/self/mountstats

  • /proc/[0-9]*/mountinfo

  • /etc/mtab

  • /etc/fstab

grub2

Kernel and system start-up configuration

  • /boot/efi/EFI/*/grub.cfg

  • /boot/grub2/grub.cfg

  • /boot/grub2/grubenv

  • /boot/grub/grub.cfg

  • /boot/loader/entries

  • /etc/default/grub

  • /etc/grub2.cfg

  • /etc/grub.d/*

hardware

Hardware details

  • /proc/interrupts

  • /proc/irq

  • /proc/dma

  • /proc/devices

  • /proc/rtc

  • /var/log/mcelog

  • /sys/class/dmi/id/*

  • /sys/class/drm/*/edid

host

Host identification

  • /etc/sos.conf

  • /etc/hostid

kernel

System log files

  • /etc/conf.modules

  • /etc/modules.conf

  • /etc/modprobe.conf

  • /etc/modprobe.d

  • /etc/sysctl.conf

  • /etc/sysctl.d

  • /lib/modules/*/modules.dep

  • /lib/sysctl.d

  • /proc/cmdline

  • /proc/driver

  • /proc/kallsyms

  • /proc/lock*

  • /proc/buddyinfo

  • /proc/misc

  • /proc/modules

  • /proc/slabinfo

  • /proc/softirqs

  • /proc/sys/kernel/random/boot_id

  • /proc/sys/kernel/tainted

  • /proc/timer*

  • /proc/zoneinfo

  • /sys/firmware/acpi/*

  • /sys/kernel/debug/tracing/*

  • /sys/kernel/livepatch/*

  • /sys/module/*/parameters

  • /sys/module/*/initstate

  • /sys/module/*/refcnt

  • /sys/module/*/taint

  • /sys/module/*/version

  • /sys/devices/system/clocksource/*/available_clocksource

  • /sys/devices/system/clocksource/*/current_clocksource

  • /sys/fs/pstore

  • /var/log/dmesg

libraries

List of shared libraries

  • /etc/ld.so.conf

  • /etc/ld.so.conf.d/*

logs

System log files

  • /etc/syslog.conf

  • /etc/rsyslog.conf

  • /etc/rsyslog.d

  • /run/log/journal/*

  • /var/log/auth.log

  • /var/log/auth.log.1

  • /var/log/auth.log.2*

  • /var/log/boot.log

  • /var/log/dist-upgrade

  • /var/log/installer

  • /var/log/journal/*

  • /var/log/kern.log

  • /var/log/kern.log.1

  • /var/log/kern.log.2*

  • /var/log/messages*

  • /var/log/secure*

  • /var/log/syslog

  • /var/log/syslog.1

  • /var/log/syslog.2*

  • /var/log/udev

  • /var/log/unattended-upgrades

lvm2

Hardware details

memory

Hardware details

  • /proc/pci

  • /proc/meminfo

  • /proc/vmstat

  • /proc/swaps

  • /proc/slabinfo

  • /proc/pagetypeinfo

  • /proc/vmallocinfo

  • /sys/kernel/mm/ksm

  • /sys/kernel/mm/transparent_hugepage/enabled

networking

Network identification

  • /etc/dnsmasq*

  • /etc/host*

  • /etc/inetd.conf

  • /etc/iproute2

  • /etc/network*

  • /etc/nftables

  • /etc/nftables.conf

  • /etc/nsswitch.conf

  • /etc/resolv.conf

  • /etc/sysconfig/nftables.conf

  • /etc/xinetd.conf

  • /etc/xinetd.d

  • /etc/yp.conf

  • /proc/net/*

  • /sys/class/net/*/device/numa_node

  • /sys/class/net/*/flags

  • /sys/class/net/*/statistics/*

pam

Sign-in security settings

  • /etc/pam.d/*

  • /etc/security

pci

Hardware details

  • /proc/bus/pci

  • /proc/iomem

  • /proc/ioports

process

List of all running processes and process details

  • /proc/sched_debug

  • /proc/stat

  • /proc/[0-9]*/smaps

processor

Hardware details

  • /proc/cpuinfo

  • /sys/class/cpuid

  • /sys/devices/system/cpu

rpm

Installed software packages

  • /var/lib/rpm/*

  • /var/log/rpmpkgs

sar

Resource and usage data

  • /var/log/sa/*

selinux

Security settings

  • /etc/sestatus.conf

  • /etc/selinux

  • /var/lib/selinux

services

All defined system services

  • /etc/inittab

  • /etc/rc.d/*

  • /etc/rc.local

ssh

SSH configuration

  • /etc/ssh/ssh_config

  • /etc/ssh/sshd_config

x11

GUI logs for the X Window System

  • /etc/X11/*

  • /var/log/Xorg.*.log

  • /var/log/Xorg.*.log.old

  • /var/log/XFree86.*.log

  • /var/log/XFree86.*.log.old

yum

Installed software packages

  • /etc/pki/consumer/cert.pem

  • /etc/pki/entitlement/*.pem

  • /etc/pki/product/*.pem

  • /etc/yum/*

  • /etc/yum.repos.d/*

  • /etc/yum/pluginconf.d/*

  • /var/log/dnf.log