1. Monitoring and Tuning the System
  2. Working With OSWatcher Black Box

3 Working With OSWatcher Black Box

Oracle OSWatcher Black Box (OSWbb) collects and archives OS and network metrics that you can use to diagnose performance issues. OSWbb operates as a set of background processes on the server and gathers data often, invoking such UNIX utilities as vmstat, mpstat, netstat, iostat, and top.

OSWbb is useful for Oracle RAC (Real Application Clusters) and Oracle Grid Infrastructure configurations. The RAC-DDT (Diagnostic Data Tool) script file includes OSWbb, but doesn't install it by default.

Installing OSWbb

To install OSWbb:

  1. Sign in to My Oracle Support (MOS) at https://support.oracle.com.

  2. Download OSWatcher from the link that is listed by Doc ID 301137.1 at https://support.oracle.com/epmos/faces/DocumentDisplay?id=301137.1.

  3. Copy the file to the directory that you want to install OSWbb, then run the following command: 

    tar xvf oswbbVERS.tar

    In the previous command, VERS represents the version number of OSWatcher, for example 832 for OSWatcher 8.32.

    Extracting the tar file creates a directory named oswbb, which contains all the directories and files that are associated with OSWbb, including the startOSWbb.sh script.

  4. To enable the collection of iostat information for NFS volumes, edit the OSWatcher.sh script in the oswbb directory, and set the value of nfs_collect to 1 as follows: 

    nfs_collect=1

Running OSWbb

To start OSWbb, run the startOSWbb.sh script from the oswbb directory. 

sudo ./startOSWbb.sh [frequency duration]

The optional frequency and duration arguments specify how often in seconds OSWbb collects data and the number of hours for which OSWbb runs. The default values are 30 seconds and 48 hours. The following example starts OSWbb recording data at intervals of 60 seconds, and has it record data for 12 hours: 

sudo ./startOSWbb.sh 60 12
...
Testing for discovery of OS Utilities...
VMSTAT found on your system.
IOSTAT found on your system.
MPSTAT found on your system.
IFCONFIG found on your system.
NETSTAT found on your system.
TOP found on your system.

Testing for discovery of OS CPU COUNT
oswbb is looking for the CPU COUNT on your system
CPU COUNT will be used by oswbba to automatically look for cpu problems

CPU COUNT found on your system.
CPU COUNT = 4

Discovery completed.

Starting OSWatcher Black Box v7.3.0  on date and time
With SnapshotInterval = 60
With ArchiveInterval = 12
...
Data is stored in directory: OSWbba_archive

Starting Data Collection...

oswbb heartbeat: date and time
oswbb heartbeat: date and time + 60 seconds
...

In the previous output, OSWbba_archive is the path of the archive directory that contains the OSWbb log files.

To stop OSWbb prematurely, run the stopOSWbb.sh script from the oswbb directory: 

sudo ./stopOSWbb.sh

OSWbb collects data in the directories that are under the oswbb/archive directory, which are described in the following table.

Directory Description

oswifconfig

Contains output from ifconfig.

oswiostat

Contains output from iostat.

oswmeminfo

Contains a listing of the contents of /proc/meminfo.

oswmpstat

Contains output from mpstat.

oswnetstat

Contains output from netstat.

oswprvtnet

If you have enable private network tracing for RAC, contains information about the status of the private networks.

oswps

Contains output from ps.

oswslabinfo

Contains a listing of the contents of /proc/slabinfo.

oswtop

Contains output from top.

oswvmstat

Contains output from vmstat.

OSWbb stores data in hourly archive files, which are named system_name_utility_name_timestamp.dat. Each entry in a file is preceded by a timestamp.

Analyzing OSWbb Archived Files

You can use the OSWbb analyzer (OSWbba) to provide information about system slowdowns, system delays, and other performance problems. You can also use OSWbba to graph data that's collected from the iostat, netstat, and vmstat utilities. OSWbba requires that you have Java version 1.4.2 or a later version installed on the system.

You can download a Java RPM for Linux by visiting http://www.java.com, or you can install Java by using the dnf command: 

sudo dnf install java-1.8.0-jdk

Run OSWbba from the oswbb directory as follows: 

sudo java -jar oswbba.jar -i OSWbba_archive

In the previous command, OSWbba_archive is the path of the archive directory that contains the OSWbb log files.

You can use OSWbba to display the following types of performance graph:

  • Process run, wait, and block queues.

  • CPU time spent running in system, user, and idle mode.

  • Context switches and interrupts.

  • Free memory and available swap.

  • Reads each second, writes each second, service time for I/O requests, and percentage usage of bandwidth for a specified block device.

You can also use OSWbba to save the analysis to a report file, which reports instances of system slowdown, spikes in run queue length, or memory shortage, describes probable causes, and offers suggestions of how to improve performance. 

sudo java -jar oswbba.jar -i OSWbba_archive -A

For more information about OSWbb and OSWbba, refer to the OSWatcher Black Box User Guide (Article ID 301137.1) and the OSWatcher Black Box Analyzer User Guide (Article ID 461053.1) on My Oracle Support (MOS) at https://support.oracle.com.