1.1 Quick Start Guide

This section explains how to install and configure Oracle ORAchk and Oracle EXAchk. Review other topics in this section for additional information.

1.1.1 Overview of Oracle ORAchk and Oracle EXAchk

Oracle ORAchk and Oracle EXAchk provide a lightweight and non-intrusive health check framework for the Oracle stack of software and hardware components.

If you have an engineered system other than Oracle Database Appliance, then use Oracle EXAchk. For all other systems, use Oracle ORAchk.

You have access to Oracle ORAchk and Oracle EXAchk as a value add-on to your existing support contract. There is no additional fee or license required to run Oracle ORAchk and Oracle EXAchk.

Features of Oracle ORAchk and Oracle EXAchk

  • Automates risk identification and proactive notification before business is impacted

  • Runs health checks based on critical and reoccurring problems

  • Runs in your environment with no need to send anything to Oracle

  • Enables you to schedule email health check reports

  • Integrates the findings into other tools of your choice

1.1.2 Prerequisites

Review the checklist for Bash requirements, SSH connectivity, and required user privileges to run health checks.

  • Bash Requirements
    Oracle ORAchk and Oracle EXAchk are written largely in Bash, and therefore Bash 3.2 or later is required to run Oracle ORAchk and Oracle EXAchk.
  • SSH Connectivity and Access
    In a clustered database environment, Oracle ORAchk and Oracle EXAchk run health checks on a single node and remotely run on all other cluster nodes. Remotely running health checks on cluster nodes involves remotely copying files to and from the targets and running commands without providing the passwords.
  • Handling of root Passwords
    Handling of root passwords depends on whether you have installed the Expect utility. Expect is a tool for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, and so on.
  • Deciding Which User Should Run Oracle ORAchk or Oracle EXAchk
    You must have root access to run health checks. You can also run health checks as the Oracle Database home owner or the Oracle Grid Infrastructure home owner. Decide which user you want to use before you run health checks.
  • Prerequisites for Running Oracle ORAchk and Oracle EXAchk
    Review Oracle ORAchk and Oracle EXAchk specific prerequisites.
  • Data Entry Terminal Considerations
    Use any supported UNIX and Linux terminal type (character mode terminal, ILOM, VNC server) to run Oracle ORAchk and Oracle EXAchk. Respond to the prompts during the interactive run, or while configuring the daemon.

1.1.2.1 Bash Requirements

Oracle ORAchk and Oracle EXAchk are written largely in Bash, and therefore Bash 3.2 or later is required to run Oracle ORAchk and Oracle EXAchk.

1.1.2.2 SSH Connectivity and Access

In a clustered database environment, Oracle ORAchk and Oracle EXAchk run health checks on a single node and remotely run on all other cluster nodes. Remotely running health checks on cluster nodes involves remotely copying files to and from the targets and running commands without providing the passwords.

Some commands might not run correctly if security restrictions block those commands. Develop alternate plans to run those commands.

To remotely run health checks on all other cluster nodes, from the database server, configure passwordless SSH equivalency for the same user on each cluster node that runs Oracle ORAchk and Oracle EXAchk on the database server. If you have not already configured passwordless SSH, then Oracle ORAchk and Oracle EXAchk automatically configures passwordless SSH. You can specify whether passwordless SSH is configured temporarily or permanently.

You cannot perform remote operations without passwordless SSH configuration in place. If it is not present, then you must run health checks on each database server in the cluster using the -localonly command line option, and then merge the results.

See Also:

My Oracle Support Note 372795.1:

How to Configure SSH for User Equivalence to install the Grid Control Agent 10.1 to 10.2.0.4

Related Topics

1.1.2.3 Handling of root Passwords

Handling of root passwords depends on whether you have installed the Expect utility. Expect is a tool for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, and so on.

If you have installed the Expect utility, then specify the root password when you run the health checks for the first time. The Expect utility stores the password and uses the stored password for subsequent sessions. The Expect utility prompts you to check if the root password is same for all of the remote components such as databases, switches, and so on.

Specify the password only once if you have configured the same root password for all of the components. If the root password is not the same for all of the components, then the Expect utility prompts you to validate the root password every time you run the health checks.

If you enter the password incorrectly or the password is changed in between the time it is entered and used, then Oracle ORAchk and Oracle EXAchk notifies you and skips the relevant checks.

Run the health checks after resolving the issues. If any of the health checks are skipped, then Oracle ORAchk and Oracle EXAchk log the details about the skipped checks in the report output.

1.1.2.4 Deciding Which User Should Run Oracle ORAchk or Oracle EXAchk

You must have root access to run health checks. You can also run health checks as the Oracle Database home owner or the Oracle Grid Infrastructure home owner. Decide which user you want to use before you run health checks.

  • (recommended) Run as root: Use root user credentials to run Oracle ORAchk and Oracle EXachk.

    The Oracle ORAchk and Oracle EXAchk processes that run as root, perform user lookups for the users who own the Oracle Database home and Oracle Grid Infrastructure home. When root access is not required, the Oracle ORAchk and Oracle EXAchk processes use the su command to run health checks as applicable Oracle Database home user or Oracle Grid Infrastructure home user. Accounts with lower privileges cannot have elevated access to run health checks that require root access.

    This approach has advantages in role-separated environments or environments with more restrictive security.

  • Run as Oracle Database home owner or Oracle Grid Infrastructure home owner: Use Oracle Database home owner or Oracle Grid Infrastructure home owner credentials to run Oracle ORAchk and Oracle EXAchk.

    The user that runs Oracle ORAchk and Oracle EXAchk must have elevated access as root to run health checks that need root access.

    This approach requires multiple runs in role-separated environments. More restrictive security requirements do not permit this elevation.

    There are a number of options:

    • Skip the checks that require root access.

    • Specify the root user ID and password when prompted.

    • Configure sudo.

      If you're using sudo, then the temporary directory used by Oracle ORAchk and Oracle EXAchk that is, $HOME, must have an entry in the /etc/sudoers file corresponding to the user who is running the health checks, for example,

      user ALL=(root) NOPASSWD:/root/.orachk/root_orachk.sh

      Or

      user ALL=(root) NOPASSWD:/root/.exachk/root_exachk.sh

    • Pre-configure passwordless SSH connectivity.

A large number of the Oracle stack health checks do not require root access. However, a subset of Oracle stack health checks need to run with root privileges.

The script used to run these root privilege checks is root_orachk.sh  or root_exachk.sh  and it depends on the tool you are using.

This root script is created in the temporary directory used by Oracle ORAchk and Oracle EXAchk, which is $HOME  by default. You can change the temporary directory by setting exporting RAT_TMPDIR.

There may be times when you want to place the root script in a different location from the temporary directory, for either security or convenience reasons.

You can choose to have the root script created outside of the standard temporary directory, in a custom directory you set through the RAT_ROOT_SH_DIR  environment variable: 
export RAT_ROOT_SH_DIR=/orahome/oradb/
You can use a specific location for sudo  remote access as follows: 
export RAT_ROOT_SH_DIR=/mylocation
Then an entry in the /etc/sudoers as follows: 
oracle ALL=(root) NOPASSWD:/mylocation/root_orachk.sh

Note:

Entries in the /etc/sudoers  file must contain full paths and do not use environment variables.

1.1.2.5 Prerequisites for Running Oracle ORAchk and Oracle EXAchk

Review Oracle ORAchk and Oracle EXAchk specific prerequisites.

For Oracle ORAchk specific prerequisites, see Oracle ORAchk Scope and Supported Environments

1.1.2.6 Data Entry Terminal Considerations

Use any supported UNIX and Linux terminal type (character mode terminal, ILOM, VNC server) to run Oracle ORAchk and Oracle EXAchk. Respond to the prompts during the interactive run, or while configuring the daemon.

Each terminal type has advantages and disadvantages. The effect of a dropped network connection varies based on the terminal type used.

For example, in an interactive run using a character mode terminal, if all the prompts are answered prior to the network drop, then the running process completes successfully even if the network connection drops. If the network connection drops before all of the input prompts are answered, then all of the running processes hang. Clean up the hung processes manually when the network connection is restored.

Using a remote connection to a VNC server running on the database where Oracle ORAchk and Oracle EXAchk are running minimizes the network drop interruptions.

If you use accessibility software or devices that prevent the use of a VNC server, and cause network failures, then you must work with your network team and system administrator to determine the root cause and adjust the environment as required.

For example, an accessibility aid might insert a suspension and restart the interactive process that runs Oracle ORAchk or Oracle EXAchk. If this causes an operating system timeout due to terminal inactivity, then lengthen the inactivity timeouts of the environment before running the commands.

The timeout caused by an assistive tool at the operating system level due to terminal inactivity is not specific to Oracle ORAchk and Oracle EXAchk. The timeout could happen to any process that is managed by the assistive technology.

1.1.3 Installing Oracle ORAchk and Oracle EXAchk

Follow these procedures to install Oracle ORAchk and Oracle EXAchk.

Note:

If your Oracle Exadata Database Machine is enrolled in the Oracle Platinum Services: Exadata Exachk Automation Project, then there is a separate installation method described in My Oracle Support Note 2043991.1:

Oracle Platinum Services: Exadata Exachk Automation Project

  1. Download the latest version of the health check tool zip file.

    Note:

    Oracle ORAchk is pre-installed with the database in the $ORACLE_HOME/suptools/orachk  directory.

    To update to the latest version, see “Updating to the Latest Version of Oracle ORAchk and Oracle EXAchk”.

    Updating to the Latest Version of Oracle ORAchk and Oracle EXAchk

  2. Copy the zip file to the installation directory on the systems that you want to check.

    Note:

    Oracle ORAchk and Oracle EXAchk are Oracle RAC database cluster aware. You only need to install Oracle ORAchk and Oracle EXAchk on one node of the cluster to check all nodes in the cluster.
  3. As the oracle software install user, extract the zip file: 
    $ unzip orachk.zip

    $ unzip exachk.zip

Note:

Oracle ORAchk and Oracle EXAchk can be staged on a shared network drive if the performance is acceptable.

To run Oracle ORAchk and Oracle EXAchk on a read-only NFS server, modify the permissions of the .cgrep directory and the scripts within it at least to 555.

chmod –R 555 .cgrep

1.1.4 Configuring the Daemon Mode

Use the daemon to configure automatic health check runs at scheduled intervals.

Note:

If you have an Oracle engineered system, then in addition to the following usage steps, follow the system-specific instructions:

  1. Set the daemon properties.
    At a minimum, set AUTORUN_SCHEDULE and NOTIFICATION_EMAIL.

    For example, to set the tool to run at 3 AM every Sunday and email the results to some.body@example.com, run the following command:

    $ ./orachk –set “AUTORUN_SCHEDULE=3 * * 0 ;NOTIFICATION_EMAIL=some.body@example.com”

    $ ./exachk –set “AUTORUN_SCHEDULE=3 * * 0 ;NOTIFICATION_EMAIL=some.body@example.com”
  2. Configure the health check daemon as described in "Automated Daemon Mode Operation".
  3. Start the daemon as root (recommended) or as the Oracle Database or Oracle Grid Infrastructure home owner. 
    # ./orachk –d start

    # ./exachk –d start
  4. Answer the questions prompted during startup.

Related Topics

1.1.5 Email Notification and Report Overview

The following sections provide a brief overview about email notifications, sections of the HTML report output, and ways to generate a diff report.

  • First Email Notification
    After completing health check runs, the daemon emails the report output as an attachment to all users that are specified in the NOTIFICATION_EMAIL list.
  • Health Check Report
    Health check reports contain the health status of each system grouped under different sections of the report.
  • Subsequent Email Notification
    For the subsequent health check runs after the first email notification, the daemon emails the diff report of the current run and the immediate past run to all users that are specified in the NOTIFICATION_EMAIL list.
  • Diff Report
    The diff report attached to the email notification shows a summary of the differences between the most recent run and the immediate past one. This helps you quickly identify the changes since the last run.

1.1.5.1 First Email Notification

After completing health check runs, the daemon emails the report output as an attachment to all users that are specified in the NOTIFICATION_EMAIL list.

Figure 1-1 First Email Notification

Description of Figure 1-1 follows
Description of "Figure 1-1 First Email Notification"

1.1.5.2 Health Check Report

Health check reports contain the health status of each system grouped under different sections of the report.

The HTML report output contains the following:

  • Health score

  • Summary of health check run

  • Table of contents

  • Controls for report features

  • Findings

  • Recommendations

Details of the report output are different on each system. The report is dynamic, and therefore certain sections are displayed only if applicable.

System Health Score and Summary

System Health Score and Summary report provides:

  • A high-level health score based on the number of checks that passed or failed.

  • A summary of the health check run, such as where and when it was run, which version was used, duration, which user it was run as, and so forth.

Figure 1-2 System Health Score and Summary

Description of Figure 1-2 follows
Description of "Figure 1-2 System Health Score and Summary"

Table of Contents and Report Feature

The Table of Contents section provides links to major sections in the report.

The Report Feature section enables you to:

  • Filter checks based on their statuses

  • Select the regions

  • Expand or collapse all checks

  • View check IDs

  • Remove findings from the report

  • Get a printable view

Figure 1-3 Table of Contents and Report Feature

Description of Figure 1-3 follows
Description of "Figure 1-3 Table of Contents and Report Feature"

Report Findings

The Report Findings section displays the result of each health check. The results are grouped by technology components and shows a row for each health check run including:

  • Check status (FAIL, WARNING, INFO or PASS)

  • Type of check

  • Check message

  • Where the check was run

  • Link to expand details for further findings and recommendation

Figure 1-4 Report Findings

Description of Figure 1-4 follows
Description of "Figure 1-4 Report Findings"

Click View for more information about the result of the health check and the recommendation.

  • What to do to solve the problem

  • Where the recommendation applies

  • Where the problem does not apply

  • Links to relevant documentation or My Oracle Support Notes

  • Example of data the recommendation is based on

Figure 1-5 View Report Findings

Description of Figure 1-5 follows
Description of "Figure 1-5 View Report Findings"

Maximum Availability Architecture (MAA) Score Card

Maximum Availability Architecture (MAA) Score Card displays the recommendations for the software installed on your system.

Figure 1-6 Maximum Availability Architecture (MAA) Score Card

Description of Figure 1-6 follows
Description of "Figure 1-6 Maximum Availability Architecture (MAA) Score Card"

Related Topics

1.1.5.3 Subsequent Email Notification

For the subsequent health check runs after the first email notification, the daemon emails the diff report of the current run and the immediate past run to all users that are specified in the NOTIFICATION_EMAIL list.

The email notification contains:

  • System Health Score of this run compared to previous

  • Summary of number of checks run and the differences between runs

  • Most recent report result as attachment

  • Previous report result as attachment

  • Diff report as attachment

Figure 1-7 Subsequent Email Notification

Description of Figure 1-7 follows
Description of "Figure 1-7 Subsequent Email Notification"

1.1.5.4 Diff Report

The diff report attached to the email notification shows a summary of the differences between the most recent run and the immediate past one. This helps you quickly identify the changes since the last run.

Generate a diff report using the following command option: 
$ ./orachk –diff report_1 report_2

When you review the diff report, you will see a baseline comparison of the two reports, and then a list of differences.

Figure 1-8 Health Check Baseline Comparison Summary

Description of Figure 1-8 follows
Description of "Figure 1-8 Health Check Baseline Comparison Summary"

Figure 1-9 Diff Report

Description of Figure 1-9 follows
Description of "Figure 1-9 Diff Report"

Related Topics

1.1.6 Recommended On-Demand Usage

Recommendations to run health checks on-demand.

Apart from scheduled health check runs, you can also run health checks on-demand by running the following commands: 
$ ./orachk

$ ./exachk

Oracle recommends that you run health checks in the following on-demand scenarios:

  • Pre or post-upgrades

  • Machine moves from one subnet to another

  • Hardware failure or repair

  • Problem troubleshooting

  • In addition to go-live testing

While running pre- or post-upgrade checks, Oracle ORAchk and Oracle EXAchk automatically detect all databases that are registered with Oracle Clusterware and presents the list of databases to check.

Run the pre-upgrade checks during the upgrade planning phase. Oracle ORAchk and Oracle EXAchk prompt you for the version to which you are planning to upgrade to: 
$ ./orachk –u –o pre

$ ./exachk –u –o pre
After upgrading, you must run the post-upgrade checks: 
$ ./orachk –u –o post

$ ./exachk –u –o post

Related Topics

1.1.7 Updating to the Latest Version of Oracle ORAchk and Oracle EXAchk

There are two methods for maintaining Oracle ORAchk and Oracle EXAchk.

1.1.7.1 Updating Oracle ORAchk and Oracle EXAchk in an Environment with an Internet Connection

If your Oracle ORAchk or Oracle EXAchk version is older than 120 days, then the tool prompts you on startup to automatically download a newer version from My Oracle Support.

The script prompts for your My Oracle Support login details, and then checks if a later version is available for download and upgrade.

You can also download manually by running the –download option:

$ ./orachk –download

$ ./exachk –download

$ ./exachk –download
Enter your my oracle support username:- some.person@acompany.com
Enter your my oracle support password:-
Started downloading…..

exachk.zip successfully downloaded to /opt/oracle.suptools/exachk/exachk_mybox_040116_043027

1.1.7.2 Updating Oracle ORAchk and Oracle EXAchk in an Environment without an Internet Connection

If you do not have a direct connection to My Oracle Support, then download the latest versions of Oracle ORAchk and Oracle EXAchk from a machine that has an Internet connection. Transfer the downloaded files to a shared network staging location, and then set the environment variable RAT_UPGRADE_LOC to point to that staging location.

The next time the Oracle ORAchk or Oracle EXAchk is started, the tool detects the latest version and prompts you to upgrade.
  1. Download the appropriate health check tool zip file:

    • For Oracle ORAchk, download orachk.zip.

    • For Oracle EXAchk, download exachk.zip.

  2. Transfer the zip file to a shared network staging directory.
  3. On each machine with a version of the tool that you want to upgrade, set the environment variable RAT_UPGRADE_LOC to point to the network staging directory. 
    $ export RAT_UPGRADE_LOC=PATH_TO_STAGING_DIRECTORY

The next time Oracle ORAchk or Oracle EXAchk is started, the tool searches the directory specified in the RAT_UPGRADE_LOC environment variable. If this directory contains the latest version of the orachk.zip or exachk.zip file, then Oracle ORAchk or Oracle EXAchk prompts you to allow it to upgrade.

$ ls /opt/oracle.SupportTools/exachk/latest
exachk.zip
$ export RAT_UPGRADE_LOC=/opt/oracle.SupportTools/exachk/latest
$ ./exachk
Latest version of exachk (EXACHK VERSION: 12.1.0.2.7_20160401) is available at /opt/oracle.SupportTools/exachk/latest/

Do you want to upgrade to the latest version of exachk? [y/n][y]

exachk has been upgraded to EXACHKVERSION:12.1.0.2.7(DEV)_20160401

Running the latest version…
If you have set RAT_UPGRADE_LOC but do not want to upgrade, then you can still run Oracle ORAchk or Oracle EXAchk using the –noupgrade option: 
$ ./orachk –noupgrade

$ ./exachk –noupgrade

Related Topics