This section explains how to install and configure Oracle ORAchk and Oracle EXAchk. Review other topics in this section for additional information.
Oracle ORAchk and Oracle EXAchk provide a lightweight and non-intrusive health check framework for the Oracle stack of software and hardware components.
If you have an Oracle Engineered System other than Oracle Database Appliance, then use Oracle EXAchk. For all other systems, use Oracle ORAchk.
You have access to Oracle ORAchk and Oracle EXAchk as a value add-on to your existing support contract. There is no additional fee or license required to run Oracle ORAchk and Oracle EXAchk.
Features of Oracle ORAchk and Oracle EXAchk
Automates risk identification and proactive notification before business is impacted
Runs health checks based on critical and reoccurring problems
Runs in your environment with no need to send anything to Oracle
Enables you to schedule email health check reports
Integrates the findings into other tools of your choice
Follow these procedures to install Oracle ORAchk and Oracle EXAchk.
Note:
If your Oracle Exadata Database machine is enrolled in the Oracle Platinum Services: Exadata Exachk Automation Project, then there is a separate installation method described in My Oracle Support Note 2043991.1.
Note:
If the performance is acceptable, then stage Oracle ORAchk and Oracle EXAchk on a shared network drive.To run Oracle ORAchk and Oracle EXAchk on a read-only NFS server, modify the permissions of the .cgrep directory and the scripts within it at least to 555.
chmod –R 555 .cgrep
Review the checklist for Bash requirements, SSH connectivity, and required user privileges to run health checks.
root passwords depends on whether you have installed the Expect utility.root. Also, run health checks as the Oracle Database home owner or the Oracle Grid Infrastructure home owner.In a clustered database environment, Oracle ORAchk and Oracle EXAchk run health checks on a single node and remotely run on all other cluster nodes.
Remotely running health checks on cluster nodes involves remotely copying files to and from the targets and running commands without providing the passwords.
If security restrictions block, then some commands fail to run. To run those commands successfully, develop alternate plans.
To run health checks remotely on all other cluster nodes from the database server:
Configure passwordless SSH equivalency for the same user on each cluster node that runs Oracle ORAchk and Oracle EXAchk on the database server
If you have not already configured passwordless SSH, then Oracle ORAchk and Oracle EXAchk automatically configure passwordless SSH. You can specify whether passwordless SSH is configured temporarily or permanently.
You cannot perform remote operations without passwordless SSH configuration in place. If passwordless SSH is not present:
Run health checks on each database server in the cluster using the -localonly command-line option
Merge the results
Handling of root passwords depends on whether you have installed the Expect utility.
Expect automates interactive applications such as Telnet, FTP, passwd, fsck, rlogin, tip, and so on.
If you have installed the Expect utility, then specify the root password when you run the health checks for the first time.
The Expect utility stores the password and uses the stored password for subsequent sessions.
The Expect utility prompts you to check if the root password is same for all the remote components such as databases, switches, and so on.
Specify the password only once if you have configured the same root password for all the components.
If root password is not the same for all the components, then the Expect utility prompts you to validate the root password every time you run the health checks.
Notify you
Skip relevant checks
Run the health checks after resolving the issues.
If Oracle ORAchk and Oracle EXAchk skip any of the health checks, then the tools log details about the skipped checks in the report output.
Related Topics
Run health checks as root. Also, run health checks as the Oracle Database home owner or the Oracle Grid Infrastructure home owner.
Many of the health checks do not require root access. However, you need root privileges to run a subset of health checks.
To run root privilege checks, Oracle ORAchk uses the script root_orachk.sh and Oracle EXAchk uses the script root_exachk.sh.
By default, the root_orachk.sh and root_exachk.sh scripts are created in the $HOME directory used by Oracle ORAchk and Oracle EXAchk. Change the directory by setting the environment variable RAT_ROOT_SH_DIR.
sudo remote access as follows:
export RAT_ROOT_SH_DIR=/mylocation
/etc/sudoers as follows:
oracle ALL=(root) NOPASSWD:/mylocation/root_orachk.sh
root scripts outside of the standard temporary directory in a custom directory. Specify the custom directory using the environment variable RAT_ROOT_SH_DIR:
export RAT_ROOT_SH_DIR=/orahome/oradb/
sudo remote access as follows:
export RAT_ROOT_SH_DIR=/mylocation
/etc/sudoers as follows:
oracle ALL=(root) NOPASSWD:/mylocation/root_orachk.sh
Note:
Specify full paths for the entries in the /etc/sudoers file. Do not use environment variables.
(recommended) Run as root: Use root user credentials to run Oracle ORAchk and Oracle EXAchk.
The Oracle ORAchk and Oracle EXAchk processes that run as root, perform user lookups for the users who own the Oracle Database home and Oracle Grid Infrastructure home. If root access is not required, then the Oracle ORAchk and Oracle EXAchk processes use the su command to run health checks as the applicable Oracle Database home user or Oracle Grid Infrastructure home user. Accounts with lower privileges cannot have elevated access to run health checks that require root access.
Running health checks as root has advantages in role-separated environments or environments with more restrictive security.
Run as Oracle Database home owner or Oracle Grid Infrastructure home owner: Use Oracle Database home owner or Oracle Grid Infrastructure home owner credentials to run Oracle ORAchk and Oracle EXAchk.
The user who runs Oracle ORAchk and Oracle EXAchk must have elevated access as root to run health checks that need root access.
Running health checks as Oracle Database home owner or Oracle Grid Infrastructure home owner requires multiple runs in role-separated environments. More restrictive security requirements do not permit elevated access.
There are several other options:
Skip the checks that require root access.
Specify the root user ID and password when prompted.
Configure sudo.
If you are using sudo, then add an entry for the root script, located in $HOME in the /etc/sudoers file that corresponds to the user who is running the health checks.
user ALL=(root) NOPASSWD:/root/root_orachk.sh
user ALL=(root) NOPASSWD:/root/root_exachk.sh
To determine what $HOME is set to, run the echo $HOME command.
For example:
user ALL=(root) NOPASSWD:/root/.orachk/root_orachk.sh
Or
user ALL=(root) NOPASSWD:/root/.exachk/root_exachk.sh
Pre-configure passwordless SSH connectivity.
Review Oracle ORAchk and Oracle EXAchk specific prerequisites.
For Oracle ORAchk specific prerequisites, see:
Oracle ORAchk Scope and Supported Environments
For Oracle EXAchk specific prerequisites, see:
Use any supported UNIX and Linux terminal type (character mode terminal, ILOM, VNC server) to run Oracle ORAchk and Oracle EXAchk.
Respond to the prompts during the interactive run, or while configuring the daemon.
Each terminal type has advantages and disadvantages. The effect of a dropped network connection varies based on the terminal type used.
For example, in an interactive run using a character mode terminal, if all the prompts are answered prior to the network drop, then the running process completes successfully even if the network connection drops. If the network connection drops before all the input prompts are answered, then all the running processes hang. Clean up the hung processes manually when the network connection is restored.
Using a remote connection to a VNC server running on the database where Oracle ORAchk and Oracle EXAchk are running minimizes the network drop interruptions.
If you use accessibility software or devices that prevent the use of a VNC server, and cause network failures, then you must work with your network team and system administrator to determine the root cause and adjust the environment as required.
For example, an accessibility aid might insert a suspension and restart the interactive process that runs Oracle ORAchk or Oracle EXAchk. If this causes an operating system timeout due to terminal inactivity, then increase the inactivity timeouts of the environment before running the commands.
The timeout caused by an assistive tool at the operating system level due to terminal inactivity is not specific to Oracle ORAchk and Oracle EXAchk. The timeout could happen to any process that assistive technology manages.
Use the daemon to configure automatic health check runs at scheduled intervals.
Note:
Daemon mode is supported only on the Linux and Solaris operating systems.
Note:
If you have an Oracle Engineered System, then in addition to the following usage steps, follow the system-specific instructions.
Related Topics
It is now possible to run multiple instances of Oracle ORAchk and Oracle EXAchk at the same time on the same machine.
As a result of this change the location of the root_orachk.sh and root_exachk.sh files have now changed. By default, they are created in the user's home directory.
If you run as non-root, then change this location in the /etc/sudoers file, for example,
/etc/sudoers entry:
oracle ALL=(root) NOPASSWD:/home/oracle/.orachk/root_orachk.sh
/etc/sudoers entry:
oracle ALL=(root) NOPASSWD:/home/oracle/root_orachk.sh
RAT_ROOT_SH_DIR.
export RAT_ROOT_SH_DIR=/some/other/location
The following sections provide a brief overview about email notifications and sections of the HTML report output.
NOTIFICATION_EMAIL list.After completing health check runs, the daemon emails the assessment report as an HTML attachment to all users that you have specified in the NOTIFICATION_EMAIL list.
Health check reports contain the health status of each system grouped under different sections of the report.
The HTML report output contains the following:
Health score
Summary of health check runs
Table of contents
Controls for report features
Findings
Recommendations
Details of the report output are different on each system. The report is dynamic, and therefore the tools display certain sections only if applicable.
System Health Score and Summary
System Health Score and Summary report provide:
A high-level health score based on the number of passed or failed checks
Name, for example, Cluster Name
Version of the operating system kernel
Path, version, name of homes, for example, CRS, DB, and EM Agent
Version of the component checked, for example, Exadata
Number of nodes checked, for example, database server, storage servers, InfiniBand switches
Version of Oracle ORAchk and Oracle EXAchk
Name of the collection output
Date and time of collection
Duration of the check
Name of the user who ran the check, for example, root
How long the check is valid
Table of Contents and Report Feature
The Table of Contents section provides links to major sections in the report:
Database Server
Storage Server
InfiniBand Switch
Cluster Wide
Maximum Availability Architecture (MAA) Scorecard
Infrastructure Software and Configuration Summary
Findings needing further review
Platinum Certification
System-wide Automatic Service Request (ASR) health check
Skipped Checks
Top 10 Time Consuming Checks
The Report Feature section enables you to:
Filter checks based on their statuses
Select the regions
Expand or collapse all checks
View check IDs
Remove findings from the report
Get a printable view
Report Findings
The Report Findings section displays the result of each health check grouped by technology components, such as Database Server, Storage Server, InfiniBand Switch, and Cluster Wide.
Each section shows:
Check status (FAIL, WARNING, INFO, or PASS)
Type of check
Check message
Where the check was run
Link to expand details for further findings and recommendation
Click View for more information about the health check results and the recommendations.
What to do to solve the problem
Where the recommendation applies
Where the problem does not apply
Links to relevant documentation or My Oracle Support notes
Example of data on which the recommendation is based
Maximum Availability Architecture (MAA) Score Card
Maximum Availability Architecture (MAA) Score Card displays the recommendations for the software installed on your system.
The details include:
Outage Type
Status of the check
Description of the problem
Components found
Host location
Version of the components compared to the recommended version
Status based on comparing the version found to the recommended version
Related Topics
For the subsequent health check runs after the first email notification, the daemon emails the summary of differences between the most recent runs.
Specify a list of comma-delimited email addresses in the NOTIFICATION_EMAIL option.
The email notification contains:
System Health Score of this run compared to the previous run
Summary of number of checks that were run and the differences between runs
Most recent report result as attachment
Previous report result as attachment
Diff report as attachment
The diff report attached to the previous email notification shows a summary of differences between the most recent runs.
To identify the changes since the last run:
$ ./orachk –diff report_1 report_2
When you review the diff report, you see a baseline comparison of the two reports and then a list of differences.
Related Topics
This section summarizes the scenarios that Oracle recommends running health checks on-demand.
$ ./orachk
$ ./exachk
Oracle recommends that you run health checks in the following on-demand scenarios:
Pre- or post-upgrades
Machine relocations from one subnet to another
Hardware failure or repair
Problem troubleshooting
In addition to go-live testing
While running pre- or post-upgrade checks, Oracle ORAchk and Oracle EXAchk automatically detect databases that are registered with Oracle Clusterware and presents the list of databases to check.
$ ./orachk –u –o pre
$ ./exachk –u –o pre
$ ./orachk –u –o post
$ ./exachk –u –o post
Related Topics
There are several methods for maintaining Oracle ORAchk and Oracle EXAchk.
Note:
Each database PSU contains the latest Oracle ORAchk version available at time of creation. When a database PSU is applied, the ORAchk zip version contained is staged in $ORACLE_HOME/suptools.
Upon the next run, Oracle ORAchk prompts you to upgrade if the version copied by the PSU is newer than installed.
If your Oracle ORAchk or Oracle EXAchk version is older than 120 days, then the tool prompts you on startup to automatically download a newer version from My Oracle Support.
The script prompts for your My Oracle Support login details, and then checks if a later version is available for download and upgrade.
You can also download manually by running the –download option:
$ ./orachk –download
$ ./exachk –download
$ ./exachk –download Enter your my oracle support username:- some.person@acompany.com Enter your my oracle support password:- Started downloading….. exachk.zip successfully downloaded to /opt/oracle.suptools/exachk/exachk_mybox_040116_043027
If you do not have a direct connection to My Oracle Support, then download the latest versions of Oracle ORAchk and Oracle EXAchk from a machine that has an internet connection.
Transfer the downloaded files to a shared network staging location, and then set the environment variable RAT_UPGRADE_LOC to point to that staging location.
The next time Oracle ORAchk or Oracle EXAchk is started, the tool searches the directory specified in the RAT_UPGRADE_LOC environment variable. If this directory contains the latest version of the orachk.zip or exachk.zip file, then Oracle ORAchk or Oracle EXAchk prompts you to allow it to upgrade.
$ ls /opt/oracle.SupportTools/exachk/latest exachk.zip $ export RAT_UPGRADE_LOC=/opt/oracle.SupportTools/exachk/latest $ ./exachk Latest version of exachk (EXACHK VERSION: 12.1.0.2.7_20160401) is available at /opt/oracle.SupportTools/exachk/latest/ Do you want to upgrade to the latest version of exachk? [y/n][y] exachk has been upgraded to EXACHKVERSION:12.1.0.2.7(DEV)_20160401 Running the latest version…
RAT_UPGRADE_LOC but do not want to upgrade, then you can still run Oracle ORAchk or Oracle EXAchk using the –noupgrade option:
$ ./orachk –noupgrade
$ ./exachk –noupgrade
Note:
Use the -noupgrade option when you have the latest version in RAT_UPGRADE_LOC and do not yet want to upgrade.
Using -noupgrade without having the latest version in RAT_UPGRADE_LOC still prompts you to download the latest version.