Understanding Oracle Exachk specifics for Oracle Exadata and Zero Data Loss Recovery Appliance

Understand the features and learn to perform tasks specific to Oracle Exachk on Oracle Exadata and Zero Data Loss Recovery Appliance.

Installation Requirements for Running Oracle Exachk on Oracle Exadata and Zero Data Loss Recovery Appliance

Understand the requirements for installing Oracle Exachk on Oracle Exadata and Zero Data Loss Recovery Appliance, either on your local database or on a remote device that is connected to a database.

Note:

For more information about installing and upgrading Oracle Autonomous Health Framework, see Installing and Upgrading Oracle Autonomous Health Framework.

Using Oracle Exachk on Oracle Exadata and Zero Data Loss Recovery Appliance

Usage of Oracle Exachk on Oracle Exadata and Zero Data Loss Recovery Appliance depends on other considerations such as virtualization, parallel run, and so on.

Database Default Access on the Client Interface

If the client interface is configured as the default access interface for the database servers, use the -clusternodes option to specify the management interface host names that Oracle Exachk should use.

For example, consider the following cluster configuration:

Interface	Database Host Names
Management	`dbadm01`, `dbadm02`, `dbadm03`, `dbadm04`
Client	`dbclnt01`, `dbclnt02`, `dbclnt03`, `dbclnt04`

To instruct Oracle Exachk to communicate over the management interface, specify:

exachk -clusternodes dbadm01,dbadm02,dbadm03,dbadm04

Note:

When using the -clusternodes option, start Oracle Exachk on the first database server specified in the list.

Virtualization Considerations

Oracle Exachk supports virtualized Oracle Exadata environments.

To run hardware and operating system-level checks for database servers, storage servers, the InfiniBand fabric, and InfiniBand switches:

Install Oracle Exachk in the management domain (DOM0).
Run Oracle Exachk as root.

When Oracle Exachk runs from DOM0, it:

Discovers compute nodes, storage servers, and InfiniBand switches across the InfiniBand fabric.
Runs the applicable checks on the discovered components.

To restrict an Oracle Exachk run from the management domain to specific components, use:

-clusternodes to specify database servers.
-cells to specify storage servers.
-ibswitches to specify InfiniBand switches.

For example, consider a full rack where only the first quarter rack is configured for virtualization, but all components share the same InfiniBand fabric.

Run the following command as root on the DOM0 database node randomadm01:

exachk -clusternodes randomadm01,randomadm02 \
       -cells randomceladm01,randomceladm02,randomceladm03 \
       -ibswitches randomsw-ibs0,randomsw-iba0,randomsw-ibb0

In addition to running Oracle Exachk in the management domain, run it separately for each cluster in the user domains (DOMU).

Within a DOMU, you do not need to specify the component-selection options because Oracle Exachk automatically discovers the nodes in the cluster.

For example, if an environment contains two clusters with four user domains in each cluster, there are eight user domains in total. However, Oracle Exachk needs to run only twice:

Once from the first node of the first cluster.
Once from the first node of the second cluster.

Oracle Exachk runs from a user domain do not include hardware or operating system-level checks for database servers, storage servers, or InfiniBand switches.

Note:

Run Oracle Exachk as root in both the management domain and user domains.

Running Serial Data Collection

By default, Oracle Exachk collects data in parallel from storage servers, InfiniBand switches, and Oracle Databases.

You can configure Oracle Exachk to collect data serially by setting environment variables for the applicable component.

To collect database server data serially:

export RAT_COMPUTE_RUNMODE=serial

To collect storage server data serially:

export RAT_CELL_RUNMODE=serial

To collect InfiniBand switch data serially:

export RAT_IBSWITCH_RUNMODE=serial

Use serial collection when required for troubleshooting or when parallel collection is not appropriate for the environment.

Using the root User in Asymmetric and Role-Separated Environments

Run Oracle Exachk as root to simplify compliance checking in asymmetric or role-separated environments.

If Oracle Database homes are not configured symmetrically across the cluster, install Oracle Exachk on enough database servers to provide access to each Oracle Database home.

For example, consider the following configuration:

Owner User ID	Oracle Database Home	Installed On	Databases
`user1`	`/path1/dbhome_1`	`db01`, `db02`, `db03`, `db04`	`dbm-a`
`user2`	`/path2/dbhome_2`	`db05`, `db06`, `db07`, `db08`	`dbm-b`, `dbm-c`
`grid`	`/path3/grid`	`db01`, `db02`, `db03`, `db04`, `db05`, `db06`, `db07`, `db08`	`+ASM`

In this example, role separation prevents user1, user2, and grid from accessing each other's database structures. Organizational policies can also require separation between system administrators and database administrators.

To run Oracle Exachk in this environment:

As root, install Oracle Exachk in the following directory on db01:
```
/tmp/exachk/121026
```
As root, install Oracle Exachk in the same directory on db05:
```
/tmp/exachk/121026
```
As root on db01, run:
```
cd /tmp/exachk/121026
exachk -clusternodes db01,db02,db03,db04
```
When prompted to select databases, select dbm-a.

As root on db05, run:

cd /tmp/exachk/121026
exachk -excludeprofiles storage,switch -clusternodes db05,db06,db07,db08

When prompted to select databases, select dbm-b and dbm-c.

If required, use the -merge option to combine the resulting reports into a single report.

Specifying a User Other Than root

Oracle Exachk provides environment variables that enable you to specify users other than root when connecting to certain Exadata components.

Specifying a Storage Server User

By default, Oracle Exachk uses root when running checks on an Oracle Exadata Storage Server.

If your security policies do not permit SSH connections to storage servers as root, set RAT_CELL_SSH_USER to an alternative user.

For example:

export RAT_CELL_SSH_USER=celladmin

Note:

When you specify RAT_CELL_SSH_USER, Oracle Exachk runs only the subset of checks permitted by the privileges of the specified user.

Specifying an InfiniBand Switch User

When Oracle Exachk runs as root on a database server, it uses root by default to run checks on InfiniBand switches.

When Oracle Exachk runs as a non-root user on a database server, it uses nm2user by default for InfiniBand switch checks.

If your security policies do not permit SSH connections to InfiniBand switches using root or nm2user, set RAT_IBSWITCH_USER to an alternative user.

For example:

export RAT_IBSWITCH_USER=ilom-admin

Note:

When you specify RAT_IBSWITCH_USER, Oracle Exachk runs only the subset of checks permitted by the privileges of the specified user.

Oracle Exachk InfiniBand Switch Processing

When Oracle Exalogic and Oracle Exadata Engineered Systems share the same InfiniBand fabric, Oracle Exachk limits switch processing according to the system from which it is run.

When Oracle Exachk runs on an Oracle Exadata database server, it excludes Oracle Exalogic gateway switches.
When Oracle Exachk runs on an Oracle Exalogic compute node, it excludes Oracle Exadata switches.

This behavior ensures that Oracle Exachk evaluates the InfiniBand switches applicable to the Engineered System on which the compliance check is being performed.

Troubleshooting Oracle Exachk on Oracle Exadata and Zero Data Loss Recovery Appliance

Follow these steps to troubleshoot and fix Oracle Exachk on Oracle Exadata and Zero Data Loss Recovery Appliance issues.

Error RC-003 - No Audit Checks Were Found

Description: While identifying the environment characteristics, Oracle Exachk

Constructs environment variables
Compares with the Oracle Exachk rules database to determine what checks to run

If one of the environment variables does not match a known profile in the rules database, then Oracle Exachk displays an error error RC-003 - no audit checks were found... and exits.

Cause: The most common case occurs when an older version of Oracle Exachk is used in an Oracle Exadata Database machine environment with recently released components. This may occur because of a delay between the release of a new component or version and when Oracle Exachk incorporates support for it.

For example, when Oracle Exachk earlier than 2.1.3_20111212 were run on an Oracle Exadata Database machine where Oracle Database release 11.2.0.3.0 was deployed, Oracle Exachk exited with the following message:

Error RC-003 - No audit checks were found for LINUXX8664OELRHEL5_112030-.
Please refer to the section for this error code in
"- Troubleshooting Scenarios" of the "Exachk User Guide".

In this example, _112030 indicates that Oracle Database release 11.2.0.3.0 was installed on the system. Since the version of Oracle Exachk used did not support 11.2.0.3.0, Oracle Exachk could not find a known match in the Oracle Exachk rules database.

How Long Should It Take to Run Oracle Exachk ?

The time it takes to run the tool varies based on the number of nodes in a cluster, CPU load, network latency, and so on. Normally the entire process takes only a few minutes per node, that is, less than 5 minutes per node. If it takes substantially more time than 5 minutes, then investigate the problem.

With the introduction of parallelized database collection in 2.2.5, the elapsed time for systems with many databases is reduced. Experience in the field is that, it normally takes about 10 minutes for a quarter rack X2-2 system with one database. On an internal X3-2 half rack with 20 storage servers, 9 InfiniBand switches, and 44 databases, the elapsed time was 44 minutes.