8.2 Troubleshooting Compliance Framework (Oracle Orachk and Oracle Exachk)

Follow the steps explained in this section to troubleshoot and fix Compliance Framework (Oracle Orachk / Oracle Exachk) related issues.

• How to Troubleshoot Oracle Orachk and Oracle Exachk Issues
  Follow these steps to fix the Oracle Orachk and Oracle Exachk related issues.
• How to Capture Debug Output
  Follow these procedures to capture debug information.
• Error Messages or Unexpected Output
  Follow these steps to troubleshoot and fix error messages and unexpected output.
• Operating System Is Not Discovered Correctly
  Oracle ORAchk and Oracle EXAchk display this message if the tools are not able to detect the operating system.
• Oracle Clusterware or Oracle Database is not Detected or Connected Issues
  Follow the procedures in this section to troubleshoot and fix Oracle Clusterware or Oracle Database issues.
• Remote Login Problems
  If Oracle Orachk and Oracle Exachk have problem locating and running SSH or SCP, then the tools cannot run any remote checks.
• Permission Problems
  You must have sufficient directory permissions to run Oracle Orachk and Oracle Exachk.
• Slow Performance, Skipped Checks, and Timeouts
  Follow these procedures to fix slow performance and other issues.
• Running Compliance Checks on a Subset of Oracle Home and Oracle Databases
  Use the -dbconfig command when Oracle Orachk and Oracle Exachk are not able to discover ORACLE_HOME, or the name of the Oracle Database and you have multiple ORACLE_HOMEs in a system and each home is running multiple Oracle Databases.
• SSH Connection Timeout
  
• Oracle Exachk Prompts to Enter Names of RoCE Fabric Switches
  
• Unable to Implement CA Certificates in Oracle Trace File Analyzer
  

8.2.1 How to Troubleshoot Oracle Orachk and Oracle Exachk Issues

Follow these steps to fix the Oracle Orachk and Oracle Exachk related issues.

1. Ensure that you are using the correct tool.

   If you have an Oracle Engineered System other than Oracle Database Appliance, then use Oracle Exachk. For all other systems, use Oracle Orachk.

2. Ensure that you are using the latest versions of Oracle Orachk and Oracle Exachk.

   New versions are released every three months.

   1. Check the version using the –v option.

      $ orachk –v

      $ exachk –v

   2. Compare your version with the latest version available here:
      1. For Oracle Orachk, refer to My Oracle Support note 2550798.1.
      2. For Oracle Exachk, refer to My Oracle Support note 1070954.1.
3. Check the FAQ  for similar problems in My Oracle Support note 1070954.1.
4. Review files within the log directory.
   1. Check applicable error.log files for relevant errors.

      This file contains stderr output captured during the run, not everything you see in here will mean you have a problem, but if you have a problem this may give more information.

      • output_dir/log/orachk _error.log
      • output_dir/log/exachk _error.log
   2. Check applicable log for other relevant information.
      • output_dir/log/orachk.log
      • output_dir/log/exachk.log
5. Review My Oracle Support notes for similar problems.
6. For Oracle Orachk issues, check My Oracle Support Community (MOSC).
7. If necessary capture debug output, log a new SR and attach the resulting zip file.

8.2.2 How to Capture Debug Output

Follow these procedures to capture debug information.

1. Before enabling debug, reproduce the problem with the least run necessary.
   • Debug captures a lot, the resulting zip file can be large so try to narrow down the amount of run necessary to reproduce the problem.

     Use relevant command line options to limit the scope of checks.

2. Enable debug.

   If you are running the tool in on-demand mode, then use –debug argument.

   If the problem area is known, then debug can be constrained to a particular module by including the –module argument too.

   $ orachk -debug [-module [ setup | discovery | execution | output ] ]

   $ exachk -debug [-module [ setup | discovery | execution | output ] ]

   When debug is enabled, Oracle Orachk and Oracle Exachk create a new debug log file in:

   • output_dir/log/orachk _debug_date_stamp_time_stamp.log

   • output_dir/log/exachk _debug_date_stamp_time_stamp.log

   The output_dir directory retains a number of other temporary files used during health checks.

   If you run health checks using the daemon, then restart the daemon with the –d start –debug option.

   Running this command generates both debug for daemon and include debug in all client runs:

   $ orachk –d start –debug

   $ exachk –d start –debug

   When debug is run with the daemon, Oracle Orachk and Oracle Exachk create a daemon debug log file in the directory the daemon was started:

   orachk_daemon_debug.log

   exachk_daemon_debug.log

3. Collect the resulting output zip file, and the daemon debug log file if applicable.

8.2.3 Error Messages or Unexpected Output

Follow these steps to troubleshoot and fix error messages and unexpected output.

• Data Entry Terminal Considerations
  
• Tool Runs without Producing Files
  
• Messages similar to “line ****: **** Killed $perl_cmd 2>> $ERRFIL?”
  
• Messages similar to “RC-001- Unable to read driver files”
  
• Messages similar to “There are prompts in user profile on [hostname] which will cause issues in [tool] successful execution”
  
• Problems Related to Remote Login
  
• Other Error Messages in orachk_error.log or exachk_error.log
  
• Space available on {node_name} at {path} is {x} MB and required space is 500 MB
  
• Running Oracle Orachk on Microsoft Windows Throws '{oratab}' is empty Error
  

8.2.3.1 Data Entry Terminal Considerations

Description:

Use any supported UNIX and Linux terminal type (character mode terminal, ILOM, VNC server) to run Oracle Orachk and Oracle Exachk.

Respond to the prompts during interactive runs, or while configuring the daemon.

Each terminal type has advantages and disadvantages. The effect of a dropped network connection varies based on the terminal type used.

For example, in an interactive run using a character mode terminal, if all the prompts are answered before the network drop, then the running process completes successfully even if the network connection drops. If the network connection drops before all the input prompts are answered, then all the running processes hang. Clean up the hung processes manually when the network connection is restored.

Using a remote connection to a VNC server running on the database where Oracle Orachk and Oracle Exachk are running minimizes the network drop interruptions.

If you use accessibility software or devices that prevent the use of a VNC server, and experience network drops, then contact your system administrator to determine the root cause and adjust the environment as necessary.

For example, if an accessibility aid inserts suspensions and restarts the interactive process running Oracle Orachk and Oracle Exachk lead to an operating system timeout due to terminal inactivity. Lengthen the inactivity timeouts of the environment before running the commands.

The timeout caused by an assistive tool at the operating system level due to terminal inactivity is not specific to Oracle Orachk and Oracle Exachk. The timeout could happen to any process managed by the assistive technology.

8.2.3.2 Tool Runs without Producing Files

Description:

Oracle Orachk and Oracle Exachk create temporary files and directories at runtime, as well as output files for data collection.

If you cancel Oracle Orachk using Ctrl+C or if Oracle Orachk fails due to an error, then Oracle Orachk cleans up the files that Oracle Orachk created while running.

If Oracle Orachk or Oracle Exachk completes health check runs, but did not generate output files, then there is an error probably near the end of the run that caused an ungraceful exit.

Action:

If the problem persists, then run the tool again in debug mode and examine the output. If necessary, contact Oracle Support for assistance.

8.2.3.3 Messages similar to “line ****: **** Killed $perl_cmd 2>> $ERRFIL?”

Description:

Oracle Orachk and Oracle Exachk have a built-in watchdog process that monitors and kills the commands that exceed default timeouts to prevent hangs.

Cause:

Killing a command results in “line ****: **** Killed $perl_cmd 2>> $ERRFIL?” error.

8.2.3.4 Messages similar to “RC-001- Unable to read driver files”

Description:

There are a number of possible causes related to not having a supported platform or not being able to read or write into temporary, working or installation directories.

Oracle Orachk and Oracle Exachk display the same error message also as, RC-002- Unable to read driver files

Action:

1. Verify that you are running on a supported platform.
2. Verify that there is sufficient diskspace available in the temporary or output directory. If necessary increase disk space or direct temporary and output files elsewhere.
3. Verify the hidden subdirectory .cgrep exists within the install location. This directory contains various support files some of which are platform-specific.
4. Verify that you are able to write into and read out of the temporary and working directory location.

8.2.3.5 Messages similar to “There are prompts in user profile on [hostname] which will cause issues in [tool] successful execution”

Description:

Oracle Orachk and Oracle Exachk exit if the tools detect prompts in the user profile.

Oracle Orachk and Oracle Exachk fetch the user environment files on all nodes. If the user environment files contain prompts, for example, read -p, or other commands that pause the running commands, then the commands timeout. The commands timeout because there is no way to respond to the messages when it is being called.

All such commands cannot be detected in the environment. However, the commands that can be detected lead to this message.

Action:

Comment all such prompts from the user profile file (at least temporarily) and test run again.

8.2.3.6 Problems Related to Remote Login

Action:

If you see messages similar to No such file or directory or /usr/bin/scp -q: No such file or directory, then refer to Remote Login Problems to fix the issues.

8.2.3.7 Other Error Messages in orachk_error.log or exachk_error.log

Description:

When examining the orachk_error.log , some messages are expected and they are not indicative of problems. These errors are redirected and absorbed into the error.log to keep them from being reported on the screen.

For example, an error similar to the following is reported numerous times, once for each Oracle software home for each node:

/bin/sh: /u01/app/11.2.0/grid/OPatch/opatch: Permission denied
chmod: changing permissions of `/u01/app/oracle_ebs/product/11.2.0.2/VIS_RAC/.patch_storage': Operation not permitted
OPatch could not open log file, logging will not be possible
Inventory load failed... OPatch cannot load inventory for the given Oracle Home.

These types of errors occur in role-separated environments when the tool runs as the Oracle Database software owner uses Opatch to list the patch inventories of homes that are owned by Oracle Grid Infrastructure or other Oracle Database home owners. When you run Opatch to list the patch inventories for other users, Opatch fails because the current user does not have permissions on the other homes. In these cases, the Opatch error is ignored and the patch inventories for those homes are gathered by other means. To avoid such errors, Oracle recommends that you run Oracle Orachk and Oracle Exachk as root in role-separated environments.

Action:

You do not need to report these types of errors to Oracle Support.

Also, ignore the errors similar to the following:

orachk: line [N]: [: : integer expression expected

The line number changes over time. However, the error indicates that the tool was expecting an integer return value and no value was found. The value was null that the shell was not able to compare the return values. The error is repeated many times for the same command, once for each node.

8.2.3.8 Space available on {node_name} at {path} is {x} MB and required space is 500 MB

Description:

Oracle Orachk displays an error message when there is no enough space in the location for temporary files and directories.

Space available on at /users/oracle is 441 MB and required space is 500 MB
Please make at least mentioned space available at above location and retry to continue.[y/n][y]?

Cause:

Oracle Orachk creates temporary files and directories during execution. The default location for temporary files and directories is the $HOME directory of the user who runs the tool.

Action:

To change the location of Oracle Orachk temporary files set the RAT_TMPDIR environment variable to the new location before running Oracle Orachk.

8.2.3.9 Running Oracle Orachk on Microsoft Windows Throws '{oratab}' is empty Error

Description:

Running Oracle Orachk commands throws the following error:

'{oratab}' is empty. 
Verify Oracle database registry entries name in '{regout}' in case Oracle database is install. 
If Oracle database registry entries name does not contains 'ORA'/'OH' 
then set registry key patterns using 'RAT_KEY_DB' environment variable.

Cause:

1. Oracle Database is not present on the system.
2. The keyword for Oracle Database registry key is different. Generally the registry key contains ORA or OH, but on some systems it can be different. Set the environment variable RAT_KEY_DB to the right Oracle Database registry keyword.
3. The initialization parameter file initSID.ora is missing from $ORACLE_HOME/database.

   Oracle Orachk looks for the initSID.ora file in $ORACLE_HOME/database for getting the instance name. If the initSID.ora file is not present, then you will encounter the aforementioned error.

Action:

Specify Oracle Database details using the -dbconfig option.

orachk -dbconfig db_home_path%db_name

8.2.4 Operating System Is Not Discovered Correctly

Oracle ORAchk and Oracle EXAchk display this message if the tools are not able to detect the operating system.

Cause:

If Oracle ORAchk and Oracle EXAchk are not able to detect the operating system, then the tools prompt:

• Data needed for the derived platform could not be found

• Improperly detecting an unsupported platform

Action:

Set the RAT_OS environment variable to the correct operating system:

$ export RAT_OS=platform

8.2.5 Oracle Clusterware or Oracle Database is not Detected or Connected Issues

Follow the procedures in this section to troubleshoot and fix Oracle Clusterware or Oracle Database issues.

• Oracle Clusterware Software is Installed, but Cannot be Found
  
• Oracle Database Software Is Installed, but Cannot Be Found
  
• Oracle Database Software Is Installed, but Version cannot Be Found
  
• Oracle ASM Software is Installed, but Cannot be Found
  
• Oracle Database Discovery Issues on Oracle Real Application Clusters (Oracle RAC) Systems
  
• Oracle Database Login Problems
  

8.2.5.1 Oracle Clusterware Software is Installed, but Cannot be Found

Description:

Oracle Orachk discovers the location of the Oracle Clusterware home from the oraInst.loc and oraInventory files.

Cause:

Oracle Clusterware discovery fails due to:

• Problems discovering the oraInst.loc and oraInventory  files
• Problems with the oraInst.loc and oraInventory  files
• One or more paths in the oraInst.loc and oraInventory  files are incorrect

Action:

1. Ensure that the oraInst.loc file is located correctly and is properly formed.
   If the file is not in the default location, then set the RAT_INV_LOC environment variable to point to the oraInventory directory:

   $ export RAT_INV_LOC=oraInventory directory

2. If necessary, set the RAT_CRS_HOME environment variable to point to the location of the Oracle Clusterware home:

   $ export RAT_CRS_HOME=CRS_HOME

8.2.5.2 Oracle Database Software Is Installed, but Cannot Be Found

Description:

Oracle Orachk and Oracle Exachk display this message if the tools cannot find the Oracle Database software installed.

Action:

If the Oracle Database software is installed, but Oracle Orachk and Oracle Exachk cannot find, then set the RAT_ORACLE_HOME environment variable to the applicable ORACLE_HOME directory.

For example, enter the following command, where your-oracle-home is the path to the Oracle home on your system:

$ export RAT_ORACLE_HOME=your-oracle-home

Oracle Orachk and Oracle Exachk perform best practice and recommended patch checks for all the databases running from the home specified in the RAT_ORACLE_HOME environment variable.

8.2.5.3 Oracle Database Software Is Installed, but Version cannot Be Found

Description:

Oracle Orachk and Oracle Exachk display this message if the tools cannot find the version of the Oracle Database software installed.

Action:

If Oracle Orachk and Oracle Exachk cannot find the correct version, then set the RAT_DB environment variable to the applicable version.

For example:

$ export RAT_DB=11.2.0.3.0

8.2.5.4 Oracle ASM Software is Installed, but Cannot be Found

Description:

Oracle Orachk and Oracle Exachk display this message if the tools cannot find the Oracle ASM software installed.

Action:

If Oracle Orachk and Oracle Exachk cannot find Oracle ASM, then set the RAT_ASM_HOME environment variable to the applicable home directory.

$ export RAT_ASM_HOME=ASM_HOME

8.2.5.5 Oracle Database Discovery Issues on Oracle Real Application Clusters (Oracle RAC) Systems

Description:

On Oracle Real Application Clusters (Oracle RAC) systems, Oracle Orachk discovers the database resources registered in the Oracle Cluster Registry.

The ORACLE_HOME for the database resources is derived from the profile of the database resources.

Cause:

If there is a problem with any of the profiles of the database resources, then Oracle Orachk cannot recognize or connect to one or more databases. Use the -dbnames option temporarily to fix the problem.

Action:

Specify the names of the database in a comma-delimited list as follows:

$ orachk -dbnames ORCL,ORADB

Alternatively, use the space-delimited environment variable RAT_DBNAMES:

$ export RAT_DBNAMES="ORCL ORADB"

Use double quotes to specify more than one database.

Note:

Configure the RAT_DBHOMES environment variable if you,

• Configure RAT_DBNAMES as a subset of databases registered in the Oracle Clusterware
• Want to check the patch inventories of ALL databases found registered in the Oracle Clusterware for recommended patches

By default, the recommended patch analysis is limited to the homes for the list of databases specified in the RAT_DBNAMES environment variable.

To perform the recommended patch analysis for additional database homes, specify space-delimited list of all database names in the RAT_DBHOMES environment variable.

For example:

export RAT_DBNAMES="ORCL ORADB"

export RAT_DBHOMES="ORCL ORADB PROD"

Best practice checks are applied to ORACL and ORADB.

Recommended patch checks are applied to ORACL, ORADB, and PROD.

8.2.5.6 Oracle Database Login Problems

Oracle Database login problems arise if you run Oracle Orachk and Oracle Exachk without sufficient privileges. If you run Oracle Orachk and Oracle Exachk as a user other than the database software installation owner, root or grid, and if you experience problems connecting to the database, then perform the following steps:

1. Log in to the system as grid (operating system).
2. Run export ORACLE_HOME=path of Oracle database home
3. Run export ORACLE_SID=database SID
4. Run export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/lib:$PATH
5. Add alias in the $ORACLE_HOME/network/admin/tnsnames.ora file fordatabase SID.
6. Connect to the database using $ORACLE_HOME/bin/sqlplus "sys@SID as sysdba", and enter the password.
7. Ensure that you have a successful connection.

If this method of connecting to the database does not work, then Oracle Orachk and Oracle Exachk do not connect either.

• If you have multiple homes owned by different users and you are not able to login to the target database as the user running Oracle Orachk independently in SQL*Plus, then Oracle Orachk does not login either.
• If the operating system authentication is not set up, then it should still prompt you for user name and password.

8.2.6 Remote Login Problems

If Oracle Orachk and Oracle Exachk have problem locating and running SSH or SCP, then the tools cannot run any remote checks.

Also, the root privileged commands do not work if:

• Passwordless remote root login is not permitted over SSH
• Expect utility is not able to pass the root password

1. Verify that the SSH and SCP commands can be found.
   • The SSH commands return the error, No such file or directory, if SSH is not located where expected.

     Set the RAT_SSHELL environment variable pointing to the location of SSH. Note that you cannot export the RAT_SSHELL environment variable, set it in the conf_file. For more information, see Running Generic Compliance Framework (Oracle Orachk and Oracle Exachk) Commands.

   • The SCP commands return the error, /usr/bin/scp -q: No such file or directory, if SCP is not located where expected.

     Set the RAT_SCOPY environment variable pointing to the location of SCP. Note that you cannot export the RAT_SCOPY environment variable, set it in the conf_file. For more information, see Running Generic Compliance Framework (Oracle Orachk and Oracle Exachk) Commands.

2. Verify that the user you are running as, can run the following command manually from where you are running Oracle Orachk and Oracle Exachk to whichever remote node is failing.

   $ ssh root@remotehostname "id"
   root@remotehostname's password:
   uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)

   • If you face any problems running the command, then contact the systems administrators to correct temporarily for running the tool.
   • Oracle Orachk and Oracle Exachk search for the prompts or traps in remote user profiles. If you have prompts in remote profiles, then comment them out at least temporarily and test run again.
   • If you can configure passwordless remote root login, then edit the /etc/ssh/sshd_config file as follows:

     n to yes

     Now, run the following command as root on all nodes of the cluster:

     hd restart

3. Enable Expect debugging.
   • Oracle Orachk uses the Expect utility when available to answer password prompts to connect to remote nodes for password validation. Also, to run root collections without logging the actual connection process by default.
   • Set environment variables to help debug remote target connection issues.
     • RAT_EXPECT_DEBUG: If this variable is set to -d , then the Expect command tracing is activated. The trace information is written to the standard output.
       For example:

       export RAT_EXPECT_DEBUG=-d

     • RAT_EXPECT_STRACE_DEBUG: If this variable is set to strace, strace calls the Expect command. The trace information is written to the standard output.
       For example:

       export RAT_EXPECT_STRACE_DEBUG=strace

   • By varying the combinations of these two variables, you can get three levels of Expect connection trace information.

Note:

Set the RAT_EXPECT_DEBUG and RAT_EXPECT_STRACE_DEBUG variables only at the direction of Oracle support or development. The RAT_EXPECT_DEBUG and RAT_EXPECT_STRACE_DEBUG variables are used with other variables and user interface options to restrict the amount of data collected during the tracing. The script command is used to capture standard output.

As a temporary workaround while you resolve remote problems, run reports local on each node then merge them together later.

On each node, run:

orachk -local

exachk -local

Then merge the collections to obtain a single report:

orachk –merge zipfile 1  zip file 2 > zip file 3 > zip file ...

exachk –merge zipfile 1  zip file 2 > zip file 3 > zip file ...

8.2.7 Permission Problems

You must have sufficient directory permissions to run Oracle Orachk and Oracle Exachk.

1. Verify that the permissions on the tools scripts orachk and exachk  are set to 755 (-rwxr-xr-x).
   If the permissions are not set, then set the permissions as follows:

   $ chmod 755 orachk

   $ chmod 755 exachk

2. If you install Oracle Orachk and Oracle Exachk as root  and run the tools as a different user, then you may not have the necessary directory permissions.

   [root@randomdb01 exachk]# ls -la
   total 14072
   drwxr-xr-x  3 root root    4096 Jun  7 08:25 .
   drwxrwxrwt 12 root root    4096 Jun  7 09:27 ..
   drwxrwxr-x  2 root root    4096 May 24 16:50 .cgrep
   -rw-rw-r--  1 root root 9099005 May 24 16:50 collections.dat
   -rwxr-xr-x  1 root root  807865 May 24 16:50 exachk
   -rw-r--r--  1 root root 1646483 Jun  7 08:24 exachk.zip
   -rw-r--r--  1 root root    2591 May 24 16:50 readme.txt
   -rw-rw-r--  1 root root 2799973 May 24 16:50 rules.dat
   -rw-r--r--  1 root root     297 May 24 16:50 UserGuide.txt

• If Oracle Clusterware is installed, then:
  • Install Oracle Exachk in /opt/oracle.SupportTools/exachk as the Oracle Grid Infrastructure home owner
  • Install Oracle Orachk in CRS_HOME/suptools/orachk as the Oracle Grid Infrastructure home owner
• If Oracle Clusterware is not installed, then:
  • Install Oracle Exachk in /opt/oracle.SupportTools/exachk as root
  • Install Oracle Orachk (in a convenient location) as root (if possible)

    or

    Install Oracle Orachk (in a convenient location) as Oracle software install user or Oracle Database home owner

8.2.8 Slow Performance, Skipped Checks, and Timeouts

Follow these procedures to fix slow performance and other issues.

When Oracle Orachk and Oracle Exachk run commands, a child process is spawned to run the command and a watchdog daemon monitors the child process. If the child process is slow or hung, then the watchdog kills the child process and the check is registered as skipped:

Figure 8-1 Skipped Checks

Description of "Figure 8-1 Skipped Checks"

The watchdog.log file also contains entries similar to killing stuck command.

Depending on the cause of the problem, you may not see skipped checks.

1. Determine if there is a pattern to what is causing the problem.
   • EBS checks, for example, depend on the amount of data present and may take longer than the default timeout.
   • If there are prompts in the remote profile, then remote checks timeout and be killed and skipped. Oracle Orachk and Oracle Exachk search for prompts or traps in the remote user profiles. If you have prompts in remote profiles, then comment them out at least temporarily, and test run again.
2. Increase the default timeout.

   • You override the default timeouts by setting the environment variables.

     Table 8-1 Timeout Controlling

     Timeout Controlling	Default Value (seconds)	Environment Variable
     Collection of all checks not run by root (most). Specify the timeout value for individual checks.	Varies per check.	RAT_{CHECK-ID}_TIMEOUT
     General timeout for all checks	90	RAT_TIMEOUT
     SSH login DNS handshake. Specify the time in seconds for checking passwords on the remote nodes.	1	RAT_PASSWORDCHECK_TIMEOUT

   • The default timeouts are lengthy enough for most cases. If it is not long enough, then it is possible you are experiencing a system performance problem that should be corrected. Many timeouts can be indicative of a non-Oracle Orachk and Oracle Exachk problem in the environment.

3. If you can not increase the timeout, then try excluding problematic checks running separately with a large enough timeout and then merging the reports back together.
4. If the problem does not appear to be down to slow or skipped checks but you have a large cluster, then try increasing the number of slave processes users for parallel database run.
   • Database collections are run in parallel. The default number of slave processes used for parallel database run is calculated automatically. You can change the default number using the options:-dbparallel slave processes, or –dbparallelmax

     The higher the parallelism the more resources are consumed. However, the elapsed time is reduced. You can raise or lower the number of parallel slaves beyond the default value. After the entire system is brought up after maintenance, but before the users are permitted on the system, use a higher number of parallel slaves to finish a run as quickly as possible.

     On a busy production system, use a number less than the default value yet more than running in serial mode to get a run more quickly with less impact on the running system.

     Turn off the parallel database run using the -dbserial option.

8.2.9 Running Compliance Checks on a Subset of Oracle Home and Oracle Databases

Use the -dbconfig command when Oracle Orachk and Oracle Exachk are not able to discover ORACLE_HOME, or the name of the Oracle Database and you have multiple ORACLE_HOMEs in a system and each home is running multiple Oracle Databases.

Syntax

orachk -dbconfig dbhome%dbname(s)[,dbhome%dbname(s)]

exachk -dbconfig dbhome%dbname(s)[,dbhome%dbname(s)]

Specify a comma-delimited list of Oracle Database homes with corresponding Oracle Database names to run health checks only on a subset of Oracle Databases. Oracle Orachk and Oracle Exachk do not prompt you for database selection while running.

Separate the Oracle Database home and the corresponding Oracle Database names with %. Specify the list of Oracle Database names associated with a particular Oracle Database home with :.

orachk -dbconfig dbhome%dbname:dbname:...[,dbhome%dbname:dbname:...]

exachk -dbconfig dbhome%dbname:dbname:...[,dbhome%dbname:dbname:...]

Example 8-1 Limiting Health Checks to a Subset of Oracle Home and Oracle Databases

orachk -dbconfig dbhome1%dbname
orachk -dbconfig dbhome1%dbname1:dbname2
orachk -dbconfig dbhome1%dbname1:dbname2,dbhome2%dbname3:dbname4

8.2.10 SSH Connection Timeout

Description: SSH connection time out as follows:

Select databases from list for checking best practices. For multiple databases, select 4 for All or comma separated number like 1,2 etc [1-5][4].
Searching out ORACLE_HOME for selected databases.
.  .  .  .  .  .  .  .  .  .
.  .  .  .  .  .  .  .  .  Connection to scag11adm02 closed by remote host.

Action: Set ServerAliveInterval to 30 in the /etc/ssh/ssh_config file on the machine from where Oracle EXAchk run started.

8.2.11 Oracle Exachk Prompts to Enter Names of RoCE Fabric Switches

Description: Oracle Exachk prompts you to enter RoCE switch names each time it's run in an Exadata X8M or higher multi-rack environment.

Action: Add a list of RoCE fabric leaf and spine switches to the switches.out file, and then run Oracle Exachk.

For example:

# exachk -showdatadir
/u01/app/grid/oracle.ahf/data/scaqan07adm01/exachk/user_root/output
# vi /u01/app/grid/oracle.ahf/data/scaqan07adm01/exachk/user_root/output/switches.out
<<add switches>>
# cat /u01/app/grid/oracle.ahf/data/scaqan07adm01/exachk/user_root/output/switches.out
scaqan07sw-rocea0
scaqan07sw-roceb0

Even if you have created a file with a list of switches, you can still pass the -switches argument to not to read from that file. Specify a comma-delimited list of switch names to run exachk on specific switches.

For example:

# exachk -switches switch1,switch2,...

If you do not provide a list of switches, then exachk prompts you to provide it.

MyTestPrompt>
MyTestPrompt> exachk
Enter RDMA Network Fabric switch names delimited by comma or skip switches using -excludeprofile switch option. (eg switch1,switch2,switch3):

To exclude switches from exachk running on them, specify a list of comma-delimited list of switches:

-excludeprofile -switches switch1,switch2,...

8.2.12 Unable to Implement CA Certificates in Oracle Trace File Analyzer

Description: After implementing TFAMain starts but ends up in client server SSL socket exceptions errors when running tfactl commands.

Cause: Combining both intermediate.pem and server.pem file into the caroot.cert.txt file results in Empty server certificate chain error.

Action: Split the caroot.cert.txt file into intermediate.pem and server.pem using the command openssl x509 -in cerfile.cer -noout -text, and then follow the keytool steps again.

1. keytool -importkeystore -destkeystore server.p12 -deststoretype pkcs12 -srckeystore serverCert.pfx

2. keytool -v -importkeystore -srckeystore server.p12 -srcstoretype PKCS12 -destkeystore server_ac.jks -deststoretype JKS

3. keytool -v -importkeystore -srckeystore server.p12 -srcstoretype PKCS12 -destkeystore client_ac.jks -deststoretype JKS

4. keytool -list -keystore client_ac.jks
   Enter keystore pswrd:  
   Keystore type: jks
   Keystore provider: SUN
   
   Your keystore contains 1 entry
   
   1, Nov 30, 2021, PrivateKeyEntry, 
   Certificate fingerprint (SHA1):
   59:BA:C8:94:97:48:9C:6C:11:23:36:F9:46:A1:1C:87:67:F7:84:19
   
   Warning:
   The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client_ac.jks -destkeystore client_ac.jks -deststoretype pkcs12".

5. # keytool -list -keystore server_ac.jks
   Enter keystore pswrd:  
   Keystore type: jks
   Keystore provider: SUN
   
   Your keystore contains 1 entry
   
   1, Nov 30, 2021, PrivateKeyEntry,
   Certificate fingerprint (SHA1):
   59:BA:C8:94:97:48:9C:6C:11:23:36:F9:46:A1:1C:87:67:F7:84:19
   
   Warning:
   The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server_ac.jks -destkeystore server_ac.jks -deststoretype pkcs12".

6. keytool -import -v -alias server-ca -file server.cert.pem -keystore client_ac.jks

7. keytool -import -v -alias client-ca -file server.cert.pem -keystore server_ac.jks

8. keytool -importcert -trustcacerts -alias inter -file intermediate.cert.pem -keystore server_ac.jks

9. keytool -list -keystore server_ac.jks
   Enter keystore pswrd:  
   Keystore type: jks
   Keystore provider: SUN
   
   Your keystore contains 3 entries
   
   inter, Nov 30, 2021, trustedCertEntry,
   Certificate fingerprint (SHA1):
   F6:E3:AA:60:E0:D0:80:69:12:72:06:E0:FA:62:7A:EB:54:38:11:55
   client-ca, Nov 30, 2021, trustedCertEntry,
   Certificate fingerprint (SHA1):
   59:BA:C8:94:97:48:9C:6C:11:23:36:F9:46:A1:1C:87:67:F7:84:19
   1, Nov 30, 2021, PrivateKeyEntry,
   Certificate fingerprint (SHA1):
   59:BA:C8:94:97:48:9C:6C:11:23:36:F9:46:A1:1C:87:67:F7:84:19
   
   Warning:
   The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server_ac.jks -destkeystore server_ac.jks -deststoretype pkcs12"

10. # keytool -list -keystore client_ac.jks
    Enter keystore pswrd:  
    Keystore type: jks
    Keystore provider: SUN
    
    Your keystore contains 2 entries
    
    1, Nov 30, 2021, PrivateKeyEntry,
    Certificate fingerprint (SHA1):
    59:BA:C8:94:97:48:9C:6C:11:23:36:F9:46:A1:1C:87:67:F7:84:19
    server-ca, Nov 30, 2021, trustedCertEntry,
    Certificate fingerprint (SHA1):
    59:BA:C8:94:97:48:9C:6C:11:23:36:F9:46:A1:1C:87:67:F7:84:19
    
    Warning:
    The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client_ac.jks -destkeystore client_ac.jks -deststoretype pkcs12".