B Troubleshooting Oracle RAC

This how diagnose problems for Oracle Real Application Clusters (Oracle RAC) components using trace and log files.

Note:

Trace and log files, similar to those generated for Oracle AI Database with Oracle RAC, are also available for the Oracle Clusterware components. For Oracle Clusterware, Oracle AI Database stores these under a unified directory log structure.

Where to Find Files for Analyzing Errors

Oracle RAC records significant events in trace files stored in the Automatic Diagnostic Repository (ADR), whose location is controlled by the DIAGNOSTIC_DEST parameter. Each instance has an alert_SID.log file containing ORA-600 errors and other operational exceptions, while background and user processes generate separate trace files with operating-system-specific names. Regular monitoring and backup of these files helps preserve diagnostic information for troubleshooting.

Managing Diagnostic Data in Oracle RAC

Oracle RAC provides ADR-based infrastructure for collecting and managing diagnostic data across instances, which is especially useful when problems require correlating or merging trace files. Placing all instance ADR homes under a shared ADR base enables ADRCI to view, correlate, and package incidents, problems, traces, dumps, health reports, and alert entries for Oracle Support. The Data Recovery Advisor can also diagnose and repair persistent failures such as corrupted blocks or missing files.

Using Instance-Specific Alert Files in Oracle RAC

Each Oracle RAC instance maintains its own alert_SID.log file in the directory specified by DIAGNOSTIC_DEST. The file records important errors, exceptions, and startup-related information, is appended to whenever the instance starts, and can be written by all process threads associated with that instance.

Enabling Tracing for Java-Based Tools and Utilities in Oracle RAC

Oracle RAC Java-based utilities, including CVU, DBCA, NETCA, and SRVCTL, are invoked through scripts named for each utility. Tracing is enabled by default for DBCA and DBUA, while setting SRVM_TRACE to TRUE enables tracing for CVU and SRVCTL. Generated logs are stored in utility-specific directories under Oracle_base/cfgtoollogs, such as the dbca and dbua directories.

Resolving Pending Shutdown Issues

When SHUTDOWN IMMEDIATE remains pending, Oracle Clusterware may still be processing the original shutdown request, so repeated immediate shutdown commands may not respond promptly. Subsequent shutdown attempts should use SHUTDOWN ABORT through SQL*Plus.

How to Determine If Oracle RAC Instances Are Using the Private Network

The private interconnect used by Oracle RAC can be examined with the oradebug ipc command, which writes network details to a trace file under DIAGNOSTIC_DEST; oradebug tracefile_name identifies that file. The V$CLUSTER_INTERCONNECTS view provides additional interconnect information, including interface names, IP addresses, status, and source, although Enterprise Manager Cloud Control is the recommended method for checking the interconnect.