1. Oracle Autonomous Health Framework User's Guide
  2. Components of Autonomous Health Framework

Components of Autonomous Health Framework

Describes the diagnostic components that are part of Oracle Autonomous Health Framework.

Oracle Autonomous Health Framework Configuration Audit Tools

Oracle Orachk and Oracle Exachk provide a lightweight, non-intrusive framework for performing health and compliance checks across Oracle software and hardware components.

Oracle Orachk and Oracle Exachk help you:

  • Automate risk identification and proactively identify issues before they affect your business.
  • Run health checks for critical and recurring problems.
  • Generate high-level reports that identify system health risks and exposure to known issues.
  • Drill down into individual findings to understand problems, recommendations, and corrective actions.
  • Schedule recurring health checks at regular intervals.
  • Receive email notifications and comparison reports when compliance checks run in daemon mode.
  • Integrate findings with Oracle Health Check Collections Manager and other tools.
  • Run health checks entirely within your environment without requiring you to send data to Oracle.

Oracle Orachk and Oracle Exachk are available as a value-added benefit of an existing Oracle support contract. No additional license or fee is required to run these tools.

Use:

  • Oracle Exachk for Oracle Engineered Systems, except Oracle Database Appliance.
  • Oracle Orachk for other supported systems, including Oracle Database Appliance.

Run Oracle Orachk or Oracle Exachk with the appropriate command-line options to perform health and compliance checks for supported Oracle products.

Oracle Trace File Analyzer

Oracle Trace File Analyzer (TFA) provides targeted diagnostic collection to simplify the collection of diagnostic data for:

  • Oracle Clusterware
  • Oracle Grid Infrastructure
  • Oracle Real Application Clusters (Oracle RAC)
  • Single-instance, non-clustered Oracle Databases

Oracle Trace File Analyzer is enabled by default and helps you:

  • Collect comprehensive first-failure diagnostic data.
  • Efficiently collect, package, and transfer diagnostic data to Oracle Support.
  • Reduce the number of interactions required between customers and Oracle Support to obtain the diagnostic information needed to investigate an issue.

By automating and targeting diagnostic data collection, Oracle Trace File Analyzer reduces the time required to gather relevant information for problem investigation and resolution.

For more information, see Oracle Autonomous Health Framework Checks and Diagnostics User's Guide.

Attention Log for Efficient Critical Issue Resolution

Oracle Database provides an attention log to improve the diagnosis of database issues.

The attention log:

  • Contains information that requires administrator attention.
  • Uses a structured XML or JSON format that makes the information easier to process and interpret.
  • Helps administrators focus on significant events without reviewing all information contained in database trace files.

Database trace files also contain information that enables trace messages to be classified more easily, including classifications related to security and data sensitivity.

These diagnosability enhancements simplify database administration, improve the identification of critical issues, and help protect sensitive diagnostic information.

AHF Insights

AHF Insights provides a consolidated view of the health and performance of your system, with the ability to drill down into diagnostic information for root cause analysis.

Note:

Starting with AHF 23.8, AHF Insights no longer has a plotly.js dependency on a Content Delivery Network (CDN). This enables AHF Insights reports to be used in environments with restricted external network access.

AHF Insights provides a web-based graphical interface that consolidates results from AHF diagnostic collectors and analyzers into a single report. The report does not require a web server to host or display its contents.

AHF performs contextual diagnostic collection for a specified period and analyzes the collected data to help assess database system performance.

The diagnostic collection can include data from AHF features in the following areas:

  • Configuration
  • Environment topology
  • Metrics
  • Logs

AHF Insights analyzes the collected diagnostic data and generates an offline report that provides information about:

  • System configuration
  • System state
  • Operating system anomalies
  • Best practices compliance
  • System traces
  • Root causes and recommended fixes for certain detected anomalies

Generating an AHF Insights Report

To generate an AHF Insights report, run:

ahf analysis create --type insights

You can specify a time range for the analysis. For example, to analyze data from the previous two hours:

ahf analysis create --type insights --last 2h

Before generating the report, you can verify the status of Oracle Trace File Analyzer:

tfactl print status

Example output:

[root@node02 ~]# tfactl print status

.-----------------------------------------------------------------------------------------------.
| Host   | Status of TFA | PID    | Port | Version    | Build ID             | Inventory Status |
+--------+---------------+--------+------+------------+----------------------+------------------+
| node02 | RUNNING       | 134679 | 5000 | 22.3.0.0.0 | 22300020221031131221 | COMPLETE         |
| node01 | RUNNING       | 128438 | 5000 | 22.3.0.0.0 | 22300020221031131221 | COMPLETE         |
'--------+---------------+--------+------+------------+----------------------+------------------'

Generate an AHF Insights report for the previous two hours:

[root@node02 ~]# ahf analysis create --type insights --last 2h

Starting analysis and collecting data for insights

Collecting data for AHF Insights
(This may take a few minutes per node)

AHF Insights report is being generated for the last 2h

From Date : 11/20/2022 01:16:41 UTC
To Date   : 11/20/2022 03:17:15 UTC

Report is generated at :
/opt/oracle.ahf/data/repository/collection_Sun_Nov_20_03_16_36_UTC_2022_node_all/cgexa-ogmn12_insights_2022_11_20_03_18_13.zip

After the analysis completes, AHF displays the location of the generated AHF Insights report ZIP file.

Oracle Cluster Health Advisor

Oracle Cluster Health Advisor continuously monitors cluster nodes and Oracle Real Application Clusters (Oracle RAC) databases for conditions that can lead to performance or availability problems. It provides early warning before detected problems become critical.

Oracle Cluster Health Advisor:

  • Detects database and cluster node performance problems.
  • Provides early-warning alerts, targeted diagnoses, and corrective actions.
  • Supports onsite model calibration to improve detection accuracy and sensitivity.

Oracle Cluster Health Advisor monitors two critical Oracle RAC subsystems:

  • Oracle Database instances
  • Host systems

It continuously determines and tracks the health of monitored systems by periodically sampling a wide range of database and operating system measurements.

Oracle Cluster Health Advisor includes models for more than a hundred database and cluster node problems. These models identify combinations of operating system and Oracle Database metrics that can indicate the development or presence of specific problems.

Oracle Cluster Health Advisor uses trained and calibrated models based on normal operating characteristics to evaluate the monitored environment.

Detecting and Diagnosing Problems

Oracle Cluster Health Advisor performs analysis multiple times per minute.

During analysis, Oracle Cluster Health Advisor:

  1. Estimates the expected value of each observed input based on its model.
  2. Compares the observed value with the expected value.
  3. Detects anomalies based on differences between the observed and expected values.
  4. Determines whether enough inputs associated with a specific problem exhibit abnormal behavior.
  5. Raises a warning when the detected conditions indicate a potential problem.
  6. Provides a targeted diagnosis and recommended corrective action.

This approach enables Oracle Cluster Health Advisor to identify developing performance and availability problems before they become critical.

Calibrating Oracle Cluster Health Advisor

Oracle Cluster Health Advisor models are conservative by default to reduce false warning notifications. However, the default models might not provide sufficient sensitivity for some critical production environments.

Use onsite model calibration to create models based on the actual workload characteristics of your production environment.

Calibration can improve:

  • Accuracy of problem detection.
  • Sensitivity of database and cluster node models.
  • Relevance of alerts to the normal workload characteristics of the monitored system.

Diagnosing Past Problems

You can also use Oracle Cluster Health Advisor to investigate and triage problems that occurred in the past.

Specify the historical time period that you want to analyze using:

  • The CHACTL command-line interface.
  • AHF Insights.
  • AHF Scope.

Historical analysis enables you to examine system behavior around a previous incident and use Oracle Cluster Health Advisor diagnostics to help identify the cause.

AHF Scope

AHF Scope is a standalone, interactive front end to Oracle Cluster Health Advisor (CHA) that supports real-time analysis while requiring a small footprint on the monitored system.

Note:

Autonomous Health Framework Scope (AHF Scope), a feature of Cluster Health Advisor, is deprecated in Oracle AI Database 26ai.

Oracle recommends using AHF Insights instead. When AHF detects issues such as node evictions, instance evictions, or database performance degradation, AHF Insights generates a comprehensive report that identifies the problem, provides root cause information, and recommends corrective actions.

The Database Anomalies Advisor also provides a timeline view of anomalies across hosts and database instances. You can drill down into individual findings to review their underlying causes and recommended resolutions.

AHF Scope is primarily designed for cluster and database experts and can efficiently process large amounts of diagnostic data. Its interface is optimized for operational efficiency, with most actions available through a positional pointer, hot keys, or a floating menu at the cursor position.

To start AHF Scope, use the ahfscope script located in:

/opt/oracle.ahf/ahfscope/bin/

When Grid Infrastructure Management Repository (GIMR) is configured, AHF Scope can connect directly to GIMR through a JDBC connection and analyze current data in real time.

AHF Scope can also operate without a connection to GIMR by using a data archive extracted from GIMR.

Note:

GIMR is optionally supported in Oracle Database 19c and is desupported in Oracle AI Database 26ai. If GIMR is configured in an existing Oracle Grid Infrastructure installation that is being upgraded to Oracle AI Database 26ai, remove GIMR.

AHF Balance

AHF Balance is a command-line utility that analyzes historical CPU consumption and Database Resource Manager (DBRM) settings for databases running in a cluster.

AHF Balance helps you:

  • Analyze historical CPU usage across databases in a cluster.
  • Identify CPU-based noisy neighbor problems.
  • Understand when and how noisy neighbor conditions occurred.
  • Review the DBRM settings associated with the databases.
  • Obtain recommended DBRM settings to reduce the risk of future noisy neighbor problems.

Use AHF Balance to evaluate how databases share CPU resources within a cluster and determine whether DBRM configuration changes can provide more effective resource isolation.

Cluster Health Monitor

Cluster Health Monitor is a component of Oracle Grid Infrastructure that continuously monitors and stores metrics for Oracle Clusterware and operating system resources.

Cluster Health Monitor is enabled by default and helps you:

  • Analyze node evictions.
  • Record process data locally.
  • Define processes to be pinned for monitoring.
  • Monitor Cluster Synchronization Services (CSS) and Grid Interprocess Communication (GIPC) events.
  • Categorize processes by type.
  • Use plug-in collectors such as traceroute, netstat, and ping.
  • Generate CSV output for easier analysis.

The metrics collected by Cluster Health Monitor also serve as a data source for other Oracle Autonomous Health Framework components, including Oracle Cluster Health Advisor.

Blocker Resolver

Blocker Resolver is an Oracle Real Application Clusters (Oracle RAC) feature that automatically detects and resolves database delays and deadlocks to help maintain resource availability.

Blocker Resolver is enabled by default and:

  • Reliably detects database delays and deadlocks.
  • Automatically resolves detected database delays and deadlocks.
  • Logs detected conditions and the actions taken to resolve them.
  • Provides a SQL interface for configuring detection sensitivity (Normal or High) and trace file sizes.

A database delay occurs when one session blocks a chain of one or more other sessions. The blocking session holds a resource, such as a lock or latch, that prevents the blocked sessions from making progress.

A chain of blocked sessions has a root blocker, or final blocking session, that ultimately prevents the other sessions in the chain from progressing.

Blocker Resolver monitors for these conditions and autonomously detects and resolves delays to restore access to the affected database resources.