17 Troubleshooting Oracle Database Appliance
Understand tools you can use to validate changes and troubleshoot Oracle Database Appliance problems.
- Resolving Errors When Updating DCS Components During Patching
Understand how to troubleshoot errors when updating DCS components during patching. - Viewing Component Information on the Appliance
View details of all the components installed on the appliance, and the RPM drift information. - Errors When Logging into the Browser User Interface
If you have problems logging into the Browser User Interface, then it may be due to your browser or credentials. - Errors when re-imaging Oracle Database Appliance
Understand how to troubleshoot errors that occur when re-imaging Oracle Database Appliance. - Using Oracle Autonomous Health Framework for Running Diagnostics
Oracle Autonomous Health Framework collects and analyzes diagnostic data collected, and proactively identifies issues before they affect the health of your system. - Running the Disk Diagnostic Tool
Use the Disk Diagnostic Tool to help identify the cause of disk problems. - Running the Oracle Database Appliance Hardware Monitoring Tool
The Oracle Database Appliance Hardware Monitoring Tool displays the status of different hardware components in Oracle Database Appliance server. - Configuring a Trusted SSL Certificate for Oracle Database Appliance
The Browser User Interface and DCS Controller use SSL-based HTTPS protocol for secure communication. Understand the implications of this added security and the options to configure SSL certificates. - Disabling the Browser User Interface
You can also disable the Browser User Interface. Disabling the Browser User Interface means you can only manage your appliance through the command-line interface. - Preparing Log Files for Oracle Support Services
If you have a system fault that requires help from Oracle Support Services, then you may need to provide log records to help Oracle support diagnose your issue.
Resolving Errors When Updating DCS Components During Patching
Understand how to troubleshoot errors when updating DCS components during patching.
.
About DCS Components
odacli update-dcscomponents
command
during patching, pre-checks for MySQL installation are automatically verified before
update of Zookeeper, MySQL, and DCS components. If any of the pre-checks fail, then
the command errors out with a reference to the pre-check report log file location
/opt/oracle/dcs/log/jobId-dcscomponentsPreCheckReport.log
.
Review the pre-check report and take corrective actions and then rerun the
odacli update-dcscomponents
command. If there are no pre-check
errors, then the patching process proceeds with updating Zookeeper, MySQL, and DCS
components such as the DCS Agent, DCS CLI, and DCS Controller.
Note:
Run theodacli
update-dcsadmin
command prior to running the odacli
update-dcscomponents
command.
When the odacli update-dcscomponents
command completes
successfully:
The command output is as follows:
# ./odacli update-dcscomponents -v 19.10.0.0.0
{
"jobId" : "3ac3667a-fa22-40b6-a832-504a56aa3fdc",
"status" : "Success",
"message" : "Update-dcscomponents is successful on all the node(s):DCS-Agent
shutdown is successful. MySQL upgrade is done before. Metadata migration is
successful. Agent rpm upgrade is successful. DCS-CLI rpm upgrade is successful.
DCS-Controller rpm upgrade is succ",
"reports" : null,
"createTimestamp" : "February 22, 2021 02:37:37 AM CST",
"description" : "Update-dcscomponents job completed and is not part of Agent
job list",
"updatedTime" : "February 22, 2021 02:39:10 AM CST"
}
The pre-check report log file at the location
/opt/oracle/dcs/log/jobId-dcscomponentsPreCheckReport.log
contains the following:
Pre-check Name: Space check
Status: Success
Comments: Required space 3 GB is available in /opt
Pre-check Name: Port check
Status: Success
Comments: Port 3306 is available for running ODA MySQL
Pre-check Name: ODA MySQL rpm installation dry-run check
Status: Success
Comments: ODA MySQL rpm dry-run passed
Pre-check Name: Check for the existence of MySQL connector/J library
Status: Success
Comments: ODA MySQL connector/J library found
Pre-check Name: Check for the existence of Metadata migration utility
Status: Success
Comments: Metadata migration utility found
When the odacli update-dcscomponents
command fails:
On Oracle Database Appliance single-node systems, the command output is as follows:
# ./odacli update-dcscomponents -v 19.10.0.0.0
DCS-10008:Failed to update DCScomponents: 19.10.0.0.0
Internal error while patching the DCS components :
DCS-10231:Cannot proceed. Pre-checks for update-dcscomponents failed. Refer to
/opt/oracle/dcs/log/jobId-dcscomponentsPreCheckReport.log
on node 0 for details.
On Oracle Database Appliance high-availability systems, the command output is as follows:
# ./odacli update-dcscomponents -v 19.10.0.0.0
Internal error while patching the DCS components :
DCS-10231:Cannot proceed. Pre-checks for update-dcscomponents failed. Refer to
/opt/oracle/dcs/log/jobId-dcscomponentsPreCheckReport.log
on node 0 and /opt/oracle/dcs/log/jobId-dcscomponentsPreCheckReport.log
on node 1 for details.
The command runs all pre-checks one by one, and errors out at the end if
any of the pre-checks is marked as Failed. When a pre-check fails, the error message
is displayed on to the console along with the reference to pre-check report log
location. The pre-check report log file is at the location
/opt/oracle/dcs/log/jobId-dcscomponentsPreCheckReport.log
.
Pre-check Name: Space check
Status: Failed
Comments: Available space in /opt is 2 GB but minimum required space in /opt is 3 GB
Pre-check Name: Port check
Status: Success
Comments: Port 3306 is available for running ODA MySQL
Pre-check Name: ODA MySQL rpm installation dry-run check
Status: Success
Comments: ODA MySQL rpm dry-run passed
Pre-check Name: Check for the existence of MySQL connector/J library
Status: Success
Comments: ODA MySQL connector/J library found
Pre-check Name: Check for the existence of Metadata migration utility
Status: Success
Comments: Metadata migration utility found
When the odacli update-dcscomponents
command fails due to space
check error:
The pre-check report log contains the following:
Pre-check Name: Space check
Status: Failed
Comments: Available space in /opt is 2 GB but minimum required space in /opt is 3 GB
Pre-check Name: Port check
Status: Success
Comments: Port 3306 is available for running ODA MySQL
Pre-check Name: ODA MySQL rpm installation dry-run check
Status: Success
Comments: ODA MySQL rpm dry-run passed
Pre-check Name: Check for the existence of MySQL connector/J library
Status: Success
Comments: ODA MySQL connector/J library found
Pre-check Name: Check for the existence of Metadata migration utility
Status: Success
Comments: Metadata migration utility found
When the odacli update-dcscomponents
command fails due to port
check error:
The pre-check report log contains the following:
Pre-check Name: Space check
Status: Success
Comments: Required space 3 GB is available in /opt
Pre-check Name: Port check
Status: Failed
Comments: No port found in the range ( 3306 to 65535 )
Pre-check Name: ODA MySQL rpm installation dry-run check
Status: Success
Comments: ODA MySQL rpm dry-run passed
Pre-check Name: Check for the existence of MySQL connector/J library
Status: Success
Comments: ODA MySQL connector/J library found
Pre-check Name: Check for the existence of Metadata migration utility
Status: Success
Comments: Metadata migration utility found
When the odacli update-dcscomponents
command fails due to MySQL
RPM installation dry-run check error:
The pre-check report log contains the following:
Pre-check Name: Space check
Status: Success
Comments: Required space 3 GB is available in /opt
Pre-check Name: Port check
Status: Success
Comments: Port 3306 is available for running ODA MySQL
Pre-check Name: ODA MySQL rpm installation dry-run check
Status: Failed
Comments: ODA MySQL rpm dry-run failed. Failed due to the following error :
Exception details are displayed below
Pre-check Name: Check for the existence of MySQL connector/J library
Status: Success
Comments: ODA MySQL connector/J library found
Pre-check Name: Check for the existence of Metadata migration utility
Status: Success
Comments: Metadata migration utility found
When the odacli update-dcscomponents
command fails due to MySQL
connector/J library check error:
The pre-check report log contains the following:
Pre-check Name: Space check
Status: Success
Comments: Required space 3 GB is available in /opt
Pre-check Name: Port check
Status: Success
Comments: Port 3306 is available for running ODA MySQL
Pre-check Name: ODA MySQL rpm installation dry-run check
Status: Success
Comments: ODA MySQL rpm dry-run passed
Pre-check Name: Check for the existence of MySQL connector/J library
Status: Failed
Comments: MySQL connector/J library does not exist. Ensure update-repository with latest serverzip bundles ran first without any issues prior to running update-dcscomponents
Pre-check Name: Check for the existence of Metadata migration utility
Status: Success
Comments: Metadata migration utility found
When the odacli update-dcscomponents
command fails due to
Metadata migration utility check error:
The pre-check report log contains the following:
Pre-check Name: Space check
Status: Success
Comments: Required space 3 GB is available in /opt
Pre-check Name: Port check
Status: Success
Comments: Port 3306 is available for running ODA MySQL
Pre-check Name: ODA MySQL rpm installation dry-run check
Status: Success
Comments: ODA MySQL rpm dry-run passed
Pre-check Name: Check for the existence of MySQL connector/J library
Status: Success
Comments: ODA MySQL connector/J library found
Pre-check Name: Check for the existence of Metadata migration utility
Status: Failed
Comments: Metadata migration utility does not exist. Ensure update-repository with latest serverzip bundles ran first without any issues prior to running update-dcscomponents
Parent topic: Troubleshooting Oracle Database Appliance
Viewing Component Information on the Appliance
View details of all the components installed on the appliance, and the RPM drift information.
Viewing the Bill of Materials in the Browser User Interface
Use the Appliance tab in the Browser User Interface to view information about your deployment and the installed components. The Advanced Information tab displays information about the following components:
-
Grid Infrastructure Version, and the home directory
-
Database Version, Home location, and Edition
-
Location and details about the databases configured
-
All patches applied to the appliance
-
Firmware Controller and Disks
-
ILOM information
-
BIOS version
-
List of RPMs
In the List of RPMs section, click Show and then click RPM Drift to view the differences between the RPMs installed on the appliance, and the RPMs shipped in the latest Oracle Database Appliance Patch Bundle Update release.
Click Download to save the components report. You can use this report to help diagnose any deployment issues.
Viewing the Bill of Materials from the Command Line
The bill of materials is also available through the command line for bare metal and virtualized platforms deployments. The information about the installed components is collected according to a set schedule, and stored in the location /opt/oracle/dcs/Inventory/
for bare metal deployments and in the /opt/oracle/oak/Inventory/
directory for virtualized platforms. The file is stored in the format oda_bom_TimeStamp.json
. Use the command describe-system
to view the bill of materials on the command line. See the Oracle Database Command-Line Interface chapter for command options and usage notes.
Example 17-1 Example Command to View the Bill of Materials from the Command Line for Bare Metal Deployments
# odacli describe-system -b
ODA Components Information
------------------------------
Component Name Component Details
--------------- -----------------------------------------------------------------------------------------------
NODE Name : oda1
Domain Name : testdomain.com
Time Stamp : April 21, 2020 6:21:15 AM UTC
RPMS Installed RPMS : abrt-2.1.11-55.0.1.el7.x86_64,
abrt-addon-ccpp-2.1.11-55.0.1.el7.x86_64,
abrt-addon-kerneloops-2.1.11-55.0.1.el7.x86_64,
abrt-addon-pstoreoops-2.1.11-55.0.1.el7.x86_64,
abrt-addon-python-2.1.11-55.0.1.el7.x86_64,
abrt-addon-vmcore-2.1.11-55.0.1.el7.x86_64,
abrt-addon-xorg-2.1.11-55.0.1.el7.x86_64,
abrt-cli-2.1.11-55.0.1.el7.x86_64,
abrt-console-notification-2.1.11-55.0.1.el7.x86_64,
abrt-dbus-2.1.11-55.0.1.el7.x86_64,
abrt-libs-2.1.11-55.0.1.el7.x86_64,
abrt-python-2.1.11-55.0.1.el7.x86_64,
abrt-tui-2.1.11-55.0.1.el7.x86_64,
acl-2.2.51-14.el7.x86_64,
adwaita-cursor-theme-3.28.0-1.el7.noarch,
adwaita-icon-theme-3.28.0-1.el7.noarch,
aic94xx-firmware-30-6.el7.noarch,
aide-0.15.1-13.0.1.el7.x86_64,
alsa-firmware-1.0.28-2.el7.noarch,
alsa-lib-1.1.8-1.el7.x86_64,
alsa-tools-firmware-1.1.0-1.el7.x86_64,
at-3.1.13-24.el7.x86_64,
at-spi2-atk-2.26.2-1.el7.x86_64,
at-spi2-core-2.28.0-1.el7.x86_64,
atk-2.28.1-1.el7.x86_64,
attr-2.4.46-13.el7.x86_64,
audit-2.8.5-4.el7.x86_64,
audit-libs-2.8.5-4.el7.x86_64,
audit-libs-python-2.8.5-4.el7.x86_64,
augeas-libs-1.4.0-9.el7.x86_64,
authconfig-6.2.8-30.el7.x86_64,
autogen-libopts-5.18-5.el7.x86_64,
avahi-libs-0.6.31-19.el7.x86_64,
basesystem-10.0-7.0.1.el7.noarch,
bash-4.2.46-33.el7.x86_64,
bash-completion-2.1-6.el7.noarch,
bc-1.06.95-13.el7.x86_64,
bind-export-libs-9.11.4-9.P2.el7.x86_64,
bind-libs-9.11.4-9.P2.el7.x86_64,
bind-libs-lite-9.11.4-9.P2.el7.x86_64,
bind-license-9.11.4-9.P2.el7.noarch,
bind-utils-9.11.4-9.P2.el7.x86_64,
binutils-2.27-41.base.0.7.el7_7.2.x86_64,
biosdevname-0.7.3-2.el7.x86_64,
blktrace-1.0.5-9.el7.x86_64,
bnxtnvm-1.40.10-1.x86_64,
boost-date-time-1.53.0-27.el7.x86_64,
boost-filesystem-1.53.0-27.el7.x86_64,
boost-iostreams-1.53.0-27.el7.x86_64,
....
....
....
Example 17-2 Example Command to View the Bill of Materials from the Command Line for Virtualized Platforms
# oakcli describe-system -b
Example 17-3 Example Command to View the Bill of Materials Report from the Stored Location
# ls -la /opt/oracle/dcs/Inventory/
total 264
-rw-r--r-- 1 root root 83550 Apr 26 05:41 oda_bom_2018-04-26_05-41-36.json
Parent topic: Troubleshooting Oracle Database Appliance
Errors When Logging into the Browser User Interface
If you have problems logging into the Browser User Interface, then it may be due to your browser or credentials.
Note:
Oracle Database Appliance uses self-signed certificates. Your browser determines how you log into the Browser User Interface. Depending on the browser and browser version, you may receive a warning or error that the certificate is invalid or not trusted because it is self-signed, or that the connection is not private. Ensure that you accept the self-signed certificate for the agent and Browser User Interface.Follow these steps to log into the Browser User Interface:
Note:
If you have any issues logging into the Oracle Database Appliance Browser User Interface on browsers such as macOS Catalina and Google Chrome, then you may need to use any workaround as described on the official site for the product.Errors when re-imaging Oracle Database Appliance
Understand how to troubleshoot errors that occur when re-imaging Oracle Database Appliance.
If re-imaging Oracle Database Appliance fails, with old header issues such as errors in storage discovery, or in running GI root scripts, or disk group RECO creation, then use the force mode with cleanup.pl
.
# cleanup.pl -f
To ensure that re-imaging is successful, remove the old headers from the storage disks by running the secure erase tool. Verify that the OAK/ASM headers are removed.
# cleanup.pl -erasedata
# cleanup.pl -checkHeader
Retry the re-imaging operation.
Related Topics
Parent topic: Troubleshooting Oracle Database Appliance
Using Oracle Autonomous Health Framework for Running Diagnostics
Oracle Autonomous Health Framework collects and analyzes diagnostic data collected, and proactively identifies issues before they affect the health of your system.
- About Installing Oracle Autonomous Health Framework
Oracle Autonomous Health Framework is installed automatically when you provision or patch to Oracle Database Appliance release 19.10. - Using the Oracle ORAchk Health Check Tool
Run Oracle ORAchk to audit configuration settings and check system health. - Generating and Viewing Oracle ORAchk Health Check Tool Reports in the Browser User Interface
Generate Oracle ORAchk Health Check Tool reports using the Browser User Interface. - Generating and Viewing Database Security Assessment Reports in the Browser User Interface
Generate and view Database Security Assessment Reports using the Browser User Interface. - Running Oracle Trace File Analyzer (TFA) Collector Commands
Understand the installed location oftfactl
and the options for the command. - Sanitizing Sensitive Information in Diagnostic Collections
Oracle Autonomous Health Framework uses Adaptive Classification and Redaction (ACR) to sanitize sensitive data. - Sanitizing Sensitive Information in Oracle Trace File Analyzer Collections
You can redact (sanitize or mask) Oracle Trace File Analyzer diagnostic collections. - Sanitizing Sensitive Information in Oracle ORAchk Output
You can sanitize Oracle ORAchk output.
Parent topic: Troubleshooting Oracle Database Appliance
About Installing Oracle Autonomous Health Framework
Oracle Autonomous Health Framework is installed automatically when you provision or patch to Oracle Database Appliance release 19.10.
When you provision or patch your appliance to Oracle Database Appliance
release 19.10, Oracle Autonomous Health Framework is
installed in the path /opt/oracle/dcs/oracle.ahf
.
[root@oak ~]# rpm -q oracle-ahf
oracle-ahf-193000-########.x86_64
Note:
When you provision or patch to Oracle Database Appliance release 19.10, Oracle Autonomous Health Framework automatically provides Oracle ORAchk Health Check Tool and Oracle Trace File Analyzer Collector.- Operating system kernel parameters and packages
- Oracle Database Database parameters, and other database configuration settings
- Oracle Grid Infrastructure, which includes Oracle Clusterware and Oracle Automatic Storage Management
- Encapsulation of diagnostic data collection for all Oracle Grid Infrastructure and Oracle RAC components on all cluster nodes into a single command, which you run from a single node
- Option to "trim" diagnostic files during data collection to reduce data upload size
- Options to isolate diagnostic data collection to a given time period, and to a particular product component, such as Oracle ASM, Oracle Database, or Oracle Clusterware
- Centralization of collected diagnostic output to a single node in Oracle Database Appliance, if desired
- On-Demand Scans of all log and trace files for conditions indicating a problem
- Real-Time Scan Alert Logs for conditions indicating a problem (for example, Database Alert Logs, Oracle ASM Alert Logs, and Oracle Clusterware Alert Logs)
Using the Oracle ORAchk Health Check Tool
Run Oracle ORAchk to audit configuration settings and check system health.
Note:
Before running ORAchk, check for the latest version of Oracle Autonomous Health Framework, and download and install it. See My Oracle Support Note 2550798.1 for more information about downloading and installing the latest verion of Oracle Autonomous Health Framework.Running ORAchk on Oracle Database Appliance 19.10 Baremetal Systems for New Installation
When you provision or upgrade to Oracle Database Appliance 19.10, ORAchk is installed using Oracle Autonomous Framework in the
directory /opt/oracle/dcs/oracle.ahf
.
[root@oak bin]# orachk
When all checks are finished, a detailed report is available. The output displays the location of the report in an HTML format and the location of a zip file if you want to upload the report. For example, you can choose the filter to show failed checks only, show checks with a Fail, Warning, Info, or Pass status, or any combination.
Review the Oracle Database Appliance Assessment Report and system health and troubleshoot any issues that are identified. The report includes a summary and filters that enable you to focus on specific areas.
Running ORAchk on Oracle Database Appliance 19.10 Virtualized Platform
When you provision or upgrade to Oracle Database Appliance 19.10, ORAchk is installed using Oracle Autonomous
Framework in the directory /opt/oracle.ahf
.
orachk
, use the following
command:[root@oak bin]# oakcli orachk
Generating and Viewing Oracle ORAchk Health Check Tool Reports in the Browser User Interface
Generate Oracle ORAchk Health Check Tool reports using the Browser User Interface.
Generating and Viewing Database Security Assessment Reports in the Browser User Interface
Generate and view Database Security Assessment Reports using the Browser User Interface.
Running Oracle Trace File Analyzer (TFA) Collector Commands
Understand the installed location of tfactl
and the options
for the command.
About Using tfactl to Collect Diagnostic Information
When you provision or upgrade to Oracle Database Appliance 19.10, Oracle Trace File Analyzer (TFA) Collector is
installed in the directory
/opt/oracle.ahf/bin/tfactl
. You can
invoke the command line utility for TFA, tfactl
from the directory /opt/oracle.ahf/bin/tfactl
,
or simply type tfactl
.
You can use the following command options to run tfactl
:
/opt/oracle.ahf/bin/tfactl diagcollect -ips|-oda|-odalite|-dcs|-odabackup|
-odapatching|-odadataguard|-odaprovisioning|-odaconfig|-odasystem|-odastorage|-database|
-asm|-crsclient|-dbclient|-dbwlm|-tns|-rhp|-procinfo|-afd|-crs|-cha|-wls|
-emagent|-oms|-ocm|-emplugins|-em|-acfs|-install|-cfgtools|-os|-ashhtml|-ashtext|
-awrhtml|-awrtext -mask -sanitize
Table 17-1 Command Options for tfactl Tool
Option | Description |
---|---|
-h |
(Optional) Describes all the options for this command. |
-ips |
(Optional) Use this option to view the diagnostic logs for the specified component. |
-oda |
(Optional) Use this option to view the logs for the entire Appliance. |
-odalite |
(Optional) Use this option to view the diagnostic logs for the odalite component. |
-dcs |
(Optional) Use this option to view the DCS log files. |
-odabackup |
(Optional) Use this option to view the diagnostic logs for the backup components for the Appliance. |
-odapatching |
(Optional) Use this option to view the diagnostic logs for patching components of the Appliance. |
-odadataguard |
(Optional) Use this option to view the diagnostic logs for Oracle Data Guard component of the Appliance. |
-odaprovisioning |
(Optional) Use this option to view provisioning logs for the Appliance. |
-odaconfig |
(Optional) Use this option to view configuration-related diagnostic logs. |
-odasystem |
(Optional) Use this option to view system information. |
-odastorage |
(Optional) Use this option to view the diagnostic logs for the Appliance storage. |
-database |
(Optional) Use this option to view database-related log files. |
-asm |
(Optional) Use this option to view the diagnostic logs for the Appliance. |
-crsclient |
(Optional) Use this option to view the diagnostic logs for the Appliance. |
-dbclient |
(Optional) Use this option to view the diagnostic logs for the Appliance. |
-dbwlm |
(Optional) Use this option to view the diagnostic logs for the specified component. |
-tns |
(Optional) Use this option to view the diagnostic logs for TNS. |
-rhp |
(Optional) Use this option to view the diagnostic logs for Rapid Home Provisioning. |
-afd |
(Optional) Use this option to view the diagnostic logs for Oracle ASM Filter Driver. |
-crs |
(Optional) Use this option to view the diagnostic logs for Oracle Clusterware. |
-cha |
(Optional) Use this option to view the diagnostic logs for the Cluster Health Monitor. |
-wls |
(Optional) Use this option to view the diagnostic logs for Oracle WebLogic Server. |
-emagent |
(Optional) Use this option to view the diagnostic logs for the Oracle Enterprise Manager agent. |
-oms |
(Optional) Use this option to view the diagnostic logs for the Oracle Enterprise Manager Management Service. |
-ocm |
(Optional) Use this option to view the diagnostic logs for the specified component. |
-emplugins |
(Optional) Use this option to view the diagnostic logs for Oracle Enterprise Manager plug-ins. |
-em |
(Optional) Use this option to view the diagnostic logs for Oracle Enterprise Manager deployment. |
-acfs |
(Optional) Use this option to view the diagnostic logs for Oracle ACFS storage. |
-install |
(Optional) Use this option to view the diagnostic logs for installation. |
-cfgtools |
(Optional) Use this option to view the diagnostic logs for the configuration tools. |
-os |
(Optional) Use this option to view the diagnostic logs for the operating system. |
-ashhtml |
(Optional) Use this option to view the diagnostic logs for the specified component. |
-ashtext |
(Optional) Use this option to view the diagnostic logs for the Appliance. |
-awrhtml |
(Optional) Use this option to view the diagnostic logs for the Appliance. |
-awrtext |
(Optional) Use this option to view the diagnostic logs for the specified component. |
|
(Optional) Use this option to choose to mask sensitive data in the log collection. |
|
(Optional) Use this option to choose to sanitize (redact) sensitive data in the log collection. |
Usage Notes
You can use Trace File Collector (the tfactl
command) to collect all log files for the Oracle Database Appliance components.
You can also use the command odaadmcli manage
diagcollect
, with similar command options, to
collect the same diagnostic information.
For more information about using the -mask
and
-sanitize
options, see the next topic.
Sanitizing Sensitive Information in Diagnostic Collections
Oracle Autonomous Health Framework uses Adaptive Classification and Redaction (ACR) to sanitize sensitive data.
After collecting copies of diagnostic data, Oracle Trace File Analyzer and Oracle ORAchk use Adaptive Classification and Redaction (ACR) to sanitize sensitive data in the collections. ACR uses a Machine Learning based engine to redact a pre-defined set of entity types in a given set of files. ACR also sanitizes or masks entities that occur in path names. Sanitization replaces a sensitive value with random characters. Masking replaces a sensitive value with a series of asterisks ("*").
- Host names
- IP addresses
- MAC addresses
- Oracle Database names
- Tablespace names
- Service names
- Ports
- Operating system user names
ACR also masks user data from the database appearing in block and redo dumps.
Example 17-4 Block dumps before redaction
14A533F40 00000000 00000000 00000000 002C0000 [..............,.]
14A533F50 35360C02 30352E30 31322E37 380C3938 [..650.507.2189.8]
14A533F60 31203433 37203332 2C303133 360C0200 [34 123 7310,...6]
Example 17-5 Block dumps after redaction
14A533F40 ******** ******** ******** ******** [****************]
14A533F50 ******** ******** ******** ******** [****************]
14A533F60 ******** ******** ******** ******** [****************]
Example 17-6 Redo dumps before redaction
col 74: [ 1] 80
col 75: [ 5] c4 0b 19 01 1f
col 76: [ 7] 78 77 06 16 0c 2f 26
Example 17-7 Redo dumps after redaction
col 74: [ 1] **
col 75: [ 5] ** ** ** ** **
col 76: [ 7] ** ** ** ** ** ** **
Redaction of Literal Values in SQL Statements in AWR, ASH and ADDM Reports
Automatic Workload Repository (AWR), Active Session History (ASH), and Automatic Database Diagnostic Monitor (ADDM) reports are HTML files, which contain sensitive entities such as hostnames, database names, and service names in the form of HTML tables. In addition to these sensitive entities, they also contain SQL statements, that can contain bind variables or literal values from tables. These literal values can be sensitive personal information (PI) stored in databases. ACR processes such reports to identify and redact both usual sensitive entities and literal values present in the SQL statements.
Sanitizing Sensitive Information Using odaadmcli Command
odaadmcli manage diagcollect
command to collect
diagnostic logs for Oracle Database Appliance components. During collection, ACR can
be used to redact (sanitize or mask) the diagnostic
logs.odaadmcli manage diagcollect [--dataMask|--dataSanitize]
In the command, the --dataMask
option blocks out the
sensitive data in all collections, for example, replaces myhost1
with *******
. The default is None. The
--dataSanitize
option replaces the sensitive data in all
collections with random characters, for example, replaces myhost1
with orzhmv1
. The default is None.
Sanitizing Sensitive Information in Oracle Trace File Analyzer Collections
You can redact (sanitize or mask) Oracle Trace File Analyzer diagnostic collections.
Enabling Automatic Redaction
To enable automatic redaction, use the command:
tfactl set redact=[mask|sanitize|none]
In the command, the -mask
option blocks out the
sensitive data in all collections, for example, replaces myhost1
with *******
. The -sanitize
option replaces the
sensitive data in all collections with random characters, for example, replaces
myhost1
with orzhmv1
. The
none
option does not mask or sanitize sensitive data in
collections. The default is none
.
Enabling On-Demand Redaction
You can redact collections on-demand, for example, tfactl diagcollect -srdc
ORA-00600 -mask or tfactl diagcollect -srdc ORA-00600 -sanitize
.
- To mask sensitive data in all
collections:
tfactl set redact=mask
- To sanitize sensitive data in all
collections:
tfactl set redact=sanitize
Example 17-8 Masking or Sanitizing Sensitive Data in a Specific Collection
tfactl diagcollect -srdc ORA-00600 -mask
tfactl diagcollect -srdc ORA-00600 -sanitize
Sanitizing Sensitive Information in Oracle ORAchk Output
You can sanitize Oracle ORAchk output.
To sanitize Oracle ORAchk output, include the -sanitize
option, for
example, orachk -profile asm -sanitize
. You can also sanitize post
process by passing in an existing log, HTML report, or a zip file, for example,
orachk -sanitize file_name
.
Example 17-9 Sanitizing Sensitive Information in Specific Collection IDs
orachk -sanitize comma_delimited_list_of_collection_IDs
Example 17-10 Sanitizing a File with Relative Path
orachk -sanitize new/orachk_node061919_053119_001343.zip
orachk is sanitizing
/scratch/testuser/may31/new/orachk_node061919_053119_001343.zip. Please wait...
Sanitized collection is:
/scratch/testuser/may31/orachk_aydv061919_053119_001343.zip
orachk -sanitize ../orachk_node061919_053119_001343.zip
orachk is sanitizing
/scratch/testuser/may31/../orachk_node061919_053119_001343.zip. Please wait...
Sanitized collection is:
/scratch/testuser/may31/orachk_aydv061919_053119_001343.zip
Example 17-11 Sanitizing Oracle Autonomous Health Framework Debug Log
orachk -sanitize new/orachk_debug_053119_023653.log
orachk is sanitizing /scratch/testuser/may31/new/orachk_debug_053119_023653.log.
Please wait...
Sanitized collection is: /scratch/testuser/may31/orachk_debug_053119_023653.log
Example 17-12 Running Full Sanity Check
orachk -localonly -profile asm -sanitize -silentforce
Detailed report (html) -
/scratch/testuser/may31/orachk_node061919_053119_04448/orachk_node061919_053119_04448.html
orachk is sanitizing /scratch/testuser/may31/orachk_node061919_053119_04448.
Please wait...
Sanitized collection is: /scratch/testuser/may31/orachk_aydv061919_053119_04448
UPLOAD [if required] - /scratch/testuser/may31/orachk_node061919_053119_04448.zip
orachk -rmap all|comma_delimited_list_of_element_IDs
You can also use orachk -rmap
to lookup a value sanitized by Oracle
Trace File Analyzer.
Example 17-13 Printing the Reverse Map of Sanitized Elements
orachk -rmap MF_NK1,fcb63u2
________________________________________________________________________________
| Entity Type | Substituted Entity Name | Original Entity Name |
________________________________________________________________________________
| dbname | MF_NK1 | HR_DB1 |
| dbname | fcb63u2 | rac12c2 |
________________________________________________________________________________
orachk -rmap all
Running the Disk Diagnostic Tool
Use the Disk Diagnostic Tool to help identify the cause of disk problems.
The tool produces a list of 14 disk checks for each node. To display details, where n represents the disk resource name, enter the following command:
# odaadmcli stordiag n
# odaadmcli stordiag pd_00
Parent topic: Troubleshooting Oracle Database Appliance
Running the Oracle Database Appliance Hardware Monitoring Tool
The Oracle Database Appliance Hardware Monitoring Tool displays the status of different hardware components in Oracle Database Appliance server.
The tool is implemented with the Trace File Analyzer collector. Use the tool both on bare-metal and on virtualized systems. The Oracle Database Appliance Hardware Monitoring Tool reports information only for the node on which you run the command. The information it displays in the output depend on the component that you select to review.
Bare Metal Platform
You can see the list of monitored components by running the command odaadmcli show -h
To see information about specific components, use the command syntax odaadmcli show component
, where component
is the hardware component that you want to query. For example, the command odaadmcli show power
shows information specifically about the Oracle Database Appliance power supply:
# odaadmcli show power
NAME HEALTH HEALTH_DETAILS PART_NO. SERIAL_NO.
Power_Supply_0 OK - 7079395 476856Z+1514CE056G
(Continued)
LOCATION INPUT_POWER OUTPUT_POWER INLET_TEMP EXHAUST_TEMP
PS0 Present 112 watts 28.000 degree C 34.938 degree C
Virtualized Platform
You can see the list of monitored components by running the command oakcli show -h
To see information about specific components, use the command syntax oakcli show component
, where component
is the hardware component that you want to query. For example, the command oakcli show power
shows information specifically about the Oracle Database Appliance power supply:
# oakcli show power
NAME HEALTH HEALTH_DETAILS PART_NO. SERIAL_NO.
Power Supply_0 OK - 7047410 476856F+1242CE0020
Power Supply_1 OK - 7047410 476856F+1242CE004J
(Continued)
LOCATION INPUT_POWER OUTPUT_POWER INLET_TEMP EXHAUST_TEMP
PS0 Present 88 watts 31.250 degree C 34.188 degree C
PS1 Present 66 watts 31.250 degree C 34.188 degree C
Note:
Oracle Database Appliance Server Hardware Monitoring Tool is enabled during initial startup of ODA_BASE on Oracle Database Appliance Virtualized Platform. When it starts, the tool collects base statistics for about 5 minutes. During this time, the tool displays the message "Gathering Statistics…" message.
Parent topic: Troubleshooting Oracle Database Appliance
Configuring a Trusted SSL Certificate for Oracle Database Appliance
The Browser User Interface and DCS Controller use SSL-based HTTPS protocol for secure communication. Understand the implications of this added security and the options to configure SSL certificates.
The Browser User Interface provides an added layer of security using certificates and encryption, when an administrator interacts with the appliance. Encryption of data ensures that:
- Data is sent to the intended recipient, and not to any malicious third-party.
- When data is exchanged between the server and the browser, data interception cannot occur nor can the data be edited.
When you connect to the Browser User Interface through HTTPS, the DCS Controller presents your browser with a certificate to verify the identity of appliance. If the web browser finds that the certificate is not from a trusted Certificate Authority (CA), then the browser assumes it has encountered an untrusted source, and generates a security alert message. The security alert dialog boxes display because Browser User Interface security is enabled through HTTPS and SSL, but you have not secured your Web tier properly with a trusted matching certificate from a Certificate Authority. It is possible to purchase commercial certificates from a Certificate Authority or create your own and register them with a Certificate Authority. However, the server and browser certificates must use the same public certificate key and trusted certificate to avoid the error message produced by the browser.
There are three options to configure your certificates:
- Create your own key and Java keystore, ensure it is signed by a Certificate Authority (CA) and then import it for use.
- Package an existing Privacy Enhanced Mail (PEM) format key and certificates in a new Java keystore.
- Convert an existing PKCS or PFX keystore to a Java keystore
and configure it for the Browser User Interface.
Note:
For Oracle Database Appliance High-Availability hardware models, run the configuration steps on both nodes.
The following topics explain how to configure these options:
- Option 1: Creating a Key and Java Keystore and Importing a Trusted Certificate
Usekeytool
, a key and certificate management utility, to create a keystore and a signing request. - Option 2: Packaging an Existing PEM-format Key and Certificates in a New Java Keystore
Use the OpenSSL tool to package Privacy Enhanced Mail (PEM) files in a PKCS keystore. - Option 3: Converting an Existing PKCS or PFX Keystore to a Java Keystore
If you have an existing PKCS or PFX keystore for your server's domain, convert it to a Java keystore. - Configuring the DCS Server to Use Custom Keystore
After packaging or converting your keystore into Java keystore, configure the DCS server to use your keystore. - Configuring the DCS Agent for Custom Certificate
After you import the certificate into the keystore, configure the DCS agent to use the same certificate.
Parent topic: Troubleshooting Oracle Database Appliance
Option 1: Creating a Key and Java Keystore and Importing a Trusted Certificate
Use keytool
, a key and certificate management utility, to
create a keystore and a signing request.
Option 2: Packaging an Existing PEM-format Key and Certificates in a New Java Keystore
Use the OpenSSL tool to package Privacy Enhanced Mail (PEM) files in a PKCS keystore.
Option 3: Converting an Existing PKCS or PFX Keystore to a Java Keystore
If you have an existing PKCS or PFX keystore for your server's domain, convert it to a Java keystore.
Configuring the DCS Server to Use Custom Keystore
After packaging or converting your keystore into Java keystore, configure the DCS server to use your keystore.
Disabling the Browser User Interface
You can also disable the Browser User Interface. Disabling the Browser User Interface means you can only manage your appliance through the command-line interface.
Parent topic: Troubleshooting Oracle Database Appliance
Preparing Log Files for Oracle Support Services
If you have a system fault that requires help from Oracle Support Services, then you may need to provide log records to help Oracle support diagnose your issue.
- Use the Bill Of Materials report saved in the
/opt/oracle/dcs/Inventory/
directory, to enable Oracle Support to help troubleshoot errors, if necessary. - You can use Trace File Collector (the
tfactl
command) to collect all log files for the Oracle Database Appliance components. - Use the command
odaadmcli manage diagcollect
to collect diagnostic files to send to Oracle Support Services.
The odaadmcli manage diagcollect
command consolidates information from log files stored on Oracle Database Appliance
into a single log file for use by Oracle Support Services. The location of the file
is specified in the command output.
Example 17-14 Collecting log file information for a time period, masking sensitive data
# odaadmcli manage diagcollect --dataMask --fromTime 2019-08-12 --toTime 2019-08-25
DataMask is set as true
FromTime is set as: 2019-08-12
ToTime is set as: 2019-08-25
TFACTL command is: /opt/oracle/tfa/tfa_home/bin/tfactl
Data mask is set.
Collect data from 2019-08-12
Collect data to 2019-08-25
Parent topic: Troubleshooting Oracle Database Appliance