6 Cluster Database

This chapter provides information about the Cluster Database (Oracle Real Application Clusters (RAC) database) metrics.

For each metric, it provides the following information:

• Description

• Metric table

  The metric table can include some or all of the following: target version, default collection frequency, default warning threshold, default critical threshold, and alert text.

6.1 Archive Area

This metric category contains the metrics representing the use of the archive areas.

If the database is running in ARCHIVELOG mode, these metrics check for available redo log destinations. If the database is not running in ARCHIVELOG mode, these metrics fail to register. For each destination, this metric category returns the total, used, and free space.

6.1.1 Archive Area Used (%)

The Archive Full (%) metric returns the percentage of space used on the archive area destination. If the space used is more than the threshold value given in the threshold arguments, then a warning or critical alert is generated.

If the database is not running in ARCHIVELOG mode or all archive destinations are standby databases for Oracle8i, this metric fails to register.

Note:

For databases that are configured to archive to the Fast Recovery Area, the Archive Area metrics (Archive Area Used(%), Archive Area Used (KB), Free Archive Area (KB), and Total Archive Area (KB)) are not applicable. Instead, use the Recovery Area Free Space(%) metric to monitor Fast Recovery Area usage. For more information, see Section 6.31.1, "Recovery Area Free Space (%)".

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 15 Minutes	80	Not Defined	%value%%% of archive area %archDir% is used.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each Archive Area Destination object.

If warning or critical threshold values are currently set for any Archive Area Destination object, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each Archive Area Destination object, use the Edit Thresholds page.

Data Source

If no quota is set for archive area, the percentage is calculated using the UNIX df -k command.

If quota is set:

archive area used (%) = (total area used / total archive area) * 100

User Action

Verify the device specified in the initialization parameter LOG_ARCHIVE_DEST is set up properly for archiving.

There are two methods you can use to specify archive destinations. These destinations can be setup using Enterprise Manager. For each database target, you can drill-down to the database Availability tab, and access the Recovery Settings page.

• The first method is to use the LOG_ARCHIVE_DEST_n parameter (where n is an integer from 1 to 10) to specify from one to ten different destinations for archival. Each numerically-suffixed parameter uniquely identifies an individual destination, for example, LOG_ARCHIVE_DEST_1, LOG_ARCHIVE_DEST_2, and so on.

• The second method, which allows you to specify a maximum of two locations, is to use the LOG_ARCHIVE_DEST parameter to specify a primary archive destination and the LOG_ARCHIVE_DUPLEX_DEST parameter to determine an optional secondary location.

If the LOG_ARCHIVE_DEST initialization parameter is set up correctly and this metric triggers, then free up more space in the destination specified by the archive destination parameters.

6.1.2 Archive Area Used (KB)

This metric represents the total space used (in KB) on the device containing the archive destination directory.

Note:

For databases that are configured to archive to the Fast Recovery Area, the Archive Area metrics (Archive Area Used(%), Archive Area Used (KB), Free Archive Area (KB), and Total Archive Area (KB)) are not applicable. Instead, use the Recovery Area Free Space(%) metric to monitor Fast Recovery Area usage. For more information, see Section 6.31.1, "Recovery Area Free Space (%)".

Target Version	Collection Frequency
Not Available	Every 15 Minutes

Data Source

If no quota is set for archive area, this is calculated through the UNIX df -k command.

total area used = quota_used * db_block_size (in KB)

User Action

Verify the device specified in the initialization parameter LOG_ARCHIVE_DEST is set up properly for archiving.

There are two methods you can use to specify archive destinations. These destinations can be setup using Enterprise Manager. For each database target, you can drill-down to the database Availability tab, and access the Recovery Settings page.

• The first method is to use the LOG_ARCHIVE_DEST_n parameter (where n is an integer from 1 to 10) to specify from one to ten different destinations for archival. Each numerically-suffixed parameter uniquely identifies an individual destination, for example, LOG_ARCHIVE_DEST_1, LOG_ARCHIVE_DEST_2, and so on.

• The second method, which allows you to specify a maximum of two locations, is to use the LOG_ARCHIVE_DEST parameter to specify a primary archive destination and the LOG_ARCHIVE_DUPLEX_DEST parameter to determine an optional secondary location.

If the LOG_ARCHIVE_DEST initialization parameter is set up correctly and this metric triggers, then free up more space in the destination specified by the archive destination parameters.

6.1.3 Free Archive Area (KB)

When running a database in ARCHIVELOG mode, the archiving of the online redo log is enabled. Filled groups of the online redo log are archived, by default, to the destination specified by the LOG_ARCHIVE_DEST initialization parameter. If this destination device becomes full, the database operation is temporarily suspended until disk space is available.

If the database is running in ARCHIVELOG mode, this metric checks for available redo log destination devices.

If the database is not running in ARCHIVELOG mode, this metric fails to register.

Note:

For databases that are configured to archive to the Fast Recovery Area, the Archive Area metrics (Archive Area Used(%), Archive Area Used (KB), Free Archive Area (KB), and Total Archive Area (KB)) are not applicable. Instead, use the Recovery Area Free Space(%) metric to monitor Fast Recovery Area usage. For more information, see Section 6.31.1, "Recovery Area Free Space (%)".

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 15 Minutes	Not Defined	Not Defined	Archive area %archDir% has %value% free KB remaining.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each Archive Area Destination object.

If warning or critical threshold values are currently set for any Archive Area Destination object, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each Archive Area Destination object, use the Edit Thresholds page.

Data Source

If the database is in NOARCHIVELOG mode, then nothing is collected.

If the database is in ARCHIVELOG mode, log_archive_destination from v$parameter is queried to obtain the current list of archivelog destinations. The results are obtained by directly checking the disk usage (df -kl).

User Action

Verify the device specified in the initialization parameter LOG_ARCHIVE_DEST is set up properly for archiving.

There are two methods you can use to specify archive destinations. These destinations can be setup using Enterprise Manager. For each database target, you can drill-down to the database Availability tab, and access the Recovery Settings page.

• The first method is to use the LOG_ARCHIVE_DEST_n parameter (where n is an integer from 1 to 10) to specify from one to ten different destinations for archival. Each numerically-suffixed parameter uniquely identifies an individual destination, for example, LOG_ARCHIVE_DEST_1, LOG_ARCHIVE_DEST_2, and so on.

• The second method, which allows you to specify a maximum of two locations, is to use the LOG_ARCHIVE_DEST parameter to specify a primary archive destination and the LOG_ARCHIVE_DUPLEX_DEST parameter to determine an optional secondary location.

If the LOG_ARCHIVE_DEST initialization parameter is set up correctly and this metric triggers, then free up more space in the destination specified by the archive destination parameters.

6.1.4 Total Archive Area (KB)

This metric represents the total space (in KB) on the device containing the archive destination directory.

Note:

For databases that are configured to archive to the Fast Recovery Area, the Archive Area metrics (Archive Area Used(%), Archive Area Used (KB), Free Archive Area (KB), and Total Archive Area (KB)) are not applicable. Instead, use the Recovery Area Free Space(%) metric to monitor Fast Recovery Area usage. For more information, see Section 6.31.1, "Recovery Area Free Space (%)".

Target Version	Collection Frequency
Not Available	Every 15 Minutes

Data Source

If no quota is set for archive area, this is calculated through the UNIX df -k command.

If quota is set:

total archive area = quota_size * db_block_size (in KB)

User Action

Oracle recommends that multiple archivelog destinations across different disks be configured. When at least one archivelog destination gets full, Oracle recommends the following:

• If tape is being used, back up archive logs to tape and delete the archive logs.

• If tape is not being used, back up the database and remove obsolete files. This also removes archive logs that are no longer needed based on the database retention policy.

• If archivelog destination quota_size is being used, raise the quota_size.

6.2 Cluster Managed Database Services

The metrics in the Cluster Managed Database Services metric category provide information about the CPU load for the managed database services.

6.2.1 % CPU Time Load (in last 5 min)

This metric provides the CPU load percentage for any cluster managed database service that exceeds the defined threshold in the last 5 minutes.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every 5 Minutes	Not Defined	Not Defined	CPU Load Percent %value% for service %name% exceeds the thresholds set.

6.3 Data Guard - 10.1 Database

The metrics in the Data Guard metrics category check the status, data not received, and data not applied for the databases in the Data Guard configuration. The metric runs on the primary database and covers conditions for all databases in the Data Guard configuration, including the primary and all physical and logical standby databases. It is applicable to both broker and non-broker Data Guard configurations.

For non-broker configurations, the metric is limited to monitoring primary database redo transport destination errors (as reflected in the ERRORS column of v$archive_dest).

For broker configurations, the metric is based on the Data Guard broker health check, which covers a much broader range of issues.

6.3.1 Data Guard Status - 10.1 Database

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Use the Data Guard Status metric to check the status of each database in the Data Guard configuration.

By default, a critical and warning threshold value was set for this metric column. Alerts will be generated when threshold values are reached. You can edit the value for a threshold as required.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR1	Every 5 Minutes	Warning	Error	The Data Guard status of %dg_name% is %value%.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

User Action

1. Check the Edit Properties General page for the primary and standby databases for detailed information.

2. Examine the database alert logs and the Data Guard broker logs for additional information.

6.3.2 Data Not Applied (logs)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The broker computes the highest applied SCN and uses its value to find the last continuous log that was successfully archived to the standby database. Redo data in all subsequent log files are counted as logs not applied. If the primary database goes down at this point, the redo data from these log files can be applied on the standby database. If there is a gap in the log files received on the standby database, any log files received after the gap cannot be applied.

For example, if log files 1, 2, 3, 6, 7, and 9 are received on the standby database and log apply services is currently applying log 1, log apply services can continue to apply up to log 3. Log apply services cannot apply any more log files because log 4 is missing. Even though log files 6, 7, and 9 are received, they cannot be applied and they will not be counted as data not applied.

If all the archived log files on the standby database are continuous, and standby redo logs are used, the standby redo logs are also counted as data not applied, unless real-time apply is turned on and log apply services is already working on the standby redo log files.

If the standby redo logs are multithreaded, the broker computes the highest applied SCN for every thread and totals the numbers. If there are multiple incarnations and the standby database is in a different incarnation from the primary database, each incarnation is computed separately and the results are then totaled.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR1	Every 5 Minutes	1	3	Standby database %dg_name% has not applied the last %value% received logs.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

6.3.3 Data Not Applied (MB)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The broker computes the highest applied SCN and uses its value to find the last continuous log that was archived to the standby database. The size of redo data in all subsequent log files are counted as data not applied. If the primary database goes down at this point, redo from these log files can be applied on the standby database. If there is a gap in the log files received on the standby database, any log files received after the gap cannot be applied.

For example, if log files 1, 2, 3, 6, 7, and 9 are received on the standby database and log apply services is currently applying log 1, log apply services can continue to apply up to log 3. Log apply services cannot apply any more log files because log 4 is missing. Even though log files 6, 7, and 9 are received, they cannot be applied and they will not be counted as data not applied. In this case, the total size of log files 1, 2, and 3 is the size of Data Not Applied.

If all the archived log files on the standby database are continuous, and standby redo log files are used, the standby redo log files are also counted as data not applied, unless real-time apply is turned on and log apply services is already working on the standby redo log files. The size of an archived log file is its file size. However, the size of a standby redo log is the size of the actual redo in the log and not the file size.

If the standby redo log files are multithreaded, the broker computes the highest applied SCN for every thread and totals the numbers. If there are multiple incarnations and the standby database is in a different incarnation from the primary database, each incarnation is computed separately and the results are then totaled.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR1	Every 5 Minutes	Not Defined	Not Defined	Standby database %dg_name% has not applied the last %value% megabytes of data received.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

6.3.4 Data Not Received (logs)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The broker computes the highest applied SCN and uses its value to find the last continuous log file that was successfully archived to the standby database. Redo data in all subsequent log files, including the current online redo log file, are counted as log files for potential data loss and will be unrecoverable if the primary database goes down at this point.

For example, if log files 1, 2, 3, 6, 7, and 9 are received on the standby database, and if log 10 is the current online log file, and if log apply services are currently applying log 1, the last continuous log after the highest applied SCN is log 3. All log files after log 3, that is log files 4 through 10, are counted as data not received. If the primary database goes down at this point, all redo data in log files 4 through 10 are lost on the standby database.

If the primary database is multithreaded (in a RAC database), the broker computes the highest applied SCN for every thread and totals the numbers. If the primary database has multiple incarnations (for example, due to a flashback operation) and the standby database is in a different incarnation from the primary database, the computation is done on each incarnation and the results are then totaled.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR1	Every 5 Minutes	1	3	Standby database %dg_name% has not received the last %value% logs from the primary database.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

6.3.5 Data Not Received (MB)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The broker computes the highest applied SCN and uses its value to find the last continuous log file that was successfully archived to the standby database. The size of redo data in all subsequent log files, including the current online redo log file, are counted as data for potential data loss and will be unrecoverable if the primary database goes down at this point. The size of an archived log file is its file size, and the size of the online redo log file is the size of the actual redo in the online log file, not the file size of the online redo log file.

For example, if log files 1, 2, 3, 6, 7, and 9 are received on the standby database, and if log 10 is the current online log file, and if log apply services is currently applying log 1, the last continuous log after the highest applied SCN is log 3. All log files after log 3, that is log files 4 through 10, are counted as data not received and the total size of redo data in these log files is the size of Data Not Received.

If the primary database is multithreaded (in a RAC database), the broker computes the highest applied SCN for every thread and totals the numbers. If the primary database has multiple incarnations (for example, due to a flashback operation) and the standby database is in a different incarnation from the primary database, the computation is done on each incarnation and the results are then totaled.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR1	Every 5 Minutes	Not Defined	Not Defined	Standby database %dg_name% has not received the last %value% megabytes of data from the primary database.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

6.4 Data Guard - 9.2 Database

The metrics in the Data Guard metrics category check the status, data not received, and data not applied for the databases in the Data Guard configuration. The metric runs on the primary database and covers conditions for all databases in the Data Guard configuration, including the primary and all physical and logical standby databases. It is applicable to both broker and non-broker Data Guard configurations.

For non-broker configurations, the metric is limited to monitoring primary database redo transport destination errors (as reflected in the ERRORS column of v$archive_dest).

For broker configurations, the metric is based on the Data Guard broker health check, which covers a much broader range of issues.

6.4.1 Data Guard Status - 9.2 Database

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Use the Data Guard Status metric to check the status of each database in the Data Guard configuration.

By default, a critical and warning threshold value was set for this metric column. Alerts will be generated when threshold values are reached. You can edit the value for a threshold as required.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2	Every 5 Minutes	Warning	Error	The Data Guard status of %dg_name% is %value%.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

User Action

1. Check the Edit Properties General page for the primary and standby databases for detailed information.

2. Examine the database alert logs and the Data Guard broker logs for additional information.

6.4.2 Data Not Applied (logs)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The broker computes the highest applied SCN and uses its value to find the last continuous log that was successfully archived to the standby database. Redo data in all subsequent log files are counted as logs not applied. If the primary database goes down at this point, the redo data from these log files can be applied on the standby database. If there is a gap in the log files received on the standby database, any log files received after the gap cannot be applied.

For example, if log files 1, 2, 3, 6, 7, and 9 are received on the standby database and log apply services is currently applying log 1, log apply services can continue to apply up to log 3. Log apply services cannot apply any more log files because log 4 is missing. Even though log files 6, 7, and 9 are received, they cannot be applied and they will not be counted as data not applied.

If all the archived log files on the standby database are continuous, and standby redo logs are used, the standby redo logs are also counted as data not applied, unless real-time apply is turned on and log apply services is already working on the standby redo log files.

If the standby redo logs are multithreaded, the broker computes the highest applied SCN for every thread and totals the numbers. If there are multiple incarnations and the standby database is in a different incarnation from the primary database, each incarnation is computed separately and the results are then totaled.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2	Every 5 Minutes	1	3	Standby database %dg_name% has not applied the last %value% received logs.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

6.4.3 Data Not Received (logs)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The broker computes the highest applied SCN and uses its value to find the last continuous log file that was successfully archived to the standby database. Redo data in all subsequent log files, including the current online redo log file, are counted as log files for potential data loss and will be unrecoverable if the primary database goes down at this point.

For example, if log files 1, 2, 3, 6, 7, and 9 are received on the standby database, and if log 10 is the current online log file, and if log apply services are currently applying log 1, the last continuous log after the highest applied SCN is log 3. All log files after log 3, that is log files 4 through 10, are counted as data not received. If the primary database goes down at this point, all redo data in log files 4 through 10 are lost on the standby database.

If the primary database is multithreaded (in a RAC database), the broker computes the highest applied SCN for every thread and totals the numbers. If the primary database has multiple incarnations (for example, due to a flashback operation) and the standby database is in a different incarnation from the primary database, the computation is done on each incarnation and the results are then totaled.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2	Every 5 Minutes	1	3	Standby database %dg_name% has not received the last %value% logs from the primary database.

Data Source

The following commands are the data sources for this metric:

Non-broker: v$archive_dest on primary database

Broker: Data Guard broker on primary database

6.5 Data Guard Fast-Start Failover

The metrics in this category are database-level metrics. For cluster databases, these metrics are monitored at the cluster database target level and not by member instances.

6.5.1 Fast-Start Failover Occurred

This metric provides the time at which a fast-start failover occurred.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2	Every 5 Minutes	Not Defined	1	A fast-start failover occurred at %dg_fs_time%.

6.6 Data Guard Fast-Start Failover Observer Status

The metrics in the Data Guard Fast-Start Failover Observer Status metric category provide information about the status of the observer process.

The observer process continuously monitors the primary and target standby databases and evaluates whether failover is necessary, and then initiates a fast-start failover when conditions warrant.

6.6.1 Observer Status

This metric displays the status of the observer process.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11g, 12c	Every Minute	Not Defined	Error	The Data Guard fast-start failover observer status is %value%.

6.7 Data Guard Fast-Start Failover Observer Status - 10.2 Database

The metrics in the Data Guard Fast-Start Failover Observer Status - 10.2 Database metric category provide information about the status of the observer process for Oracle Database 10gR2 only.

The observer process continuously monitors the primary and target standby databases and evaluates whether failover is necessary, and then initiates a fast-start failover when conditions warrant.

6.7.1 Observer Status - 10.2 Database

This metric displays the status of the observer process for Oracle Database 10gR2.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2	Every 5 Minutes	Not Defined	Error	The Data Guard fast-start failover observer status is %value%.

6.8 Data Guard Performance

This metric category provides the Data Guard Performance metric.

6.8.1 Apply Lag (seconds)

This metric displays (in seconds) how far the standby is behind the primary.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 5 Minutes	Not Defined	Not Defined	The standby database is approximately %value% seconds behind the primary database.

Data Source

The data source for this metric is the following command:

v$dataguard_stats('apply lag')

6.8.2 Estimated Failover Time (seconds)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric shows the approximate number of seconds required to failover to this standby database. This accounts for the startup time, if necessary, plus the remaining time required to apply all the available redo on the standby. If a bounce is not required, it is only the remaining apply time.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 5 Minutes	Not Defined	Not Defined	The estimated time to failover is approximately %value% seconds.

Data Source

The data source for this metric is the following command:

v$dataguard_stats ('estimated startup time','apply finish time','standby has been open')

6.8.3 Redo Apply Rate (KB/second)

Displays the Redo Apply Rate in KB/second on this standby.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 5 Minutes	Not Defined	Not Defined	The estimated time to failover is approximately %value% seconds.

6.8.4 Redo Generation Rate (KB/second)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11g, 12c	Every 5 Minutes	Not Defined	Not Defined	The redo generation rate is %value% KB/sec.

6.8.5 Transport Lag (seconds)

The approximate number of seconds of redo not yet available on this standby database. This may be because the redo has not yet been shipped or there may be a gap.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 5 Minutes	Not Defined	Not Defined	There are approximately %value% seconds of redo not yet available on this standby database.

Data Source

The data source for this metric is the following command:

v$dataguard_stats('transport lag')

6.9 Data Guard Performance - 11.1 and Prior Database

This metric category includes the Data Guard Performance metrics for Oracle Database 11g and Oracle Database 10gR2.

6.9.1 Apply Lag (seconds) - 11.1 and Prior Database

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric displays (in seconds) how far the standby is behind the primary.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11gR1	Every 5 Minutes	Not Defined	Not Defined	The standby database is approximately %value% seconds behind the primary database.

Data Source

The data source for this metric is the following command:

v$dataguard_stats('apply lag')

6.9.2 Estimated Failover Time (seconds) - 11.1 and Prior Database

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric shows the approximate number of seconds required to failover to this standby database. This accounts for the startup time, if necessary, plus the remaining time required to apply all the available redo on the standby. If a bounce is not required, it is only the remaining apply time.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11gR1	Every 5 Minutes	Not Defined	Not Defined	The estimated time to failover is approximately %value% seconds.

Data Source

The data source for this metric is the following command:

v$dataguard_stats ('estimated startup time','apply finish time','standby has been open')

6.9.3 Transport Lag (seconds) - 11.1 and Prior Database

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The approximate number of seconds of redo not yet available on this standby database. This may be because the redo has not yet been shipped or there may be a gap.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11gR1	Every 5 Minutes	Not Defined	Not Defined	There are approximately %value% seconds of redo not yet available on this standby database.

Data Source

The data source for this metric is the following command:

v$dataguard_stats('transport lag')

6.9.4 Redo Apply Rate (KB/second) - 11.1 and Prior Database

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric displays the Redo Apply Rate in KB/second on this standby.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11gR1	Every 5 Minutes	Not Defined	Not Defined	The redo apply rate is %value% KB/sec

6.10 Data Guard Status

The metrics in the Data Guard metrics category check the status, data not received, and data not applied for the databases in the Data Guard configuration.

6.10.1 Data Guard Status

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Use the Data Guard Status metric to check the status of each database in the Data Guard configuration.

By default, a critical and warning threshold value was set for this metric column. Alerts will be generated when threshold values are reached. You can edit the value for a threshold as required.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11g, 12c	Every 5 Minutes	Warning	Error	The Data Guard status of %dg_name% is %value%.

Data Source

1. Check the Edit Properties General page for the primary and standby databases for detailed information.

2. Examine the database alert logs and the Data Guard broker logs for additional information.

6.11 Database Cardinality

This metric category contains the metrics that monitor the number of active instances of a cluster database.

6.11.1 Open Instance Count

This metric monitors how many instances are in an open state.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 5 Minutes	Not Defined	Not Defined	%value% instance(s) out of %total_count% are up.

6.12 Database Job Status

This metric category contains the metrics that represent the health of database jobs registered through the DBMS_SCHEDULER interface.

6.12.1 Broken Job Count

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The Oracle Server job queue is a database table that stores information about local jobs such as the PL/SQL call to execute for a job such as when to run a job. Database replication is also managed by using the Oracle job queue mechanism using jobs to push deferred transactions to remote master sites, to purge applied transactions from the deferred transaction queue, or to refresh snapshot refresh groups.

A job can be broken in two ways:

Oracle has failed to successfully execute the job after sixteen attempts. The job has been explicitly marked as broken by using the procedure DBMS_ JOB.BROKEN.

This metric checks for broken DBMS jobs. A critical alert is generated if the number of broken jobs exceeds the value specified by the threshold argument.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 30 Minutes	0	Not Defined	%value% job(s) are broken.

Data Source

The data source for this metric is the following command:

SELECT COUNT(*) 
  FROM dba_jobs 
  WHERE broken < > 'N' 

User Action

Check the ALERT log and trace files for error information. Correct the problem that is preventing the job from running. Force immediate re-execution of the job by calling DBMS_SCHEDULER.RUN.

6.12.2 Failed Job Count

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The Oracle Server job queue is a database table that stores information about local jobs such as the PL/SQL call to execute for a job such as when to run a job. Database replication is also managed by using the Oracle job queue mechanism using jobs to push deferred transactions to remote master sites, to purge applied transactions from the deferred transaction queue or to refresh snapshot refresh groups.

If a job returns an error while Oracle is attempting to execute it, the job fails. Oracle repeatedly tries to execute the job doubling the interval of each attempt. If the job fails sixteen times, Oracle automatically marks the job as broken and no longer tries to execute it.

This metric checks for failed DBMS jobs. An alert is generated if the number of failed job exceeds the value specified by the threshold argument.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 30 Minutes	0	Not Defined	%value% job(s) have failed.

Data Source

The data source for this metric is the following command:

SELECT COUNT(*) 
  FROM dba_jobs 
  WHERE NVL(failures, 0) < >  0" 

User Action

Check the ALERT log and trace files for error information. Correct the problem that is preventing the job from running.

6.13 Database Wait Bottlenecks

This metric category contains the metrics that approximate the percentage of time spent waiting by user sessions across instances for the cluster database. This approximation takes system-wide totals and discounts the effects of sessions belonging to background processes.

6.13.1 Active Sessions Using CPU

This metric represents the active sessions using CPU.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Collection Frequency
10g, 11g, 12c	Every 15 Minutes

6.13.2 Active Sessions Waiting: I/O

This database-level metric represents the active sessions waiting for I/O. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Collection Frequency
10g, 11g, 12c	Every 15 Minutes

6.13.3 Active Sessions Waiting: Other

This database-level metric represents all the waits that are neither idle nor user I/O. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Collection Frequency
10g, 11g, 12c	Every 15 Minutes

6.13.4 Average Database CPU (%)

This metric represents the average database CPU across instances as a percentage.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

6.13.5 Host CPU Utilization (%)

This metric represents the percentage of CPU being used across hosts.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Collection Frequency
10g, 11g, 12c	Every 15 Minutes

6.13.6 Load Average

This metric reports the sum of the current CPU load for all cluster database hosts.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

6.13.7 Maximum CPU

This metric represents the total CPU count across all the cluster database hosts.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Collection Frequency
10g, 11g, 12c	Every 15 Minutes

6.13.8 Wait Time (%)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric represents the percentage of time spent waiting, database-wide, for resources or objects during this sample period.

This test checks the percentage time spent waiting, database-wide, for resources or objects during this sample period. If the % Wait Time is greater than or equal to the threshold values specified by the threshold arguments, and the number of occurrences exceeds the value specified in the Number of Occurrences parameter, then a warning or critical alert is generated.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every 15 Minutes	Not Defined	Not Defined	Generated By Database Server

Data Source

The data source for this metric is the following formula where:

• DeltaTotalWait: Difference of 'sum of time waited for all wait events in v$system_event' between sample end and start.

• DeltaCpuTime: Difference of 'select value from v$sysstat where name='CPU used by this session' between sample end and start.

DeltaTotalWait / (DeltaTotalWait + DeltaCpuTime)

User Action

Investigate further into which specific wait events are responsible for the bulk of the wait time. Individual wait events may identify unique problems within the database. Diagnosis will be tailored where appropriate through drilldowns specific to individual wait events.

6.14 Database Vault Attempted Violations - Command Rules

The metrics in the Database Vault Attempted Violations - Command Rules metric category provides information about the attempted Database Vault command rule violations.

6.14.1 Database Vault Attempted Violations Count - Command Rules

This metric raises an alert, which helps the Oracle Database Vault security analyst to monitor violation attempts on the Database Vault database. This user can select the command rules to be affected by the alert and filter these command rules based on the different types of attempts by using error codes.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2,10g, 11g, 12c	Every Hour	Not Defined	Not Defined	%ACTION_OBJECT_NAME% got violated at %VIOLATIONTIMESTAMP%

6.15 Database Vault Attempted Violations - Realms

The metrics in the Database Vault Attempted Violations - Realms metric category provide information about realm violations (for example, when an unauthorized user tries to modify an object that is protected by the realm).

6.15.1 Database Vault Attempted Violations Count - Realms

This metric raises an alert, which helps the Oracle Database Vault security analyst to monitor violation attempts on the Database Vault database. This user can select the realms to be affected by the alert and filter these realms based on the different types of attempts by using error codes.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2,10g, 11g, 12c	Every Hour	Not Defined	Not Defined	%ACTION_OBJECT_NAME% got violated at %VIOLATIONTIMESTAMP%.

6.16 Database Vault Configuration Issues - Realms

The metrics in the Database Vault Configuration Issues - Realms metric category provide information about configuration issues in the realm. The Oracle Database Vault realm protects configuration information in the Oracle Database Vault.

6.16.1 Database Vault Configuration Issues Count - Realms

This metric tracks and raises an alert if users misconfigure realms.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2,10g, 11g, 12c	Every Hour	Not Defined	0	%ACTION_OBJECT_NAME% has configuration issues.

6.17 Database Vault Configuration Issues - Command Rules

The metrics in the Database Vault Configuration Issues - Command Rules metric category provide information about configuration issues in the command rules. A command rule is a rule that you create to protect SELECT, ALTER SYSTEM, database definition language (DDL), and data manipulation language (DML) statements that affect one or more database objects

6.17.1 DV (Command Rule) - Configuration Issue Count

This metric tracks and raises an alert if users misconfigure command rules.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2,10g, 11g, 12c	Every Hour	Not Defined	0	%ACTION_OBJECT_NAME% has configuration issues.

6.18 Database Vault Policy Changes

The metrics in the Database Vault Policy Changes metric category provide information about any changes to a Database Vault Policy.

6.18.1 Database Vault Policy Changes Count

This metric raises an alert on any change to any Database Vault policy, such as policies for realms and command rules.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2,10g, 11g, 12c	Every Hour	Not Defined	0	%ACTION_OBJECT_NAME% has configuration issues.

6.19 Deferred Transactions

This metric category contains the metrics associated with this distributed database's deferred transactions.

6.19.1 Deferred Transaction Count

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Oracle uses deferred transactions to propagate data-level changes asynchronously among master sites in an advanced replication system as well as from an updatable snapshot to its master table.

This metric checks for the number of deferred transactions. An alert is generated if the number of deferred transactions exceeds the value specified by the threshold argument.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 5 Minutes	100	Not Defined	Number of deferred transactions is %value%.

Data Source

The data source for this metric is the following command:

SELECT count(*) 
  FROM sys.deftran 

User Action

When the advanced replication facility pushes a deferred transaction to a remote site, it uses a distributed transaction to ensure that the transaction has been properly committed at the remote site before the transaction is removed for the queue at the local site. If transactions are not being pushed to a given remote site, verify that the destination for the transaction was correctly specified. If you specify a destination database when calling DBMS_DEFER_SYS.SCHEDULE_EXECUTION using the DBLINK parameter, or DBMS_DEFER_SYS.EXECUTE using the DESTINATION parameter, make sure the full database link is provided.

Wrong view destinations can lead to erroneous deferred transaction behavior. Verify that the DEFCALLEST and DEFTRANDEST views are the definitions from the CATREPC.SQL and not those from CATDEFER.SQL.

6.19.2 Deferred Transaction Error Count

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Oracle uses deferred transactions to propagate data-level changes asynchronously among master sites in an advanced replication system as well as from an updatable snapshot to its master table. If a transaction is not successfully propagated to the remote site, Oracle rolls back the transaction, logs the transaction in the SYS.DEFERROR view in the remote destination database.

This metric checks for the number of transactions in SYS.DEFERROR view and raises an alert if it exceeds the value specified by the threshold argument.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 5 Minutes	0	Not Defined	Number of deferred transactions with errors is %value%.

Data Source

The data source for this metric is the following command:

SELECT count(*) 
  FROM sys.deferror 

User Action

An error in applying a deferred transaction may result from a database problem, such as a lack of available space in the table to be updated, or may be the result of an unresolved insert, update, or delete conflict. The SYS.DEFERROR view provides the ID of the transaction that could not be applied. Use this ID to locate the queued calls associated with the transaction. These calls are stored in the SYS.DEFCALL view. You can use the procedures in the DBMS_DEFER_QUERY package to determine the arguments to the procedures listed in the SYS.DEFCALL view.

6.20 Exadata Module Version Failure

This metric category provides information about any Exadata module version errors.

6.20.1 Error Count

This metric tracks and raises an alert when a defined number of Exadata module version errors occur.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 24 Hours	0	Not Defined	%errorCode% occurrences of %errorCount%.

6.21 Failed Logins

The metric in this metric category checks for the number of failed logins on the target database. This check is performed every ten minutes and returns the number of failed logins for that ten-minute interval. This metric will only work for databases where the audit_trail initialization parameter is set to DB or XML and the session is being audited.

6.21.1 Failed Login Count

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric checks for the number of failed logins on the target database. This check is performed every ten minutes and returns the number of failed logins for that ten-minute interval. This metric will only work for databases where the audit_trail initialization parameter is set to DB or XML and the session is being audited.

If the failed login count crosses the values specified in the threshold arguments, then a warning or critical alert is generated. Because it is important to know every time a significant number of failed logins occurs on a system, this metric will generate a new alert for any ten-minute interval where the thresholds are crossed. You can manually clear these alerts. They will not automatically clear after the next collection.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 30 Minutes	150	300	Number of failed login attempts exceeds threshold value.

Data Source

The database stores login information in different views based on the audit_trail setting. The database views used are:

• DB or DB_EXTENDED: DBA_AUDIT_SESSION

• XML (10g Release 2 only): DBA_COMMON_AUDIT_TRAIL

6.22 Fast Recovery

The metrics in the Fast Recovery metrics category relate to the fast recovery area.

6.22.1 Fast Recovery Area

Formerly referred to as flash recovery area, this metric returns an optional disk location that you can use to store recovery-related files such as control file and online redo log copies, archived redo log files, flashback logs, and RMAN backups.

Oracle Database and RMAN manage the files in the fast recovery area automatically. You can specify the disk quota, which is the maximum size of the fast recovery area.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is the following command:

SELECT value 
  FROM v$parameter 
  WHERE name='db_recovery_file_dest';

User Action

No user action is required.

6.22.2 Fast Recovery Area Size

This metric returns the Fast Recovery Area Size.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is the following command:

SELECT value 
  INTO 1_fast_recovery_size
  FROM v$parameter 
  WHERE name='db_recovery_file_dest_size';

User Action

No user action is required.

6.22.3 Flashback On

This metric returns whether or not flashback logging is enabled - YES, NO, or RESTORE POINT ONLY. For the RESTORE POINT ONLY option, flashback is ON but you can only flashback to guaranteed restore points.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is the following command:

SELECT flashback_on 
  FROM v$database;

User Action

No user action is required.

6.22.4 Log Mode

This metric returns the log mode of the database - ARCHIVELOG or NOARCHIVELOG.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is the following command:

SELECT log_mode 
  FROM v$database;

User Action

No user action is required.

6.22.5 Non-Reclaimable Fast Recovery Area (%)

This metric represents the percentage of space non-reclaimable (spaced used minus space reclaimable) in the fast recovery area.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is one of the following commands:

Non-reclaimable = space used - space reclaimable

Space Used:
  SELECT SUM(PERCENT_SPACE_USED 
     FROM v$fast_recovery_area_usage;

Space Reclaimable:
  SELECT SUM(PERCENT_SPACE_RECLAIMABLE) 
     FROM v$fast_recovery_area_usage;

User Action

No user action is required.

6.22.6 Oldest Flashback Time

This metric returns the oldest point-in-time to which you can flashback your database.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is the following command:

SELECT to_char(oldest_flashback_time, 'YYYY-MM-DD HH24:MI:SS') 
  FROM v$flashback_database_log;

User Action

No user action is required.

6.22.7 Reclaimable Fast Recovery Area (%)

This metric represents the percentage of space reclaimable in the fast recovery area.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is the following command:

Space Reclaimable:
  SELECT SUM(PERCENT_SPACE_RECLAIMABLE) 
     FROM v$fast_recovery_area_usage;

User Action

No user action is required.

6.22.8 Usable Fast Recovery Area (%)

This metric represents the percentage of space usable in the fast recovery area. The space usable is composed of the space that is free in addition to the space that is reclaimable.

Target Version	Collection Frequency
10g, 11g	Every 15 Minutes

Data Source

The data source for this metric is the following command:

SELECT (CASE WHEN PERCENT_USED > 100 THEN 0 ELSE (100-PERCENT_USED) END) 
PERCENT_FREE
   FROM (SELECT (SUM(PERCENT_SPACE_USED)-SUM(PERCENT_SPACE_RECLAIMABLE)) 
PERCENT_USED
   FROM V$FAST_RECOVERY_AREA_USAGE);

User Action

No user action is required.

6.23 Invalid Objects

This metric category contains the metrics associated with invalid objects.

6.23.1 Total Invalid Object Count

This metric represents the total invalid object count.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
All Versions	Every 24 Hours	Not Defined	Not Defined	%value% object(s) are invalid in the database.

6.24 Invalid Objects by Schema

This metric category contains the metrics that represent the number of invalid objects in each schema.

6.24.1 Owner's Invalid Object Count

This metric represents the invalid object count by owner.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
All Versions	Every 24 Hours	Not Defined		%value% object(s) are invalid in the %owner% schema.

Data Source

The data source for each metric index is the following command:

SELECT count(1)

User Action

View the status of the database objects in the schema identified by the Invalid Object Owner metric. Recompile objects as necessary.

6.25 Messages Per Buffered Queue

The metrics in the Messages Per Buffered Queue metrics category monitor the age and state of the first (top of the queue) message for each buffered queue in the database except for the system queues. Queues that are in the schema of SYS, SYSTEM, DBSNMP, and SYSMAN are defined as system level queues.

6.25.1 Average age of messages per buffered queue (seconds)

This metric provides the average age (in seconds) of the messages in the buffered queue for all nonsystem queues in the database.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Average age of messages in %schema%.%queue_name% queue is %value% seconds.

6.25.2 First Message Age in Buffered Queue Per Queue (Seconds)

This metric gives the age (in seconds) of the first message in the buffered queue for all non-system queues in the database.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Age of first message in %schema%.%queue_name% buffered queue is %value% seconds.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Schema Name and Queue Name objects.

If warning or critical threshold values are currently set for any unique combination of Schema Name and Queue Name objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Schema Name and Queue Name objects, use the Edit Thresholds page.

Data Source

This metric is calculated by finding the age of the first message in all the subscribers of the queue and then the oldest amongst all is taken.

The following views and tables are used for the calculation:

1. <SCHEMA>.AQ$<QUEUE_TABLE>

2. v$buffered_queues

User Action

When using buffered queues for storing and propagating messages, monitor this metric to get the age of first message in the queue.

6.25.3 Messages processed per buffered queue (%)

This metric gives the messages processed percentage per minute per buffered queue in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed for queue %schema%.%queue_name% is %value% percent.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Schema Name and Queue Name objects.

If warning or critical threshold values are currently set for any unique combination of Schema Name and Queue Name objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Schema Name and Queue Name objects, use the Edit Thresholds page.

Data Source

This is calculated as the percent of total number of messages processed per minute and total number of messages received per minute in the last collection interval per buffered queue.

User Action

When using queues for storing/propagating messages, monitor this metric to get the messages processed percent (or throughput) per minute in the last collection interval for the queue.

6.25.4 Messages processed per buffered queue (%) per minute

This metric gives the messages processed percentage per minute in the last interval per buffered queue in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed per minute in the last interval for queue %schema%.%queue_name% is %value% .

6.25.5 Spilled Messages

This metric displays the current number of overflow messages spilled to disk from the buffered queue.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Current number of overflow messages spilled to disk from the buffered queue %schema%.%queue_name% is %value%

6.25.6 Total Messages Processed per Buffered Queue per Minute

This metric gives the total number of messages processed per minute per buffered queue in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Total messages processed per minute in the last interval for queue %schema%.%queue_name% is %value% .

6.25.7 Total Messages Received per Buffered Queue per Minute

This metric gives the total number of messages received or enqueued into the buffered queue per minute in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Total messages received per minute in the last interval for queue %schema%.%queue_name% is %value% .

6.26 Messages Per Buffered Queue Per Subscriber

This metric category monitors the messages for buffered queues per subscriber in the database.

6.26.1 Average Age of Messages Per Buffered Queue Per Subscriber (Seconds)

This metric display's the average age of messages in the buffered queue per queue in seconds.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Average age of messages for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% seconds.

6.26.2 First Message Age in Buffered Queue per Subscriber (Seconds)

This metric displays the age of the first message in the buffered queue per queue per subscriber in seconds.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Age of first message for subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% seconds.

6.26.3 Messages Processed Per Buffered Queue (%) Per Subscriber Per Minute

This metric gives the total number of messages processed per minute per buffered queue subscriber in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed per minute in the last interval for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value%

6.26.4 Messages Processed Per Buffered Queue Per Subscriber (%)

This metric gives the messages processed percentage for the buffered queue per subscriber. Messages processed percent is calculated as the percent of the total number messages processed or dequeued to the total number of messages received or enqueued.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% percent.

6.26.5 Total Messages Processed Per Buffered Queue Per Subscriber Per Minute

This metric gives the total number of messages processed per minute per buffered queue subscriber in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Total messages processed per minute in the last interval for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% .

6.26.6 Total Messages Received Per Buffered Queue Per Subscriber Per Minute

This metric gives the total number of messages received or enqueued into the queue per subscriber per minute in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Total messages received per minute in the last interval for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% .

6.27 Messages Per Persistent Queue

The metrics in the Messages Per Persistent Queue metrics category monitor the age and state of the first (top of the queue) message for each persistent queue in the database except for the system queues. Queues that are in the schema of SYS, SYSTEM, DBSNMP, and SYSMAN are defined as system level queues.

6.27.1 Average Age of Messages Per Persistent Queue (Seconds)

This metric displays the average age of messages in the persistent queue per queue in seconds.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Average age of messages in %schema%.%queue_name% queue is %value% seconds.

6.27.2 Age of The First Message in Persistent Queue Per Queue (Seconds)

This metric gives the age (in seconds) of the first message in the persistent queue for all non-system queues in the database.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Age of first message in %schema%.%queue_name% queue is %value% seconds.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Schema Name and Queue Name objects.

If warning or critical threshold values are currently set for any unique combination of Schema Name and Queue Name objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Schema Name and Queue Name objects, use the Edit Thresholds page.

Data Source

This metric is calculated by finding the age of the first message in all the subscribers of the queue and then the oldest amongst all is taken.

The following views/tables are used for the calculation:

1. <SCHEMA>.AQ$_<QUEUE_TABLE>_S

2. <SCHEMA>.AQ$_<QUEUE_TABLE>_I

3. <SCHEMA>.AQ$<QUEUE_TABLE>

User Action

When using persistent queues for storing and propagating messages, monitor this metric to get the age of first message in the queue.

6.27.3 Messages Processed Per Persistent Queue (%)

This metric gives the messages processed percentage for the persistent queue. Messages processed percent is calculated as the percent of the total number messages processed or dequeued to the total number of messages received or enqueued.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed for queue %schema%.%queue_name% is %value% percent.

6.27.4 Messages Processed Per Persistent Queue (%) Per Minute

This metric gives the messages processed percentage per minute per persistent queue in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed per minute in the last interval for queue %schema%.%queue_name% is %value%

6.27.5 Total Messages Processed per Persistent Queue per Minute

This metric gives the total number of messages processed per minute per persistent queue in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Total messages processed per minute in the last interval for queue %schema%.%queue_name% is %value% .

6.27.6 Total Messages Received per Persistent Queue per Minute

This metric gives the total number of messages received or enqueued into the queue per minute in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Total messages received per minute in the last interval for queue %schema%.%queue_name% is %value% .

6.28 Messages Per Persistent Queue Per Subscriber

The metrics in the Messages Per Persistent Queue Per Subscriber metrics category monitor the age and state of the first (top of the queue) message for each persistent queue per queue subscriber in the database except for the system queues. Queues that are in the schema of SYS, SYSTEM, DBSNMP, and SYSMAN are defined as system level queues.

6.28.1 Average Age of Messages Per Persistent Queue Per Subscriber (Seconds)

This metric display's the average age of messages in the persistent queue per queue in seconds.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Average age of messages for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% seconds.

6.28.2 First Message Age in Persistent Queue per Subscriber (Seconds)

This metric gives the age (in seconds) of the first message in the persistent queue per subscriber for all non-system queues in the database.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Age of first message for subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% seconds.

6.28.3 Messages Processed Per Persistent Queue (%) Per Subscriber Per Minute

This metric gives the messages processed percentage per minute per persistent queue subscriber in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed per minute in the last interval for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% .

6.28.4 Messages Processed Per Persistent Queue Per Subscriber (%)

This metric gives the messages processed percentage for the persistent queue per subscriber. Messages processed percent is calculated as the percent of the total number messages processed or dequeued to the total number of messages received or enqueued.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% percent.

6.28.5 Total Messages Processed Per Persistent Queue Per Subscriber Per Minute

This metric gives the messages processed percentage per minute per persistent queue subscriber in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages processed per minute in the last interval for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% .

6.28.6 Total Messages Received Per Persistent Queue Per Subscriber Per Minute

This metric gives the total number of messages received or enqueued into the queue per subscriber per minute in the last collection interval of the metric.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Total messages received per minute in the last interval for the subscriber %subs_name% %subs_address% in %schema%.%queue_name% queue is %value% .

6.29 QoS Management - Performance Satisfaction

Oracle Database Quality of Service (QoS) Management is an automated, policy-based product that monitors the workload requests for an entire system.

For more information, see Oracle Database Quality of Service Management User's Guide.

6.29.1 Negative PSM Duration (seconds)

This metric tracks the negative PSM duration and raises an alert when it exceeds its threshold.

Performance Satisfaction Metric (PSM) is a normalized numeric value that indicates how well a particular Performance Objective is being met, and which enables Oracle Database QoS Management to compare the performance of the system for widely differing Performance Objectives.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 5 Minutes	Not Defined	Not Defined	Negative PSM Duration value %value% for Performance Class %PC% has crossed the threshold.

6.30 Recovery

This metric category contains the metrics representing database recovery.

6.30.1 Corrupt Data Block Count

This metric represents the count of corrupt data blocks.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Metric Summary 9iR2 or higher Evaluated and Collected every 15 minutes Operator > Warning Threshold - 0 Critical Threshold - Not Defined Number of corrupt data blocks is %value%.

Data Source

The data source for this metric is the following command:

SELECT count(unique(file#)) 
  FROM v$database_block_corruption;

User Action

Perform a database recovery.

6.30.2 Missing Media File Count

This metric represents the count of missing media files.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Metric Summary 8i or higher Evaluated and Collected every 15 minutes Operator > Warning Threshold - 0 Critical Threshold - Not Defined Number of missing media files is %value%.

Data Source

This metric is calculated with the following command:

SELECT count(file#) 
  FROM v$datafile_header 
  WHERE recover ='YES' OR error is not null;

User Action

You should perform a database recovery.

6.31 Recovery Area

This metric category contains the recovery area metrics.

6.31.1 Recovery Area Free Space (%)

Use the Recovery Area Free Space (%) metric to monitor Fast Recovery Area usage. This metric represents the recovery area free space as a percentage. It is a database-level metric that is evaluated by the database server every 15 minutes or during a file creation, whichever occurs first. The metric data is also printed in the alert log. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.The Critical Threshold is set for < 3% and the Warning Threshold is set for < 15%. You cannot customize these thresholds. An alert is returned the first time the alert occurs, and the alert is not cleared until the available space rises above 15%.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every 15 Minutes or during file creation, whichever occurs first	15% (Cannot be changed)	3% (Cannot be changed)	db_recovery_file_dest_size of N bytes is N% used, and has N remaining bytes available.

User Action

View the latest Automatic Database Diagnostic Monitor (ADDM) report. For a more detailed analysis, run ADDM from the Advisor Central link on the Database Home page.

6.32 SCN Growth Statistics

This metric category provides information about the Systems Change Number (SCN) in the database environment and reports on the health of the SCN growth in the database.

6.32.1 SCN Health

This metric displays the status of the SCN health.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every Hour	62	10	The SCN health is %scn_health%.

6.33 SCN Max Statistics

This metric category provides information about the maximum value of the SCN.

6.33.1 Max SCN Jump in one second (last 24 hours)

This metric displays the maximum SCN jump in one second over the previous 24 hours.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every Hour	Not Defined	Not Defined	The maximum SCN jump in one second (last 24 hours) is %scn_max_jump%.

6.34 Segment Advisor Recommendations

This metric category contains metrics related to the Automatic Segment Advisor job.

Oracle uses the Automatic Segment Advisor job to detect segment issues regularly within maintenance windows. It determines whether the segments have unused space that can be released. The Number of recommendations is the number of segments that have Reclaimable Space. The recommendations come from all runs of the automatic segment advisor job and any user-scheduled segment advisor jobs.

6.34.1 Number of recommendations

Oracle uses the Automatic Segment Advisor job to detect segment issues regularly within maintenance windows. It determines whether the segments have unused space that can be released. The Number of recommendations is the number of segments that have Reclaimable Space. The recommendations come from all runs of the automatic segment advisor job and any user-scheduled segment advisor jobs.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

6.35 Session Suspended

This metric category contains the metrics that represent the number of resumable sessions that are suspended due to a correctable error.

6.35.1 Session Suspended by Data Object Limitation

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric represents the session suspended by a data object limitation.

Target Version	Collection Frequency
10g, 11g, 12c	Every 5 Minutes

User Action

View the latest Automatic Database Diagnostic Monitor (ADDM) report. For a more detailed analysis, run ADDM from the Advisor Central link on the Database Home page.

6.35.2 Session Suspended by Quota Limitation

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric represents the session suspended by a quota limitation.

Target Version	Collection Frequency
10g, 11g, 12c	Every 5 Minutes

User Action

View the latest Automatic Database Diagnostic Monitor (ADDM) report. For a more detailed analysis, run ADDM from the Advisor Central link on the Database Home page.

6.35.3 Session Suspended by Rollback Segment Limitation

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric represents the session suspended by a rollback segment limitation.

Target Version	Collection Frequency
10g, 11g, 12c	Every 5 Minutes

User Action

View the latest Automatic Database Diagnostic Monitor (ADDM) report. For a more detailed analysis, run ADDM from the Advisor Central link on the Database Home page.

6.35.4 Session Suspended by Tablespace Limitation

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric represents the session suspended by a tablespace limitation.

Target Version	Collection Frequency
10g, 11g, 12c	Every 5 Minutes

User Action

View the latest Automatic Database Diagnostic Monitor (ADDM) report. For a more detailed analysis, run ADDM from the Advisor Central link on the Database Home page.

6.36 Snapshot Too Old

This metric category contains the snapshot too old metrics.

6.36.1 Snapshot Too Old due to Rollback Segment Limit

This database-level metric represents snapshots too old because of the rollback segment limit. This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

User Action

View the latest Automatic Database Diagnostic Monitor (ADDM) report. For a more detailed analysis, run ADDM from the Advisor Central link on the Database Home page.

6.36.2 Snapshot Too Old due to Tablespace Limit

This database-level metric represents snapshots too old because of the tablespace limit. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

User Action

View the latest Automatic Database Diagnostic Monitor (ADDM) report. For a more detailed analysis, run ADDM from the Advisor Central link on the Database Home page.

6.37 Space Usage by Buffered Queues

This metric category monitors the space usage of buffered queues with respect to the streams pool size.

6.37.1 Queue Size (MB)

This metric display's the size of buffered queue, which is the total number of Mega bytes allocated for all messages and metadata.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Size of buffered queue %schema%.%queue_name% is %value% MB.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Schema Name and Queue Name objects.

If warning or critical threshold values are currently set for any unique combination of Schema Name and Queue Name objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Schema Name and Queue Name objects, use the Edit Thresholds page.

Data Source

The source of this metric is the INSTANCE_NAME column from GV$INSTANCE view.

User Action

When using queues for storing or propagating messages, monitor this metric to get the instance in which the buffered queue is available.

6.37.2 Space Usage of Buffered Queue With Respect to Streams Pool Size (%)

This metric gives the space usage percentage of buffered queue with respect to streams pool size per buffered queue.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Buffered queue %schema%.%queue_name% has consumed %value% percent of streams pool size.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Schema Name and Queue Name objects.

If warning or critical threshold values are currently set for any unique combination of Schema Name and Queue Name objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Schema Name and Queue Name objects, use the Edit Thresholds page.

Data Source

The source of this metric is the QUEUE_SIZE AND CURRENT_SIZE columns from GV$BUFFERED_QUEUES and GV$SGA_DYNAMIC_COMPONENTS views.

User Action

When using buffered queues for storing or propagating messages, monitor this metric to get the space usage percentage of buffered queue with respect to the allocated streams pool size.

6.38 Streams Apply Queue - Buffered

The metrics in the Streams Apply Queue - Buffered metrics category show the current total number of messages in a buffered queue to be dequeued by each apply process and the total number of messages to be dequeued by each apply process that have spilled from memory into the persistent queue table.

6.38.1 Streams Apply - (%)Spilled Messages

This metric usually indicates that transactions are staying longer in memory.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
11gR2, 12c	Every 30 Minutes	Not Defined	Not Defined	Spilled messages for Apply process [%APPLY_NAME%] queue is %value% percent.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each Apply Name object.

If warning or critical threshold values are currently set for any Apply Name object, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each Apply Name object, use the Edit Thresholds page.

Data Source

The source for this metric is the target database in the gv$buffered_queues and gv$buffered_subscribers tables.

User Action

Either increase Streams Pool size and /or increase Apply Parallelism to speed up Apply processing.

6.39 Streams Apply Queue - Persistent

The metrics in the Streams Apply Queue - Persistent metrics category show the number of messages in a persistent queue in READY state and WAITING state for each apply process.

6.39.1 Streams Apply - (%)Messages in Waiting State

This metric shows the percentage of messages in a wait state.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2, 10g, 11g, 12c	Every 30 Minutes	Not Defined	Not Defined	Messages waiting for Apply process [%APPLY_NAME%] queue is %value% percent.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Apply Name and Messages Delivery Mode objects.

If warning or critical threshold values are currently set for any unique combination of Apply Name and Messages Delivery Mode objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Apply Name and Messages Delivery Mode objects, use the Edit Thresholds page.

Data Source

The data source for this metric is Target Database and Apply Queue.

User Action

No user action is required.

6.40 Streams Apply Reader Statistics

The reader server for an apply process dequeues messages from the queue. The reader server computes dependencies between LCRs and assembles messages into transactions. The reader server then returns the assembled transactions to the coordinator, which assigns them to idle apply servers.

The metrics in this metric category show the total number of messages dequeued by the reader server for the apply process since the last time the apply process was started.

6.40.1 Rate at Which Messages Are Getting Spilled (Per Sec)

The reader server for an apply process dequeues messages from the queue. The reader server computes dependencies between LCRs and assembles messages into transactions. The reader server then returns the assembled transactions to the coordinator, which assigns them to idle apply servers.

This metric shows the rate at which message are getting spilled (per second) by the reader server for the apply process since the last time the apply process was started.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9iR2, 10g, 11g, 12c	Every 30 Minutes	Not Defined	Not Defined	Total number of spilled messages for Apply Process [%APPLY_NAME%] is %value% .

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each Apply Name object.

If warning or critical threshold values are currently set for any Apply Name object, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each Apply Name object, use the Edit Thresholds page.

Data Source

For this metric, the data source is Target database, gv$streams_apply_reader view.

User Action

No user action is required.

6.41 Streams Capture Queue Statistics

The metrics in this metric category show the current total number of messages in a buffered queue that were enqueued by each capture process and the total number of messages enqueued by each capture process that have spilled from memory into the queue spill table.

If queue publishers other than the capture process enqueue messages into a buffered queue, then the values shown can include messages from these other queue publishers.

6.41.1 Streams Capture - (%)Spilled Messages

Queue spill indicates the messages are staying in memory longer. It can also indicate that the Propagation or Apply Process is slow to consume the enqueued messages.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every 30 Minutes	Not Defined	Not Defined	Spilled messages for Capture process %CAPTURE_NAME% queue is %value% percent.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each Capture Name object. If warning or critical threshold values are currently set for any Capture Name object, those thresholds can be viewed on the Metric Detail page for this metric. To specify or change warning or critical threshold values for each Capture Name object, use the Edit Thresholds page.

Data Source

The source of this metric is the target database in the gv$buffered_queues table view.

User Action

Increase Streams Pool Size to avoid queue spills.

6.42 Streams Latency and Throughput

The metrics in the Streams Latency and Throughput metrics category collect information about latency and throughput for each capture, propagation and apply component in the database. Latency and throughput are important indicators for the overall performance of the streams path.

6.42.1 Latency

High Latency indicates that the components are slow.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11g, 12c	Every 30 Minutes	Not Defined	Not Defined	Latency for Streams %streams_process_type% Process %streams_process_name% is %value% seconds.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Streams Process Name and Streams Process Type objects.

If warning or critical threshold values are currently set for any unique combination of Streams Process Name and Streams Process Type objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Streams Process Name and Streams Process Type objects, use the Edit Thresholds page.

Data Source

The data source for this metric is the target database in the gv$streams_capture, gv$propagation_sender, and gv$streams_apply_server views.

User Action

Identify and correct the least performing component in the streams configuration.

6.42.2 Throughput (per sec)

This metric collects information about throughput for each capture, propagation and apply component in the database.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10gR2, 11g, 12c	Every 30 Minutes	Not Defined	Not Defined	Throughput for Streams %streams_process_type% Process %streams_process_name% is %value% messages/sec.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each unique combination of Streams Process Name and Streams Process Type objects.

If warning or critical threshold values are currently set for any unique combination of Streams Process Name and Streams Process Type objects, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each unique combination of Streams Process Name and Streams Process Type objects, use the Edit Thresholds page.

Data Source

Not available.

User Action

The required actions are specific to your site.

6.43 Streams Processes Count

The metrics in this metric category show the total number of Streams capture processes, propagations, and apply processes at the local database. This metric also shows the number of capture processes, propagations, and apply processes that have encountered errors.

6.43.1 Apply Processes Having Errors

This metric shows the number of apply processes that have encountered errors at the local database.

Target Version	Collection Frequency
10g, 11g, 12c	Every 10 Minutes

Data Source

The information in this metric is in the DBA_APPLY data dictionary view.

User Action

If an apply process has encountered errors, then correct the conditions that caused the errors.

6.43.2 Capture Processes Having Errors

This metric shows the number of capture processes that have encountered errors at the local database.

Target Version	Collection Frequency
10g, 11g, 12c	Every 10 Minutes

Data Source

The information in this metric is in the DBA_CAPTURE data dictionary view.

User Action

If a capture process has encountered errors, then correct the conditions that caused the errors.

6.43.3 Number of Apply Processes

This metric shows the number of apply processes at the local database.

Target Version	Collection Frequency
10g, 11g, 12c	Every 10 Minutes

Data Source

The information in this metric is in the DBA_APPLY data dictionary view.

User Action

Use this metric to determine the total number of apply processes at the local database.

6.43.4 Number of Capture Processes

This metric shows the number of capture processes at the local database.

Target Version	Collection Frequency
10g, 11g, 12c	Every 10 Minutes

Data Source

The information in this metric is in the DBA_CAPTURE data dictionary view.

User Action

Use this metric to determine the total number of capture processes at the local database.

6.43.5 Number of Propagation Jobs

This metric shows the number of propagations at the local database.

Target Version	Collection Frequency
10g, 11g, 12c	Every 10 Minutes

Data Source

The information in this metric is in the DBA_PROPAGATION data dictionary view.

User Action

Use this metric to determine the total number of propagations at the local database.

6.43.6 Propagation Errors

This metric shows the number of propagations that have encountered errors at the local database.

Target Version	Collection Frequency
10g, 11g, 12c	Every 10 Minutes

Data Source

The information in this metric is in the DBA_PROPAGATION data dictionary view.

User Action

If a propagation has encountered errors, then correct the conditions that caused the errors.

6.44 Streams Propagation - Message State Stats

The metrics in this metric category collect the number of messages in Ready and Waiting state for each Propagation process.

6.44.1 Streams Prop - (%)Messages in Waiting State

This metric collects the number of messages in Ready state for each Propagation process.

Target Version	Collection Frequency
9iR2, 10g, 11g, 12c	Every 30 Minutes

Data Source

The source of the data for this metric is the target database in the source and destination queues.

User Action

No user action is required.

6.45 Suspended Session

The metrics in this metric category contain the metrics that represent the number of resumable sessions that are suspended due to a correctable error.

6.45.1 Suspended Session Count

This metric represents the number of resumable sessions currently suspended in the database.

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9i	Every 5 Minutes	0	Not Defined	%value% session(s) are suspended.

Data Source

This metric is calculated with the following command:

SELECT count(*) 
  FROM v$resumable 
  WHERE status = 'SUSPENDED' and enabled = 'YES' 

User Action

Query the v$resumable view to see what the correctable errors are that are causing the suspension. The method to correct each error depends on the nature of the error.

6.46 Tablespace Allocation

The metrics in this metric category check the amount of space used and the amount of space allocated to each tablespace. The used space can then be compared to the allocated space to determine how much space is unused in the tablespace. This metric is intended for reporting, rather than alerts. Historical views of unused allocated free space can help DBAs to correctly size their tablespaces, eliminating wasted space.

6.46.1 Tablespace Allocated Space (MB)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The allocated space of a tablespace is the sum of the current size of its data files. A portion of this allocated space is used to store data while some may be free space. If segments are added to a tablespace, or if existing segments grow, they will use the allocated free space. The allocated free space is only available to segments within the tablespace. If, over time, the segments within a tablespace are not using this free space, the allocated free space is not being used.

This metric calculates the space allocated for each tablespace. It is not intended to generate alerts. Rather it should be used in conjunction with the Allocated Space Used (MB) metric to produce a historical view of the amount of space being used and unused by each tablespace.

Data Source

Tablespace Allocated Space (MB) is calculated by looping though the tablespace�s data files and totalling the size of the data files.

6.46.2 Tablespace Used Space (MB)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

The allocated space of a tablespace is the sum of the current size of its data files. Some of this allocated space is used to store data, and some of it may be free space. If segments are added to a tablespace, or if existing segments grow, they will use the allocated free space. The allocated free space is only available to segments within the tablespace. If, over time, the segments within a tablespace are not using this free space, then the allocated free space is being wasted.

This metric calculates the space used for each tablespace. It is not intended to generate alerts. Rather it should be used in conjunction with the Tablespace Allocated Space (MB) metric to produce a historical view of the amount of space being used and unused by each tablespace.

Data Source

Tablespace Used Space (MB) is derived from Tablespace Allocated Space (MB)� Tablespace Allocated Free Space (MB) where:

Tablespace Allocated Space (MB) is calculated by looping through the tablespace�s data files and totaling the size of the data files.

Tablespace Allocated Free Space (MB) is calculated by looping through the tablespace�s data files and totaling the size of the free space in each data file.

6.47 Tablespaces Full

The metrics in this metric category check for the amount of space used by each tablespace. The used space is then compared to the available free space to determine tablespace fullness. The available free space accounts for the maximum data file size as well as available disk space. This means that a tablespace will not be flagged as full if data files can extend and there is enough disk space available for them to extend.

6.47.1 Tablespace Free Space (MB)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

As segments within a tablespace grow, the available free space decreases. If there is no longer any available free space, meaning data files reached their maximum size or there is no more disk space, then the creation of new segments or the extension of existing segments will fail.

This metric checks for the total available free space in each tablespace. This metric is intended for larger tablespaces, where the Available Space Used (%) metric is less meaningful. If the available free space falls below the size specified in the threshold arguments, then a warning or critical alert is generated.

If the version of the monitored database target is Oracle Database 10g Release 1 or later and the tablespace uses Local Extent Management, then the Oracle Database Server evaluates this metric internally every 10 minutes. Alternatively, if the version of the monitored Database target is Oracle 9i or earlier, or the tablespace uses Dictionary Extent Management, then the Oracle Management Agent tests the value of this metric every 30 minutes.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
8i, 9i	Every 30 Minutes	Not Defined	Not Defined	Tablespace [%name%] has [%value% mbytes] free
10g, 11g, 12c	Every 30 Minutes	Not Defined	Not Defined	Generated By Database Server

Data Source

The source of the data for this metric is MaximumSize� Total Used Space where:

• TotalUsedSpace: Total used space in MB of tablespace.

• MaximumSize: Maximum size (in MB) of the tablespace. The maximum size is determined by looping through the tablespace�s data files, as well as additional free space on the disk that would be available for the tablespace should a data file autoextend.

User Action

Perform one of the following:

• Increase the size of the tablespace by: Enabling automatic extension for one of its existing data files, manually resizing one of its existing data files, or adding a new data file.

• If the tablespace is suffering from tablespace free space fragmentation problems, consider reorganizing the entire tablespace.

• Relocate segments to another tablespace, thereby increasing the free space in this tablespace.

• Run the Segment Advisor on the tablespace.

6.47.2 Tablespace Space Used (%)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

As segments within a tablespace grow, the available free space decreases. If there is no longer any available free space, meaning data files have reached their maximum size or there is no more disk space, then the creation of new segments or the extension of existing segments will fail.

This metric checks the Available Space Used (%) for each tablespace. If the percentage of used space is greater than the values specified in the threshold arguments, then a warning or critical alert is generated.

If the version of the monitored database target is Oracle Database 10g Release 1 or later and the tablespace uses Local Extent Management, then the Oracle Database Server evaluates this metric internally every 10 minutes. Alternatively, if the version of the monitored Database target is Oracle 9i or earlier, or the tablespace uses Dictionary Extent Management, then the Oracle Management Agent tests the value of this metric every 30 minutes.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
8i, 9i	Every 30 Minutes	85	97	Tablespace [%name%] is [%value% percent] full
10g, 11g, 12c	Every 30 Minutes	85	97	Generated By Database Server

Data Source

This metric is calculated with the following command where:

• TotalUsedSpace: total used space in MB of tablespace.

• MaximumSize: Maximum size (in MB) of the tablespace. The maximum size is determined by looping through the tablespace�s data files, as well as additional free space on the disk that would be available for the tablespace should a data file autoextend.

(TotalUsedSpace / MaximumSize) * 100

For additional information about the data source, refer to the fullTbsp.pl Perl script located in the sysman/admin/scripts directory.

User Action

Perform one of the following:

• Increase the size of the tablespace by: Enabling automatic extension for one of its existing data files, manually resizing one of its existing data files, or adding a new data file.

• If the tablespace is suffering from tablespace free space fragmentation problems, consider reorganizing the entire tablespace.

• Relocate segments to another tablespace, thus increasing the free space in this tablespace.

• Run the Segment Advisor on the tablespace.

6.48 Tablespaces Full (dictionary managed)

The metrics in this metric category check for the amount of space used by each tablespace. The used space is then compared to the available free space to determine tablespace fullness. The available free space accounts for the maximum data file size as well as available disk space. This means that a tablespace will not be flagged as full if data files can extend, and there is enough disk space available for them to extend.

6.48.1 Tablespace Free Space (MB) (dictionary managed)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

As segments within a tablespace grow, the available free space decreases. If there is no longer any available free space, meaning data files have reached their maximum size or there is no more disk space, then the creation of new segments or the extension of existing segments will fail.

This metric checks for the total available free space in each tablespace. This metric is intended for larger tablespaces, where the Available Space Used (%) metric is less meaningful. If the available free space falls below the size specified in the threshold arguments, then a warning or critical alert is generated.

If the version of the monitored database target is Oracle Database 10g Release 1 or later and the tablespace uses Local Extent Management, then the Oracle Database Server evaluates this metric internally every 10 minutes. Alternatively, if the version of the monitored Database target is Oracle 9i or earlier, or the tablespace uses Dictionary Extent Management, then the Oracle Management Agent tests the value of this metric every 30 minutes.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every 30 Minutes	Not Defined	Not Defined	Tablespace [%name%] has [%value% mbytes] free.

Data Source

The source of the data for this metric is MaximumSize� Total Used Space where:

• TotalUsedSpace: Total used space in MB of tablespace.

• MaximumSize: Maximum size (in MB) of the tablespace. The maximum size is determined by looping through the tablespace�s data files, as well as additional free space on the disk that would be available for the tablespace should a data file autoextend.

6.48.2 Tablespace Space Used (%) (dictionary managed)

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

As segments within a tablespace grow, the available free space decreases. If there is no longer any available free space, meaning data files have reached their maximum size or there is no more disk space, then the creation of new segments or the extension of existing segments will fail.

This metric checks the Available Space Used (%) for each tablespace. If the percentage of used space is greater than the values specified in the threshold arguments, then a warning or critical alert is generated.

If the version of the monitored database target is Oracle Database 10g Release 1 or later and the tablespace uses Local Extent Management, then the Oracle Database Server evaluates this metric internally every 10 minutes. Alternatively, if the version of the monitored Database target is Oracle 9i or earlier, or the tablespace uses Dictionary Extent Management, then the Oracle Management Agent tests the value of this metric every 30 minutes.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
10g, 11g, 12c	Every 30 Minutes	85	97	Tablespace [%name%] is [%value% percent] full

Data Source

The source of the data for this metric is (TotalUsedSpace / MaximumSize) * 100 where:

• TotalUsedSpace: Total used space in MB of tablespace.

• MaximumSize: Maximum size (in MB) of the tablespace. The maximum size is determined by looping through the tablespace�s data files, as well as additional free space on the disk that would be available for the tablespace should a data file autoextend.

User Action

Perform one of the following:

• Increase the size of the tablespace by: Enabling automatic extension for one of its existing data files, manually resizing one of its existing data files, or adding a new data file.

• If the tablespace is suffering from tablespace free space fragmentation problems, consider reorganizing the entire tablespace.

• Relocate segments to another tablespace, thereby increasing the free space in this tablespace.

• Run the Segment Advisor on the tablespace.

6.49 Tablespaces With Problem Segments

The metrics in this metric category check for the following:

• The largest chunk-free space in the tablespace. If any table, index, cluster, or rollback segment within the tablespace cannot allocate one additional extent, then an alert is generated.

• Whether any of the segments in the tablespace are approaching their maximum extents. If, for any segment, the maximum number of extents minus the number of existing extents is less than 2, an alert is generated.

Only the tablespaces with problem segments are returned as results.

6.49.1 Segments Approaching Maximum Extents Count

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric checks for segments nearing the upper limit of the number of maximum extents. If the number of segments is greater than the values specified in the threshold arguments, a warning or critical alert is generated.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
All Versions	Every 24 Hours	0	Not Defined	%value% segments in %name% tablespace approaching max extents.

Data Source

The source of the data for this metric is the number of segments for which the maximum number of extents minus the number of existing extents is less than 2.

For additional information about the data source, refer to the problemTbsp.pl Perl script located in the sysman/admin/scripts directory.

User Action

If possible, increase the value of the segment�s MAXEXTENTS storage parameter. Otherwise, rebuild the segment with a larger extent size ensuring the extents within a segment are the same size by using a locally managed tablespace. For a dictionary managed tablespace, specify STORAGE parameters where NEXT=INITIAL and PCTINCREASE = 0.

6.49.2 Segments Not Able to Extend Count

This is a database-level metric. For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric checks for segments that cannot allocate an additional extent. If the number of segments is greater than the values specified in the threshold arguments, a warning or critical alert is generated.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
All Versions	Every 24 Hours	0	Not Defined	%value% segments in %name% tablespace unable to extend.

Data Source

After checking for the largest chunk free space in the tablespace, this is the number of segments that cannot allocate an additional extent.

For additional information about the data source, refer to the problemTbsp.pl Perl script located in the sysman/admin/scripts directory.

User Action

Perform one of the following:

• Increase the size of the tablespace by enabling automatic extension for one of its existing data files, manually resizing one of its existing data files, or adding a new data file.

• If the tablespace is suffering from tablespace free space fragmentation problems, consider reorganizing the entire tablespace.

• Relocate segments to another tablespace, thereby increasing the free space in this tablespace.

6.50 Temporary File Status

This metric category contains the Temporary File Status metric.

6.50.1 Temporary File Id

The absolute file number of the temporary file, used to join with other database tables and views to retrieve additional information.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9i, 10g, 11g, 12c	Every 15 Minutes	OFFLINE	Not Defined	The temporary file %NAME% is %STATUS%.

6.51 Total Objects by Schema

The metrics in this metric category contain the metric that provides the number of database objects in a schema.

6.51.1 Total Object Count

This metric displays the total number of database objects in a schema.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 24 Hours	Not Defined	Not Defined	%value% object(s) exist in the %owner% schema.

6.52 Total Tables by Schema

The metrics in this metric category provide the number of tables in a schema.

6.52.1 Total Table Count

This metric displays the total number of tables in a schema.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 24 Hours	Not Defined	Not Defined	%value% table(s) exist in the %owner% schema.

6.53 User Block

The metrics in this metric category contain the metrics that tell to what extent, and how consistently, a given session is blocking multiple other sessions.

6.53.1 Blocking Session Count

For cluster databases, this metric is monitored at the cluster database target level and not by member instances.

This metric signifies that a database user is blocking at least one other user from performing an action, such as updating a table. An alert is generated if the number of consecutive blocking occurrences reaches the specified value. The sessions being blocked can come from different instances.

Note: The catblock.sql script needs to be run on the managed database prior to using the User Blocks test. This script creates some additional tables, view, and public synonyms that are required by the User Blocks test.

Target Version	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
Not Available	Every 5 Minutes	0	Not Defined	Session %sid% blocking %value% other sessions for all instances.

Data Source

This metric is calculated using the following command:

SELECT blocking_sid, num_blocked 
  FROM ( SELECT blocking_sid, SUM(num_blocked) num_blocked 
  FROM ( SELECT l.id1, l.id2, 
                MAX(DECODE(l.block, 1, i.instance_name||'-'||l.sid, 
                2, i.instance_name||'-'||l.sid, 0 )) blocking_sid, 
                SUM(DECODE(l.request, 0, 0, 1 )) num_blocked 
          FROM gv$lock l, gv$instance i 
          WHERE ( l.block!= 0 OR l.request > 0 ) AND 
                  l.inst_id = i.inst_id 
                  GROUP BY l.id1, l.id2) 
  GROUP BY blocking_sid 
  ORDER BY num_blocked DESC) 
  WHERE num_blocked != 0 

User Action

Either have the user who is blocking other users rollback the transaction, or wait until the blocking transaction has been committed.

6.54 User Locks

The metrics in this metric category provide information regarding user locks.

Enterprise Manager will issue the alert when the Maximum Blocked Session Count or maximum blocked DB time (seconds) of transactional locks: TM, TX, UL reach the threshold.

6.54.1 Maximum Blocked DB Time (seconds)

This metric represents the maximum time wasted in any given lock chain, not for the total time wasted by everyone in any lock chain.

Target Version	Key	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9i, 10g, 11g, 12c	lockType= TM	Every 10 Minutes	Not Defined	Not Defined	%value% seconds in DB Time is spent waiting for %lockType% lock.
9i, 10g, 11g, 12g	lockType= TX	Every 10 Minutes	Not Defined	Not Defined	%value% seconds in DB Time is spent waiting for %lockType% lock.
9i, 10g, 11g, 12c	lockType = UL	Every 10 Minutes	Not Defined	Not Defined	%value% seconds in DB Time is spent waiting for %lockType% lock.

Multiple Thresholds

For this metric you can set different warning and critical threshold values for each User Lock Type object.

If warning or critical threshold values are currently set for any User Lock Type object, those thresholds can be viewed on the Metric Detail page for this metric.

To specify or change warning or critical threshold values for each User Lock Type object, use the Edit Thresholds page.

Data Source

The data for the metric is retrieved from database view gv$session.

User Action

You can set the threshold for warning alert or critical alert for maximum Blocked DB Time (seconds). When maximum time wasted in any given lock chain reaches the threshold, Enterprise Manager will issue the alert.

6.54.2 Maximum Blocked Session Count

This metric represents the maximum length of any lock chain, not for the total number of people stuck in lock chains.

Target Version	Key	Evaluation and Collection Frequency	Default Warning Threshold	Default Critical Threshold	Alert Text
9i, 10g, 11g, 12c	lockType= TM	Every 10 Minutes	Not Defined	Not Defined	%value% sessions are blocked by %lockType% lock.
9i, 10g, 11g, 12c	lockType= TX	Every 10 Minutes	Not Defined	Not Defined	%value% sessions are blocked by %lockType% lock.
9i, 10g, 11g, 12c	lockType= UL	Every 10 Minutes	Not Defined	Not Defined	%value% sessions are blocked by %lockType% lock.