Streaming with Apache Kafka Metrics
Using metrics, you can monitor Kafka clusters and endpoints in OCI Streaming with Apache Kafka.
Use the OCI Monitoring service to create custom queries and alarms to notify you when these metrics meet alarm-specified triggers.
We recommend that you monitor client metrics using your own custom dashboards.
Available Metrics
A namespace is a container for metrics. The namespace for OCI Streaming with Apache Kafka is oci_kafka.
Use the following Streaming with Apache Kafka metrics with the following three dimensions:
brokerId: The ID of the Kafka cluster broker.resourceId: The OCID of the Kafka cluster.resourceName: The name of the Kafka cluster.
| Metric | Unit | Description |
|---|---|---|
BrokerActiveControllerCount |
Count | Number of active controllers for each broker. Must always be 1. |
BrokerAvailableDiskSpace |
Percentage | Percentage of available disk space in each broker. |
BrokerBytesInTotal |
Bytes | Total Number of bytes in for a broker |
BrokerBytesOutTotal |
Bytes | Total Number of bytes out for a broker |
BrokerConnectionCountPerListener |
Count | Total number of active connections to a Kafka broker, broken down by listener (such as PLAINTEXT and SSL). |
BrokerCPUUsage |
Percentage | Total CPU usage for each broker. |
BrokerFailedFetchRequests |
Count | Cumulative count of Fetch requests that the broker returned with an error, signalling client-visible read failures. |
BrokerFailedProduceRequests |
Count | Tracks the total number of failed produce (write) requests received by the Kafka broker. |
BrokerMemoryUsed |
Percentage | Total heap + non-heap memory usage for each broker. |
BrokerNetworkProcessorAvgIdlePercentage |
Percentage | The average idle percentage of the Kafka network I/O threads (also known as network processors) over time. |
BrokerOfflinePartitionsCount |
Count | Number of partitions assigned to a broker that are currently offline. |
BrokerOpenFileDescriptors |
Count | Total file descriptors that are open on a broker. |
BrokerRequestHandlerAvgIdlePercentage |
Percentage | Measures the average percentage of time that Kafka request handler threads are idle, indicating available processing capacity. |
BrokerTotalFetchRequests |
Count | Cumulative count of every Fetch request the broker has processed, reflecting overall read-side traffic volume. |
BrokerTotalIncomingMessages |
Count | Counts the total number of messages received by the Kafka broker across all topics and partitions. |
BrokerTotalProduceRequests |
Count | Tracks the total number of produce (write) requests received by the Kafka broker. |
BrokerUnderReplicatedPartitions |
Count | Shows the number of partitions that are under replicated in a broker. |
Recommendations
View the recommendations for the available metrics to ensure healthy operation of the Kafka clusters.
You must create alarms to monitor the values of these metrics.
| Metric | Alert Threshold Recommendation |
|---|---|
BrokerCPUUsage |
|
BrokerAvailableDiskSpace |
|
BrokerMemoryUsed |
|
BrokerActiveControllerCount |
|
BrokerUnderReplicatedPartitions |
|
BrokerOfflinePartitionsCount |
|