Using Oracle Coherence Metrics

4 Using Oracle Coherence Metrics

Oracle Coherence Metrics enables an alternative means to monitor a running Coherence cluster than JMX MBeans.

This chapter includes the following sections:

About Coherence Metrics

As of version 12.2.1.4, Coherence provides a metrics endpoint which extends the metrics capability by allowing scraping of metrics via metrics gathering systems such as Prometheus. The Coherence metrics endpoints are enabled on each Coherence Cache server, and a Prometheus scraping job is configured with the location of these endpoints.

If an application requires the Coherence Metrics endpoints to be secure, it is possible to configure the Coherence Metrics endpoint to be secure and to configure the Prometheus scraping job with the necessary security information. See Securing Oracle Coherence Metrics in Securing Oracle Coherence.

The Coherence Metrics endpoint supports content-encoding of gzip. By default, the metrics endpoint returns metrics in text, or Prometheus format, but can also return metrics in JSON format if the Accepts: application/json HTTP header is set.

To secure Oracle Coherence Metrics endpoint, see Securing Oracle Coherence Metrics in Securing Oracle Coherence.

In this release of Coherence, the coherence-metrics module has been deprecated and all the metrics functionality has been moved into coherence.jar. Unlike previous versions, Coherence metrics no longer require third-party dependencies. Therefore, all applications should remove any dependency on coherence-metrics.jar as this module will be removed in a future release.

Enabling Coherence Metrics Endpoint

To enable the Coherence Metrics endpoint, set the system property coherence.metrics.http.enabled to true when starting the Coherence cache server.

Using Metrics System Properties

The metrics system properties are defined in the metrics-http-config.xml file in coherence.jar.

This section contains the following topics:

Coherence Metrics System Properties

The following Coherence Metrics system properties are defined in the default metrics-http-config.xml file in coherence.jar.

Table 4-1 Coherence Metrics system properties

System Property	Description	Default
`coherence.metrics.http.enabled`	Determines if metrics endpoint enabled	`false`
`coherence.metrics.http.address`	Specifies the address to listen on	`0.0.0.0` (All interfaces)
`coherence.metrics.http.port`	Specifies the port to listen on (0 indicates the ephemeral port)	`9612`
`coherence.metrics.http.auth`	Configures the authentication mechanism for the HTTP server `basic`, `auth`, `basic+auth`or `not set`	not set
`coherence.metrics.http.provider`	Defines the id of a `<socket-provider>` in coherence operational override file	none
`coherence.metrics.http.path`	Specifies the metrics context root path.	not set

You can override the default metrics-http-config.xml by placing an override file with same file name in classpath before coherence.jar.

Note:

If multiple Coherence cache servers with metrics endpoint enabled are started on same machine, the coherence.metrics.http.port must be unique for each server. If not, the subsequent ones with same coherence.metrics.http.port value will get an address already in use and will result in an error with starting up the MetricsHttpProxy service.

The Coherence cache server will start up but the Coherence Metrics endpoint will only be available on one Coherence cache server.

Other Metrics System Properties

The other metrics system property includes:

Table 4-2 Other metrics system properties

System Property	Description	Default
`coherence.metrics.extended`	Includes extended information (type and/or description) when publishing metrics.	`false`

The extended information for Prometheus metrics is #Help, which contains the metrics name. If more data follows, the #Help is considered the docstring for that metric name. For Coherence Metrics, the docstring is typically the JMX MBean attribute description from which the metric value was derived. There is one or no #Help per metrics value in a Prometheus response.

Configuring Prometheus to Scrape Coherence Metrics Endpoint

Configuring a scraping job to scrape the Coherence Metrics endpoints for a Coherence cluster is done using the static_configs parameter of a Prometheus configuration file. You need to specify the IP address and coherence.metrics.http.port for each Coherence Metrics endpoint in the static_configs of a Prometheus configuration file. Once configured and started, the Prometheus server will scrape the configured static targets at the configured interval, collecting metrics data for the Coherence cluster.

See Prometheus configuration documentation.

Note:

The plain text format metric names published by the Coherence metrics endpoint (for example when being scraped by Prometheus) are in the vendor:coherence_<metric_name> format. This legacy name format has been deprecated, and will be changed in a future Coherence release to be coherence_<metric_name>. That is, the vendor: prefix will be removed. If you want to use the future metric name format, set the coherence.metrics.legacy.names system property to false when starting Coherence.

Visualizing Metrics in Grafana

After you have configured Prometheus to scrape metrics, you can visualize these metrics using the pre-built Grafana dashboards available in the Coherence Operator. See coherence-operator.

These dashboards provide detailed insight into your Coherence cluster by using the collected metrics and are a valuable tool in monitoring and managing the health of you cluster.

For more information about the dashboards and how to load them, see the Coherence Operator documentation.

For an example of the Coherence Summary Dashboard, see Use dashboards.

Note:

Oracle recommends you to use the minimum Grafana version 8.5.13 to view the dashboards. At this point, due to the many issues with Grafana 9.x, do not use this version yet.

Querying for Coherence Metrics

The metrics endpoint supports Prometheus and JSON formats. It also supports querying for specific metrics by name and tags.

This section contains the following topics:

Basic Querying

The coherence.metrics.http.port is assumed to be set to 9612 for the below listed basic querying examples.

http://localhost:9612/metrics

By default, this will produce Prometheus' simple text-based exposition format. If the caller uses the Accepts: application/json header, then JSON is returned. The endpoint also supports content negotiation by appending the required type to the url.

http://localhost:9612/metrics/.txt returns Prometheus data whereas http://localhost:9612/metrics/.json returns JSON data.

The coherence.metrics.extended system property controls the inclusion of additional help or descriptions. By default, the coherence.metrics.extended is set to false and additional information is not included. By adding the extended=true query parameter, additional data can be included.

http://localhost:9612/metrics?extended=true (this applies to both Prometheus and JSON formats).

Querying for Specific Metrics

To query for specific metrics append the metric name to the URL.

To query for number of members in a cluster:

http://localhost:9612/metrics/Coherence.Cluster.Size

To query for all cache size metrics:

http://localhost:9612/metrics/Coherence.Cache.Size

To query for a metric with matching tags add the tag key and value as query parameters. To query for the size metric for a specific cache where the cache name is cache_name.

http://localhost:9612/metrics/Coherence.Cache.Size?name=cache_name

Additionally, if the cache (for example near cache) is included so that there are front and back tiers, then the request could be further restricted.

http://localhost:9612/metrics/Coherence.Cache.Size?name=cache_name&tier=back

http://localhost:9612/metrics/Coherence.Cache.Size?name=cache_name&tier=front

Adding Custom Metrics

You can add your own metrics if you annotate the MBeans that you add to the Coherence Registry.

For example:

import com.tangosol.net.management.annotation.MetricsValue;
import com.tangosol.net.management.annotation.MetricsTag;   

public interface CustomMBean
    {
    @MetricsValue("custom_value")
    long getValueOne();

    @MetricsValue
    long getValueTwo();

    long getValueThree();

    @MetricsTag("custom_tag")
    String getTagValueOne();

    @MetricsTag
    String getTagValueTwo();
    }

The method getValueOne is annotated with @MetricValue. Hence, it will be a metric and the name of the metric will be custom_value.
The method getValueTwo is annotated with @MetricValue. Hence, it will be a metric. The annotation does not specify a name and the name will be generated from the attribute name (in this case value_two).
The method getValueThree is not annotated. Hence, though being an MBean attribute it will not be used as a metric or as a tag.
The method getTagValueOne is annotated with @MetricTag. Hence, it will be a metric tag and the name of the tag will be custom_tag.
The method getTagValueTwo is annotated with @MetricTag. Hence, it will be a metric tag. The annotation does not specify a name and the name will be generated from the attribute name (in this case tag_value_two).

For annotations to be detected by Coherence when registering the MBean, the annotations must be wrapped in an AnnotatedStandardMBean.

For example:

Registry registry   = CacheFactory.ensureCluster().getManagement();
Custom   custom     = new Custom();
String   sMBeanName = registry.ensureGlobalName(“type=Custom,name=bar”);

registry.register(sMBeanName, new AnnotatedStandardMBean(custom, CustomMBean.class));

The code above will register the Mbean with the Object name similar to Coherence:type=Custom,name=bar,nodeId=1

The metrics name produced will be prefixed with the value from the type key in the ObjectName, in this case Custom. This will produce the following metrics:

Custom_custom_value from method getValueOne
Custom_value_two from method getValueTwo

Both metrics will have common tags such as cluster name, nodeId, machine, role, site, and so on, as well as the custom tags name=bar from the ObjectName, custom_tag from the annotated method getTagValueOne and tag_value_two from the annotated method getTagValueTwo. The method getTagValueTwo is annotated with @MetricTag. Hence it will be a metric tag. The annotation does not specify a name and the name will be generated from the attribute name (in this case tag_value_two).

Using Coherence Micrometer Metrics

The coherence-micrometer module provides integration between Coherence metrics and Micrometer allowing Coherence metrics to be published through any of the Micrometer registries.

This section includes the following topics:

Enabling the Use of Coherence Micrometer Metrics

To use Coherence Micrometer metrics, you should declare the module as a dependency in the pom.xml file and bind the Micrometer registry with the Coherence metrics adapter:

<dependency>
    <groupId>${coherence.groupId}</groupId>
    <artifactId>coherence-micrometer</artifactId>
    <version>${coherence.version}</version>
</dependency>

The coherence-micrometer provides a Micrometer MeterBinder implementation class called CoherenceMicrometerMetrics. This class is a singleton and cannot be constructed. To access this class, use the CoherenceMicrometerMetrics.INSTANCE field.

Micrometer provides many registry implementations to support different metrics applications and formats. For example, to bind Coherence metrics to the Micrometer PrometheusMeterRegistry, create the PrometheusMeterRegistry. For more information, see the Micrometer Documentation, and call the CoherenceMicrometerMetrics class’s bindTo method:

PrometheusMeterRegistry prometheusRegistry = new PrometheusMeterRegistry(PrometheusConfig.DEFAULT);

// complete the registry configuration...

CoherenceMicrometerMetrics.INSTANCE.bindTo(prometheusRegistry);

You can bind the Micrometer registries to Coherence at any time, before or after Coherence starts. As Coherence creates or removes metrics, they will be registered with or removed from the Micrometer registries.

Automatic Global Registry Binding

Micrometer has a global registry available which Coherence binds to automatically if you set the coherence.micrometer.bind.to.global system property to true (this property is false by default).