High Performance Computing Metrics

The compute instance metrics help you measure the activity level and throughput of compute instances. The metrics listed in the following table are available for any monitoring-enabled compute instance. To get these metrics, enable monitoring on the instance.

The metrics in this namespace are aggregated across all the related resources on the instance. For example, DiskBytesRead is aggregated across all the instance's attached storage volumes, and NetworkBytesIn is aggregated across all the instance's attached VNICs.

For metrics emitted by the metric namespace gpu_infrastructure_health, data points are sampled every ten seconds. A batch of six of data points is emitted every minute. Therefore, for every minute granularity, the aggregate count is always six, the aggregate sum is the sum of the six data points, and the aggregate average is the average of the six data points.

You also can use the Monitoring service to create custom queries.

Each metric includes the following dimensions :

component

GPU or rdma_nic

timestamp

UTC time when the payload/heartbeat is emitted

version

The payload version number for compatibility

Fault Metrics: gpu_infrastructure_health

The compute instance metrics help you measure activity level and throughput of compute instances. The metrics listed in the following table are available for any monitoring-enabled compute instance. To get these metrics, enable monitoring on the instance.

The metrics in this namespace are aggregated across all the related resources on the instance. For example, DiskBytesRead is aggregated across all the instance's attached storage volumes, and NetworkBytesIn is aggregated across all the instance's attached VNICs.

For metrics emitted by the metric namespace rdma_infrastructure_health, data points are sampled every ten seconds. A batch of six of data points is emitted every minute. Therefore, for every minute granularity, the aggregate count is always six, the aggregate sum is the sum of the six data points, and the aggregate average is the average of the six data points.

You also can use the Monitoring service to create custom queries.

Each metric includes the following dimensions :

component

GPU or rdma_nic

timestamp

UTC time when the payload/heartbeat is emitted

version

The payload version number for compatibility

Fault Metrics: rdma_infrastructure_health

Metric	Metric Display Name	Unit	Description	Dimensions
RdmaLinkSpeedFault	Faults	count	Detects if a link speed fault is present. If the value is 0, there are no faults. If the value is 1, faults are detected.	availabilityDomain faultDomain imageId instancePoolId pcieAddress rdmaId region resourceDisplayName resourceId shape
RdmaPcieAddressFault	Faults	count	Detects if a PCIE address fault is present. If the value is 0, there are no faults. If the value is 1, faults are detected.
RdmaPcieBerCheckFault	Faults	count	Detects if a PCIE BER fault is present. If the value is 0, there are no faults. If the value is 1, faults are detected.
RdmaPcieCableFlapFault	Faults	count	Detects if a PCIE cable flap fault is present. If the value is 0, there are no faults. If the value is 1, faults are detected.
RdmaPcieCablePlugFault	Faults	count	Detects if a PCIE cable plug fault is present. If the value is 0, there are no faults. If the value is 1, faults are detected.
RdmaPcieCableStateFault	Faults	count	Detects if a PCIE cable state fault is present. If the value is 0, there are no faults. If the value is 1, faults are detected.

The compute instance metrics help you measure activity level and throughput of compute instances. The metrics listed in the following table are available for any monitoring-enabled compute instance. To get these metrics, enable monitoring on the instance.

For metrics emitted by the metric namespace oci_gpu_infrastructure, data points are sampled every minute. A batch of 13 data points is emitted every minute. For a host with N RDMA NICs, each metric produces N datapoints/minute.

For the 13 new metrics in this table, the total datapoints per host per minute are:

You also can use the Monitoring service to create custom queries.

Each metric includes the following dimensions :

component

GPU or rdma_nic

timestamp

UTC time when the payload/heartbeat is emitted

version

The payload version number for compatibility

Metric Counters: oci_gpu_infrastructure

Metric	Metric Display Name	Unit	Description	Dimensions
NpEcnMarkedRocePacketsCount	Count	count	ECN-marked RoCEv2 packets received (ECN=11). RDMA HW counter.	baremetal_cluster_ocid customer_network_block_ocid gpu_shape host_serial hpc_island_id poolname port_pci tailnode_serial
OutOfSequenceCount	Count	count	Out-of-sequence packets received. Error counter.
NpCnpSentCount	Count	count	CNP packets sent by Notification Point. RDMA HW counter.
RpCnpHandledCount	Count	count	CNP packets handled by Reaction Point. RDMA HW counter.
SymbolErrorCount	Count	count	Minor link errors on physical lanes. Port counter.
TxPacketsPhyCount	Count	count	Packets transmitted on physical port. Ethtool counter.
RxPacketsPhyCount	Count	count	Packets received on physical port. Ethtool counter.
RxMulticastPhyCount	Count	count	Multicast packets received. Ethtool counter.
RxCrcErrorsPhyCount	Count	count	Packets dropped due to CRC errors. Error counter.
RxDiscardsPhyCount	Count	count	Packets dropped due to lack of buffers. Congestion indicator.
TxDiscardsPhyCount	Count	count	Packets discarded on transmit. Congestion indicator.
LinkDownEventsPhyCount	Count	count	Link transitioned to DOWN (flapping). Link health.
RxPcsSymbolErrPhyCount	Count	count	PCS symbol errors not corrected by FEC. Link health.