Sun Studio 12: Performance Analyzer

Hardware Counter Overflow Profiling Data

Hardware counters keep track of events like cache misses, cache stall cycles, floating-point operations, branch mispredictions, CPU cycles, and instructions executed. In hardware counter overflow profiling, the Collector records a profile packet when a designated hardware counter of the CPU on which an LWP is running overflows. The counter is reset and continues counting. The profile packet includes the overflow value and the counter type.

Various CPU families support from two to eighteen simultaneous hardware counter registers. The Collector can collect data on one or more registers. For each register the Collector allows you to select the type of counter to monitor for overflow, and to set an overflow value for the counter. Some hardware counters can use any register, others are only available on a particular register. Consequently, not all combinations of hardware counters can be chosen in a single experiment.

Hardware counter overflow profiling data is converted by the Performance Analyzer into count metrics. For counters that count in cycles, the metrics reported are converted to times; for counters that do not count in cycles, the metrics reported are event counts. On machines with multiple CPUs, the clock frequency used to convert the metrics is the harmonic mean of the clock frequencies of the individual CPUs. Because each type of processor has its own set of hardware counters, and because the number of hardware counters is large, the hardware counter metrics are not listed here. The next subsection tells you how to find out what hardware counters are available.

One use of hardware counters is to diagnose problems with the flow of information into and out of the CPU. High counts of cache misses, for example, indicate that restructuring your program to improve data or text locality or to increase cache reuse can improve program performance.

Some of the hardware counters provide similar or related information. For example, branch mispredictions and instruction cache misses are often related because a branch misprediction causes the wrong instructions to be loaded into the instruction cache, and these must be replaced by the correct instructions. The replacement can cause an instruction cache miss, or an instruction translation lookaside buffer (ITLB) miss, or even a page fault.

Hardware counter overflows are often delivered one or more instructions after the instruction which caused the event and the corresponding event counter to overflow: this is referred to as “skid” and it can make counter overflow profiles difficult to interpret. In the absence of hardware support for precise identification of the causal instruction, an apropos backtracking search for a candidate causal instruction may be attempted.

When such backtracking is supported and specified during collection, hardware counter profile packets additionally include the PC (program counter) and EA (effective address) of a candidate memory-referencing instruction appropriate for the hardware counter event. (Subsequent processing during analysis is required to validate the candidate event PC and EA.) This additional information about memory-referencing events facilitates various data-oriented analyses.

Backtracking and recording of a candidate event PC and EA can also be specified for clock-profiling.

Hardware Counter Lists

Hardware counters are processor-specific, so the choice of counters available to you depends on the processor that you are using. The performance tools provide aliases for a number of counters that are likely to be in common use. You can obtain a list of available hardware counters on any particular system from the Collector by typing collect with no arguments in a terminal window on that system. If the processor and system support hardware counter profiling, the collect command prints two lists containing information about hardware counters. The first list contains “well-known” (aliased) hardware counters; the second list contains raw hardware counters.

Here is an example that shows the entries in the counter list. The counters that are considered well-known are displayed first in the list, followed by a list of the raw hardware counters. Each line of output in this example is formatted for print.

Well known HW counters available for profiling:
cycles[/{0|1}],9999991 (’CPU Cycles’, alias for Cycle_cnt; CPU-cycles)
insts[/{0|1}],9999991 (’Instructions Executed’, alias for Instr_cnt; events)
dcrm[/1],100003 (’D$ Read Misses’, alias for DC_rd_miss; load events)
Raw HW counters available for profiling:
Cycle_cnt[/{0|1}],1000003 (CPU-cycles)
Instr_cnt[/{0|1}],1000003 (events)
DC_rd[/0],1000003 (load events)

Format of the Well-Known Hardware Counter List

In the well-known hardware counter list, the first field (for example, cycles) gives the alias name that can be used in the -h counter... argument of the collect command. This alias name is also the identifier to use in the er_print command.

The second field lists the available registers for the counter; for example, [/{0|1}]. For well-known counters, the default value has been chosen to provide a reasonable sample rate. Because actual rates vary considerably, you might need to specify a non-default value.

The third field, for example, 9999991, is the default overflow value for the counter.

The fourth field, in parentheses, contains type information. It provides a short description (for example, CPU Cycles), the raw hardware counter name (for example, Cycle_cnt), and the type of units being counted (for example, CPU-cycles), which can include up to two words.

If the first word of type information is:

If the second or only word of the type information is:

In the well-known hardware counter list in the example, the type information contains one word, CPU-cycles for the first counter and events for the second counter. For the third counter, the type information contains two words, load events.

Format of the Raw Hardware Counter List

The information included in the raw hardware counter list is a subset of the information in the well-known hardware counter list. Each line includes the internal counter name as used by cpu-track(1), the register number(s) on which that counter can be used, the default overflow value, and the counter units, which can be either CPU-cycles or Events.

If the counter measures events unrelated to the program running, the first word of type information is not-program-related. For such a counter, profiling does not record a call stack, but instead shows the time being spent in an artificial function, collector_not_program_related . Thread and LWP ID’s are recorded, but are meaningless.

The default overflow value for raw counters is 1000003. This value is not ideal for most raw counters, so you should specify timeout values when specifying raw counters.