Performance Analyzer Source Tab Layout

The Source tab is divided into columns, with fixed-width columns for individual metrics on the left and the annotated source taking up the remaining width on the right.

Identifying the Original Source Lines

All lines displayed in black in the annotated source are taken from the original source file. The number at the start of a line in the annotated source column corresponds to the line number in the original source file. Any lines with characters displayed in a different color are either index lines or compiler commentary lines.

Index Lines in the Source Tab

A source file is any file compiled to produce an object file or interpreted into byte code. An object file normally contains one or more regions of executable code corresponding to functions, subroutines, or methods in the source code. The Analyzer analyzes the object file, identifies each executable region as a function, and attempts to map the functions it finds in the object code to the functions, routines, subroutines, or methods in the source file associated with the object code. When the analyzer succeeds, it adds an index line in the annotated source file in the location corresponding to the first instruction in the function found in the object code.

The annotated source shows an index line for every function—including inline functions—even though inline functions are not displayed in the list displayed by the Function tab. The Source tab displays index lines in red italics with text in angle-brackets. The simplest type of index line corresponds to the function’s default context. The default source context for any function is defined as the source file to which the first instruction in that function is attributed. The following example shows an index line for a C function icputime.

0.       0.         600. int
0.       0.         601. icputime(int k)
0.       0.         602. {
0.       0.              <Function: icputime>

As can be seen from the above example, the index line appears on the line following the first instruction. For C source, the first instruction corresponds to the opening brace at the start of the function body. In Fortran source, the index line for each subroutine follows the line containing the subroutine keyword. Also, a main function index line follows the first Fortran source instruction executed when the application starts, as shown in the following example:

0.       0.       1. ! Copyright 02/04/2000 Sun Microsystems, Inc. All Rights Reserved
0.       0.       2. !
0.       0.       3. ! Synthetic f90 program, used for testing openmp directives and 
0.       0.       4. ! the analyzer
0.       0.       5.
0.       0.       6. !$PRAGMA C (gethrtime, gethrvtime)
0.       0.       7.
0.       81.497    [  8] 9000    format(’X ’, f7.3, 7x, f7.3, 4x, a)
0.       0.             <Function: main>

Sometimes, the Analyzer might not be able to map a function it finds in the object code with any programming instructions in the source file associated with that object code; for example, code may be #included or inlined from another file, such as a header file.

If the source for the object code is not the default source context for a function contained in the object code, then the annotated source corresponding to the object code contains a special index line cross-referring to the function’s default source context. For example, compiling the synprog demo creates an object module, endcases.o corresponding to source file endcases.c. The source in endcases.c declares function inc_func, which is defined in header inc_func.h . Headers often include inline function definitions in this way. Because endcases.c declares function inc_func, but no source line within endcases.c corresponds to the instructions of inc_func, the top of the annotated source file for endcases.c displays a special index line, as follows:

0.650     0.650       <Function: inc_func, instructions from source file inc_func.h>

The metrics on the index line indicates that part of the code from the endcases.o object module does not have line-mappings (within the source file endcases.c).

The Analyzer also adds a standard index line in the annotated source for inc_func.h, where the function inc_func is defined.

.

0.       0.         2.
0.       0.         3. void
0.       0.         4. inc_func(int n)
0.       0.         5. {
0.       0.            <Function: inc_func>

Similarly, if a function has an alternate source context, then an index line cross referring to that context is displayed in the annotated source for the default source context.

0.            0.        142. inc_body(int n)
0.650     0.650       <Function: inc_body, instructions from source file inc_body.h>
0.            0.        143. {
0.            0.                  <Function: inc_body>
0.            0.        144. #include "inc_body.h"
0.            0.        145. }

Double-clicking on an index line referring to another source context will display the contents of that source file in the source window.

Also displayed in red are special index lines and other special lines that are not compiler commentary. For example, as a result of compiler optimization, a special index line might be created for a function in the object code that does not correspond to code written in any source file. For details, refer to Special Lines in the Source, Disassembly and PCs Tabs.

Compiler Commentary

Compiler commentary indicates how compiler-optimized code has been generated. Compiler commentary lines are displayed in blue, to distinguish them from index lines and original source lines. Various parts of the compiler can incorporate commentary into the executable. Each comment is associated with a specific line of source code. When the annotated source is written, the compiler commentary for any source line appears immediately preceding the source line.

The compiler commentary describes many of the transformations which have been made to the source code to optimize it. These transformations include loop optimizations, parallelization, inlining and pipelining. The following shows an example of compiler commentary.

                    Function freegraph inlined from source file ptraliasstr.c into 
                    the code for the following line
0.       0.         47.       freegraph();
                    48.    }
0.       0.         49.    for (j=0;j<ITER;j++) {

                    Function initgraph inlined from source file ptraliasstr.c into 
                    the code for the following line
0.       0.         50.       initgraph(rows);
        
                    Function setvalsmod inlined from source file ptraliasstr.c into 
                    the code for the following line
                    Loop below fissioned into 2 loops
                    Loop below fused with loop on line 51
                    Loop below had iterations peeled off for better unrolling and/or 
                    parallelization
                    Loop below scheduled with steady-state cycle count = 3
                    Loop below unrolled 8 times
                    Loop below has 0 loads, 3 stores, 3 prefetches, 0 FPadds, 
                    0 FPmuls, and FPdivs per iteration
                    51.       setvalsmod();

Note that the commentary for line 51 includes loop commentary because the function setvalsmod() contains loop code, and the function has been inlined.

You can set the types of compiler commentary displayed in the Source tab using the Source/Disassembly tab in the Set Data Presentation dialog box; for details, see Setting Data Presentation Options.

Common Subexpression Elimination

One very common optimization recognizes that the same expression appears in more than one place, and that performance can be improved by generating the code for that expression in one place. For example, if the same operation appears in both the if and the else branches of a block of code, the compiler can move that operation to just before the if statement. When it does so, it assigns line numbers to the instructions based on one of the previous occurrences of the expression. If the line numbers assigned to the common code correspond to one branch of an if structure, and the code actually always takes the other branch, the annotated source shows metrics on lines within the branch that is not taken.

Loop Optimizations

The compiler can do several types of loop optimization. Some of the more common ones are as follows:

• Loop unrolling

• Loop peeling

• Loop interchange

• Loop fission

• Loop fusion

Loop unrolling consists of repeating several iterations of a loop within the loop body, and adjusting the loop index accordingly. As the body of the loop becomes larger, the compiler can schedule the instructions more efficiently. Also reduced is the overhead caused by the loop index increment and conditional check operations. The remainder of the loop is handled using loop peeling.

Loop peeling consists of removing a number of loop iterations from the loop, and moving them in front of or after the loop, as appropriate.

Loop interchange changes the ordering of nested loops to minimize memory stride, to maximize cache-line hit rates.

Loop fusion consists of combining adjacent or closely located loops into a single loop. The benefits of loop fusion are similar to loop unrolling. In addition, if common data is accessed in the two pre-optimized loops, cache locality is improved by loop fusion, providing the compiler with more opportunities to exploit instruction-level parallelism.

Loop fission is the opposite of loop fusion: a loop is split into two or more loops. This optimization is appropriate if the number of computations in a loop becomes excessive, leading to register spills that degrade performance. Loop fission can also come into play if a loop contains conditional statements. Sometimes it is possible to split the loops into two: one with the conditional statement and one without. This can increase opportunities for software pipelining in the loop without the conditional statement.

Sometimes, with nested loops, the compiler applies loop fission to split a loop apart, and then performs loop fusion to recombine the loop in a different way to increase performance. In this case, you see compiler commentary similar to the following:

    Loop below fissioned into 2 loops
    Loop below fused with loop on line 116
    [116]    for (i=0;i<nvtxs;i++) {

Inlining of Functions

With an inline function, the compiler inserts the function instructions directly at the locations where it is called instead of making actual function calls. Thus, similar to a C/C++ macro, the instructions of an inline function are replicated at each call location. The compiler performs explicit or automatic inlining at high optimization levels (4 and 5). Inlining saves the cost of a function call and provides more instructions for which register usage and instruction scheduling can be optimized, at the cost of a larger code footprint in memory. The following is an example of inlining compiler commentary.

                Function initgraph inlined from source file ptralias.c 
                    into the code for the following line
0.       0.         44.       initgraph(rows);

The compiler commentary does not wrap onto two lines in the Source tab of the Analyzer.

Parallelization

If your code contains Sun, Cray, or OpenMP parallelization directives, it can be compiled for parallel execution on multiple processors. The compiler commentary indicates where parallelization has and has not been performed, and why. The following shows an example of parallelization computer commentary.

0.       6.324       9. c$omp  parallel do shared(a,b,c,n) private(i,j,k)
0.       0.    
                   Loop below parallelized by explicit user directive
                   Loop below interchanged with loop on line 12
0.010    0.010     [10]            do i = 2, n-1

                   Loop below not parallelized because it was nested in a parallel loop
                   Loop below interchanged with loop on line 12
0.170    0.170      11.               do j = 2, i

For more details about parallel execution and compiler-generated body functions, refer to Overview of OpenMP Software Execution.

Special Lines in the Annotated Source

Several other annotations for special cases can be shown under the Source tab, either in the form of compiler commentary, or as special lines displayed in the same color as index lines. For details, refer to Special Lines in the Source, Disassembly and PCs Tabs.

Source Line Metrics

Source code metrics are displayed, for each line of executable code, in fixed-width columns. The metrics are the same as in the function list. You can change the defaults for an experiment using a .er.rc file; for details, see Commands That Set Defaults. You can also change the metrics displayed and highlighting thresholds in the Analyzer using the Set Data Presentation dialog box; for details, see Setting Data Presentation Options.

Annotated source code shows the metrics of an application at the source-line level. It is produced by taking the PCs (program counts) that are recorded in the application’s call stack, and mapping each PC to a source line. To produce an annotated source file, the Analyzer first determines all of the functions that are generated in a particular object module (.o file) or load object, then scans the data for all PCs from each function. In order to produce annotated source, the Analyzer must be able to find and read the object module or load object to determine the mapping from PCs to source lines, and it must be able to read the source file to produce an annotated copy, which is displayed. The Analyzer searches for the source file, object file, and executable files in the following default locations in turn, and stops when it finds a file of the correct basename:

• The archive directories of experiments

• The current working directory

• The absolute pathname as recorded in the executables or compilation objects

The default can be changed by the addpath or setpath directive, or by the Analyzer GUI.

If a file cannot be found using the path list set by addpath or setpath, you can specify one or more path remappings with the pathmap command. With the pathmap command you specify an old-prefix and new-prefix. In any pathname for a source file, object file, or shared object that begins with the prefix specified with old-prefix, the old prefix is replaced by the prefix specified with new-prefix. The resulting path is then used to find the file. Multiple pathmap commands can be supplied, and each is tried until the file is found.

The compilation process goes through many stages, depending on the level of optimization requested, and transformations take place which can confuse the mapping of instructions to source lines. For some optimizations, source line information might be completely lost, while for others, it might be confusing. The compiler relies on various heuristics to track the source line for an instruction, and these heuristics are not infallible.

Interpreting Source Line Metrics

Metrics for an instruction must be interpreted as metrics accrued while waiting for the instruction to be executed. If the instruction being executed when an event is recorded comes from the same source line as the leaf PC, the metrics can be interpreted as due to execution of that source line. However, if the leaf PC comes from a different source line than the instruction being executed, at least some of the metrics for the source line that the leaf PC belongs to must be interpreted as metrics accumulated while this line was waiting to be executed. An example is when a value that is computed on one source line is used on the next source line.

The issue of how to interpret the metrics matters most when there is a substantial delay in execution, such as at a cache miss or a resource queue stall, or when an instruction is waiting for a result from a previous instruction. In such cases the metrics for the source lines can seem to be unreasonably high, and you should look at other lines in the code to find the line responsible for the high metric value.

Metric Formats

The four possible formats for the metrics that can appear on a line of annotated source code are explained in Table 8–1.