All experiments must have three files:
A log file (log.xml), an XML file that contains information about what data was collected, the versions of various components, a record of various events during the life of the target, and the word size of the target
A map file (map.xml), an XML file that records the time-dependent information about what load objects are loaded into the address space of the target, and the times at which they are loaded or unloaded
An overview file, which is a binary file containing usage information recorded at every sample point in the experiment
In addition, experiments have binary data files representing the profile events in the life of the process. Each data file has a series of events, as described in Interpreting Performance Metrics. Separate files are used for each type of data, but each file is shared by all threads in the target.
For clock profiling or hardware counter overflow profiling, the data is written in a signal handler invoked by the clock tick or counter overflow. For synchronization tracing, heap tracing, I/O tracing, MPI tracing, or OpenMP tracing, data is written from libcollector routines that are interposed by the LD_PRELOAD environment variable on the normal user-invoked routines. Each such interposition routine partially fills in a data record, then invokes the normal user-invoked routine, and fills in the rest of the data record when that routine returns, and writes the record to the data file.
All data files are memory-mapped and written in blocks. The records are filled in such a way as to always have a valid record structure, so that experiments can be read as they are being written. The buffer management strategy is designed to minimize contention and serialization between threads.
An experiment can optionally contain an ASCII file with the name notes. This file is automatically created when using the -C comment argument to the collect command. You can create or edit the file manually after the experiment has been created. The contents of the file are prepended to the experiment header.
Each experiment has an archives directory that contains binary files describing each load object referenced in the map.xml file. These files are produced by the er_archive utility, which runs at the end of data collection. If the process terminates abnormally, the er_archive utility might not be invoked, in which case, the archive files are written by the er_print utility or Performance Analyzer when first invoked on the experiment.
Archive directories can also include copies of shared objects or of source files, depending on the options used to archive the experiment.
Subexperiments are created when multiple processes are profiled, such as when you follow descendant processes, collect an MPI experiment, or profile the kernel with user processes.
Descendant processes write their experiments into subdirectories within the founder-experiment directory. These new subexperiments are named to indicate their lineage as follows:
An underscore is appended to the creator's experiment name.
One of the following code letters is added: f for fork, x for exec, and c for other descendants. On Linux, C is used for a descendant generated by clone(2).
A number to indicate the index of the fork or exec is added after the code letter. .
The experiment suffix, .er is appended to complete the experiment name.
For user processes, if the experiment name for the founder process is test.1.er, the experiment for the descendant process created by its third fork is test.1.er/_f3.er. If that descendant process executes a new image, the corresponding experiment name is test.1.er/_f3_x1.er. Descendant experiments consist of the same files as the parent experiment, but they do not have descendant experiments (all descendants are represented by subdirectories in the founder experiment), and they do not have archive subdirectories (all archiving is done into the founder experiment).
Experiments on the kernel by default are named ktest.1.er rather than test.1.er. When data is also collected on user processes, the kernel experiment contains subexperiments for each user process being followed. The kernel subexperiments are named using the format _process-name_PID_process-id.1.er. For example an experiment run on a sshd process running under process ID 1264 would be named ktest.1.er/_sshd_PID_1264.1.er.
Data for MPI programs are collected by default into test.1.er, and all the data from the MPI processes are collected into subexperiments, one per rank. The Collector uses the MPI rank to construct a subexperiment name with the form M_rm.er, where m is the MPI rank. For example, MPI rank 1 would have its experiment data recorded in the test.1.er/M_r1.er directory.
An experiment where the target creates dynamic functions has additional records in the map.xml file describing those functions. An additional file, dyntext, contains a copy of the actual instructions of the dynamic functions. The copy is needed to produce annotated disassembly of dynamic functions.
A Java experiment has additional records in the map.xml file, both for dynamic functions created by the JVM software for its internal purposes and for dynamically compiled (HotSpot) versions of the target Java methods.
In addition, a Java experiment includes a JAVA_CLASSES file, containing information about all of the user’s Java classes invoked.
Java tracing data is recorded using a JVMTI agent, which is part of libcollector.so. The agent receives events that are mapped into the recorded trace events. The agent also receives events for class loading and HotSpot compilation, that are used to write the JAVA_CLASSES file, and the Java-compiled method records in the map.xml file.