Parallel Makefile Processing With qmake

qmake is a replacement for the standard UNIX make facility. qmake extends make by enabling the distribution of independent make steps across a cluster of suitable machines. qmake is built around the popular GNU-make facility gmake. See the information that is provided in sge-root/3rd_party for details on the involvement of gmake.

To ensure that a distributed make process can run to completion, qmake first allocates the required resources in a way analogous to a parallel job. qmake then manages this set of resources without further interaction with the scheduling. qmake distributes make steps as resources become available, using the qrsh facility with the -inherit option.

qrsh provides standard output, error output, and standard input handling as well as terminal control connection to the remotely executing make step. Therefore, only three noticeable differences exist between executing a make procedure locally and using qmake:

• Provided that individual make steps have a certain duration and that enough independent make steps exist to process, the parallelization of the make process will speed up significantly.

• In the make steps to be started up remotely, an implied small overhead exists that is caused by qrsh and the remote execution.

• To take advantage of the make step distribution of qmake, the user must specify as a minimum the degree of parallelization. That is, the user must specify the number of concurrently executable make steps. In addition, the user can specify the resource characteristics required by the make steps, such as available software licenses, machine architecture, memory, or CPU-time requirements.

The most common use of make is the compilation of complex software packages. Compilation might not be the major application for qmake, however. Program files are often quite small as a matter of good programming practice. Therefore, compilation of a single program file, which is a single make step, often takes only a few seconds. Furthermore, compilation usually implies significant file access. Nested include files can cause this problem. File access might not be accelerated if done for multiple make steps in parallel because the file server can become a bottleneck. Such a bottleneck effectively serializes all the file access. Therefore, the compilation process sometimes cannot be accelerated in a satisfactory manner.

Other potential applications of qmake are more appropriate. An example is the steering of the interdependencies and the workflow of complex analysis tasks through makefiles. Each make step in such environments is typically a simulation or data analysis operation with nonnegligible resource and computation time requirements. A considerable acceleration can be achieved in such cases.

qmake Usage

The command-line syntax of qmake looks similar to the syntax of qrsh:

% qmake [-pe pe-name pe-range][options] \
 -- [gnu-make-options][target]

The -inherit option is also supported by qmake, as described later in this section.

Pay special attention to the use of the -pe option and its relation to the gmake -j option. You can use both options to express the amount of parallelism to be achieved. The difference is that gmake provides no possibility with -j to specify something like a parallel environment to use. Therefore, qmake assumes that a default environment for parallel makes is configured that is called make. Furthermore, gmake ´s -j allows for no specification of a range, but only for a single number. qmake interprets the number that is given with -j as a range of 1-n. By contrast, -pe permits the detailed specification of all these parameters. Consequently the following command line examples are identical:

% qmake -- -j 10
% qmake -pe make 1-10 --

The following command lines cannot be expressed using the -j option:

% qmake -pe make 5-10,16 --
% qmake -pe mpi 1-99999 --

Apart from the syntax, qmake supports two modes of invocation: interactively from the command line without the -inherit option, or within a batch job with the -inherit option. These two modes start different sequences of actions:

• Interactive – When qmake is invoked on the command line, the make process is implicitly submitted to the grid engine system with qrsh. The process takes the resource requirements that are specified in the qmake command line into account. The grid engine system then selects a master machine for the execution of the parallel job that is associated with the parallel make job. The grid engine system starts the make procedure there. The procedure must start there because the make process can be architecture-dependent. The required architecture is specified in the qmake command line. The qmake process on the master machine then delegates execution of individual make steps to the other hosts that are allocated for the job. The steps are passed to qmake through the parallel environment hosts file.

• Batch – In this case, qmake appears inside a batch script with the -inherit option. If the -inherit option is not present, a new job is spawned, as described in the first case earlier. This results in qmake making use of the resources already allocated to the job into which qmake is embedded. qmake uses qrsh -inherit directly to start make steps. When calling qmake in batch mode, the specification of resource requirements, the -pe option and the -j option are ignored.

  ---

  Note –

  Single CPU jobs also must request a parallel environment:

  qmake -pe make 1 --

  If no parallel execution is required, call qmake with gmake command-line syntax without grid engine system options and without --. This qmake command behaves like gmake.

  ---

See the qmake(1) man page for further details.