Dependency Checking: make vs. Shell Scripts

While it is possible to use a shell script to assure consistency in trivial cases, scripts to build software projects are often inadequate. On the one hand, you don't want to wait for a simple minded script to compile every single program or object module when only one of them has changed. On the other hand, having to edit the script for each iteration can defeat the goal of consistency. Although it is possible to write a script of sufficient complexity to recompile only those modules that require it, make does this job better.

make allows you to write a simple, structured listing of what to build and how to build it. It uses the mechanism of dependency checking to compare each module with the source or intermediate files it derives from. make only rebuilds a module if one or more of these prerequisite files, called dependency files, has changed since the module was last built.

To determine whether a derived file is out of date with respect to its sources, make compares the modification time of the (existing) module with that of its dependency file. If the module is missing, or if it is older than the dependency file, make considers it to be out of date, and issues the commands necessary to rebuild it. A module can be treated as out of date if the commands used to build it have changed.

Because make does a complete dependency scan, changes to a source file are consistently propagated through any number of intermediate files or processing steps. This lets you specify a hierarchy of steps in a top-to-bottom fashion.

You can think of a makefile as a recipe. make reads the recipe, decides which steps need to be performed, and executes only those steps that are required to produce the finished module. Each file to build, or step to perform, is called a target. The makefile entry for a target contains its name, a list of targets on which it depends, and a list of commands for building it.

The list of commands is called a rule. make treats dependencies as prerequisite targets, and updates them (if necessary) before processing its current target. The rule for a target need not always produce a file, but if it does, the file for which the target is named is referred to as the target file. Each file from which a target is derived (for example, the file the target depends on) is called a dependency file.

If the rule for a target produces no file by that name, make performs the rule and considers the target to be up-to-date for the remainder of the run.

make assumes that only it will make changes to files being processed during the current run. If a source file is changed by another process while make is running, the files it produces might be in an inconsistent state.

Writing a Simple Makefile

The basic format for a makefile target entry is:

target .  .  .  : [ dependency .  .  .  ] 
            [ command ]
             . . .

In the first line, the list of target names is terminated by a colon. This, in turn, is followed by the dependency list, if there is one. If several targets are listed, this indicates that each such target is to be built independently using the rule supplied.

Subsequent lines that start with a TAB are taken as the command lines that comprise the target rule. A common error is to use SPACE characters instead of the leading TAB.

Lines that start with a # are treated as comments up until the next (unescaped) NEWLINE and do not terminate the target entry. The target entry is terminated by the next non-empty line that begins with a character other than TAB or #, or by the end of the file.

A trivial makefile might consist of just one target shown in the following figure:

test: 
     ls test 
     touch test

When you run make with no arguments, it searches first for a file named makefile, or, if there is no file by that name, Makefile. If either of these files is under SCCS control, make checks the makefile against its history file. If it is out of date, make extracts the latest version.

If make finds a makefile, it begins the dependency check with the first target entry in that file. Otherwise, you must list the targets to build as arguments on the command line. make displays each command it runs while building its targets.

$ make 
ls test
test not found
touch test
ls test
test

Because the file test was not present (and therefore out of date), make performed the rule in its target entry. If you run make a second time, it issues a message indicating that the target is now up to date and skips the rule:

$ make 
`test' is up to date.

make invokes a Bourne shell to process a command line if that line contains any shell metacharacters, such as a semicolon (;) redirection symbols (<, >, >>, |), substitution symbols (*, ?, [], $, =), or quotes, escapes or comments (", ', `, \, #, etc.:), If a shell is not required to parse the command line, make exec()'s the command directly.

Line breaks within a rule are significant in that each command line is performed by a separate process or shell.

This means that a rule such as:

test: 
       cd /tmp 
       pwd

behaves differently than you might expect, as shown below.

$ make test
cd /tmp 
pwd
/usr/tutorial/waite/arcana/minor/pentangles 

You can use semicolons to specify a sequence of commands to perform in a single shell invocation:

test: 
cd /tmp ; pwd 

Or, you can continue the input line onto the next line in the makefile by escaping the NEWLINE with a backslash (\). The escaped NEWLINE is treated as white space by make.

The backslash must be the last character on the line. The semicolon is required by the shell.

test: 
       cd /tmp ; \ 
       pwd 

Basic Use of Implicit Rules

When no rule is given for a specified target, make attempts to use an implicit rule to build it. When make finds a rule for the class of files the target belongs to, it applies the rule listed in the implicit rule target entry.

In addition to any makefile(s) that you supply, make reads in the default makefile, /usr/share/lib/make/make.rules, which contains the target entries for a number of implicit rules, along with other information.

Implicit rules were hand-coded in earlier versions of make.

There are two types of implicit rules: Suffix and Pattern-matching. Suffix rules specify a set of commands for building a file with one suffix from another file with the same base name but a different suffix. Pattern-matching rules select a rule based on a target and dependency that match respective wild-card patterns. The implicit rules provided by default are suffix rules.

In some cases, the use of suffix rules can eliminate the need for writing a makefile entirely. For instance, to build an object file named functions.o from a single C source file named functions.c, you could use the command:

$ make functions.o
cc -c functions.c -o functions.o 

This would work equally well for building the object file nonesuch.o from the source file nonesuch.c.

To build an executable file named functions (with a null suffix) from functions.c, you need only type the command:

$ make functions
cc -o functions functions.c

The rule for building a .o file from a .c file is called the .c.o (pronounced "dot-see-dot-oh") suffix rule. The rule for building an executable program from a .c file is called the .c rule. The complete set of default suffix rules is listed in Table 4-8.

Processing Dependencies

After make begins, it processes targets as it encounters them in its depth-first dependency scan. For example, with the following makefile:

batch: a b 
     touch batch 
b: 
     touch b 
a: 
     touch a 
c: 
     echo "you won't see me"

make starts with the target batch. Since batch has some dependencies that have not been checked, namely a and b, make defers batch until after it has checked a and b against any dependencies they might have.

Since a has no dependencies, make processes it; if the file is not present, make performs the rule in its target entry.

$ make 
touch a 
...

Next, make works its way back up to the parent target batch. Since there is still an unchecked dependency b, make descends to b and checks it.

b also has no dependencies, so make performs its rule:

...
touch b 
...

Finally, now that all of the dependencies for batch have been checked and built (if needed), make checks batch.

Since it rebuilt at least one of the dependencies for batch, make assumes that batch is out of date and rebuilds it; if a or b had not been built in the current make run, but were present in the directory and newer than batch, make's time stamp comparison would also result in batch being rebuilt:

...
touch batch

Target entries that are not encountered in a dependency scan are not processed. Although there is a target entry for c in the makefile, make does not encounter it while performing the dependency scan for batch, so its rule is not performed. You can select an alternate starting target like c by entering it as an argument to the make command.

In the next example, the batch target produces no file. Instead, it is used as a label to group a set of targets.

batch: a b c 
a: a1 a2 
      touch a
b: 
      touch b 
c: 
      touch c
a1: 
      touch a1
a2: 
      touch a2

In this case, the targets are checked and processed, as shown in the following diagram:

Essentially, make then:

1. Checks batch for dependencies, notices that there are three, and so defers it.

2. Checks a, the first dependency, and notices that it has two dependencies of its own. Continuing in like fashion, make:

   1. Checks a1, and if necessary, rebuilds it.

   2. Checks a2, and if necessary, rebuilds it.

3. Determines whether to build a.

4. Checks b and rebuilds it if necessary.

5. Checks and rebuilds c if necessary.

6. After traversing its dependency tree, make checks and processes the topmost target, batch. If batch contains a rule, make performs that rule. Here batch has no rule, therefore make performs no action, but notes that batch has been rebuilt; any targets depending on batch would also be rebuilt.

Null Rules

If a target entry contains no rule, make attempts to select an implicit rule to build it. If make cannot find an appropriate implicit rule and there is no SCCS history from which to retrieve it, make concludes that the target has no corresponding file, and regards the missing rule as a null rule.

You can use a dependency with a null rule to force the target rule to be executed. The conventional name for such is FORCE.

With this makefile:

haste: FORCE 
        echo "haste makes waste"
FORCE:

make performs the rule for making haste, even if a file by that name is up to date:

$ touch haste 
$ make haste 
echo "haste makes waste"
haste makes waste

Special Targets

make has several built-in special targets that perform special functions. For example, the .PRECIOUS special target directs make to preserve library files when make is interrupted.

Special targets:

• Begin with a period (.)

• Have no dependencies

• Can appear anywhere in a makefile

Table 4-3 includes a list of special targets.

Unknown Targets

If a target is named either on the command line or in a dependency list, and it

• Is not a file present in the working directory

• Has no target or dependency entry

• Does not belong to a class of files for which an implicit rule is defined

• Has no SCCS history file, and

• There is no rule specified for the .DEFAULT special target

Then make stops processing and issues an error message.

$ make believe 
make: Fatal error: Don't know how to make target `believe'.

However, if the -k option is in effect, make continues with the other targets that do not depend on the one in which the error occurred.

Duplicate Targets

Targets can appear more than once in a makefile. For example,

foo:  dep_1
foo:  dep_2
foo:
     touch foo

is the same as

foo:  dep_1 dep_2
      touch foo

However, many people feel that it's preferable to have a target appear only once, for ease of reading.

Reserved make Words

The words in the following table are reserved by make:

Running Commands Silently

You can inhibit the display of a command line within a rule by inserting an @ as the first character on that line. For example, the following target:

quiet: 
       	@echo you only see me once 

produces:

$ make quiet 
         you only see me once

If you want to inhibit the display of commands during a particular make run, you can use the -s option. If you want to inhibit the display of all command lines in every run, add the special target .SILENT to your makefile.

.SILENT:
quiet: 
         echo you only see me once

Special-function targets begin with a dot (.). Target names that begin with a dot are never used as the starting target, unless specifically requested as an argument on the command line. make normally issues an error message and stops when a command returns a nonzero exit code. For example, if you have the target:

rmxyz: 
     rm xyz 

and there is no file named xyz, make halts after rm returns its exit status.

$ ls xyz 
xyz not found
$ make rmxyz 
rm xyz
rm: xyz: No such file or directory
*** Error code 1
make: Fatal error: Command failed for target `rmxyz' 

If - and @ are the first two such characters, both take effect.

To continue processing regardless of the command exit code, use a dash character (-) as the first non-TAB character:

rmxyz:
        -rm xyz

In this case you get a warning message indicating the exit code make received:

$make rmxyz  
rm xyz 
rm: xyz: No such file or directory 
*** Error code 1 (ignored)

Unless you are testing a makefile, it is usually a bad idea to ignore non-zero error codes on a global basis

Although it is generally ill-advised to do so, you can cause make to ignore error codes entirely with the -i option. You can also cause make to ignore exit codes when processing a given makefile, by including the .IGNORE special target, though this too should be avoided.

If you are processing a list of targets, and you want make to continue with the next target on the list rather than stopping entirely after encountering a non-zero return code, use the -k option.

Automatic Retrieval of SCCS Files

When source files are named in the dependency list, make treats them like any other target. Because the source file is presumed to be present in the directory, there is no need to add an entry for it to the makefile.

When a target has no dependencies, but is present in the directory, make assumes that file is up to date. If, however, a source file is under SCCS control, make does some additional checking to ensure that the source file is up to date. If the file is missing, or if the history file is newer, make automatically issues the following command to retrieve the most recent version:

sccs get -s filename -Gfilename 

With other versions of make, automatic sccs retrieval was a feature only of certain implicit rules. Also, unlike earlier versions, make only looks for history (s.) files in the sccs directory; history files in the current working directory are ignored.

However, if the source file is writable by anyone, make does not retrieve a new version.

$ ls SCCS/* 
SCCS/s.functions.c
$ rm -f functions.c 
$ make functions 
sccs get -s functions.c -Gfunctions.c 
cc -o functions functions.c 

make checks the time stamp of the retrieved version against the time stamp of the history file. It does not check to see if the version present in the directory is the most recently checked-in version. So, if someone had done a get by date (sccs get -c), make would not discover this fact, and you might unwittingly build an older version of the program or object file. To be absolutely sure that you are compiling the latest version, you can precede make with an sccs get SCCS` or an sccs clean command.

Suppressing SCCS Retrieval

The command for retrieving SCCS files is specified in the rule for the .SCCS_GET special target in the default makefile. To suppress automatic retrieval, simply add an entry for this target with an empty rule to your makefile:

# Suppress sccs retrieval. 
       	.SCCS_GET:

Passing Parameters: Simple make Macros

The make macro substitution comes in handy when you want to pass parameters to command lines within a makefile. Suppose that you want to compile an optimized version of the program program, using the cc -O option. You can lend this sort of flexibility to your makefile by adding a macro reference, such as the following example, to the target for functions:

functions: functions.c 
         cc $(CFLAGS) -o functions functions.c

The macro reference acts as a placeholder for a value that you define, either in the makefile itself, or as an argument to the make command. If you then supply make with a definition for the CFLAGS macro, make replaces its references with the value you have defined.

$rm functions 
$ make functions "CFLAGS= -O"
cc -O -o functions functions.c

There is a reference to the CFLAGS macro in both the .c and the .c.o implicit rules.

The command-line definition must be a single argument, hence the quotes in this example.

If a macro is undefined, make expands its references to an empty string.

You can also include macro definitions in the makefile itself. A typical use is to set CFLAGS to -O, so that make produces optimized object code by default:

CFLAGS= -O 
functions: functions.c 
         cc $(CFLAGS) -o functions functions.c

A macro definition supplied as a command line argument to make overrides other definitions in the makefile. Conditionally defined macros are an exception to this.

For instance, to compile functions for debugging with dbx or dbxtool, you can define the value of CFLAGS to be -g on the command line:

$ rm functions 
$ make CFLAGS=-g 
cc -g -o functions functions.c

To compile a profiling variant for use with gprof, supply both -O and -pg in the value for CFLAGS.

A macro reference must include parentheses when the name of the macro is longer than one character. If the macro name is only one character, the parentheses can be omitted. You can use curly braces, { and }, instead of parentheses. For example, `$X', `$(X)', and `${X}' are equivalent.

.KEEP_STATE and Command Dependency Checking

In addition to the normal dependency checking, you can use the special target .KEEP_STATE to activate command dependency checking. When activated, make not only checks each target file against its dependency files, it compares each command line in the rule with those it ran the last time the target was built. This information is stored in the .make.state file in the current directory (see "State File").

With the makefile:

CFLAGS= -O 
.KEEP_STATE:

functions: functions.c 
        	cc -o functions functions.c

the following commands work as shown:

$ make 
cc -O -o functions functions.c
$ make CFLAGS=-g 
cc -g -o functions functions.c 
$ make "CFLAGS= -O -pg" 
cc -O -pg -o functions functions.c

This ensures that make compiles a program with the options you want, even if a different variant is present and otherwise up to date.

The first make run with .KEEP_STATE in effect recompiles all targets in order to gather and record the necessary information. The KEEP_STATE variable, when imported from the environment, has the same effect as the .KEEP_STATE target.

Suppressing or Forcing Command Dependency Checking for Selected Lines

To suppress command dependency checking for a given command line, insert a question mark as the first character after the TAB.

Command dependency checking is automatically suppressed for lines containing the dynamic macro $?. This macro stands for the list of dependencies that are newer than the current target, and can be expected to differ between any two make runs.

To force make to perform command dependency checking on a line containing this macro, prefix the command line with a ! character (following the TAB).

State File

When .KEEP_STATE is in effect, make writes out a state file named .make.state, in the current directory. This file lists all targets that have ever been processed while .KEEP_STATE has been in effect, along with the rules to build them, in makefile format. In order to ensure that this state file is maintained consistently, after you have added .KEEP_STATE to a makefile, it is recommended that you leave it in effect.

Since this target is ignored in earlier versions of make, it does not introduce any compatibility problems. Other versions treat it as a superfluous target that no targets depend on, with an empty rule and no dependencies of its own. Because it starts with a dot, it is not used as the starting target.

.KEEP_STATE and Hidden Dependencies

When a C source file contains #include directives for interpolating headers, the target depends just as much on those headers as it does on the sources that include them. Because such headers might not be listed explicitly as sources in the compilation command line, they are called hidden dependencies. When .KEEP_STATE is in effect, make receives a report from the various compilers and compilation preprocessors indicating which hidden dependency files were interpolated for each target.

It adds this information to the dependency list in the state file. In subsequent runs, these additional dependencies are processed just like regular dependencies. This feature automatically maintains the hidden dependency list for each target; it ensures that the dependency list for each target is always accurate and up to date. It also eliminates the need for the complicated schemes found in some earlier makefiles to generate complete dependency lists.

A slight inconvenience can arise the first time make processes a target with hidden dependencies, because there is as yet no record of them in the state file. If a header is missing, and make has no record of it, make does not know that it needs to retrieve it from SCCS before compiling the target.

Even though there is an SCCS history file, the current version won't be retrieved because it doesn't yet appear in a dependency list or the state file. When the C preprocessor attempts to interpolate the header, it won't find it; the compilation fails.

Supposing that a #include directive for interpolating the header hidden.h is added to functions.c, and that the file hidden.h is somehow removed before the subsequent make run. The results would be:

$ rm -f hidden.h 
$ make functions 
cc -O -o functions functions.c 
functions.c: 2: Can't find include file hidden.h 
make: Fatal error: Command failed for target `functions'

A simple workaround might be to make sure that the new header is extant before you run make. Or, if the compilation should fail (and assuming the header is under SCCS), you could manually retrieve it from SCCS:

$ sccs get hidden.h 
1.1 
10 lines 
$ make functions 
cc -O -o functions functions.c

In all future cases, should the header turn up missing, make will know to build or retrieve it for you because it will be listed in the state file as a hidden dependency.

Note that with hidden dependency checking, the $? macro includes the names of hidden dependency files. This might cause unexpected behavior in existing makefiles that rely on $?.

.INIT and Hidden Dependencies

The problem with both of these approaches is that the first make in the local directory might fail due to a random condition in some other (include) directory. This might entail forcing someone to monitor a (first) build. To avoid this, you can use the .INIT target to retrieve known hidden dependencies files from SCCS. .INIT is a special target that, along with its dependencies, is built at the start of the make run. To be sure that hidden.h is present, you could add the following line to your makefile

.INIT:     hidden.h

Displaying Information About a make Run

Running make with the -n option displays the commands make is to perform, without executing them. This comes in handy when verifying that the macros in a makefile are expanded as expected. With the following makefile:

CFLAGS= -O 

.KEEP_STATE:

functions: main.o data.o 
         $(LINK.c) -o functions main.o data.o

make -n displays:

$ make -n 
cc -O -c main.c 
cc -O -c data.c 
cc -O -o functions main.o data.o

There is an exception however. make executes any command line containing a reference to the MAKE macro (such as $(MAKE) or ${MAKE}), regardless of -n. It is a bad idea to include a line such as the following in your makefile: $(MAKE) ; rm -f *

Setting an environment variable named MAKEFLAGS can lead to complications, since make adds its value to the list of options. To prevent puzzling surprises, avoid setting this variable.

make has some other options that you can use to keep abreast of what it's doing and why:

-d

Displays the criteria by which make determines that a target is be out-of-date. Unlike -n, it does process targets, as shown in the following example. This option also displays the value imported from the environment (null by default) for the MAKEFLAGS macro, which is described in detail in a later section.

$ make -d 
MAKEFLAGS value: 
    Building main.o using suffix rule for .c.o because it is out of date relative to main.c 
cc -O -c main.c 
    Building functions because it is out of date relative to main.o 
    Building data.o using suffix rule for .c.o because it is out of date relative to data.c 
cc -O -c data.c 
    Building functions because it is out of date relative to data.o 
cc -O -o functions main.o data.o

-dd

This option displays all dependencies make checks, including any hidden dependencies, in vast detail.

-D

Displays the text of the makefile as it is read.

-DD

Displays the makefile and the default makefile, the state file, and hidden dependency reports for the current make run.

-f makefile

make uses the named makefile (instead of makefile or Makefile).

Several -f options indicate the concatenation of the named makefiles.

-Kmakestatefile

If makestatefile is a directory, make writes the KEEP_STATE information into a .make.state file in that directory. If makestatefile is a file, make will write the KEEP_STATE information into the makestatefile

-p

Displays the complete set of macro definitions and target entries.

-P

Displays the complete dependency tree for the default target or the specified target.

An option that can be used to shortcut make processing is the -t option. When run with -t, make does not perform the rule for building a target. Instead it uses touch to alter the modification time for each target that it encounters in the dependency scan. It also updates the state file to reflect what it built. This often creates more problems than it solves, and it is recommended that you exercise caution if you do use it. Note that if there is no file corresponding to a target entry, touch creates it.

Due to its potentially troublesome side effects, it is recommended that you not use the -t (touch) option for make.

The following is one example of how not to use make -t. Suppose you have a target named clean that performed housekeeping in the directory by removing target files produced by make:

clean: 
         rm functions main.o data.o

clean is the conventional name for a target that removes derived files. It is useful when you want to start a build from scratch

If you give the nonsensical command:

$ make -t clean
touch clean 
$ make clean 
`clean' is up to date

you then have to remove the file clean before your housekeeping target can work again.

-q

Invokes the question mode, and returns a zero or non-zero status code, depending on whether or not the target file is up-to-date.

-r

Suppresses reading in of the default makefile, /usr/share/lib/make/make.rules.

-S

Undoes the effect of the -K option by stopping processing when a non-zero exit status is returned by a command.