Basic Concepts
Distributed make (dmake) allows you to concurrently distribute the process of building large projects, consisting of many programs, over a number of workstations and, in the case of multiprocessor systems, over multiple CPUs. The dmake utility parses your makefiles and:
-
Determines which targets can be built concurrently
-
Distributes the build of those targets over a number of hosts designated by you
The dmake utility is a superset of the make utility.
To understand dmake, you should know about:
-
Configuration files (runtime and build server)
-
The dmake host
-
The build server
Configuration Files
The dmake utility consults two files to determine to which build servers jobs are distributed and how many jobs can be distributed to each.
Runtime Configuration File
The dmake utility searches for a runtime configuration file on the dmake host to know where to distribute jobs. Generally, this file is in your home directory on the dmake host and is named .dmakerc. It consists of a list of build servers and the number of jobs to be distributed to each build server. See "The dmake Host" for more information.
Build Server Configuration File
The /etc/opt/SPROdmake/dmake.conf file is in the file system of build servers. It specifies the maximum total number of dmake jobs that can be distributed to each build server by all dmake users. In addition, it may specify the "nice" priority under which all dmake jobs should run.
See "The Build Server" for more information.
The dmake Host
The dmake utility searches for a runtime configuration file to determine where to distribute jobs. Generally, this file must be in your home directory on the dmake host and is named .dmakerc. The dmake utility searches for the runtime configuration file in these locations and in the following order:
-
The path name you specify on the command line using the -c option
-
The path name you specify using the DMAKE_RCFILE makefile macro
-
The path name you specify using the DMAKE_RCFILE environment variable
-
$(HOME)/.dmakerc
If a runtime configuration file is not found, the dmake utility distributes two jobs to the dmake host.
You edit the runtime configuration file so that it consists of a list of build servers and the number of jobs you want distributed to each build server. The following is an example of a .dmakerc file:
# My machine. This entry causes dmake to distribute to it.
falcon { jobs = 1 }
hawk
eagle { jobs = 3 }
# Manager's machine. She's usually at meetings
heron { jobs = 4 }
avocet
-
The entries: falcon, hawk, eagle, heron, and avocet are listed build servers.
-
You can specify the number of jobs you want distributed to each build server. The default number of jobs is two.
-
Any line that begins with the # character is interpreted as a comment.
Note -This list of build servers includes falcon which is also the dmake host. The dmake host can also be specified as a build server. If you do not include it in the runtime configuration file, no dmake jobs are distributed to it.
You can also construct groups of build servers in the runtime configuration file. This provides you with the flexibility of easily switching between different groups of build servers as circumstances warrant. For instance, you may define groups of build servers for builds under different operating systems, or you may define groups of build servers that have special software installed on them.
The following is an example of a runtime configuration file that contains groups of build servers:
earth { jobs = 2 }
mars { jobs = 3 }
group lab1 {
host falcon { jobs = 3 }
host hawk
host eagle { jobs = 3 }
}
group lab2 {
host heron
host avocet { jobs = 3 }
host stilt { jobs = 2 }
}
group labs {
group lab1
group lab2
}
group sunos5.x {
group labs
host jupiter
host venus { jobs = 2 }
host pluto { jobs = 3 }
}
-
Formal groups are specified by the group directive and lists of their members are delimited by braces ({}).
-
Build servers that are members of groups are specified by the optional host directive.
-
Groups can be members of other groups.
-
Individual build servers can be listed in runtime configuration files that also contain groups of build servers; in this case, dmake treats these build servers as members of the unnamed group.
In order of precedence, the dmake utility distributes jobs to the following:
-
The formal group specified on the command-line as an argument to the -g option
-
The formal group specified by the DMAKE_GROUP makefile macro
-
The formal group specified by the DMAKE_GROUP environment variable
-
The first group specified in the runtime configuration file
The dmake utility allows you to specify a different execution path for each build server. By default dmake looks for the dmake support binaries on the build server in the same logical path as on the dmake host. You can specify alternate paths for build servers as a host attribute in the .dmakerc file. For example:
group lab1 {
host falcon { jobs = 10 , path = "/set/dist/sparc-S2/bin" }
host hawk { path = "/opt/SUNWspro/bin" }
}
You can use double quotation marks to enclose the names of groups and hosts in the .dmakerc file. This allows you more flexibility in the characters that you can use in group names. Digits are allowed, as well as alphabetic characters. Names that start with digits should be enclosed in double quotes. For example:
group "123_lab" {
host "456_hawk" { path = "/opt/SUNWspro/bin" }
}
The Build Server
The /etc/opt/SPROdmake/dmake.conf file is in the file system of build servers. Use this file to limit the maximum number of dmake jobs (from all users) that can run concurrently on a build server and to specify the "nice" priority under which all dmake jobs should run. The following is an example of an /etc/opt/SPROdmake/dmake.conf file. This file sets the maximum number of dmake jobs permitted to run on a build server (from all dmake users) to be eight.
max_jobs: 8 nice_prio: 5
Note -
If the /etc/opt/SPROdmake/dmake.conf file does not exist on a build server, no dmake jobs will be allowed to run on that server.
- © 2010, Oracle Corporation and/or its affiliates