ChorusOS 4.0 Production Guide

Chapter 4 Building with mkmk

Read this Chapter only if you want to modify components built with the mkmk tool. Three of these components are the NUCLEUS, the OS and the IOM components.


Note -

The mkmk tool is more complex than imake. Use the mkmk tool only if it is mandatory.


This Chapter provides an overview of the mkmk build rules and the utilization of the mkmk tool.

As you receive the IOM component as source files, even in binary deliveries, the examples in this Chapter will refer to building an IOM component, as outlined in Chapter 2, System Generation.

The mkmk Build Process

The mkmk tool enables you to select a set of files from the source tree, in order to build an operating system for a particular target system. The mkmk tool also provides specific build rules which manage the system configuration through tunables, parameters, and features options.

The mkmk build process can be divided into five steps:

  1. Create a link to the source files in the build directory. This operation is called merging and uses the mkmerge utility. The links you have created enable mkmk to locate your source files in <src_dir>.

  2. Build the Makefiles in the component's build directory, using mkmk.

  3. Build the binary files.

  4. Create the dependency files that are used for further builds by the makedepend and getexport utilities.

  5. Build the system images with the mkimage command. This phase is described in more detail in the ChorusOS 4.0 Porting Guide.

Build Profile

The build profile used by the configure command is an argument of the mkmerge command. There is one default profile for each board, as shown in Table 2-1. For non standard system generation, customize the build profile by using the information on the file entries described here.

The build profile contains entries of the following type:

The range of <value> values depends on the variable. Note that:

Table 4-1 Entry Type for Build Profile, Defined for PowerPC/ppc60x
 merge_dir=<dir>Specifies the path of the merged tree. <dir> will be created if necessary.
 tree=<dir1dir2,...> Specifies directory paths of the split source tree that you want to merge.
 target=<ppc> Specifies the target platform.
 debug=<off|on>The entry is required if the kernel is to be compiled with the DEBUG option set. Default is off.
 optim=<off|on|size|speed>Specifies if and how you want to optimize the compilation of your kernel. size focuses on size, speed on speed while on is a good compromise. Setting optim to speed may make symbolic debugging impossible. Default is on.
 profile=<off|on>Needed if you want to profile your system. Default is off.
 locks=<rt|gp> Specifies the implementation of locks within the core executive.

mkmk Files

With the mkmk tool, you can write portable description files. The files are independent of both the host system you use and the target for which the system you are creating is destined. The mkmk tool also provides high level abstraction rules which ensure the portability of the application build files.

The mkmk command uses three types of build description files, suffixed by .df, .bf, and .mf, to produce Makefiles and generate all.dp dependencies. With mkmk, you create a .mf file for each host. In this way the mkmk tool ensures portability during cross compilation and so ensures the portability of hosts and targets.

.df Files

The .df files are used as shell scripts that are launched when the Makefile is created to adjust variables before they are written into the Makefile. The .df files define the following variables:

The .df files have similar properties to the Project.tmpl files of the imake tool. See ChorusOS 4.0 Introduction for further information. They affect the building of the Makefiles for all the subdirectories. With mkmk, you can have several .df files in each directory of the source tree. This contrasts with the imake tool, which has only one Project.tmpl file per component.

The inheritance mechanism of the mkmk tool is different from that of the imake tool. With mkmk, every Makefile is dependent on the Makefile present in the parent directory. However, with imake, every Makefile is dependent on the Project.tmpl file present in the top-level of the component and on the few variables which are inherited from the parent directory. The variable and macros are defined in the production tools. Table 4-2 lists those which you must not modify and Table 4-3 lists those which you may modify.

Table 4-2 Variables for the .df files, that must not be changed
 TARGETMKRULES<bin_dir>/tools/tgt-make/mktgt.rf
 MKRULES <bin_dir>/tools/tgt-make/mktgt.rf
 DTL<bin_dir>/tools/host/bin/
 GROOT  ../../
 BDIR <bin_dir>
 BNAME kern
 HOST LINUX
 FAMILY PowerPC
 COMPILER  gcc
 BFILESList of .bf files found in your work directory
 MFILESList of .lf files found in your work directory
 DFILESkern.df

Table 4-3 Variables in the .df files that you can modify
 DEFINES List of macro definitions to use in every compilation rule
 INCLUDES List of directories to search for header files
 VARIABLESThe list of user defined variables to export in Makefiles produced in subdirectories
 SUB_DIRS The list of directories to use when compiling; these are usually the subdirectories of your work directory
 EXTRA_DIRS A list of directories to use, in addition to SUB_DIRS, when creating libraries or linking actors (usually empty)
 MODULES The name of the module to which the object in your work directory (and subdirectories) belong; this is only useful in configurable actors
 FEXCEPTION  (ON/OFF); Compile with exception support
 FFPU  (ON/OFF): Compile with FPU support
 FOPTIM  (ON/OFF): Compile with optimization
 FPROF  (ON/OFF): Support for profiling
 FREMOTEDEB  (ON/OFF): Compile with debug options, to enable debug with XRAY
 FVERBOSE (ON/OFF): Verbose compilations
 FWARN (STRICT/ON/OFF): Enable warnings production during compilation; with STRICT, warnings are treated as errors, the NUCLEUS component uses STRICT

.bf Files

The .bf files are equivalent to the Imakefiles. They define source files, binary files and compilation options. The .bf files contain link directives and source definitions, such as C__SRCS. You can have several .bf files per directory. The mkmk tool concatenates the .bf files inside the Makefile product. The .bf file is preprocessed by the macro processor m4(1).

Table 4-4 lists the macros used in a .bf file. Actor, ConfigurableActor, Library and BigObject, either directly, or indirectly use getExport to get the list of object files. DistFile and Export are defined in $(DEVTOOLS_DIR)/tgt-make/mkrules.m4.

Table 4-4 Macros used in .bf files.
 Actor(actor, libs)  build 'actor' using libs
 ConfigurableActor(actor, libs)  build 'actor' using libs
 Library(lib)  build 'lib'
 BigObject(obj) build a relocatable object file
 DistFile(file,dir)  copy 'file' into 'dir'
 Export(file,dir)  same as DistFile

.mf Files

The .mf files contain lists of source files. The .mf files are used when the binary file to be built uses many subdirectories, each subdirectory containing a variable number of files to be compiled.

Another form of .mf files are .lf files. The variables and macros used in the .mf and .lf files are listed in Table 4-5.

Table 4-5 Variables and macros used in .mf and .lf files
 C++SRCS  C++ source files
 C__SRCS  C source files
 AS_SRCS  assembly code source files
 M4_SRCS  M4 source files (assembly code source files preprocessed with m4).
 OF_SRCS. C++ source files used to produce offset files

Merging

The role of mkmerge is to link the source files and the component's build directory. For further information, see the mkmerge(1CC) man page.

The code for ChorusOS components which contain family specific code is organized in split trees. Within the split trees, there are subtrees for:

Before generation begins, subtrees are merged into a merged tree. There is only one merged tree in each merged component. Initially, it contains symbolic, or hard, links that point to files in the split trees. The build takes place in the merged tree, so the split trees remain free of generated files, such as object files and Makefiles. You can delete a merged tree at any time. This will not affect the source code, as it remains clean in the split tree.

From a given set of subtrees of the split tree, different merged trees can be built. This allows you to produce several system configurations concurrently.

Building the merged tree is called the merge operation. When performing this operation, you provide a number of first-level options, called generation options. These options correspond to fundamental production choices, such as the choice of a development system or the target family and are typically found in the build profile given in Table 4-1.

If you are using the mkmk tool on a system running the Solaris operating environment, mkmerge populates the merge directory with symbolic links. These links point to the source directories required for your build.

If you are using the mkmk tool on a Windows NT system, mkmerge creates hard links between the source directories and your <work_dir>. The use of hard links permits you to use Windows NT hosts to edit files in the <work_dir>.

The Makefiles created by mkmk Tool

During generation, mkmk goes down the build tree and creates a Makefile, for each build directory. The Makefiles are used to build the binary files and each of these Makefiles is autonomous. This means that you can set make running from any directory and all files located in the subdirectories attached to that directory are automatically built. The Makefile content depends on the mkmk production files.

In each directory, where mkmk produces a Makefile, mkmk:

The list of files to be compiled varies depending on the merge operation. Writing a Makefile that will work on several configurations can be complicated. The solution offered by mkmk is described by the following two steps:

The .mf files do not have an equivalent within the imake tool.

The mkmk tool will regenerate the Makefile produced, if the .df, .bf or .mf files used to build it change.

Managing Dependencies

Once a file has been modified, for example with an editor, or by a previous compilation, the files that depend on the altered file must be rebuilt. Most development tools only deal with the dependencies between source and binary files. The mkmk system goes further, taking into account any changes which occur in the source files during the build process.

The relationship between the different components is managed through the components' Makefile.bin and Makefile.src files and through the build paths found in the Paths file in your work directory. For example, applications can access both the kernel API and the OS API through the variables set in the Paths file:

The information that is accessed between components is the exported information, which is usually present in the binary deliveries.

The IOM Build Directory

To look at a particular example of a built component, list the contents of the build directory for the IOM component, <work_dir>/build-IOM. This component's build directory contains five directories, three symbolic links and six files.

Table 4-6 Description of the Directories in the IOM Component's Build Directory
 Directories Description
include header files exported by the IOM component
lib libraries used by the IOM component
obj  object files needed to link the IOM component
src source files of the IOM component
confconfiguration files of the IOM component (used by mkmerge)

These five directories will be found in each source component's build directory.

Table 4-7 Symbolic Links Between the IOM Build Directory and the Source Files
 Symbolic Link Description
Makefile.bin exports interface with other components
Makefile.src describes how to build the IOM component
src.df build description file

Table 4-8 Files Generated in the IOM Build Directory
 File Description
DONE created by Makefile.src when compilation is complete
SUMproduced by Makefile.src
profile  created by the merge process, it contains merge options
merge.log created by the merge process, it contains all mkmerge output
export.lst the list of files to export; mkmake uses this file to determine which files it should copy and where to copy them to
Makefile the top level Makefile, produced by mkmk

Examples of IOM Build Files

For the IOM component, the IOM source files are merged into the component's build directory, <work_dir>/build_IOM. In this directory, exports.1st is produced by the mkmerge command. This directory relies on the profile file to ensure that the NUCLEUS component is merged before the IOM component.

Look in further detail at the IOM build directory by listing the contents of the <work_dir>/build-IOM/src/os/iom/sys directory. This directory contains four files and two symbolic links:

Table 4-9 Files Generated in src/os/iom/sys
 File Description
Makefilethe Makefile of the directory, produced by mkmk
N_iom.r the IOM actor
all.dpdependency file, produced by make depend and getExport
sys_agglo.mkinternal Makefile, used when linking the configurable actor, to copy files from the src directory to the obj directory

Table 4-10 Symbolic links in src/os/iom/sys
 Symbolic Link Description
sys.bf the build file
sys.df the definition file

The contents of the symbolic link files, sys.bf and sys.df, the Makefile and the /sys/lib/gen/common.mf file are given here.

sys.df

The sys.df file, defines the INCLUDES and DEFINES variables.

INCLUDES=" 			\
     -Isys/include			\
     -Ibsd				\
     -Ibsd/sys			\
     -I${OS_DIR}/include/sys	\
     -Ibsd/machine			\
     -I${OS_DIR}/include/machine	\
     -I${OS_DIR}/include/chorus/iom	\
     -I${OS_DIR}/include/chorus	\
     -I${OS_DIR}/include/chorus/cx	\
     -I${NUCLEUS_DIR}/include/chorus	\
     -I${IOM_DIR}/include		\
     -I${OS_DIR}/include		\
     -I${NUCLEUS_DIR}/include	\
     -I${OS_DIR}/include/stdc	\
     -I${NUCLEUS_DIR}/include/stdc"
    
    DEFINES="$DEFINES -DKERNEL  -D_KERNEL -D__FreeBSD__ /
    -DINET -DNO_CACHE -DMSDOSFS -DNFS -DIOM_MALLOC  -DSHARED_FD"

sys.bf

In this example, S_LIBS contains the list of libraries used to link the IOM component. The ConfigurableActor rule is also used to link the IOM component.

S_LIBS = $(IOM_DIR)/lib/os/iom/sys/lib/ufs.a \
    	 $(IOM_DIR)/lib/os/iom/sys/lib/disk.a \
    	 $(IOM_DIR)/lib/os/iom/sys/lib/gen.a \
    	 $(IOM_DIR)/lib/os/iom/sys/lib/mem.a \
    	 $(IOM_DIR)/lib/os/iom/sys/lib/unresolved.a \
    	 $(NUCLEUS_DIR)/lib/stdc/libC.a \
    	 $(NUCLEUS_DIR)/lib/embedded/libebd.s.a \
    	 $(NUCLEUS_DIR)/lib/cpu/cpu.s.a \
    	 $(OS_DIR)/lib/classix/libcx.a \
    	 $(NUCLEUS_DIR)/lib/classix/libsys.s.a 
    
    ConfigurableActor(N_iom.r, $(S_LIBS))

Makefile

The Makefile generated by mkmk contains two parts. View the contents of the Makefile. The first part has a list of the definitions of variables as defined in Table 4-2, Table 4-3, Table 4-4 and Table 4-5. The second part contains preprocessed copies of the .bf, .lf, and .mf files found in the work directory. In this example there is only a .bf file present in the work directory.

    #
    #		This makefile is generated automatically
    #			in build-IOM/src/os/iom/sys
    #
    ...
    BDIR =  <work_dir>
    BNAME = sys
    MPATH = build-IOM/src/os/iom/sys
    DEFINES = -DNDEBUG -DKERNEL -D_KERNEL -D__FreeBSD__ -DINET \
    -DNO_CACHE -DMSDOSFS -DNFS -DIOM_MALLOC -DSHARED_FD
    INCLUDES = -Isys/include -Ibsd -Ibsd/sys 
    -I/<work_dir>/build-OS/include/sys -Ibsd/machine \
    -I/<work_dir>/build-OS/include/machine \
    -I/<work_dir>/build-OS/include/chorus/iom \
    ...
    FEXCEPTION = OFF
    FFPU = ON
    FOPTIM = ON
    FPROF = OFF
    ...
    FAMILY = ppc60x
    COMPILER = gcc
    VARIABLES = OS_DIR IOM_DIR NUCLEUS_DIR DRV_DIR
    SUB_DIRS = bsd sys
    EXTRA_DIRS = 
    BFILES = sys.bf
    MFILES = 
    DFILES = sys.df
    MODULES = 
    OS_DIR = <work_dir>/build-OS
    IOM_DIR = <work_dir>/build-IOM
    NUCLEUS_DIR = <work_dir>/build-NUCLEUS
    DRV_DIR = <work_dir>/build-DRV
    
    include $(BDIR)/Paths
    
    include $(DEVTOOLS_DIR)/tgt-make/shared.rf
    
    # produced from sys.bf -- begin
     S_LIBS = $(IOM_DIR)/lib/os/iom/sys/lib/ufs.a \
    	         $(IOM_DIR)/lib/os/iom/sys/lib/disk.a \
    	         $(IOM_DIR)/lib/os/iom/sys/lib/gen.a \
    	         $(IOM_DIR)/lib/os/iom/sys/lib/mem.a \
    	         $(IOM_DIR)/lib/os/iom/sys/lib/unresolved.a \
    	         $(NUCLEUS_DIR)/lib/stdc/libC.a \
    	         $(NUCLEUS_DIR)/lib/embedded/libebd.s.a \
    	         $(NUCLEUS_DIR)/lib/cpu/cpu.s.a \
    	         $(OS_DIR)/lib/classix/libcx.a \
    	         $(NUCLEUS_DIR)/lib/classix/libsys.s.a 
     
     BINARY += N_iom.r
     
     N_iom.r: $(S_LIBS)
    	        $(MKLINK) -r -c -e _start -o N_iom.r -B $(S_LIBS)
    # produced from sys.bf -- end
    
    include all.dp
    include $(DEVTOOLS_DIR)/tgt-make/mktgt.rf

common.mf File

To examine a directory that has a .mf file, go to the <work_dir>/build-IOM/src/os/iom/sys/sys/lib/gen directory. List the contents of the common.mf file:

...
    C__SRCS = \
            insremque.c ovbcopy.c memstat.c iomRqTask.c util.c

The .mf files contain the definition of variables and nothing else. These variables are listed in Table 4-5.

all.dp Dependency File

The Makefile listed above for the IOM component, located at <work_dir>/build-IOM/src/os/iom/sys, includes a dependency file, all.dp. This dependency file is located in the same directory as the Makefile. In the IOM component, the all.dp file contains the list of files upon which the linked binary files, such as actors or libraries, are dependent. The -export lines at the top of the file are used by getExport during the link phase. In this example, as the directory contains no object file, these lines are empty.

host% head all.dp
 #	automatically generated file
    # -export ALL :      
    # -export SUP :         
    # -export USR :         
    N_iom.r: bsd/dev/conf/all.dp
    N_iom.r: bsd/dev/conf/conf.o
    N_iom.r: bsd/dev/console/all.dp
    N_iom.r: bsd/dev/console/console.o
    N_iom.r: bsd/dev/flash/all.dp
    N_iom.r: bsd/dev/flash/flashdrv.o
    ...
    N_iom.r: all.dp

Building an IOM Component

The make process occurs in three phases

  1. Build source files (this phase is only present in the NUCLEUS component).

  2. Compile source files and create dependency files.

  3. Link object files.

When make runs, it launches mkmake. The mkmake tool uses the all.dp dependency file to determine in which directories make will run. Only the directories accessed are displayed.

host% make
...
<bin_dir>/tools/host/bin/mkmake 
<work_dir>/build-NUCLEUS/ sources +l +b -- sources
>> In src/os/iom/sys sources
<< In src/os/iom/sys sources done
<bin_dir>/tools/host/bin/mkmake 
<work_dir>/build-NUCLEUS/ prod +l +m +b -- local_prod
>> In src/os/iom/sys prod
<< In src/os/iom/sys prod done
<bin_dir>/tools/host/bin/mkmake 
<work_dir>/build-NUCLEUS/ link +b -- local_link
>> In src/os/iom/sys/bsd/kern/disk link
<< In src/os/iom/sys/bsd/kern/disk link done
...

Relink of the IOM Actor

To relink the IOM actor, remove it and then run the make command to regenerate the component.

host% rm <work_dir>/build-IOM/src/os/iom/sys/N_iom.r
host% make

The output you get displays the regeneration of the N_iom.r actor.

Recompilation of a Source File

Recompile a source file, for example pathName.C, by removing the object file, <work_dir>/build-IOM/src/os/iom/sys/sys/lib/mem/pathName.o and running the make command. You get a very long output which includes the following:

host% rm <work_dir>/build-IOM/src/os/iom/sys/sys/lib/mem/pathName.o
host% make
>> In src/os/iom/sys/sys/lib/mem prod
    	...
     CC pathName.C
    	...
     ar -> mem.a
     ...
     sh <bin_dir>/tools/host/bin//../../tgt-make/genLink \
     <work_dir>/build-NUCLEUS/ <work_dir> -r -c -e _start -o N_iom.r -B \
     ...<work_dir>/build-NUCLEUS/lib/classix/libsys.s.a
     
     <bin_dir>/tools/host/bin/configurator \
     -c <work_dir>/conf/ChorusOS.xml -action configure
       
     <bin_dir>/tools/powerpc/solaris/5.00/powerpc-elf/bin/gcc \
     ...
     bsd/kern/vfs/vfs_vnops.o \
     ...
     <bin_dir>/tools/host/bin/mkctors -T <work_dir>/obj/os/iom/sys/tunables.k \
     <work_dir>/obj/os/iom/sys/N_iom.r.xpO > <work_dir>/obj/os/iom/sys/N_iom.r.CT.s
     <bin_dir>/tools/powerpc/solaris/5.00/powerpc-elf/bin/gcc
     ...
     <work_dir>/obj/os/iom/sys/N_iom.r.CT.o
     ...
     <bin_dir>/tools/powerpc/solaris/5.00/powerpc-elf/bin/gcc
     ...
     <work_dir>/obj/os/iom/sys/N_iom.r.CT.o
     ...

Look at this output as occurring in several steps.

  1. For step 1:

        ...
         >> In src/os/iom/sys/sys/lib/mem prod
        	...
         CC pathName.C
        	...
         ar -> mem.a
         ...

    The <work_dir>/build-IOM/src/os/iom/sys/sys/lib/mem directory builds a library used during the IOM link. The object file is generated, the library is updated, and the mkmake command continues.

  2. For step 2:

          ...
          sh <bin_dir>/tools/host/bin//../../tgt-make/genLink \
          <work_dir>/build-NUCLEUS/ <work_dir> -r -c -e _start -o N_iom.r -B \
          ...<work_dir>/build-NUCLEUS/lib/classix/libsys.s.a
    

    The linked actor is copied back to the source directory.

  3. Step 3 calls the configurator command:

         <bin_dir>/tools/host/bin/configurator \
         -c <work_dir>/conf/ChorusOS.xml -action configure
    

    This gets the IOM configuration settings from the XML configuration file.

  4. The last step is the link itself.

     <bin_dir>/tools/powerpc/solaris/5.00/powerpc-elf/bin/gcc    ...
     bsd/kern/vfs/vfs_vnops.o \...
     <bin_dir>/tools/host/bin/mkctors -T <work_dir>/obj/os/iom/sys/tunables.k \
     <work_dir>/obj/os/iom/sys/N_iom.r.xpO > <work_dir>/obj/os/iom/sys/N_iom.r.CT.s
     <bin_dir>/tools/powerpc/solaris/5.00/powerpc-elf/bin/gcc
     ...
     <work_dir>/obj/os/iom/sys/N_iom.r.CT.o
     ...
     <bin_dir>/tools/powerpc/solaris/5.00/powerpc-elf/bin/gcc
     ...
     <work_dir>/obj/os/iom/sys/N_iom.r.CT.o
     ...
    

    This is done after several loops of gcc, and a call to mkctors, in order to set the values of the tunable parameters.

Link of Configurable Actors

Steps 2 to 4 comprise the link of a configurable actor; in this instance, N_iom.r. The link of a configurable actor is unusual, as instead of linking the actor in the current work directory, object files are copied into the obj directory, located in build-IOM. The actor is linked to your work directory and then copied back to the source tree.

The first step is to call genLink, which will:

Verbose Mode

See how the mkmake command is called for each phase by running the make command in verbose mode as follows:

host% make SILENT=
 /<bin_dir>/tools/host/bin/mkmake \
   /<work_dir>/build-NUCLEUS/ sources +l +b -- sources
   >> In <bin_dir>/bin/tools/host/bin/mkmake \
   <work_dir>/build-NUCLEUS/ prod +l +m +b -- local_prod
   <bin_dir>/tools/host/bin/mkmake \
   <work_dir>/build-NUCLEUS/ link +b -- local_link
   >> In src/os/iom/sys/sys/lib/mem link
   << In src/os/iom/sys/sys/lib/mem link done
  

depend and all

The make depend command rebuilds dependencies. This make target is used when the list of header files in a source file is changed. As seen earlier, if the Makefile content changes, dependencies are reconstructed. In the <work_dir>/build-IOM/src/os/iom/sys/sys/lib/gen directory, use the make depend command.

host% make depend
     >> In src/os/iom/sys/sys/lib/gen depend
    ...
    Makefile -> all.dp
    << In src/os/iom/sys/sys/lib/gen depend done

Note -

In this directory, there are object files that are used in linking the IOM component. In this case, the -export lines in all.dp are not empty.


#	automatically generated file
    # -export ALL :   insremque.o ovbcopy.o memstat.o iomRqTask.o util.o
    # -export SUP :   insremque.o ovbcopy.o memstat.o iomRqTask.o util.o                                  
    # -export USR :   insremque.o ovbcopy.o memstat.o iomRqTask.o util.o 

This information is used by getExport, which reads all the all.dp files in the source tree to determine which object files to use.

make Targets for mkmk

The complete list of make targets that you can use with the mkmk tool are shown here. The section above shows depend and all make targets.

Table 4-11 Description of make Targets used with mkmk
 Target Description
 all  default target, builds everything
 Makefile rebuild the Makefile in current directory
 makemk rebuild the Makefiles (recursively)
 merge  remerge the current component
 makesrc  build offset files (recursively)
 sources  build offset files in the current directory
 makeprod  build object files and libraries (recursively)
 prod build object files and libraries in the current directory
 makelink link libraries and actors (recursively)
 link  link libraries and actors in the current directory
 depend generate dependencies (recursively)
 cleanremove files produced (recursively), with the exception of Makefiles and dependency files
 clobber remove files produced (recursively)

make mkmk

By using the make mkmk command, the Makefiles can be built or rebuilt.