Makefiles

and Dependency Generation

Back to CDF Run2 document


Caveat This plan relies heavily on UNIX utilities. This is not necessarily wise. Unfortunately we have little experience supporting Windows NT, and no knowledge of what OS's will be fashionable in 2001.


Library Maintenance

Layout

We expect the structure of our software Releases to be that of a single directory tree, with subdirectories for the utility packages, for the analysis support code, and for auxilliary and setup utilities. The tree will probably contain links to vetted versions of foreign code (CERNLIB, etc).

The top directory contains a Makefile which invokes Makefiles for each of the subdirectories in the appropriate order. The top directory Makefile (the Master Makefile) has to be put together by the Configuration Manager and must reflect the order things need to be assembled in. There will be inevitably some trial and error in creating this, but once the correct order is discovered the Master Makefile shouldn't change much from Release to Release. This is not the case for Makefiles governing the libraries of analysis support code.

Libraries

An analysis support library will contain code written in Fortran and C++, together with header files to support the C++ and include files to support the Fortran. We leave to the Language Support group the decision of how to maintain the interface between Fortran and C++. We have historically kept verb definition files, module help files, and some special-purpose database files in analysis support libraries. We want to also maintain test routines and test scripts in each library.

We can simplify the Makefile's job by sequestering the test routines and scripts in a subdirectory and handing off their management to a Makefile in the test subdirectory. However, from Release to Release (and from day to day for Development) the source and include files will change, and the Makefile must be rebuilt to reflect this.

C and C++

So long as the compilation options don't vary too much within a library, the X11 distribution program makedepend can help us maintain the C and C++ files. The C++ sources could be treated as a block as in

SOURCES = accum.c alpha.c alphabuf.c attrib.c bitmap.c blend.c bresenhm.c 

OBJECTS = $(SOURCES:.c=.o)

##### RULES #####

.c.o:
        $(CC) -c -I$(INCDIR) $(CFLAGS) $<

include ../Make-config

include depend

#
# Run 'make depend' to update the dependencies if you change what's included
# by any source file.
#
dep: $(SOURCES)
        makedepend -fdepend -Y -I../include $(SOURCES)

The above was taken from the Makefile for Mesa.

We can generally assume that all .CC files in a library (not in the test subdirectory) should be compiled and put in an object library, but if this isn't true we can make a veto file which holds file names we don't want included. A simple script can use this file to generate a `wanted file' list and update the Makefile with it.

Fortran

The problem of generating dependencies with Fortran is similar but not as severe, since we've avoided having include files include other include files. We have the first pass at a tool to generate these dependencies; fmakedepend, written by Dejan Vucinic. At the moment it diddles with the actual Makefile, but this is straightforward to fix.


User File Maintenance

A user can select which Release he wants to use, and change his mind the next day. If the dependency paths are hard-coded into his Makefile and Makefile.cdepend and Makefile.fdepend files, he may have problems when he selects a new Release. The object libraries he links against no longer correspond to the include files he uses for compilation! The correct thing to do is rerun the makedepend and fmakedepend before using the Makefile, but we don't believe most users will reliably remember to do this. We therefore want to bullet-proof the Makefile as much as we possibly can.

There is a fairly simple approach to keeping consistency. It relies on the user using either a standard template Makefile or standard includes within a Makefile. This seems reasonable.

Keep a special file in the working directory containing the relevant environmental variables defined the last time the Makefile was regenerated. On envocation, the Makefile checks compares the current environmental variables with those in the file, and envokes make depend if they do not agree. Then it touches those files which depend on the analysis support libraries (or touches them all--easier!).

(One could imagine putting a wrapper around makedepend and fmakedepend to dump out the current environmental variables, and then parseing the generated Makefile.xinclude files to substitute the matching environmental variable name for the hard-coded path name. Unfortunately, since the include files in each Release will probably be older than the .o file, make will fail to recompile things as it ought unless the .c files have been touched; and it won't know to touch the .c files unless it knows that the environmental variables have changed, so we're back where we were before.)


Required Tools

We require the following tools, some of which exist. The rest seem fairly straightforward.

Pictures

In a sample user's subdirectory, we find the following files:

Makefile
Makefile.cdepend
Makefile.environment
Makefile.fdepend
Makefile.sources	# may just write in Makefile instead
dimu_die.h
fmutrk.f
fmutrk.o
higgsfind
higgsmutop.c
higgsmutop.o
nobel_acceptance.tex

An administrator creating a new Release on a remote machine would take steps resembling these:

echo "V1.1 /disk7/offline" >> /usr/local/cdf/offline/where_version
			# add location of new release to cdf setup file
cd /disk7
mkdir offline
setup cdf-cvs		# make sure cvs is setup and the repository defined
cvs -co -rV1.1 offline		# get copy of entire offline from cvs
cd offline
make_makefiles *	# generate Makefiles and auxilliary files in all
			# revision directories (a lot is hidden here)
gmake all		# invoke all the Makefiles in all subdirectories
gmake clean		# invoke all the Makefiles to do cleanup
gmake test		# invoke the (handmade, I fear) Makefiles in all
			# the /test subdirectories of the Revision main
			# subdirectories.

Note that the above leaves all the source code lying around. This is rather useful for people trying to use the debugger.