How to Make My Makefiles Better

Good practices of Makefile

What I normally do is have a Makefile in the src directory (which can be invoked from the top level Makefile if you like) and then use rules like this:

D_BIN = ../bin
$(D_BIN)/%.o: %.cpp

You could also experiment with just a makefile in the top level dir, and use rules that look like this:

D_BIN = bin
D_SRC = src
$(D_BIN)/%.o: $(D_SRC)/%.cpp

but I have not used such rules, so I don't know the pros/cons vs the way I normally do it. The way I normally do it works fine, I even have rules that build depends like so:

$(D_BIN)/%.d: %.cpp

and the link rule would be like:

../dist/outexe: $(F_OBJ)

Using a foreach is usually frowned upon because it does not make use of all the features built into normal makefile rules (i.e. there is no depends check on a per file basis, either you build everything or nothing), and as such foreach should only be used as a last resort, but in this case you will be able to get it to work without the foreach.

In addition to this there are much easier ways to build your file lists, you don't need to use the shell or sed.

F_CPP = $(wildcard *.cpp)
F_OBJ = $(F_CPP:.cpp=.o)

Update: This is how I normally issue recursive makes:

SUBDIRS = src
.PHONY: $(SUBDIRS)
all: $(SUBDIRS)

$(SUBDIRS):
    @echo "Building $@..."
    $(MAKE) -C $@ $(MFLAGS)

Then indeed in your submake, you would need to use ../bin for example.

However with a project as simple as yours, you might be better off just having one makefile at the root level and using rules like this:

D_BIN = bin
D_SRC = src
$(D_BIN)/%.o: $(D_SRC)/%.cpp

recursive makefiles are ok (ok but not great) if you have a really complex directory structure, where you will be adding/removing/modifying new dir trees as time goes on. But for a simple project where you just want to have separate directories for code and objs, it is probably overkill.

Best practice for building a make file

Here is an example makefile for you, tested on your github project.

Features:

Automatic dependency generation.
Automatic rebuild when Makefile is changed.
Debug/release builds in different directories. Debug build is the default, use make BUILD=release for release builds.
Supports gcc and clang. gcc is the default, use make COMPILER=clang for clang.
clean target works by removing the entire build directory.
run target to build and run, e.g. make run_othello.

Limitations:

Assumes that all source files are in the same directory. In a bigger project there are going to be multiple directories each with multiple build targets (executable, static or shared library). An isomorphic build directory structure is required to support this. Which is totally doable but requires more complexities in example Makefile below, so that it omitted for brevity.

When archiving you may like to archive only the source files and the makefile. No need to include any build artefacts (like object files, libraries and executables).

# ==== Begin prologue boilerplate.
all : # The canonical default target.
BUILD := debug
build_dir := ${CURDIR}/${BUILD}
exes := # Executables to build.
# ==== End prologue boilerplate.

# ==== Begin define executable othello.
exes += othello
objects.othello = main.o game.o Myothello.o space.o
-include ${objects.othello:%.o=${build_dir}/%.d} # Include auto-generated dependencies.
# ==== End define executable othello.

# ==== Begin define another executable.
# ... as for othello
# ==== End define another executable.

# ==== Begin rest of boilerplate.
SHELL := /bin/bash
COMPILER=gcc

CXX.gcc := /bin/g++
CC.gcc := /bin/gcc
LD.gcc := /bin/g++
AR.gcc := /bin/ar

CXX.clang := /bin/clang++
CC.clang := /bin/clang
LD.clang := /bin/clang++
AR.clang := /bin/ar

CXX := ${CXX.${COMPILER}}
CC := ${CC.${COMPILER}}
LD := ${LD.${COMPILER}}
AR := ${AR.${COMPILER}}

CXXFLAGS.gcc.debug := -Og -fstack-protector-all
CXXFLAGS.gcc.release := -O3 -march=native -DNDEBUG
CXXFLAGS.gcc := -pthread -std=gnu++14 -march=native -W{all,extra,error} -g -fmessage-length=0 ${CXXFLAGS.gcc.${BUILD}}

CXXFLAGS.clang.debug := -O0 -fstack-protector-all
CXXFLAGS.clang.release := -O3 -march=native -DNDEBUG
CXXFLAGS.clang := -pthread -std=gnu++14 -march=native -W{all,extra,error} -g -fmessage-length=0 ${CXXFLAGS.clang.${BUILD}}

CXXFLAGS := ${CXXFLAGS.${COMPILER}}
CFLAGS := ${CFLAGS.${COMPILER}}

LDFLAGS.debug :=
LDFLAGS.release :=
LDFLAGS := -fuse-ld=gold -pthread -g ${LDFLAGS.${BUILD}}
LDLIBS := -ldl

COMPILE.CXX = ${CXX} -c -o $@ ${CPPFLAGS} -MD -MP ${CXXFLAGS} $(abspath $<)
PREPROCESS.CXX = ${CXX} -E -o $@ ${CPPFLAGS} ${CXXFLAGS} $(abspath $<)
COMPILE.C = ${CC} -c -o $@ ${CPPFLAGS} -MD -MP ${CFLAGS} $(abspath $<)
LINK.EXE = ${LD} -o $@ $(LDFLAGS) $(filter-out Makefile,$^) $(LDLIBS)
LINK.SO = ${LD} -shared -o $@ $(LDFLAGS) $(filter-out Makefile,$^) $(LDLIBS)
LINK.A = ${AR} rsc $@ $(filter-out Makefile,$^)

all : ${exes:%=${build_dir}/%} # Build all exectuables.

.SECONDEXPANSION:
# Build an executable.
${exes:%=${build_dir}/%} : ${build_dir}/% : $$(addprefix ${build_dir}/,$${objects.$$*}) Makefile | ${build_dir}
    $(strip ${LINK.EXE})

# Run an executable. E.g. make run_othello
${exes:%=run_%} : run_% : ${build_dir}/%
    @echo "---- running $< ----"
    /usr/bin/time --verbose $<

# Create the build directory on demand.
${build_dir} :
    mkdir $@

# Compile a C++ source into .o.
# Most importantly, generate header dependencies.
${build_dir}/%.o : %.cc Makefile | ${build_dir}
    $(strip ${COMPILE.CXX})

# Compile a C source into .o.
# Most importantly, generate header dependencies.
${build_dir}/%.o : %.c Makefile | ${build_dir}
    $(strip ${COMPILE.C})

clean :
    rm -rf ${build_dir}

.PHONY : clean all run_%

# ==== End rest of boilerplate.

How could I improve this Makefile?

You really don't need the $(CC) $(CFLAGS) $(DEBUG) $(LDFLAGS) $(FREQ_OBJECTS) -o $@ lines. make already knows how to build binaries.

If your filenames are constant for different binaries (binary.c and binary_lib.c), you can also create a general rule for that:

FOO := $(shell ls *_lib.c)
BIN = $(FOO:%_lib.c=%)

$(BIN) : % : %.o %_lib.o

EDIT: Here's how it works:

FOO is the list of all files ending with _lib.c
BIN is the same list, with the "_lib.c" suffixes removed, so it's the list of your binaries
The last line is your make rule. The rule states each foo in $(BIN) depends on foo.o and foo_lib.o

Why don't makefiles behave more like shell scripts within recipes?

I don't really know how to answer your question. Probably that means it's not really appropriate for StackOverflow.

The requirement for using $$ instead of $ is obvious. The reasoning for using a separate shell for each logical line of a makefile instead of passing the entire recipe to a single shell, is less clear. It could have worked either way, and this is the way it was chosen.

There is one advantage to the way it works now, although maybe most people don't care about it: you only have to indent the first recipe line with TAB, if you use backslash newline to continue each line. If you don't use backslash newline, then every line has to be indented with TAB else you don't know where the recipe ends.

If your question is, could Stuart Feldman have made very different syntax decisions that would have made it easier to write long/complex recipes in makefiles, then sure. Choosing a more obscure character than $ as a variable introducer would reduce the amount of escaping (although, shell scripting uses pretty much every special character somewhere so "reduce" is the best you can do). Choosing an explicit "start/stop" character sequence for recipes would make it simpler to write long recipes, possibly at the expense of some readability.

But that's not how it was done.

How to properly make my makefile to compile and run?

Running from the makefile is a bit unusual. Are you, perhaps, trying to duplicate the "Compile and Run" Menu item that some IDE provide? Make is not well equipped to do that.

All the stuff that happens in the target commands happens in sub-processes that are not attached directly to the terminal, which is why make receives your key stroke.

Another thing to look at: usually the object-file to executable stage (linking) uses a different set of flags (LDFLAGS and LIBS) then the compile stage. In this simple example you can get away with it, but if you copy this makefile for use in a more complicated case you'll run into trouble.

C++ Makefile, is it possible to factorize it even more?

Mostly your makefile is pretty good. There are some simplifications you can make, but they're just syntax and not really performance etc.:

DEP_DIR := .dep/

You never use this by itself so if you change its definition to:

DEP_DIR := $(BUILD_DIR).dep/

you can simplify the references to it.

DEPENDS := $(patsubst %.o, $(BUILD_DIR)$(DEP_DIR)%.d, $(notdir $(wildcard $(BUILD_DIR)*.o)))

-include $(DEPENDS)

this seems complex. Why not get rid of DEPENDS and just write:

include $(wildcard $(DEP_DIR)*.d)

This:

@$(CXX) $(CXXFLAGS) -MMD -MP -MF $(BUILD_DIR)$(DEP_DIR)$(notdir $(basename $@).d) -c $< -o $@

is also complex. You can write it (if you simply DEP_DIR) as:

@$(CXX) $(CXXFLAGS) -MMD -MP -MF $(DEP_DIR)$(@F:.o=.d) -c $< -o $@

For:

.PRECIOUS: $(BUILD_DIR)%.o

I would definitely NOT use this. .PRECIOUS should be rarely, if ever, used. If you're trying to avoid object files being considered intermediate it's best to just list them directly as prerequisites, such as:

keep : $(EXE:$(BIN_DIR)%=$(BUILD_DIR)%.o)

But unless you have special need to look at these object files it doesn't hurt to let make delete them.

Regarding your question about shortcuts: the reason you see the behavior you do is that your target definition:

fileX-test: $(BIN_DIR)fileX-test

has no recipe attached to it, so make will try to find a recipe using an implicit rule. It finds built-in recipe for % : %.c, and because you set vpath it can find a %.c file that matches, so it uses it. To avoid this you can just give an empty recipe; replace the above with:

fileX-test: $(BIN_DIR)fileX-test ;

(note added semicolon).

Your main question is how to simplify this:

EXE := $(addprefix $(BIN_DIR), file1-test file2-test)

OBJS_1 := $(addprefix $(BUILD_DIR), file1.o)
OBJS_2 := $(addprefix $(BUILD_DIR), file1.o file2.o)

all: $(EXE)

$(BIN_DIR)file1-test: $(OBJS_1)
$(BIN_DIR)file2-test: $(OBJS_2)

You can do this automatically but doing so requires knowing the deeper parts of GNU make. You might find this set of blog posts interesting: http://make.mad-scientist.net/category/metaprogramming/ (start with the bottom / oldest and work your way up).

Replace the above with:

# Write one of these for each program you need:

file1-test_OBJECTS = file1.o
file2-test_OBJECTS = file1.o file2.o

# Now everything below here is boilerplate

EXE = $(patsubst %_OBJECTS,%,$(filter %_OBJECTS,$(.VARIABLES)))

all: $(EXE:%=$(BIN_DIR)%)

$(foreach E,$(EXE),$(eval $(BIN_DIR)$E: $($E_OBJECTS)))
$(foreach E,$(EXE),$(eval $E: $(BIN_DIR)$E ;))
.PHONY: $(EXE)