Parallel Make: Set -J8 as The Default Option

Parallel make: set -j8 as the default option

Your question is not about threads, but processes (jobs) executed by make.

The simple, way to set this, when make is used from the console is adding:

alias make="/usr/bin/make -j 8"

to your .profile file.

You can also use setenv MAKEFLAGS '-j 8', but MAKEFLAGS can ignore this parameter in some scenarios, because keeping desired number of processes requires communicating with recursive make calls. Happily this method works with current versions of GNU Make.

Make make default to make -j 8

alias make="make -j 8", assuming bash shell

From CMake setup 'make' to use '-j' option by default

Via setting the CMAKE_MAKE_PROGRAM variable you want to affect the build process. But:

1. This variable affects only the build via cmake --build, not on native tool (make) call:

   The CMAKE_MAKE_PROGRAM variable is set for use by project code. The value is also used by the cmake(1) --build and ctest(1) --build-and-test tools to launch the native build process.

2. This variable should be a CACHEd one. It is used in such way by make-like generators:

   These generators store CMAKE_MAKE_PROGRAM in the CMake cache so that it may be edited by the user.

   That is, you need to set this variable with

   set(CMAKE_MAKE_PROGRAM <program> CACHE PATH "Path to build tool" FORCE)
   

3. This variable should refer to the executable itself, not to a program with arguments:

   The value may be the full path to an executable or just the tool name if it is expected to be in the PATH.

   That is, value "make -j 2" cannot be used for that variable (splitting arguments as list

   set(CMAKE_MAKE_PROGRAM make -j 2 CACHE PATH "Path to build tool" FORCE)
   

   wouldn't help either).

In summary, you may redefine the behavior of cmake --build calls with setting the CMAKE_MAKE_PROGRAM variable to the script, which calls make with parallel options. But you may not affect the behavior of direct make calls.

GNU make's -j option

I think make -j will respect the dependencies you specify in your Makefile; i.e. if you specify that objA depends on objB and objC, then make won't start working on objA until objB and objC are complete.

Most likely your Makefile isn't specifying the necessary order of operations strictly enough, and it's just luck that it happens to work for you in the single-threaded case.

Automatically setting jobs (-j) flag for a multicore machine?

It appears that the MAKEFLAGS environment variable can pass flags that are part of every make run (at least for GNU make). I haven't had much luck with this myself, but it might be possible to use -l rather than -j to automatically run as many jobs as are appropriate for the number of cores you have available.

Does setting -j8 in MAKEFLAGS cause problems for recursive make?

If your operating system supports it, it's not problematic. If for some reason it doesn't, or your make isn't gnu make, it's not a good idea. Also, it's not clear if you execute a program in your make that executes another make, whether or not the two makes will communicate.

Using make with -j4 or -j8

As you say the -j flag tells make that it is allowed to spawn the provided amount of 'threads'. Ideally each thread is executed on its own core/CPU, so your multi-core/CPU environment is used to its fullest.

make itself does not compile the source files. This is done by a compiler (gcc). The Makefile (input for make) contains a set of targets. Each target has a set of dependencies (on other targets) and rules how to build the target. make reads the Makefile(s) and manages all targets, dependencies, and build rules. Besides compiling source files you can use make to perform any task that can be described by shell commands.

If you set the allowed number of threads too high, it is not possible to schedule each thread on its own core. Additional scheduling (context) switches are required to let all threads execute. This additional resource usage obviously result in lower performance.

There are multiple rules-of-thumb, but I guess that setting to total amount to <number of cores> + 1 is the most common. The idea behind this is that all cores have their own thread and there is one additional managing thread that handles the targets and which is next to be built.

How do I set default values for make?

You can set the environment variable MAKEFLAGS to values you want to have "always set":

MAKEFLAGS=-j8
export MAKEFLAGS

---

Related Topics