What Techniques Can Be Used to Speed Up C++ Compilation Times

How to speed up C++ compilation

Use a fast SSD setup. Or even create a ram disk, if suitable on your system.
Project -> Properties -> Configuration Properties -> C/C++ -> General -> Multi-processor Compilation: Yes (/MP)
Tools -> Options -> Projects and Solutions -> Build and Run: and set the maximum number of parallel project builds. (already set to 8 on my system, probably determined on first run of VS2013)

Are there techniques to greatly improve C++ building time for 3D applications?

Compilation speed is something, that can be really boosted, if you know how to. It is always wise to think carefully about project's design (especially in case of large projects, consisted of multiple modules) and modify it, so compiler can produce output efficiently.

1. Precompiled headers.

Precompiled header is a normal header (.h file), that contains the most common declarations, typedefs and includes. During compilation, it is parsed only once - before any other source is compiled. During this process, compiler generates data of some internal (most likely, binary) format, Then, it uses this data to speed up code generation.

This is a sample:

#pragma once

#ifndef __Asx_Core_Prerequisites_H__
#define __Asx_Core_Prerequisites_H__

//Include common headers
#include "BaseConfig.h"
#include "Atomic.h"
#include "Limits.h"
#include "DebugDefs.h"
#include "CommonApi.h"
#include "Algorithms.h"
#include "HashCode.h"
#include "MemoryOverride.h"
#include "Result.h"
#include "ThreadBase.h"
//Others...

namespace Asx
{

    //Forward declare common types
    class String;
    class UnicodeString;

    //Declare global constants
    enum : Enum
    {
        ID_Auto     = Limits<Enum>::Max_Value,
        ID_None     = 0
    };

    enum : Size_t
    {
        Max_Size            = Limits<Size_t>::Max_Value,
        Invalid_Position    = Limits<Size_t>::Max_Value
    };

    enum : Uint
    {
        Timeout_Infinite    = Limits<Uint>::Max_Value
    };

    //Other things...

}

#endif /* __Asx_Core_Prerequisites_H__ */

In project, when PCH is used, every source file usually contains #include to this file (I don't know about others, but in VC++ this actually a requirement - every source attached to project configured for using PCH, must start with: #include PrecompiledHedareName.h). Configuration of precompiled headers is very platform-dependent and beyond the scope of this answer.

Note one important matter: things, that are defined/included in PCH should be changed only when absolutely necessary - every chnge can cause recompilation of whole project (and other depended modules)!

More about PCH:

Wiki

GCC Doc

Microsoft Doc

2. Forward declarations.

When you don't need whole class definition, forward declare it to remove unnecessary dependencies in your code. This also implicates extensive use of pointers and references when possible. Example:

#include "BigDataType.h"

class Sample
{
protected:
    BigDataType _data;
};

Do you really need to store _data as value? Why not this way:

class BigDataType; //That's enough, #include not required

class Sample
{
protected:
    BigDataType* _data; //So much better now
};

This is especially profitable for large types.

3. Do not overuse templates.

Meta-programming is a very powerful tool in developer's toolbox. But don't try to use them, when they are not necessary.

They are great for things like traits, compile-time evaluation, static reflection and so on. But they introduce a lot of troubles:

Error messages - if you have ever seen errors caused by improper usage of std:: iterators or containers (especially the complex ones, like std::unordered_map), than you know what is this all about.
Readability - complex templates can be very hard to read/modify/maintain.
Quirks - many techniques, templates are used for, are not so well-known, so maintenance of such code can be even harder.
Compile time - the most important for us now:

Remember, if you define function as:

template <class Tx, class Ty>
void sample(const Tx& xv, const Ty& yv)
{
    //body
}

it will be compiled for each exclusive combination of Tx and Ty. If such function is used often (and for many such combinations), it can really slow down compilation process. Now imagine, what will happen, if you start to overuse templating for whole classes...

4. Using PIMPL idiom.

This is a very useful technique, that allows us to:

hide implementation details
speed up code generation
easy updates, without breaking client code

How does it work? Consider class, that contain a lot of data (for example, representing person). It could look like this:

class Person
{
protected:
    string name;
    string surname;
    Date birth_date;
    Date registration_date;
    string email_address;
    //and so on...
};

Our application evolves and we need to extend/change Person definition. We add some new fields, remove others... and everything crashes: size of Person changes, names of fields change... cataclysm. In particular, every client code, that depends on Person's definition needs to be changed/updated/fixed. Not good.

But we can do it the smart way - hide the details of Person:

class Person
{
protected:
    class Details;
    Details* details;
};

Now, we do few nice things:

client cannot create code, that depends on how Person is defined
no recompilation needed as long as we don't modify public interface used by client code
we reduce the compilation time, because definitions of string and Date no longer need to be present (in previous version, we had to include appropriate headers for these types, that adds additional dependencies).

5. #pragma once directive.

Although it may give no speed boost, it is clearer and less error-prone. It is basically the same thing as using include guards:

#ifndef __Asx_Core_Prerequisites_H__
#define __Asx_Core_Prerequisites_H__

//Content

#endif /* __Asx_Core_Prerequisites_H__ */

It prevents from multiple parses of the same file. Although #pragma once is not standard (in fact, no pragma is - pragmas are reserved for compiler-specific directives), it is quite widely supported (examples: VC++, GCC, CLang, ICC) and can be used without worrying - compilers should ignore unknown pragmas (more or less silently).

6. Unnecessary dependencies elimination.

Very important point! When code is being refactored, dependencies often change. For example, if you decide to do some optimizations and use pointers/references instead of values (vide point 2 and 4 of this answer), some includes can become unnecessary. Consider:

#include "Time.h"
#include "Day.h"
#include "Month.h"
#include "Timezone.h"

class Date
{
protected:
    Time time;
    Day day;
    Month month;
    Uint16 year;
    Timezone tz;

    //...
};

This class has been changed to hide implementation details:

//These are no longer required!
//#include "Time.h"
//#include "Day.h"
//#include "Month.h"
//#include "Timezone.h"

class Date
{
protected:
    class Details;
    Details* details;

    //...
};

It is good to track such redundant includes, either using brain, built-in tools (like VS Dependency Visualizer) or external utilities (for example, GraphViz).

Visual Studio has also a very nice option - if you click with RMB on any file, you will see an option 'Generate Graph of include files' - it will generated a nice, readable graph, that can be easily analyzed and used to track unnecessary dependencies.

Sample graph, generated inside my String.h file:

Sample Graph

How to speed up g++ compile time (when using a lot of templates)

What has been most useful for me:

Build on a RAM filesystem. This is trivial on Linux. You may want to keep a copy of common header files (precompiled or the actual .h files) on the RAM filesystem as well.
Precompiled headers. I have one per (major) library (e.g. Boost, Qt, stdlib).
Declare instead of include classes where possible. This reduces dependencies, thus reduces the number of files which need to be recompiled when you change a header file.
Parallelize make. This usually helps on a case-by-case basis, but I have -j3 globally for make. Make sure your dependency graphs are correct in your Makefile, though, or you may have problems.
Use -O0 if you're not testing execution speed or code size (and your computer is fast enough for you not to care much about the (probably small) performance hit).
Compile each time you save. Some people don't like this, but it allows you to see errors early and can be done in the background, reducing the time you have to wait when you're done writing and ready to test.

Why does C++ compilation take so long?

Several reasons

Header files

Every single compilation unit requires hundreds or even thousands of headers to be (1) loaded and (2) compiled.
Every one of them typically has to be recompiled for every compilation unit,
because the preprocessor ensures that the result of compiling a header might vary between every compilation unit.
(A macro may be defined in one compilation unit which changes the content of the header).

This is probably the main reason, as it requires huge amounts of code to be compiled for every compilation unit,
and additionally, every header has to be compiled multiple times
(once for every compilation unit that includes it).

Linking

Once compiled, all the object files have to be linked together.
This is basically a monolithic process that can't very well be parallelized, and has to process your entire project.

Parsing

The syntax is extremely complicated to parse, depends heavily on context, and is very hard to disambiguate.
This takes a lot of time.

Templates

In C#, List<T> is the only type that is compiled, no matter how many instantiations of List you have in your program.
In C++, vector<int> is a completely separate type from vector<float>, and each one will have to be compiled separately.

Add to this that templates make up a full Turing-complete "sub-language" that the compiler has to interpret,
and this can become ridiculously complicated.
Even relatively simple template metaprogramming code can define recursive templates that create dozens and dozens of template instantiations.
Templates may also result in extremely complex types, with ridiculously long names, adding a lot of extra work to the linker.
(It has to compare a lot of symbol names, and if these names can grow into many thousand characters, that can become fairly expensive).

And of course, they exacerbate the problems with header files, because templates generally have to be defined in headers,
which means far more code has to be parsed and compiled for every compilation unit.
In plain C code, a header typically only contains forward declarations, but very little actual code.
In C++, it is not uncommon for almost all the code to reside in header files.

Optimization

C++ allows for some very dramatic optimizations.
C# or Java don't allow classes to be completely eliminated (they have to be there for reflection purposes),
but even a simple C++ template metaprogram can easily generate dozens or hundreds of classes,
all of which are inlined and eliminated again in the optimization phase.

Moreover, a C++ program must be fully optimized by the compiler.
A C# program can rely on the JIT compiler to perform additional optimizations at load-time,
C++ doesn't get any such "second chances". What the compiler generates is as optimized as it's going to get.

Machine

C++ is compiled to machine code which may be somewhat more complicated than the bytecode Java or .NET use (especially in the case of x86).
(This is mentioned out of completeness only because it was mentioned in comments and such.
In practice, this step is unlikely to take more than a tiny fraction of the total compilation time).

Conclusion

Most of these factors are shared by C code, which actually compiles fairly efficiently.
The parsing step is a lot more complicated in C++, and can take up significantly more time, but the main offender is probably templates.
They're useful, and make C++ a far more powerful language, but they also take their toll in terms of compilation speed.

How to improve Visual C++ compilation times?

One thing that slows down the VC++ compiler is if you have a header file that initializes concrete instances of non-trival const value types. You may see this happen with constants of type std::string or GUIDs. It affects both compilation and link time.

For a single dll, this caused a 10x slowdown. It helps if you put them in a precompiled header file, or, just declare them in a header and initialize them in a cpp file.

Do take a look into the virus scanner, and be sure to experiment with precompiled headers, without it you won't see VC++ at its best.

Oh yeah, and make sure the %TMP% folder is on the same partition as where your build is written to, as VC++ makes temp files and moves them later.

What strategies have you used to improve build times on large projects?

Forward declaration
pimpl idiom
Precompiled headers
Parallel compilation (e.g. MPCL add-in for Visual Studio).
Distributed compilation (e.g. Incredibuild for Visual Studio).
Incremental build
Split build in several "projects" so not compile all the code if not needed.

[Later Edit]
8. Buy faster machines.

How to speed up Compile Time of my CMake enabled C++ Project?

Here's what I had good results with using CMake and Visual Studio or GNU toolchains:

Exchange GNU make with Ninja. It's faster, makes use of all available CPU cores automatically and has a good dependency management. Just be aware of
a.) You need to setup the target dependencies in CMake correctly. If you get to a point where the build has a dependency to another artifact, it has to wait until those are compiled (synchronization points).
```
$ time -p cmake -G "Ninja" ..
-- The CXX compiler identification is GNU 4.8.1
...
real 11.06
user 0.00
sys 0.00

$ time -p ninja
...
[202/202] Linking CXX executable CMakeTest.exe
real 40.31
user 0.01
sys 0.01
```
b.) Linking is always such a synchronization point. So you can make more use of CMake's Object Libraries to reduce those, but it makes your CMake code a little bit uglier.
```
$ time -p ninja
...
[102/102] Linking CXX executable CMakeTest.exe
real 27.62
user 0.00
sys 0.04
```
Split less frequently changed or stable code parts into separate CMake projects and use CMake's ExternalProject_Add() or - if you e.g. switch to binary delivery of some libraries - find_library().
Think of a different set of compiler/linker options for your daily work (but only if you also have some test time/experience with the final release build options).
a.) Skip the optimization parts
b.) Try incremental linking
If you often do changes to the CMake code itself, think about rebuilding CMake from sources optimized for your machine's architecture. CMake's officially distributed binaries are just a compromise to work on every possible CPU architecture.
When I use MinGW64/MSYS to rebuild CMake 3.5.2 with e.g.
```
cmake -DCMAKE_BUILD_TYPE:STRING="Release"
      -DCMAKE_CXX_FLAGS:STRING="-march=native -m64 -Ofast -flto" 
      -DCMAKE_EXE_LINKER_FLAGS:STRING="-Wl,--allow-multiple-definition"
      -G "MSYS Makefiles" .. 
```
I can accelerate the first part:
```
$ time -p [...]/MSYS64/bin/cmake.exe -G "Ninja" ..
real 6.46
user 0.03
sys 0.01
```
If your file I/O is very slow and since CMake works with dedicated binary output directories, make use of a RAM disk. If you still use a hard drive, consider switching to a solid state disk.
Depending of your final output file, exchange the GNU standard linker with the Gold Linker. Even faster than Gold Linker is lld from the LLVM project. You have to check whether it supports already the needed features on your platform.
Use Clang/c2 instead of Visual C++ compiler. For the Visual C++ compiler performance recommendations are provided from the Visual C++ team, see https://blogs.msdn.microsoft.com/vcblog/2016/10/26/recommendations-to-speed-c-builds-in-visual-studio/
Increadibuild can boost the compilation time.

References

CMake: How to setup Source, Library and CMakeLists.txt dependencies?
Replacing ld with gold - any experience?
Is the lld linker a drop-in replacement for ld and gold?