What Strategies Have You Used to Improve Build Times on Large Projects

What strategies have you used to improve build times on large projects?

1. Forward declaration
2. pimpl idiom
3. Precompiled headers
4. Parallel compilation (e.g. MPCL add-in for Visual Studio).
5. Distributed compilation (e.g. Incredibuild for Visual Studio).
6. Incremental build
7. Split build in several "projects" so not compile all the code if not needed.

[Later Edit]
8. Buy faster machines.

What are ways of improving build/compile time?

The biggest improvement we made for our large C++ project was from distributing our builds. A couple of years ago, a full build would take about half an hour, while it's now about three minutes, of which one third is link time.

We're using a proprietary build system, but IncrediBuild is working fine for a lot of people (we couldn't get it to work reliably).

Compilation in VS2010 takes too long

The time to compile C++ probably varies more than with any other language I've ever used.

One thing that can make a significant difference is what headers you're including. Even though your code may only be 800 lines, if a few of those are #includes, the compiler may easily be looking at thousands of lines (just for reference, #include <windows.h>, by itself, generally means the compiler will look at over 10,000 lines).

A few of us in the C++ chat room were recently doing some tests on a particularly nasty piece of code that has a lot of recursive templates. Even though it's only about 30 lines of code, depending on the parameters you set, it's pretty easy to get compiles times of an hour or more -- and with most compilers (including VC++10 and 11/2012) it's pretty easy to outright crash the compiler.

If the code has little or nothing in the way of headers and/or templates (especially things like recursive templates), then 7.5 seconds to compile seems fairly excessive. Just for comparison, I did a quick test compiling a program I had lying around that's close to the same size (926 lines). That took 0.3 seconds. My machine is something like 5 or 6 years old, so its speed isn't even close to cutting edge either. At the same time, I should add that for compiling that small an amount of code, CPU speed probably isn't the main determining factor. I'd expect an SSD to make a lot more difference than a faster CPU.

I want to reduce my VS.NET project's compile time - what are your ideas for how to do this?

I'm surprised that 60k lines of code take 2 minutes to compile. I have an application that is 500,000 lines of code, and it only takes about a minute and a half. Make sure you are not doing full rebuilds each time, and make sure you are not cleaning the solution between builds. A normal build should perform an incremental build, only recompiling code that has changed since the last build (along with anything affected by that change.)

Perhaps some other factors might include heavy use of large resources (images?), broad-sweeping changes in the lowest level libraries (i.e. those used by everything else), etc. Generally speaking, on a relatively modern machine, compiling 60,000 lines of C# code should take less than a minute on average, unless you are rebuilding the entire solution.

How to decrease build times / speed up compile time in Xcode?

Often, the largest thing you can do is to control your inclusion of header files.

Including "extra" header files in source code dramatically slows down the compilation. This also tends to increase the time required for dependency checking.

Also, using forward declaration instead of having headers include other headers can dramatically reduce the number of dependencies, and help all of your timings.

Ways to speed up build time? (C#/Unmanaged C++)

Focus on the C++ projects - they are almost guaranteed to be the largest time drains for building.

Some tips on getting the C++ build times down:

• Make sure that you're only including headers that you need in the C++ projects!
• Use forward declarations whenever possible in headers instead of including other headers
• Use the /MP switch to build in parallel, when possible
• Use abstraction effectively
• Be sparing in the use of inline functions, since these cost more at compile time
• Get the dependencies correct, so you're not building more often that required
• Use pre-compiled headers appropriately

Aside from that, if you're talking 2 hour build times, often there is a simple, cheap (in a big picture way) solution, as well:

• Upgrade your hardware to help reduce the computation times

How to manage Build time in TDD

I believe it was in "Working Effectively with Legacy Code" that he said if your test suite takes longer than a couple minutes it will slow developers down too much and the tests will start getting neglected. Sounds like you are falling into that trap.

Are your test cases running against a database? Then that's most likely your biggest source of performance problems. As a general rule, test cases shouldn't ever be doing I/O, if possible. Dependency Injection can allow you to replace a database object with mock objects that simulate the database portion of your code. That allows you test the code without worrying whether the database is setup correctly.

I highly recommend Working Effectively with Legacy Code by Michael Feathers. He discusses how to handle a lot of the headaches that you seem to be running into without having to refactor the code all at once.

UPDATE:

A another possible help would be something like NDbUnit. I haven't used it extensively yet, but it looks promising: http://code.google.com/p/ndbunit/

Very slow compile times on Visual Studio 2005

The Chromium.org team listed several options for accelerating the build (at this point about half-way down the page):

In decreasing order of speedup:

• Install Microsoft hotfix 935225.
• Install Microsoft hotfix 947315.
• Use a true multicore processor (ie. an Intel Core Duo 2; not a Pentium 4 HT).
• Use 3 parallel builds. In Visual Studio 2005, you will find the option in Tools > Options... > Projects and Solutions > Build and Run > maximum number of parallel project builds.
• Disable your anti-virus software for .ilk, .pdb, .cc, .h files and only check for viruses on modify. Disable scanning the directory where your sources reside. Don't do anything stupid.
• Store and build the Chromium code on a second hard drive. It won't really speed up the build but at least your computer will stay responsive when you do gclient sync or a build.
• Defragment your hard drive regularly.
• Disable virtual memory.

---

Related Topics