C++ Include test: full matrix

Finally I am able to present some include experiment results! Previously I wrote about Code Generator and now I can actually run this tool and get some numbers out of it. I compared VC11.0 (Visual Studio 2012 For Desktop) and GCC 4.7.2 (using MinGW32 4.7.2 and DevCpp).

Basics of experiment

VC11.0 - Visual Studio 2012 For Desktop, Release Mode, 32bit, no optimizations.
- I turned build timings on to have detailed build performance.
GCC 4.7.2 32bit - MinGW 4.7.2 version and run from DevCpp 5.4.1
- g++.exe -c testHeaders.cpp -o testHeaders.o -ftime-report
The machine: Core i5, 4 cores, 4GB RAM, Windows 8 64bit
- compilation will probably take place only on one core only (there will be one translation unit only)

I run each test 3 times and then compute the average. At the end of post there is a link to detailed spreadsheet.

Code structure

The overall code structure is not a real case scenario. It was my first attempt to do such experiment and it was actually easy to create such hierarchy.

testHeader.cpp includes N header files
m-th header file includes N-1 other header files (so we have "cross" include)
each header file has its proper include guard

Note: For 199 GCC will return an error "#include nested too deeply". You can read more about it in gamesfromwithin.com. In general GCC thinks that 199 include level is unreal and will block it.

Test N headers

A header can look like this in this tests:

#ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H 
    #include "header_0.h"
    #include "header_1.h"
    #include "header_2.h"
    #include "..."

generator.exe N 100 includeTestOutput/

N = 100

GCC	3,95s
GCC Compilation	2,99s
VC11.0	2,90s
VC11.0 Compilation	2,68s

N = 132

GCC	5,37s
GCC Compilation	3,98s
VC11.0	4,31s
VC11.0 Compilation	4,11s

N = 164

GCC	6,49s
GCC Compilation	4,92s
VC11.0	6,10s
VC11.0 Compilation	5,91s

N = 192

GCC	7,40s
GCC Compilation	5,77s
VC11.0	7,98s
VC11.0 Compilation	7,77s

Test N headers - additional ifDef

A header can look like this in this tests:

#ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H
    #ifndef _INCLUDED_HEADER_HEADER_0_H
        #include "header_0.h"
    #endif
    #ifndef _INCLUDED_HEADER_HEADER_1_H
        #include "header_1.h"
    #endif
    #ifndef _INCLUDED_HEADER_HEADER_2_H
        #include "header_2.h"
    #endif
    #include "..."

generator.exe N 100 includeTestOutput/ ifDef

N = 100

GCC	3,91s
GCC Compilation	2,96s
VC11.0	1,44s
VC11.0 Compilation	1,22s

N = 132

GCC	5,35s
GCC Compilation	3,91s
VC11.0	1,71s
VC11.0 Compilation	1,51s

N = 164

GCC	6,41s
GCC Compilation	4,86s
VC11.0	1,98s
VC11.0 Compilation	1,77s

N = 192

GCC	7,31s
GCC Compilation	5,69s
VC11.0	2,16s
VC11.0 Compilation	1,96s

Test N headers - #pragma once

A header can look like this in this tests:

#pragma once
    #ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H
    #include "header_0.h"
    #include "header_1.h"
    #include "header_2.h"
    #include "..."

generator.exe N 100 includeTestOutput/ pragmaOnce

N = 100

GCC	4,02s
GCC Compilation	3,08s
VC11.0	1,48s
VC11.0 Compilation	1,28s

N = 132

GCC	5,42s
GCC Compilation	4,06s
VC11.0	1,84s
VC11.0 Compilation	1,65s

N = 164

GCC	6,64s
GCC Compilation	5,08s
VC11.0	2,06s
VC11.0 Compilation	1,86s

N = 192

GCC	7,60s
GCC Compilation	5,98s
VC11.0	2,39s
VC11.0 Compilation	2,20s

Conclusion

The code structure is rather theoretical and does not represent 'common' structures that may appear in projects.
GCC and VC build code a bit different. GCC linker phase is much longer than in VC.
GCC uses lots of optimization for header files. There is almost no need to do any 'tricks' with includes. Header guards are enough.
VC likes header files 'tricks'!
- There is above 2X to 3X speedup using additional include guards or pragma once
- Pragma once for such code structure seems to be a bit slower than additional include guards: from 5% to even 10% slower.
All in all VC11.0 is faster than GCC.

What I have learnt

VC Release mode is not the same as GCC basic compilation
Make sure you set up proper experiment base
Automate as much as possible
Tool for spreadsheet is a must have-software
It is valuable to figure out how to break the compiler

Links

Link to repo

Link to spreadsheet

Hint: For improved responsiveness ensure Javascript is enabled and choose 'Normal' from the Layout dropdown and hit 'Update'.
You must Sign In to use this message board.

Search
	Per page