XMake - A Console build system based on XML

• Download source files - 66 Kb
• Download demo project - 12 Kb

Introduction

So why another build tool? Well that's a great question now isn't it! Basically I hate make files, and the whole make file syntax. I find it incredibly difficult to read, it is error prone to edit, and just seems to me to be incredibly messy. Obviously I am not alone in this feeling, since there are a number of projects out there to create alternatives to make (Ant for Java, and Jam, come to mind immediately).

So, since the VCF is now building on linux/Unix systems (so far the FoundationKit has been built on linux and Solaris ), it was thought it sure would be nice to have a single build file that could be used on any system for GCC, VC++, etc. Since XML syntax is easy to read (compared to a traditional make file), and the scope of the program would be fairly limited, i.e. it is designed to compiled and link C++ programs, and not a whole lot more, I thought it was worth the effort to try and write a make like program that would accept this new syntax. As it turned out it wasn't very hard, and maybe a couple of days of programming time (plus probably a week of testing) has gone into writing xmake so I think it is worth it . 

Another reason to write xmake was the desire to be able to have other developers quickly be able to build the framework, even on system that do not have good IDE's. I felt that it is a LOT easier to explain a simple set of XML tags to developers with a background in VC++ and GUI IDE's than trying to struggle to explain makefiles. 

While there are a couple of features that are not yet finished, notably the preBuild/postBuild tags, and the project level dependencies, the core of xmake works quite well and I have been regularly using it to build the VCF on linux as I work on porting it. This includes the build the FoundationKit as a shared library and the various test projects used to test the various parts of the port. 

In addition to a command line tool, the xmake engine will become the underlying build tool for the VCFBuilder, so VCFBuilder workspaces/projects will share the same XML syntax as xmake does (though the VCFBuilder will include a number of extra tags that will simply be ignored by the xmake engine). 

Basic Usage

xmake is designed to work on one or more projects. A project generally results in the output of a single binary, such as an executable (.exe) or a library (.lib) or a dynamic library (.dll or .so depending on your system). Within in a project, you can specify one or more configurations to build. For example, you could have a project build a "Debug GCC" and a "Debug VC++" and a "Debug BCC" (please note you can call the configuration anything you want), that would build a binary file using different compilers and linkers depending on the configuration. Configurations hold most of the key settings for the compiler and linker, as well as allowing you to specify additional build tools for specials file types (such as a resource compiler for .RC files on Win32 systems).  Thus when using xmake you tell it to build a specific configuration for a given project in a make file (usually makefile.xml). xmake is smart enough to automatically check file dependencies for you, so if you have recently modified a header that two of the five source files in your project depend on, it will only build the two affected source files.

Projects also hold a list of one or more source files that are necessary to build the project. These source files hold information such as the file name (the exact path is determined dynamically, relative to the path where is xmake is invoked), the output name, whether or not the file should be built (this can be useful, to allow you to turn "off" certain files ), and finally the configuration the source belongs to. Source files may belong to one or more configurations, with each configuration name separated by a "|" character (for example, "Debug GCC|Debug VC++") .

The command line usage is :

xmake [projectName] -config configurationName [-f makefileName] || [-install] || [-clean]

You can invoke the xmake program from the command line. You control the program by the following options :

• projectName - is the name of the project to build in this makefile. If left blank then the first project found is the one that is built.
• -config configurationName - this tells the xmake program which configuration to build. The -config option must be followed by a configuration name so xmake knows what to build. Specify the wrong name and you will not build what you expected.
• -f makefileName - the makefile to use. If this is not specified then xmake looks for a file named "makefile.xml" in the current directory. If this is not found then an error results and xmake exits.
• -install - currently unimplemented
• -clean - removes all binary files produced by the make file for the specified configuration

An example of it's usage, for example to build the FoundationKit for the VCF in linux using GCC, is this:

xmake -config  "GCC Debug"        

XML syntax

Since the xmake makefile is based on XML it follows all XML syntax rules. Please note that if you need to have double quotes in a value, use the single quote, which will then be replaced with a double quote during the build process. 

<make>

This is the outermost tag. Every xmake makefile must begin and end with this tag.

<substitutions>

This is where you can specify variable names that will be substituted when the make file is parsed and executed by xmake. For example you can specify a common include path, or a common output folder. Inside of a substitutions tag you can have zero or more variable tags that have 2 attributes, a name and a value. The name attribute is the name of the variable as it will be referenced throughout the make file, and the value attribute specifies the string that will be substituted wherever the parser encounters the variable. System environment variables may also be used, so if you have defined VCF_INCLUDE as one of your system's environment variables, then the parser is smart enough to attempt to try and check to see if the variable exists.

To reference the variable you do so as follows:

• $(<variable name>), for example if you have defined a variable called MyIncludePath, then you would reference it $(MyIncludePath)
• %<variable name>%, for example if you have defined a variable called MyIncludePath, then you would reference it %MyIncludePath%

An example of this tag section looks like this:

<substitutions>

    <variable name="VCF_INCLUDE" value="e:\code\vcf\include"/>

    <variable name="INC" value="../../../include"/>

    <variable name="SRC" value="../../../src"/>

</substitutions>

Please note: Variables can be used anywhere you are specifying a value for a attribute. 

<project>

The project holds all the important nodes. It also has a series of attributes, as follows:

• name - the name of the project
• path - the path of the project. This is optional right now and doesn't do much...
• outputAs - the output path of the project. This is used to determine the binary that represents a successful build of the project, and is used when your "clean" your project to specify the binary to get rid of. For example it could be "FooBar.exe", or "MyLib.dll", or "MySharedCode.so". 

Note that the outputAs attribute is optional. With most compilers/linkers you can specify where the binary output will go. However, as mentioned above, the output name is used so that xmake can do a complete "clean" of the project.

<dependencies>

Not implemented yet. These will allow for specifying what project(s) must be built before this one can be built. Again this is very similar to how dependencies work in VC++.

<configurations>

Configurations are the heart of xmake. If you have used Microsoft's Visual C++ then you'll be right at home here. Basically a configuration is a complete set of settings for compiler and linker in order to build the project. A single project may have multiple configurations, for example you might have a "Win32 Debug", "Win32 Release", "Linux Debug", "Linux Release", "Mac Debug", and "Mac Release". Thus, in order for xmake to do it's job it needs to know the configuration you want to build. The <configurations> tag is used to indicate the collection, and then a series of 1 or more <config> sub tags for each configuration. A configuration has the following attributes:

• name - The configuration name identifies the configuration, and is used by source files to identify which configuration they belong to. A configuration name can only have the characters [a..z, A..Z, 0..9] the "|" is used to separate multiple configurations.
• srcBinaryExt - the srcBinaryExt is used to identify the extension of compiled source files. Many compilers use ".o" while many others use ".obj". For example, GCC/G++ uses ".o" for it's extension for object files, while VC++ uses ".obj". This allows you to specify just the name of your output files (without the "." extension) and let xmake append the correct extension based on the configuration currently being built. 

An example looks something like this:

<config name="VC++ Debug" srcBinaryExt=".obj">

.... more stuff follows

Besides the above attributes, the configuration also has the following sub tags:

• includes - The includes section informs xmake of where to look for include files not in the current directory. This is used during the dependency scan for source files. 
• tools - The tools sections allows you to specify special build tools for build source files that are not able to be compiled using the <samp>compiler</samp> and linker in the configuration.
• compiler - This specifies the compiler used to compile source files (unless the source file specifies another tool).
• linker - This specifies the linker used to create the final project binary. 
• preBuild - not implemented yet.
• postBuild - not implemented yet.

<includes>

This allows you to specify a set of directories to look in when resolving dependencies for what to build. Each include directory requires an include tag. Each include tag has a single attribute:

• path - Where path is a directory path that will get added to the list of directory paths to search for when determining dependencies.

An example looks something like this:

<includes>
    <include path="../../MyDir/includes"/>
</includes>

.... more stuff follows

By default the directory where the source file lives is search in the dependency check.

<tools>

 The tools tag allows you to specify 0 or more tools that can be used to build a specific type of file. The association of tool to a source file is done with a source's "tool" attribute  having the same value as the tool's "id" attribute. Each tool is described in a <tool> tag with the following attributes:

• name - the name of the tool executable, such as "rc.exe". You can specify a full or relative path.
• id - the text name of the tool. This is the name that the various source entries will refer to the tool as. It can be anything you want, so long as you consistently refer to it.

Each tool tag can have flags associated with it. This is done by using the <flags> tag to indicate the collection, and then a <flag> sub tag. Each flag has the following attributes:

• value - the text parameters you want to pass to the tool

The flags are then appended to on another to create a command line that is then sent to the tool when needed.

An example looks something like this:

<tools>
    <tool name="rc.exe" id="rc" > <!-- here we are using just the resource compilers file 
                                       name, assuming it can be found on the system path -->
            <flags>
            <flag value="/i'..\..\MyResIncludes' \d '_DEBUG'"/>
        </flags>
    </tool>        
</tools>
.... more stuff follows

<compiler>

The compiler tag allows you to specify the executable to use as your compiler (for a given configuration) and a set of flags to send to the compiler when processing source files.  The compiler tag has only a single attribute:

• name - the relative or full path name to the executable to use for compiling source files. If the name is just the short name (i.e. "cl.exe"), then it is assumed that it can be found on the system path.

The compiler can have flags associated with it. This is done by using the <flags> tag to indicate the collection, and then a <flag> sub tag. Each flag has the following attributes:

• value - the text parameters you want to pass to the compiler

The flags are then appended to on another to create a command line that is sent to the compiler when it is invoked for each source file.

An example looks something like this:

<compiler name="cl">
  <flags>
    <flag value="/I $(INC)"/>
    <flag value="/nologo /MDd /W3 /Gm /GR /GX /ZI /Od"/>
    <flag value="/D WIN32 /D _DEBUG /D _CONSOLE /D _MBCS /D FRAMEWORK_DLL /D FRAMEWORK_EXPORTS"/>
    <flag value="/c"/>
  </flags>
</compiler >

.... more stuff follows

Note the use of the variable INC in the first flag. Also note that you could have just as easily put everything in one flag, but breaking it up makes it easier to read for others.

Please note: Use single quote's where you would use double quotes. Single quote characters will be expanded to double quote when the string is parsed and prepared to send to the compiler. 

<linker>

The linker tag allows you to specify the executable to use as your linker (for a given configuration) and a set of flags to send to the linker when linking all the compiler object code.  The linker tag has only a single attribute:

• name - the relative or full path name to the executable to use for linking object files. If the name is just the short name (i.e. "link.exe"), then it is assumed that it can be found on the system path.

The linker can have flags associated with it. This is done by using the <flags> tag to indicate the collection, and then a <flag> sub tag. Each flag has the following attributes:

• value - the text parameters you want to pass to the linker 

The flags are then appended to on another to create a command line that is sent to the linker when it is invoked to build the final binary image for the project.

An example looks something like this:

<linker name="link.exe">
    <flags>
        <flag value="/nologo"/>
        <flag value="/subsystem:console"/>                    
        <flag value="/incremental:yes"/>
        <flag value="/pdb:'Debug/xmake.pdb'"/>
        <flag value="/debug"/>
        <flag value="/machine:I386"/>
        <flag value="/out:'Debug/xmake.exe'"/>
        <flag value="/pdbtype:sept"/>        
    </flags>
</linker>

.... more stuff follows

Please note: Use single quote's where you would use double quotes. Single quote characters will be expanded to double quote when the string is parsed and prepared to send to the linker . 

<preBuild>

Currently this is not implemented yet. When it is, it will allow you to specify a series of command line actions to execute prior to the compile phase of the build process. preBuild is part of the configuration tag and is thus associated with a specific configuration. 

The proposed syntax will probably look something like this:

<prebuild>
    <exec commandLine="copy Foo.cpp ../../outputSrc"/>
    <!-- ...other commands -->
</prebuild>

.... more stuff follows

<postBuild>

Currently this is not implemented yet. When it is, it will allow you to specify a series of command line actions to execute after the link phase of the build process. postBuild is part of the configuration tag and is thus associated with a specific configuration. 

The proposed syntax will probably look something like this:

<postbuild>
    <!-- in this example we use doxygen to autogenerate
        source documentation-->
    <exec commandLine="doxygen ../../help/Doxyfile"/> 
    <!-- ...other commands -->
</postbuild>

.... more stuff follows

<sources>

After the configurations are all specified you list you source files that are required to build your project. The sources tag is a collection of 1 or more source sub tags, with each source sub tag representing an individual source file to build. The source tag has the following attributes:

• name - this is the name of the source file, relative to the directory where the project is being built. This attribute is required.
• partOfConfig - this indicates which configuration the source file will compile under. The name of the configuration must exactly match the name of the configuration specified in one of the config sections that was defined earlier in the makefile. If the source file can be compiled under multiple configurations, then each configuration name must be separated by the "|" character. This attribute is required. 
• build - indicates whether the file should even be built for this configuration. The possible values for this attribute are "yes" or "no". This attribute is optional. If you do not specify this then it defaults to "yes", meaning the file will be compiled.
• outputAs - this tells xmake where the compiled object file should go. This will override the settings in the compiler flags. Generally you want the two to match. By specifying this xmake knows where to go to clean up the object files during a "clean". This attribute is optional. If you do not specify this then it defaults to the directory where the project's makefile is. When specifying the name of the output file you can leave off the extension (like foo.o or baz.obj) and xmake will determine the proper extension from the configuration's srcBinaryExt attribute. 
• tool - tells xmake that the source file is not compiled using the default compiler, instead it is to be "compiled" using the tool whose id attribute matches this attribute's value. The two values must match exactly. If a tool does not exist in the makefile with a matching id attribute then the source file is not compiled. This attribute is required. 

An example looks something like this:

<sources>
    <source name="Test1Make.rc" partOfConfig="VC++ Debug" 
            outputAs="Debug/Test1Make.res" tool="rc"/>
    <source name="StdAfx.cpp" partOfConfig="VC++ Debug" />
    <source name="MainFrm.cpp" partOfConfig="VC++ Debug" 
            outputAs="Debug/MainFrm.obj"/>
    <source name="ChildView.cpp" partOfConfig="VC++ Debug" 
            outputAs="Debug/ChildView.obj"/>
    <source name="Test1Make.cpp" partOfConfig="VC++ Debug" 
            outputAs="Debug/Test1Make.obj"/>                        
</sources>

Note the use of the tool attribute in the first source tag.

Building and installing xmake

You can get xmake from here at the Source Forge VCF project page. You can build for Win32 systems using the VC++ workspace that is included. Alternately it can be built using GCC on 'nix systems using the traditional configure, make, make install commands (a configure and Makefile are provided for this).

It has been built and tested on Windows 2000, Windows NT sp4 (it is a fairly simple command line tool so it should work fine in Windows 98 and Windows XP), linux 2.4 (RH7.1 distro), SparcStation with Solaris 8 (2.8), and I have heard it runs on MacOSX and VMS but I have not personally verified this.

To install it under Win32 systems just put the executable where you want, preferably someplace that is on your system path. For 'nix systems the make install command will put it in /usr/bin, or you can place it somewhere else if you want.

Credits

I'd like to thank Chris Losinger for his CCmdLine class which  I used for the command line parsing.

I'd also like to thank Peter Sulyok, who wrote the original code for the file dependency checker which I appropriated and reworked a bit. This was originally in the file MAKEDEP.C, © 1994 Peter Sulyok.