Flex/Bison on MSBuild

• Download project - 2.2 KB

Introduction

The UNIX programs lex and yacc are powerful tools in the hands of a compiler or parser programmer. GNU clones of these tools, flex and bison, are freely available, and can be used as a part of a build toolchain in order to speed development of any program where token lexing and syntax parsing are needed.

Using flex and bison in a makefile on a linux system is of course a well-documented process. What follows is a recipe for using versions of these tools compiled for Windows, as part of a toolchain leveraging MSBuild and Visual Studio.

Using the Code 

For configurability's sake, set the location of FlexBinary and BisonBinary as properties that the gnu-flex and gnu-bison targets will refer to.

<PropertyGroup>
    <FlexBinary>C:\Program Files (x86)\GnuWin32\bin\flex.exe</FlexBinary>
    <BisonBinary>C:\Program Files (x86)\GnuWin32\bin\bison.exe</BisonBinary>
 </PropertyGroup> 

Include the flex and bison .l and .y files in the project. The <None> tag as a child of <ItemGroup> can be used for this, but any other new <ItemGroup> child tag name will do. Additional metadata added to these items will tell the project how to process them later.

 <ItemGroup>
    <None Include="lexer.l">
      <GnuType>Flex</GnuType>
    </None>
    <None Include="parser.y">
      <GnuType>Bison</GnuType>
    </None>
  </ItemGroup> 

This preparation step will create an environment friendly to flex and bison on your build machine. It uses a directory symlink, which can only be done as an administrator. Note that it is set to run BeforeTargets of gnu-bison and gnu-flex.

<Target Name="gnu-build-prep" 
BeforeTargets="gnu-bison;gnu-flex;" Condition=" 
$(BisonBinary.Contains(' ')) ">
    <!-- GnuWin32 bison not only needs to see itself on the path, 
    but it also must be run with the working directory set to its binary location.
         The working directory additionally may not contain spaces. -->
    <!-- We create and softlink a working directory at a known path 
    guaranteed to fit these requirements, so that we can build from there later. -->
    <!-- A side effect of the linking process is that this project 
    must be built as administrator the first time it is run on the host machine. -->
    <PropertyGroup>
      <GnuBuild>c:\temp\Build\GnuWin32</GnuBuild>
      <BisonBinDir>$([System.IO.Path]::GetDirectoryName($(BisonBinary)))</BisonBinDir>
    </PropertyGroup>
    <Exec Command="mklink /D $(GnuBuild) 
    &quot;$(BisonBinDir)\..\&quot;" Condition=" 
    !Exists('$(GnuBuild)\bin\$([MSBuild]::MakeRelative($(BisonBinDir),$(BisonBinary)))') " />
    <PropertyGroup>
      <BisonBinary>$(GnuBuild)\bin\$([MSBuild]::MakeRelative
      ($(BisonBinDir),$(BisonBinary)))</BisonBinary>
      <BisonBinDir>$([System.IO.Path]::GetDirectoryName
      ($(BisonBinary)))</BisonBinDir>
    </PropertyGroup>
  </Target> 

The gnu-flex target uses an Exec command to run flex on files with '%(GnuType)' == 'Flex'.

<Target Name="gnu-flex">
    <ItemGroup>
      <FlexDefinition Include="@(None)" Condition=" '%(GnuType)' == 'Flex' " />
    </ItemGroup>
    <Message Text="Pre-procssing with GNU Flex: `$(FlexBinary)`" Importance="High" />
    <Message Text="&quot;$(FlexBinary)&quot; 
    -o@(FlexDefinition -> '%(Filename).yy.c')  
    @(FlexDefinition)" Importance="High" />
    <Exec Command="&quot;$(FlexBinary)&quot; 
    -o@(FlexDefinition -> '%(Filename).yy.c')  
    @(FlexDefinition)" WorkingDirectory="
    @(FlexDefinition-&gt;DirectoryName())" 
    Outputs="@(FlexDefinition -> '@(Filename).yy.c')" />
  </Target> 

The gnu-bison target uses an Exec command to run bison on files with '%(GnuType)' == 'Bison'. Note the additional inline command to add the BisonBinDir to the path before invoking the bison command.

 <Target Name="gnu-bison" BeforeTargets="Build">
    <ItemGroup>
      <BisonDefinition Include="@(None)" Condition=" '%(GnuType)' == 'Bison' " />
    </ItemGroup>
    <Message Text="Pre-procssing with GNU Bison: 
    `$(BisonBinary)`" Importance="High" />
    <Message Text="set PATH=%PATH%;$(BisonBinDir)&amp;$(BisonBinary) 
    -o@(BisonDefinition -> '%(Filename).tab.c') --defines=@(BisonDefinition 
    -> '%(Filename).h') @(BisonDefinition)" Importance="High" />
    <Exec Command="set PATH=%PATH%;$(BisonBinDir)&amp;$(BisonBinary) 
    -o@(BisonDefinition -> '%(Filename).tab.c') --defines=@(BisonDefinition 
    -> '%(Filename).h') @(BisonDefinition)" 
    WorkingDirectory="@(BisonDefinition-&gt;DirectoryName())" 
    Outputs="@(BisonDefinition -> '@(Filename).tab.c')" />
  </Target>  

Lastly, hook the new targets into the build process by modifying the $(BuildDependsOn) variable.

<PropertyGroup>
    <BuildDependsOn>gnu-flex; gnu-bison;$(BuildDependsOn)</BuildDependsOn>
</PropertyGroup> 

Points of Interest

Note that escaping of commands in the MSBuild .vcxproj XML is a precise art. Be sure to download the attached .vcxproj, which is an entire template project, for context.

History

• 2013-12-20 - Original article creation and code explanation
• 2014-01-13 - Fixed some incorrect copy/paste errors in explanation text