Mad, Beautiful Ideas

It would be unfair of me to claim to be an expert in project build systems. When I was first learning C, I was pretty much just stuck to batch scripts and the command line, when I first started playing with Borland C++ or Visual Studio 6, there were the integrated tools, though it was hard to call those a ‘system’. No, my first build system was the simple GNU Make. To this day, it’s still what I’m most familiar with, though I’m starting to learn Ant, and now MSBuild.

Build systems have, for me at least, always been a matter of convenience. I’m a Linux user, so Make has always been available, the files easy to write and understand, but as I work more and more on other platforms, and with code that I want to be cross-platform, I’m finding myself looking for other systems. Since my current job is based entirely on .NET on Windows, using Visual Studio, MSBuild was my obvious choice, though I did consider NAnt, I decided against it since Visual Studio has already done quite a bit of the work for me, MSBuild wise.

It turns out that was probably a more-than-ignorant statement, because while the entire Visual Studio 2005/2008 build system is MSBuild-based, MSBuild is, by far, the most convoluted Build System I have ever seen.

Hence the title. There are things that I really do love about MSBuild, but at the same time, there are some things that they did that just make me really hate working with MSBuild. As for a how any of this relates to Team Foundation Server’s Team Build, I suspect that system is just MSBuild plus a bit, but I can’t say for sure.

As for some background, the project that I’m working on right now is a Silverlight Application which communicates via a Web Application Project to a set of Windows Communcation Foundation (WCF) web services. In implementing our build system we have the following goals:

1. Update the ServiceReferences.ClientConfig in the Silverlight Project to point to the correct URLs for deployment
2. Update the Web.Config files based on deployment criteria
3. Update the Service References in the Silverlight after building the Web Application Project, to ensure the Services are configured correctly.
4. Deploy the project, if appropriate

Ideally, we’ll be implementing this into a Continuous Integration system soon, which will ensure that our Testing website is always up to date with the mainline in our version control. We’re not there just yet, but for what I have accomplished I’ll be going through the good, bad, and ugly of MSBuild in this post.

First, replacing URLs in the ServiceReference.ClientConfig. For this, I decided to modify the Pre- and Post-Build Tasks in the Silverlight project. One thing that I find really nice about MSBuild is that the projects are basically like onions. If you import a MSBuild Targets file, anything you define in the outer file, will trump what’s in the inner file. This has led to a significant amount of customization in the system, and in fact, pretty much everything is based around a single file, MSBuild.Common.Targets.

This file is the strength of MSBuild, and in fact there are literally dozens of targets declared in this file. Microsoft did an amazing job of breaking down each and every step of the build process into very discrete steps. If you want to redefine any of these small confined (or larger meta) steps, you can do that pretty easily. The basics of these MSBuild files is well described by this Hands-on Lab from PDC 2005.

To be honest, I believe that most things that need to be done, can be done only by modifying the BeforeBuild and AfterBuild Tasks, which are commented out in the proj files in your Visual Studio solution. Two words of warning: You won’t be able to open the proj files in the same instance as the project is loaded for normal work, and visual studio will scream about your custom changes to the files. Just accept them, and you should be fine.

So, onto the changes. To modify the ServiceReference.ClientConfig file, I began to use the MSBuildExtensionPack from CodePlex. I perform a Regular Expression replace taking the ‘localhost’ URL, and pointing it either at our Test environment, or our Production environment depending on the build configuration selected. I do make sure that in the AfterBuild step, I replace the ServiceReference.ClientConfig file to the way it was, so that Debug builds still work as expected.

Second, the Web.config replacements. Actually, for this one, I take a task from the Web Deployment Projects. It already has a task to replace sections of a web.config file with parts of a different file. The configuration is fairly simple:

<ItemGroup Condition="'$(Configuration)|$(Platform)' == 'Release|AnyCPU'>
    <WebConfigReplacementFiles Include="connectionStrings.config">
        <Section>connectionStrings</Section>
    </WebConfigReplacementFiles>
</ItemGroup>

<Target Name="UpdateConfig">
    <Microsoft.WebDeployment.Tasks.WebConfigReplacement
        TargetDir="$(DeployDir)"
        ReplacementSections="@(WebConfigReplacementFiles)" />
</Target>

What is really unfortunate about this system, is that you will sometimes needs to have recreate large sections of your Web.config for replacement, like the sections to secure our WCF services, there is a lot of almost, but not quite, identical code in there. This is a minor annoyance though.

Updating the service references is a much harder problem, one which is currently unsolved. There are a few parts to this. First, the Web Application Project, by default, depends on the Silverlight project. This is useless. I’ve broken that relationship, and in the AfterBuild step of the Silverlight project, it copies the resulting XAP to the WebApplication Project ClientBin folder, again to ensure that debugging works correctly. I am in the process of figuring out the best way to generate the service references at this stage in the build, and I’ll be sure to post a full solution for that problem later.

Regarding deployment, I copy the necessary files to the appropriate directory (over a UNC path), fairly straightforward. Actually, this is (aside from the service references thing) the most annoying part of the whole process. It is suprisingly difficult to copy an entire directory anywhere. For instance, the Web Application Project has four sets of files I’m interested in:

<ItemGroup>
    <App_DataFiles Include="./WebProject1/App_Data/**/*.*" />
    <BinFiles Include="./WebProject1/Bin/**/*.dll" />
    <ClientBinFiles Include="./WebProject1/ClientBin/**/*.*" />
    <RootFiles Include="*.svc" />
    <RootFiles Include="*.aspx" />
    <RootFiles Include="web.config" />
</ItemGroup>

The ‘/**/’ syntax tells MSBuild to recurse through the subdirectories, returning a list of files with subdirectories that you’ll need. The really lousy part, is that it requires multiple Copy commands to move all the files, like so:

<Target Name="Deploy">
    <Copy 
        Files="@(App_DataFiles)" 
        DestinationDirectory="$(DestDir)\App_Data\%(RecurseDir)" />
    <Copy 
        Files="@(BinFiles)" 
        DestinationDirectory="$(DestDir)\Bin\%(RecurseDir)" />
    <Copy 
        Files="@(ClientBinFiles)" 
        DestinationDirectory="$(DestDir)\ClientBin\%(RecurseDir)" />
    <Copy 
        Files="@(RootFiles)" 
        DestinationDirectory="$(DestDir)\" />
</Target>

The syntax for variables is really Perl-ish. Frankly, I love that, but it really does surprise me. Anyway, that’s just a side note. In several of those cases (App_Data, and ClientBin) in particular, I really want to be able to copy those entire directories, and a copy dir command would be really nice.

Luckily, if I wanted it badly enough, it’s fairly easy to develop your own MSBuild Tasks, they’re just .NET code after all. The PDC Lab linked above walks through that process.

MSBuild is very, very customizable, which is fantastic. However, that complexity comes at a cost: Tasks which don’t appear to have been designed with ease of use in mind, a level of complexity which is frankly intimidating when first looking at it. It’s a build system that seems to present a “don’t touch” front. It’s highly capable, but unlike other build systems I’m familiar with, even simple tasks seem difficult in MSBuild.

I don’t think it’s a system I’d use if I wasn’t using Visual Studio, but if Visual Studio is already doing most of the heavy lifting for you, it is far easier to tap into that work, than throw it all away.