This is my first attempt at editing a build template in Team Build 2010. Previously I have expressed a desire to see Team Build move away from cryptic XML files and go to a GUI. Well I got my wish, but instead of cryptic XML, we get a cryptic GUI in the form of Windows Workflow foundation. Now maybe I’m not supposed to go in a edit the default build template, but that’s how I learn. I start with what I know works and start tweaking stuff.

How bad is it? Well the entire workflow will not fit on my screen at 25% zoom. Here are a couple of screen shots.

Now to be fair, the default build template allows you to configure a lot of settings using a property grid like GUI so you never have to look at the WF diagram. It’s only when you need to add additional functionality that you have to go into the land of WF. I guess I, along with Microsoft are expecting the community to step up and provide some new build templates. I also found out that if you double click on the blocks in the WF designer view, you jump "into" the block and that’s all you see.

None the less, I set about the task of modifying the default build template to execute XUnit tests instead of MS Tests. Before I go into that, let me explain why I am setting myself up for this torture. My immediate alternatives are to use a different build server, such as Team City, or to use MSTest. If I use Team City, I loose the tight integration with TFS and I do not wish to explain to people about multiple tools. If I use MSTest, I’ll still have to edit the build template to get automated deployment working, as so far I haven’t seen a MS Deploy activity. I’m not sure if this is the best solution. Maybe I’ll get some feedback, but more then likely, I’ll end up borrowing someone else’s solution down the road.

So what do I have to start with? The default build template has an option to scan for assemblies that match a certain naming convention (***test*.dll) which does match my naming convention (this pattern is also configurable in the build definition GUI under BasicAutomated TestsTest Assembly). Looking at the log file for the build, I can see that my two test assembly projects are loaded, but no tests are executed. Once I found where in the WF sequence this was occurring, I realized I had a pretty good starting point. 

Where to Start

First I had to find where I could switch out the call to MSTest with a call to Xunit using the list of test assemblies found. It’s buried pretty deep, so keep double clicking along the following path.

• Sequence
• Run On Agent
• Try Compile, Test, and Associate Changesets and Work items
• Sequence
• Compile, Test, and Associate Changesets and Work items
• Try Compile and Test
• Compile and test
• For Each Configuration in BuildSettings.PlatformConfigurations
• Compile and Test for Configuration
• If Not DisableTests
• Run Tests
• If Not TestSpecs Is Nothing
• For Each TestSpec in TestSpecs
• Try Run Tests
• If spec Is TestMetadataFileSpec
• Run MSTest for Test Assemblies
• If Test Assemblies Found (started inserting here)
• If testAssembly.HasTestSettingsFile (deleted)

Run XUnit on all test assemblies

Since I don’t have a test settings file with XUnit, I delete the check for HasTestSettingsFile and added an ForEach with an Invoke Process activity to use the xunit console runner. So my "If Test Assemblies Found" block now looks like:

The syntax to use the console runner is xunit.console Assembly.dll /nunit , which looks pretty easy to use. To get the Xunit console on my build server, I could either add it to source control, or put it in a known location on the build server. I opted for the later because it was the easiest.

To get the XUnitPath into the build template, I created a new Argument called XUnitPath and took the easy way out by specifying the default as where I had installed XUnit. I did this because I couldn’t figure out how to get it to show in the GUI initially. You need to check your build template in and re-open the build definition. After doing this, my new argument showed up under section 4. Misc.

I then went back to my Invoke Process activity and set the file name property to my argument "variable" XUnitPath.

For the arguments, I specified item, which is the ForEach variable for the assembly, as well as an option to output the results as an nUnit formatted xml file.

Before I attempt to merge the XML results file, I need to be able to trigger my build to fail if my tests fail. In the invoke process, you can grab the output using the stdOut variable, which I did. I then added an If statement to check for the text "Tests failed", which the XUnit console outputs to when it encounters a failed test. Unfortunately, every line output fires off my If statement, so my build log file looks a little verbose. However, this works, and after fixing my tests, my build went from partially succeeded to succeeded. 

At this point, here is what my "If Test Assemblies Found" block looks like. The Assign blocks under Standard Output and Standard error assign BuildDetail.TestStatus to Microsoft.TeamFoundation.Build.Client.BuildPhaseStatus.Failed.

Publish XUnit results to TFS

A couple of months ago I went about integrating XUnit tests and results into TFS 2008, so I figured that those same steps would work more or less. After going down that path initially, I decided it would be cleaner to just create a new XSLT template based on the NUnit4MSBuild template that takes an XUnit results file and transforms it into a MSTest trx file directly. This cuts out an extra step in my sequence, and eliminates the need for an external dependency on nxslt3. I wrote up a separate blog post on how to Transform XUnit to MSTest. Like the information presented in this post, it’s a work in progress.

After getting the results file in a compatible format I still needed to get them published to TFS. You can use an Invoke activity to MSTest (which requires Visual Studio to be installed on your build server) to publish your results. The filename argument I used was:

"""" & System.Environment.GetFolderPath(Environment.SpecialFolder.ProgramFilesX86) & "Microsoft Visual Studio 10.0Common7IDEMSTest.exe"""

and the arguments I used where:

"/publish:""" & BuildDetail.BuildServer.TeamProjectCollection.Uri.ToString() & """" & _
" /publishbuild:""" & BuildDetail.BuildNumber & """" & _
" /publishresultsfile:" & """" & item & "_XunitResults.trx" & """" & _
" /teamproject:""" & BuildDetail.TeamProject & """" & _
" /platform:""" & platformConfiguration.Platform & """" & _
" /flavor:""" & platformConfiguration.Configuration & """"

Final Thoughts and What’s Next

Well it works, but it’s what I consider to be a hack. However, I’m going to invoke YAGNI in that this is manageable given the number of projects, builds and workflow I am working with. I expect that I will have to work on this as my Greenfield project matures. I also expect that once TFS 2010 is released and more people start using it, there will be more community contributions. I would like to wrap this all up into an XUnit workflow activity, but it depends on how much extra time I have. Since this works, the next thing on my to do list is to get automated web deployments working using MS Deploy.

Right click and choose Save As: