Invariably, every time I complain about XBuild, some long-time user of it tells me it’s not so bad and it actually works pretty fine if you know its limitations. So I set out to document those, since in my experience, they are so many as to cause beautifully and carefully crafted MSBuild targets to become an entanglement of obsolete elements for the sake of satisfying XBuild (i.e. usage of CreateProperty and CreateItem tasks, when declarative items and property groups are perfect in MSBuild).

Most of the worst bugs are in handling items, which is precisely what’s typically most useful and powerful, such as item function and item metadata reference and augmentation.

Item Functions

I won’t enumerate them all, but here are several scenarios where Item Functions are totally the right solution and just don’t work.

Separating Items According to their Metadata

Say you have some items (i.e. EmbeddedResource) and you want to process those that have a certain Generator metadata (i.e. those that have ResXFileCodeGenerator or PublicResXFileCodeGenerator, meaning those are resources that are accessed via code generated by VS):

<ItemGroup>
	<EmbeddedResource Include="NonCode.resx" />
	<EmbeddedResource Include="NonPublic.resx">
		<Generator>ResXFileCodeGenerator</Generator>
	</EmbeddedResource>
	<EmbeddedResource Include="Public.resx">
		<Generator>PublicResXFileCodeGenerator</Generator>
	</EmbeddedResource>
</ItemGroup>

You can easily filter the elements using WithMetadataValue item function:

<ItemGroup>
	<ResxCode Include="@(EmbeddedResource -> WithMetadataValue('Generator', 'ResXFileCodeGenerator'))">
		<Public>False</Public>
	</ResxCode>
	<ResxCode Include="@(EmbeddedResource -> WithMetadataValue('Generator', 'PublicResXFileCodeGenerator'))">
		<Public>True</Public>
	</ResxCode>
</ItemGroup>

Here’s the output when printing out the resulting item groups:

<Target Name="Build">
	<Message Importance="high" Text="Resx with code: %(ResxCode.Identity) (Public=%(ResxCode.Public))" />
	<!-- 
MSBuild Output:
	Resx with code: NonPublic.resx (Public=False)
	Resx with code: Public.resx (Public=True)		

XBuild:
	Resx with code: @(EmbeddedResource -> WithMetadataValue('Generator', 'ResXFileCodeGenerator')) (Public=False)
	Resx with code: @(EmbeddedResource -> WithMetadataValue('Generator', 'PublicResXFileCodeGenerator')) (Public=True)
	-->	
</Target>

Clearly, XBuild didn’t understand at all the WithMetadataValue item function, and just used the entire string in the Include as the ItemSpec for those two ResxCode items. You might suggest that I can change the implementation so that the ItemGroup lives inside the Build target, and use a condition in the ResxCode based on the metadata:

<Target Name="Build">
	<ItemGroup>
		<ResxCode Include="@(EmbeddedResource)" Condition=" '%(Generator)' == 'ResXFileCodeGenerator' ">
			<Public>False</Public>
		</ResxCode>
		<ResxCode Include="@(EmbeddedResource)" Condition=" '%(Generator)' == 'PublicResXFileCodeGenerator' ">
			<Public>True</Public>
		</ResxCode>
	</ItemGroup>

	<Message Importance="high" Text="Resx with code: %(ResxCode.Identity) (Public=%(ResxCode.Public))" />
</Target>

That doesn’t work however, unless every EmbeddedResource has a Generator metadata item defined, which is the very thing that WithMetadataValue solves elegantly. And no, ItemDefinitionGroup doesn’t work either (see below).

Calculating Properties from Item Groups

It’s not uncommon to use item functions to populate property values, such as counts of items, of plain lists of items with certain metadata, such as:

<ItemGroup>
	<None Include="a.txt">
		<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
	</None>
	<None Include="b.txt" />
</ItemGroup>

If you want to concatenate in a property (say to pass this to a command line tool of some sort) all None items that have a CopyToOutputDirectory value:

<Target Name="Build">
	<PropertyGroup>
		<NoneWithCopy>@(None -> HasMetadata('CopyToOutputDirectory'))</NoneWithCopy>
	</PropertyGroup>

	<Message Importance="high" Text="$(NoneWithCopy)" />
	<!-- 
MSBuild Output:
	a.txt
  
XBuild Output:
	@(None -> HasMetadata('CopyToOutputDirectory'))

	-->
</Target>

Again, XBuild completely ignored the expression that contained an item function and treated it as a plain string.

String Item Functions

If you just take the entire sample on String Item Functions from MSDN and run it:

<ItemGroup>
	<theItem Include="andromeda;tadpole;cartwheel" />
</ItemGroup>

<Target Name="Build">
	<Message Text="IndexOf  @(theItem->IndexOf('r'))" />
	<Message Text="Replace  @(theItem->Replace('tadpole', 'pinwheel'))" />
	<Message Text="Length   @(theItem->get_Length())" />
	<Message Text="Chars    @(theItem->get_Chars(2))" />
</Target>

<!--
MSBUild Output:
    IndexOf  3;-1;2
    Replace  andromeda;pinwheel;cartwheel
    Length   9;7;9
    Chars    d;d;r
	
XBuild Output:
	IndexOf  @(theItem->IndexOf('r'))
	Replace  @(theItem->Replace('tadpole', 'pinwheel'))
	Length   @(theItem->get_Length())
	Chars    @(theItem->get_Chars(2))
-->

Enough said.

Count Items

This is another useful one every now and then, which also doesn’t work:

<ItemGroup>
	<None Include="a.txt" />
	<None Include="b.txt" />
</ItemGroup>

<Target Name="Build">
	<Message Text="Count @(None -> Count())" />
</Target>

<!--
MSBuild Output:
	Count 2

XBuild Output:
	Count @(None -> Count())
-->

Distinct Items

This is another one that when you absolutely need it, it’s painful to work around:

<ItemGroup>
	<None Include="a.txt" />
	<None Include="a.txt" />
</ItemGroup>

<Target Name="Build">
	<Message Text="None: @(None -> Distinct())" />
</Target>

<!--
MSBuild Output:
	None: a.txt

XBuild Output:
	None: @(None -> Distinct())
-->

A gist with all the files in this post is available as a gist

The Entire Intrinsic Item Funcions Fail

To avoid writing a sample and broken output for each one, the rest of the list of Intrinsic Item Functions is broken too.

Other

There are myriad others that are also equally painful. Here are some:

Item Definition Groups

ItemDefinitionGroup are a way to define the items you’ll use in your projects, and assign default metadata values if they are not provided when declaring an item.

For example, if you want to define a File item for your projects, which will (say) have some codegen associated, you can specify that the codegen will by default be public unless overriden for a particular file:

<ItemDefinitionGroup>
	<!-- Say you want to provide a default value for all, unless explicitly overriden -->
	<File>
		<IsPublic>true</IsPublic>
	</File>
</ItemDefinitionGroup>

This allows you to simplify the items declaration, since you only need to specify the IsPublic metadata if you want to override the default:

<ItemGroup>
	<File Include="Default.resx" />
	<File Include="NonPublic.resx">
		<IsPublic>false</IsPublic>
	</File>
	<File Include="Public.resx">
		<IsPublic>true</IsPublic>
	</File>
</ItemGroup>

<Target Name="Build">
	<Message Importance="high" Text="%(File.Identity): IsPublic=%(File.IsPublic)" />
	<!-- 
MSBuild Output:
	Default.resx: IsPublic=true
	NonPublic.resx: IsPublic=false
	Public.resx: IsPublic=true
  
XBuild:
	Default.resx: IsPublic=
	NonPublic.resx: IsPublic=false
	Public.resx: IsPublic=true
	-->
</Target>

Note how the default value was never applied in XBuild’s case. Moreover, the treatment of the missing value is highly inconsistent, since with the exact same items above, adding a filtered item group inside the Build target:

<Target Name="Build">
    <ItemGroup>
        <PublicFile Include="@(File)" Condition=" '%(IsPublic)' == 'true' " />
    </ItemGroup>

Causes the build to fail with:

error : Error building target Build: Metadata named 'IsPublic' not found in item named Default.resx in item list named File

I’m not sure I should be thankful or not about the extra looseness when referring to item metadata in task attributes, but the inconsistency is definitely not welcomed. It’s one more of those “if you know its limitations” thing that you have to constantly remember.

Property Functions in Attributes

Say you have some property, and you want to apply a replacement string right before passing it to a task attribute:

<PropertyGroup>
	<Content>Hello $Name$</Content>
</PropertyGroup>

<Target Name="Build">
	<!-- This works on XBuild -->
	<Message Importance="high" Text="$(Content.Replace('$Name$', 'Foo'))" />

Initially, it seems like it works, but some other tasks fail:

	<!-- This doesn't work on XBuild -->
	<WriteLinesToFile File="out.cs" Lines="$(Content.Replace('$Name$', 'Foo'))" Overwrite="true" />

In this case, it fails with:

error : Error executing task WriteLinesToFile: Error converting Property named 'Lines' with value '$(Content.Replace('$Name$', 'Foo'))' to type Microsoft.Build.Framework.ITaskItem[]: The method or operation is not implemented.

It seems the expression was not evaluated before passing it to Lines.

Property Functions in Item Metadata

This is a very powerful way to augment user-declared items with additional metadata as you process the items across multiple targets. For example, say you need to repeatedly access the content of the None text files in a project. For performance reasons, you might want to read the content once and attach it as metadata on the items themselves, so that further targets can directly access it without unnecessarily reading over and over the same files:

<ItemGroup>
	<None Include="a.txt" />
	<None Include="b.txt" />
</ItemGroup>

	<ItemGroup>
		<None>
			<Content>$([System.IO.File]::ReadAllText('%(Identity)'))</Content>
		</None>
	</ItemGroup>
	
	<Message Importance="high" Text="%(None.Identity)=%(None.Content)" />
</Target>

If the content of a.txt is ‘A’ and of b.txt is ‘B’, this works flawlessly in MSBuild, but fails on XBuild:

MSBuild Output:
	a.txt=A
	b.txt=B
  
XBuild (with /v:diag):
	: error : Error building target Build: Exception has been thrown by the target of an invocation.
	Error building target Build: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. -> System.IO.FileNotFoundException: Could not find file "/Volumes/C/Code/Personal/xbuildsucks/%(Identity)".
	File name: '/Volumes/C/Code/Personal/xbuildsucks/%(Identity)'

Item Metadata Augmentation

The problem above goes beyond property functions. You can’t really reference the current item metadata when you’re augmenting it at all. So given a simple item group:

<ItemGroup>
	<None Include="a.txt" />
	<None Include="b.txt" />
</ItemGroup>

A trivial metadata addition such as:

<Target Name="Build">
	<ItemGroup>
		<None>
			<OriginalIdentity>%(Identity)</OriginalIdentity>
		</None>
	</ItemGroup>
	
	<Message Importance="high" Text="%(None.Identity)=%(None.OriginalIdentity)" />
</Target>

Results in:

MSBuild Output:
	a.txt=a.txt
	b.txt=b.txt
  
XBuild (with /v:diag):
	: error : Error building target Build: Object reference not set to an instance of an object
	Error building target Build: System.NullReferenceException: Object reference not set to an instance of an object
	  at Microsoft.Build.BuildEngine.Project.GetMetadataBatched (System.String itemName, System.String metadataName) <0x2b7ee08 + 0x00037> in <filename unknown>:0 

Generating Code

I left this one for last, but it’s really the one that drives me totally crazy, since it’s super powerful to generate code at build-time using MSBuild.

Say you have some template code that you apply a replacement before emitting it at compile time:

<PropertyGroup>
	<Content>
        public static class ThisAssembly
        {
        	public const string AssemblyName = "$AssemblyName$";
        }		
	</Content>
</PropertyGroup>

And let’s say this is in an imported .targets, and the user specifies in his .csproj:

<PropertyGroup>
	<AssemblyName>Foo</AssemblyName>
</PropertyGroup>

During build, you want to just replace the $AssemblyName$ token with the $(AssemblyName) property value, and write that to a file:

<Target Name="Build">
	<PropertyGroup>
		<!-- Need to do the replacement in another property, since it doesn't work 
			 in the Lines attribute, as shown in 'Property Functions in Attributes' section -->
		<Replaced>$(Content.Replace('$AssemblyName$', '$(AssemblyName)'))</Replaced>
	</PropertyGroup>

	<WriteLinesToFile File="out.cs" Lines="$(Replaced)" Overwrite="true" />
	<Message Text="$([System.IO.File]::ReadAllText('out.cs'))" />
</Target>

You’d never guess what the output is:

MSBuild Output:
	public static class ThisAssembly
	{
		public const string AssemblyName = "Foo";
	}

XBuild Output:
	public static class ThisAssembly
	{
		public const string AssemblyName = "Foo"
	}

Yep, that’s right, there’s a missing semicolon! What’s more, you can’t use the %3B encoding, or escape the semicolon in any way, it invariable gets dropped. So unless you switch to a language with no semicolons, you’re basically screwed.

Wait, some would way, that’s easy to solve! You an just cook up a nice regex and use something like sed in MacOS X to append the missing semicolon! Read on…

Indiscriminate Backslash Conversion

Before you jump to give me the fancy sed command line to solve all my problems, which is:

sed 's/\(.*\)"/\1";/' out.cs

Let me tell you why that won’t work. If you have the following content:

<PropertyGroup>
	<Content>Hello \ Bye</Content>
</PropertyGroup>

Even doing this:

<Target Name="Build">
	<Message Text="$(Content)" />
</Target>

On XBuild results in:

Hello / Bye

And I could find NO way to escape the backslashes so they are left as-is. As you can see, it’s not that they are converted only on places where it might be (sometimes) useful, such as in <Exec Command="" />, but it’s replaced everywhere in any task attributes (as far as I can tell). That’s a Message task, what does that backslash replacement have to do there??

So, your carefully crafted regex to fixup the other messed up codegen in XBuild, is unfortunately doomed before you even start. Don’t bother.

All this took me down the path of repackaging the xplat branch of the open sourced MSBuild as a nuget package so that I can do all the ugly tricks in a separate targets file specifically for XBuild where I just install this package during build and redefine the targets to invoke MSBuild directly on a Mac, as I explained some time ago on my blog.

And one day, when Microsoft (with Xamarin’s help) is done porting MSBuild to the Mac, I can just breathe and kill a single line of code and forget all this pain ever happened.

Tags: msbuild xbuild

/kzu dev↻d