Item to copy to output

[KirillOsenkov]

If we need to copy a file to output directory, the current pattern seems to be:

<ItemGroup>
  <None Include="myfile.txt">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </None>
</ItemGroup>

It works by first assigning the target path in the AssignTargetPaths (plural!) target. The output of the AssignTargetPath task (singular!) is copied to the NoneWithTargetPath item:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 3197 to 3199 in dbf8d12

	<AssignTargetPath Files="@(None)" RootFolder="$(MSBuildProjectDirectory)">
	<Output TaskParameter="AssignedFiles" ItemName="_NoneWithTargetPath" />
	</AssignTargetPath>

All the AssignTargetPath task does is append the TargetPath metadata to the item, usually just the name. It is the relative path inside the project output directory where to copy the file. Without it, you'll get an error saying the destination file path is a directory (because the target file path is empty).

msbuild/src/Tasks/AssignTargetPath.cs

Line 122 in dbf8d12

AssignedFiles[i].SetMetadata(ItemMetadataNames.targetPath, EscapingUtilities.Escape(targetPath));

Then NoneWithTargetPath gets copied to _ThisProjectItemsToCopyToOutputDirectory:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 5176 to 5179 in dbf8d12

	<ItemGroup>
	<_ThisProjectItemsToCopyToOutputDirectory KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_NoneWithTargetPath->'%(FullPath)')" Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always' AND '%(_NoneWithTargetPath.MSBuildSourceProjectFile)'==''"/>
	<_ThisProjectItemsToCopyToOutputDirectory KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_NoneWithTargetPath->'%(FullPath)')" Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest' AND '%(_NoneWithTargetPath.MSBuildSourceProjectFile)'==''"/>
	</ItemGroup>

Then the target returns and the items go into _ThisProjectItemsToCopyToOutputDirectory:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 5195 to 5197 in dbf8d12

	<CallTarget Targets="_GetCopyToOutputDirectoryItemsFromThisProject">
	<Output TaskParameter="TargetOutputs" ItemName="_ThisProjectItemsToCopyToOutputDirectory" />
	</CallTarget>

Then the items flow into _SourceItemsToCopyToOutputDirectory:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 5204 to 5214 in dbf8d12

	<_TransitiveItemsToCopyToOutputDirectoryAlways KeepDuplicates=" '$(_GCTODIKeepDuplicates)' != 'false' " KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_TransitiveItemsToCopyToOutputDirectory->'%(FullPath)')" Condition="'%(_TransitiveItemsToCopyToOutputDirectory.CopyToOutputDirectory)'=='Always'"/>
	<_TransitiveItemsToCopyToOutputDirectoryPreserveNewest KeepDuplicates=" '$(_GCTODIKeepDuplicates)' != 'false' " KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_TransitiveItemsToCopyToOutputDirectory->'%(FullPath)')" Condition="'%(_TransitiveItemsToCopyToOutputDirectory.CopyToOutputDirectory)'=='PreserveNewest'"/>
	<_ThisProjectItemsToCopyToOutputDirectoryAlways KeepDuplicates=" '$(_GCTODIKeepDuplicates)' != 'false' " KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_ThisProjectItemsToCopyToOutputDirectory->'%(FullPath)')" Condition="'%(_ThisProjectItemsToCopyToOutputDirectory.CopyToOutputDirectory)'=='Always'"/>
	<_ThisProjectItemsToCopyToOutputDirectoryPreserveNewest KeepDuplicates=" '$(_GCTODIKeepDuplicates)' != 'false' " KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_ThisProjectItemsToCopyToOutputDirectory->'%(FullPath)')" Condition="'%(_ThisProjectItemsToCopyToOutputDirectory.CopyToOutputDirectory)'=='PreserveNewest'"/>
	<!-- Append the items from this project last so that they will be copied last. -->
	<_SourceItemsToCopyToOutputDirectoryAlways Include="@(_TransitiveItemsToCopyToOutputDirectoryAlways);@(_ThisProjectItemsToCopyToOutputDirectoryAlways)"/>
	<_SourceItemsToCopyToOutputDirectory Include="@(_TransitiveItemsToCopyToOutputDirectoryPreserveNewest);@(_ThisProjectItemsToCopyToOutputDirectoryPreserveNewest)"/>
	<AllItemsFullPathWithTargetPath Include="@(_SourceItemsToCopyToOutputDirectoryAlways->'%(FullPath)');@(_SourceItemsToCopyToOutputDirectory->'%(FullPath)')"/>

Finally the copy happens in _CopyOutOfDateSourceItemsToOutputDirectory:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 5264 to 5282 in dbf8d12

	<Target
	Name="_CopyOutOfDateSourceItemsToOutputDirectory"
	Condition=" '@(_SourceItemsToCopyToOutputDirectory)' != '' "
	Inputs="@(_SourceItemsToCopyToOutputDirectory)"
	Outputs="@(_SourceItemsToCopyToOutputDirectory->'$(OutDir)%(TargetPath)')">
	<!--
	Not using SkipUnchangedFiles="true" because the application may want to change
	one of these files and not have an incremental build replace it.
	-->
	<Copy
	SourceFiles = "@(_SourceItemsToCopyToOutputDirectory)"
	DestinationFiles = "@(_SourceItemsToCopyToOutputDirectory->'$(OutDir)%(TargetPath)')"
	OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
	Retries="$(CopyRetryCount)"
	RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
	UseHardlinksIfPossible="$(CreateHardLinksForAdditionalFilesIfPossible)"
	UseSymboliclinksIfPossible="$(CreateSymbolicLinksForAdditionalFilesIfPossible)"
	>

Note the destination is $(OutDir)%(TargetPath). This is what the TargetPath metadata was needed for.

======

Now, the problem with the None item is that it's considered an input by the Visual Studio Fast Up-To-Date Check. So if you are generating an item as part of the project build, and then add it to the None item to ensure it gets copied, you have a situation where the project's output is also its input, so the FUTDC will always consider the project not up-to-date, because the generated file was written to after the primary output assembly, but it's now an input, so we have an input newer than output.

I was looking for a loophole to find a better way to do this. I first tried to directly add to the _ThisProjectItemsToCopyToOutputDirectory item, but without the TargetPath metadata I got an error from the copy task because the destination file name was empty.

The only thing I found that works is instead of None to add it to _CompileItemsToCopy:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 5163 to 5174 in dbf8d12

	<ItemGroup>
	<_CompileItemsToCopy Include="@(Compile->'%(FullPath)')" Condition="('%(Compile.CopyToOutputDirectory)'=='Always' or '%(Compile.CopyToOutputDirectory)'=='PreserveNewest') AND '%(Compile.MSBuildSourceProjectFile)'==''"/>
	</ItemGroup>
	<AssignTargetPath Files="@(_CompileItemsToCopy)" RootFolder="$(MSBuildProjectDirectory)">
	<Output TaskParameter="AssignedFiles" ItemName="_CompileItemsToCopyWithTargetPath" />
	</AssignTargetPath>
	<ItemGroup>
	<_ThisProjectItemsToCopyToOutputDirectory KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_CompileItemsToCopyWithTargetPath)" Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
	<_ThisProjectItemsToCopyToOutputDirectory KeepMetadata="$(_GCTODIKeepMetadata)" Include="@(_CompileItemsToCopyWithTargetPath)" Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
	</ItemGroup>

Conveniently, AssignTargetPath is being called for this item, so it acquires the TargetPath metadata.

However, obviously, it's a hack.

I wonder what's the official blessed pattern for ensuring that a file generated by this project gets copied to output. If we don't have one, we should make one and make it easy and fool-proof.

[KirillOsenkov]

also the benefit of using this is that it flows transitively to referencing projects. For example things like <Reference Include="..." Private="true" /> do not flow transitively. Would be nice to have control over the transitive behavior as well (copy to output of this project vs. copy to output for all referencing projects too)

[KirillOsenkov]

My recommendation would be to literally clone _CompileItemsToCopy, name it something sensible like CopyToOutput and set the default metadata CopyToOutputDirectory=PreserveNewest.

The biggest risk here is someone already having an item named CopyToOutput, not sure what to do in this case. I guess we'll just break them?