[5.0.100-rc.2.20464.10] Single File publish failed for dnspy app after retarget .NET5

[jiangzeng01]

Application Name: dnspy
OS: Windows 10 RS5
CPU: X64
.NET Build Number: 5.0.100-rc.2.20464.10
App Source :
\appcompatsrv\Appcompat\DotNetCore\AppSource\SingleFileP0AppsTarget5\AppSources\dnSpy
DevDiv issue: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1215382

Repro steps:
Install .NET5.0.100-rc.2.20464.10 SDK on local machine.
Target .NET5
Copy '\appcompatsrv\Appcompat\DotNetCore\AppSource\SingleFileP0AppsTarget5\AppSources\dnSpy' to local machine.
Open command prompt window and navigate to %localpath%\dnSpy\dnSpy\dnSpy folder.
Run "dotnet publish dnSpy.csproj -r win-x64 -p:PublishSingleFile=true --self-contained true --framework net5.0-windows".

Expected Result:
dnSpy app should be published successful.

Actual Result:
Get error, results the publish process failed.

Findings :
If we publish without "-p:PublishSingleFile=true" flag, the publish process is successful.
After add the "-p:PublishSingleFile=true" flag, the publish is failed.

We attached the failed publish log (WithSingleFileFlag.txt) with "-p:PublishSingleFile=true" and the successful publish log (WithoutSingleFileFlag.txt) without "-p:PublishSingleFile=true" in Attachemenet.
WithSingleFileFlag.txt
WithoutSingleFileFlag.txt

Exception message for failed publish :

C:\Program Files\dotnet\sdk\5.0.100-rc.2.20464.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1012,5): error MSB4018: The "GenerateBundle" task failed unexpectedly. [C:\DonetCoreApps\dnSpy\dnSpy\dnSpy\dnSpy.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20464.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1012,5): error MSB4018: System.ArgumentException: Invalid input specification: Found multiple entries with the same BundleRelativePath [C:\DonetCoreApps\dnSpy\dnSpy\dnSpy\dnSpy.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20464.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1012,5): error MSB4018:    at Microsoft.NET.HostModel.Bundle.Bundler.GenerateBundle(IReadOnlyList`1 fileSpecs) [C:\DonetCoreApps\dnSpy\dnSpy\dnSpy\dnSpy.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20464.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1012,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateBundle.ExecuteCore() [C:\DonetCoreApps\dnSpy\dnSpy\dnSpy\dnSpy.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20464.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1012,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [C:\DonetCoreApps\dnSpy\dnSpy\dnSpy\dnSpy.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20464.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1012,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [C:\DonetCoreApps\dnSpy\dnSpy\dnSpy\dnSpy.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20464.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1012,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [C:\DonetCoreApps\dnSpy\dnSpy\dnSpy\dnSpy.csproj]

cc @dotnet-actwx-bot

[ghost]

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

[LakshanF]

We throw here because we try to add 2 items in FilesToBundle that gets resolved to the same relative path, Microsoft.DiaSymReader.Native.amd64.dll.

.nuget\packages\microsoft.diasymreader.native\1.7.0\runtimes\win\native\Microsoft.DiaSymReader.Native.amd64.dll
    CopyToPublishDirectory = PreserveNewest
    RelativePath = Microsoft.DiaSymReader.Native.amd64.dll
    TargetPath = Microsoft.DiaSymReader.Native.amd64.dll

.nuget\packages\microsoft.diasymreader.native\1.7.0\runtimes\win-x64\native\Microsoft.DiaSymReader.Native.amd64.dll
    AssetType = native
    CopyLocal = true
    CopyToPublishDirectory = PreserveNewest
    DestinationSubPath = Microsoft.DiaSymReader.Native.amd64.dll
    NuGetPackageId = Microsoft.DiaSymReader.Native
    NuGetPackageVersion = 1.7.0
    PathInPackage = runtimes/win-x64/native/Microsoft.DiaSymReader.Native.amd64.dll
    RelativePath = Microsoft.DiaSymReader.Native.amd64.dll

Need to find why rid fallback logic picked both win-x64 and win

[LakshanF]

The csproj file for the project, dnSpy.csproj, contains a package reference to Microsoft.DiaSymReader.Native and the reason for the duplicate files being included above for the self-contained SingleFile build failure (the FX assemblies also includes this file).

Removing the reference to Microsoft.DiaSymReader.Native in the app proj file or building a FDD SingleFile app will be successful. However, the app will fail given the multiple use of System.Reflection.Assemly.Location in the app code which will return an empty string.

Still need to track if SingleFile should take care of scenarios like this but likely on a different issue so as not to confuse.

[PriyaPurkayastha]

Same error is also reported with NugetPackageExplorer. For NPE, we checked the latest source code & built with RC2.
The app built and published as FDD successfully, but failed to publish as Single-File.

Error logs:
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20468.21\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1017,5): error MSB4018: The "GenerateBundle" task failed unexpectedly. [C:\DonetCoreApps\NuGetPackageExplorer\PackageExplorer\NuGetPackageExplorer.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20468.21\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1017,5): error MSB4018: System.ArgumentException: Invalid input specification: Found multiple entries with the same BundleRelativePath [C:\DonetCoreApps\NuGetPackageExplorer\PackageExplorer\NuGetPackageExplorer.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20468.21\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1017,5): error MSB4018: at Microsoft.NET.HostModel.Bundle.Bundler.GenerateBundle(IReadOnlyList`1 fileSpecs) [C:\DonetCoreApps\NuGetPackageExplorer\PackageExplorer\NuGetPackageExplorer.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20468.21\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1017,5): error MSB4018: at Microsoft.NET.Build.Tasks.GenerateBundle.ExecuteCore() [C:\DonetCoreApps\NuGetPackageExplorer\PackageExplorer\NuGetPackageExplorer.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20468.21\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1017,5): error MSB4018: at Microsoft.NET.Build.Tasks.TaskBase.Execute() [C:\DonetCoreApps\NuGetPackageExplorer\PackageExplorer\NuGetPackageExplorer.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20468.21\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1017,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [C:\DonetCoreApps\NuGetPackageExplorer\PackageExplorer\NuGetPackageExplorer.csproj]
C:\Program Files\dotnet\sdk\5.0.100-rc.2.20468.21\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(1017,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [C:\DonetCoreApps\NuGetPackageExplorer\PackageExplorer\NuGetPackageExplorer.csproj]

[agocke]

This looks like a problem with the product build -- the GenerateBundle task is verifying that none of the output files have the same name, but that doesn't seem to be the case here.

This is a bug in publishing, sometimes called "double write", where binaries with the same name or content are referenced twice by a project. This tends not to be a fatal error for MSBuild as the files are simply copied over one another, but it is an incorrect build graph.

This is, however, a fatal error for single-file, as we cannot choose which file is the "right" one to include in the bundle.

[stevenbrix]

I'm a bit confused, @agocke has this issue been fixed then? Or are we using microsoft/CsWinRT#442 to track this?

[agocke]

I don't see why there is a bug here, since single-file publish has never worked for dnspy before. There isn't an expectation that single-file publish always works out of the box, so right now I don't think there's any runtime-specific bug to track here.

(The R2R bug looks separate, can't speak to that)