MSBuild MSB3030 errors from apparent race condition on ARM64 Linux at commit e03255e

[RobertHenry6bev]

Running on ARM64 Linux Ubuntu 22.04 with dotnet 7.0., about 50% of the time, I see crank failing because a msbuild it kicks off fails.

Here's a reformatted error message from crank(?): exhibiting MSB3030

mnt/root_0/dotnet_bench/dotnethome/sdk/7.0.400/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Publish.targets(267,5):
error MSB3030: Could not copy the file
"/mnt/root_0/root_crank_0/benchmarks-server-171334/ROBHENRY_APP/src/BenchmarksApps/TechEmpower/PlatformBenchmarks/published/published/global.json"
because it was not found.
[/mnt/root_0/root_crank_0/benchmarks-server-171334/ROBHENRY_APP/src/BenchmarksApps/TechEmpower/PlatformBenchmarks/PlatformBenchmarks.csproj]

I did a careful git bisect, and it seems that e03255e
is to blame.

[RobertHenry6bev]

I don't use docker.

[sebastienros]

@caaavik-msft it's the multi-source, and there also seems to be a "published" added to the path ... might be the same issue as #628

[caaavik-msft]

I'll take a look at both of these issues today

[caaavik-msft]

The issue I believe might be caused due to the use of a custom sourcekey and potential backwards compatibility issues around the cached sources when upgrading to a newer version of the crank controller. We have been struggling to reproduce this, so for now will leave this open, but if we are able to reproduce this occurring where the crank versions have not changed we can take a look into this again.

[sebastienros]

Because nobuild is not working correctly (single source at least) the builds keep being executed on cached folders and the published folders are recursively copied on each build. At some point there is a conflict between the copies as it tries to publish in the same folder.