[release/2.1][release/2.2][perf] Windows_NT I/O error reading or writing PE file

[michellemcdaniel]

We are getting the following error when building corefx on Windows for the perf runs in release/2.1 and release/2.2:

error : C:\j\w\perf_windows_---7c7bede1\bin/obj/ref/System.Threading.Tasks.Dataflow/4.6.3.0/netstandard/System.Threading.Tasks.Dataflow.dll: I/O error reading or writing PE file: Could not load file or assembly 'System.Collections.Immutable, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. [C:\j\w\perf_windows_---7c7bede1\src\System.Threading.Tasks.Dataflow\ref\System.Threading.Tasks.Dataflow.csproj]

I did a little digging and it looks like this is happening in the regular corefx builds as well. See: https://ci.dot.net/job/dotnet_corefx/job/release_2.1/job/windows_nt_release/192/console

It is unclear when this failure started, but the last passing run was August 30.

[michellemcdaniel]

@billwert @brianrob @danmosemsft

[karelz]

@wtgodbe @weshaggard are you aware of this?

[wtgodbe]

I was not, I can take a look

[wtgodbe]

Hmm, I wasn't able to get this to repro with the build command from https://ci.dot.net/job/dotnet_corefx/job/release_2.1/job/windows_nt_release/192/consoleText, build.cmd -Release -os:Windows_NT -buildArch:x64 -framework:netcoreapp. @adiaaida can you link one of the perf jobs failing with this?

[michellemcdaniel]

@wtgodbe https://ci2.dot.net/job/dotnet_corefx/job/perf/job/release_2.1/job/perf_windows_nt_release/191/

[wtgodbe]

@adiaaida I'm still having trouble repro-ing the failure, is there a way I can get on to one of the perf machines to try it out in that environment?

[michellemcdaniel]

@wtgodbe I thought I had responded, but it looks like I haven't. Sorry. Yes, you can absolutely get onto a machine. I'll email you instructions

[wtgodbe]

@adiaaida I think this is the VS2015/VS2017 thing again - we're missing System.Collections.Immutable, which stems from dotnet/buildtools@1a79896. I think the fix is to get the perf machines upgraded to use VS2017 (would also explain why the failure isn't repro-ing on non-perf builds). @weshaggard does that sound right?

[michellemcdaniel]

Ah, of course. Thanks. We will up the priority of this on our side.

@DrewScoggins FYI. It looks like we really need to up the priority of moving the windows machines to VS2017.

[DrewScoggins]

OK, I can get a ticket filed today to start with the first half of the machines.

[maririos]

@DrewScoggins Should we move this issue to core-eng repo instead?

[maririos]

Issue: https://github.com/dotnet/core-eng/issues/4366