[master] Update dependencies from dotnet/arcade #243

dotnet-maestro · 2019-11-25T13:24:49Z

This pull request updates the following dependencies

From https://github.com/dotnet/arcade

Build: 20191125.7
Date Produced: 11/26/2019 4:27 AM
Commit: 549475dff607986e6d9626f0dc8678e9736b7d0c
Branch: refs/heads/master
Updates:
- Microsoft.DotNet.XUnitExtensions -> 5.0.0-beta.19575.7
- Microsoft.DotNet.XUnitConsoleRunner -> 2.5.1-beta.19575.7
- Microsoft.DotNet.VersionTools.Tasks -> 5.0.0-beta.19575.7
- Microsoft.DotNet.ApiCompat -> 5.0.0-beta.19575.7
- Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.19575.7
- Microsoft.DotNet.Build.Tasks.Configuration -> 5.0.0-beta.19575.7
- Microsoft.DotNet.Build.Tasks.Feed -> 5.0.0-beta.19575.7
- Microsoft.DotNet.Build.Tasks.Packaging -> 5.0.0-beta.19575.7
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk -> 5.0.0-beta.19575.7
- Microsoft.DotNet.CodeAnalysis -> 5.0.0-beta.19575.7
- Microsoft.DotNet.GenAPI -> 5.0.0-beta.19575.7
- Microsoft.DotNet.GenFacades -> 5.0.0-beta.19575.7
- Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.19575.7
- Microsoft.DotNet.RemoteExecutor -> 5.0.0-beta.19575.7

…124.1 - Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19574.1 - Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19574.1 - Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19574.1 - Microsoft.DotNet.ApiCompat - 5.0.0-beta.19574.1 - Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19574.1 - Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19574.1 - Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19574.1 - Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19574.1 - Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19574.1 - Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19574.1 - Microsoft.DotNet.GenAPI - 5.0.0-beta.19574.1 - Microsoft.DotNet.GenFacades - 5.0.0-beta.19574.1 - Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19574.1 - Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19574.1

ViktorHofer · 2019-11-25T14:10:33Z

@safern @tmat what's the status regarding the break in Arcade?

tmat · 2019-11-25T18:35:00Z

Is this change included? dotnet/arcade#4424

tmat · 2019-11-25T18:36:44Z

Seems like it is :(.

ViktorHofer · 2019-11-25T18:36:55Z

yes: https://dnceng.visualstudio.com/internal/_build/results?buildId=436505

tmat · 2019-11-25T18:38:31Z

Is there a binlog for the failed build? I see just a text log file

safern · 2019-11-25T18:39:07Z

https://dev.azure.com/dnceng/_apis/resources/Containers/2897850?itemPath=Windows_NT_Build%20x64_Debug%2FBuild.binlog

safern · 2019-11-25T18:41:22Z

I've been trying to repro this locally and haven't been able to. It seems like a race condition because of the BeforeTargets as the TargetPath doesn't exist yet as the Exec task logs Couldn't find the specified path.

safern · 2019-11-25T18:43:44Z

Also note that it doesn't fail for the second inner build (net46) but it does for the first one (netstandard2.0). This project cross compiles for those two target frameworks.

tmat · 2019-11-25T18:47:29Z

The whole trouble with this target are the race conditions (or rather correct build target ordering).

tmat · 2019-11-25T18:48:31Z

I am running out of ideas :( We do chain after CoreBuild precisely to have the PDB available before the target runs.

safern · 2019-11-25T18:51:49Z

In the meantime I set the installer.tasks as IsShipping=false so that this target doesn't run. We can re-enable once this is fixed.

a911e7c

Based on my understanding of the binlog the chaining is not happening correctly

CoreCompile seems to me to be running in parallel.

safern · 2019-11-25T18:54:42Z

Is it because PrepareForRun is the one that actually copies over the output files to bin from obj?

And it is part of Compile not part of CoreCompile?

EDIT: I got confused you're chaining to CoreBuild not CoreCompile

safern · 2019-11-25T18:56:39Z

I also see this message in the evaluation of the project:

The target "AfterBuild" listed in a BeforeTargets attribute at "D:\a\1\s.packages\microsoft.dotnet.arcade.sdk\5.0.0-beta.19574.1\tools\SymStore.targets (12,11)" does not exist in the project, and will be ignored.

dagood · 2019-11-25T19:00:10Z

Based on my understanding of the binlog the chaining is not happening correctly

CoreCompile seems to me to be running in parallel.

When an error happens, the binlog parenting gets really messed up, in my experience. But something to note is that targets within a single project never run in parallel, only projects can run in parallel to each other in MSBuild.

Setting IsShipping false makes sense to me. This may be another thing that is missing now that installer.tasks was moved and doesn't inherit src/installer/Directory.Build.* anymore. (It may have been fixed a different way there?)

tmat · 2019-11-25T19:06:14Z

Well, seems like more things are gonna be broken:

safern · 2019-11-25T19:08:29Z

Setting IsShipping false makes sense to me. This may be another thing that is missing now that installer.tasks was moved and doesn't inherit src/installer/Directory.Build.* anymore. (It may have been fixed a different way there?)

I just searched for that in core-setup and it didn't define it. This target I believe was running differently. This is the change that caused this issue to show up: dotnet/arcade#4379

tmat · 2019-11-25T19:08:40Z

This might be one of the problems:

<MSBuild Projects="@(RepoTaskProjects)"
             Properties="Configuration=Debug;Platform=AnyCPU"
             Targets="Restore;Build"/>

AFAIK, it is not ok to run Restore and Build targets at the same time.

ViktorHofer · 2019-11-25T19:12:24Z

As was stated, you cannot parallelise at the task level or even at the target level. MSBuild only will build projects (i.e. MSBuild project files) in parallel.

from https://stackoverflow.com/questions/997047/how-to-run-tasks-in-parallel-in-msbuild

tmat · 2019-11-25T19:13:41Z

Can we move this project to Arcade repo to avoid this pre-build logic, as the comment in Build.props suggests?

 <!--
    Use this extensibility point to build custom tasks during Build.proj, before any restoring or
    building happens in the repo. These DLLs would ideally live in Arcade and be restored as tools,
    so the idea is building them here is somewhat equivalent.

    Also create the host RID props file so the main build can use it statically.

    Use synthetic inputs/outputs to avoid building it all the time. This should let devs build with
    MSBuild node reuse enabled (the Arcade default). If it were built every time, it would hit file
    locking issues vs. the persistent nodes that loaded the task DLL for the previous build. It
    isn't particularly accurate, but better than nothing.
  -->

safern · 2019-11-25T19:15:23Z

Note that this is also happening for some of coreclr projects:

F:\workspace.5_work\1\s.packages\microsoft.dotnet.arcade.sdk\5.0.0-beta.19574.1\tools\SymStore.targets(57,5): error MSB3073: The command ""F:\workspace.5_work\1\s.packages\microsoft.diasymreader.pdb2pdb\1.1.0-beta2-19521-03\tools\Pdb2Pdb.exe" "F:\workspace.5_work\1\s\artifacts\bin\coreclr\Windows_NT.x86.Debug\runincontext.dll" /out "F:\workspace.5_work\1\s\artifacts\SymStore\Debug\runincontext\netcoreapp3.0\runincontext.pdb" /srcsvrvar SRC_INDEX=public" exited with code 3. [F:\workspace.5_work\1\s\src\coreclr\src\tools\runincontext\runincontext.csproj]

https://github.com/dotnet/runtime/blob/be980b71efadc622b5720a36867696758e59e71c/src/coreclr/src/build.proj#L10

ViktorHofer · 2019-11-25T19:16:47Z

Can we move this project to Arcade repo to avoid this pre-build logic, as the comment in Build.props suggests?

Sure we can (and we should at some point) but how would that solve the issue?

tmat · 2019-11-25T19:23:02Z

Sure we can (and we should at some point) but how would that solve the issue?

I think the problem is in the way how build targets are invoked in Build.props. So, this would eliminate that.

I'd suggest separating Restore and Build for now.

 <MSBuild Projects="@(RepoTaskProjects)"
             Properties="Configuration=Debug;Platform=AnyCPU;__BuildPhase=Restore"
             Targets="Restore"/>
 <MSBuild Projects="@(RepoTaskProjects)"
             Properties="Configuration=Debug;Platform=AnyCPU;__BuildPhase=Build"
             Targets="Build"/>

Like we do in Arcade's build.proj. I remember keeping these together used to cause problems.

ViktorHofer · 2019-11-25T19:25:56Z

I'd suggest separating Restore and Build for now. I remember keeping these together used to case problems.

Thanks but I don't think that's the issue. I just opened the log in sequential order (a new feature in msbuild log viewer):

Also as stated above, there is no parallelism on a target level.

safern · 2019-11-25T19:26:33Z

I'd suggest separating Restore and Build for now.

What about the coreclr projects that are also failing that I pointed above? Those don't have that behavior, so I think that's not the problem.

Should those not be ProjectReferences of build.proj?

tmat · 2019-11-25T19:38:50Z

If that is not the problem then I don't know what is. Up to you to figure out now. I am out of ideas.

dotnet-maestro · 2019-11-25T19:39:19Z

Auto-Merge Status

This pull request has not been merged because Maestro++ is waiting on the following merge policies.

❓ Waiting on checks: runtime-libraries, runtime-coreclr, runtime-coreclr (CoreCLR Pri0 Test Run Windows_NT arm checked), runtime-coreclr (CoreCLR Pri0 Test Run Windows_NT x86 checked), runtime-coreclr (CoreCLR Pri0 Test Run R2R Windows_NT x86 checked), runtime-coreclr (CoreCLR Pri0 Test Run Windows_NT arm64 checked), runtime-coreclr (CoreCLR Pri0 Test Run Linux arm64 checked), runtime-coreclr (CoreCLR Pri0 Test Run Windows_NT x64 checked), runtime-coreclr (CoreCLR Pri0 Test Run R2R Windows_NT x64 checked), runtime-coreclr (CoreCLR Pri0 Test Run Linux arm checked), runtime-coreclr (CoreCLR Pri0 Test Run Linux_musl x64 checked), runtime-coreclr (Test crossgen-comparison Linux arm checked), runtime-libraries (Windows Build x64_Debug)
✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

…125.7 - Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19575.7 - Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19575.7 - Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19575.7 - Microsoft.DotNet.ApiCompat - 5.0.0-beta.19575.7 - Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19575.7 - Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19575.7 - Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19575.7 - Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19575.7 - Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19575.7 - Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19575.7 - Microsoft.DotNet.GenAPI - 5.0.0-beta.19575.7 - Microsoft.DotNet.GenFacades - 5.0.0-beta.19575.7 - Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19575.7 - Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19575.7

ViktorHofer · 2019-12-17T13:46:48Z

I'd suggest separating Restore and Build for now.

@tmat just wanted to say thank you for the recommendation, this helped in another case in coreclr where the design time properties and targets weren't imported correctly. I also found the issue that discusses the problem: dotnet/msbuild#2811

Dotnet-GitSync-Bot added the area-Infrastructure-libraries label Nov 25, 2019

This comment has been minimized.

Sign in to view

Set installer.tasks to IsShipping=false

3c6fe5f

safern force-pushed the darc-master-f0dd7523-7206-4385-9cd1-33e61a9ca531 branch from a911e7c to 3c6fe5f Compare November 25, 2019 19:25

This comment has been minimized.

Sign in to view

safern mentioned this pull request Nov 25, 2019

Disable tests on CoreCLR using new attribute #118

Closed

safern force-pushed the darc-master-f0dd7523-7206-4385-9cd1-33e61a9ca531 branch from 411b452 to a1b3623 Compare November 26, 2019 01:13

safern mentioned this pull request Nov 26, 2019

SymStore targets disabled for System.Private.CoreLib #273

Closed

Temporarily disable SymStore targets for System.Private.CoreLib

4f71435

safern force-pushed the darc-master-f0dd7523-7206-4385-9cd1-33e61a9ca531 branch from a1b3623 to 4f71435 Compare November 26, 2019 08:22

ViktorHofer merged commit 3a9f8ae into master Nov 26, 2019

ViktorHofer deleted the darc-master-f0dd7523-7206-4385-9cd1-33e61a9ca531 branch November 26, 2019 14:35

ghost locked as resolved and limited conversation to collaborators Dec 11, 2020

danmoseley added area-codeflow for labeling automated codeflow and removed area-Infrastructure-libraries labels Jul 28, 2021

[master] Update dependencies from dotnet/arcade #243

[master] Update dependencies from dotnet/arcade #243

Conversation

dotnet-maestro bot commented Nov 25, 2019 • edited Loading

From https://github.com/dotnet/arcade

This comment has been minimized.

ViktorHofer commented Nov 25, 2019

This comment has been minimized.

tmat commented Nov 25, 2019

tmat commented Nov 25, 2019

ViktorHofer commented Nov 25, 2019

tmat commented Nov 25, 2019

This comment has been minimized.

safern commented Nov 25, 2019

safern commented Nov 25, 2019

This comment has been minimized.

safern commented Nov 25, 2019

tmat commented Nov 25, 2019 • edited Loading

tmat commented Nov 25, 2019

This comment has been minimized.

safern commented Nov 25, 2019

This comment has been minimized.

safern commented Nov 25, 2019 • edited Loading

safern commented Nov 25, 2019

This comment has been minimized.

dagood commented Nov 25, 2019

This comment has been minimized.

tmat commented Nov 25, 2019

safern commented Nov 25, 2019

tmat commented Nov 25, 2019

This comment has been minimized.

ViktorHofer commented Nov 25, 2019 • edited Loading

tmat commented Nov 25, 2019 • edited Loading

This comment has been minimized.

safern commented Nov 25, 2019

ViktorHofer commented Nov 25, 2019

This comment has been minimized.

tmat commented Nov 25, 2019 • edited Loading

This comment has been minimized.

ViktorHofer commented Nov 25, 2019

safern commented Nov 25, 2019 • edited Loading

This comment has been minimized.

tmat commented Nov 25, 2019

dotnet-maestro bot commented Nov 25, 2019 • edited Loading

Auto-Merge Status

ViktorHofer commented Dec 17, 2019

dotnet-maestro bot commented Nov 25, 2019 •

edited

Loading

tmat commented Nov 25, 2019 •

edited

Loading

safern commented Nov 25, 2019 •

edited

Loading

ViktorHofer commented Nov 25, 2019 •

edited

Loading

tmat commented Nov 25, 2019 •

edited

Loading

tmat commented Nov 25, 2019 •

edited

Loading

safern commented Nov 25, 2019 •

edited

Loading

dotnet-maestro bot commented Nov 25, 2019 •

edited

Loading