Update SDK #43028
Update SDK #43028
Conversation
MackinnonBuck
added
the
area-infrastructure
|
Hey @dotnet/aspnet-build, looks like this PR is something you want to take a look at. |
|
The Not totally sure what the cause is at the moment. See https://dev.azure.com/dnceng/public/_build/results?buildId=1916161&view=logs&j=2f0d093c-1064-5c86-fc5b-b7b1eca8e66a&t=d2b639a6-cb19-5f1f-66fd-8047f66b3026&l=126 |
|
I'd recommend checking out the binlog to see if there's more info (here). Looks like it's only in the source-build leg, so maybe there was a relevant change there in dotnet/sdk or dotnet/installer recently. @MichaelSimons might know more |
|
cc @dotnet/source-build-internal |
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
I'll try a later SDK until this works. |
|
@Tratcher this SDK didn't work. Suggest watching https://maestro-prod.westus2.cloudapp.azure.com/2237/https:%2F%2Fgithub.com%2Fdotnet%2Finstaller/latest/graph for a build that includes dotnet/msbuild@f2d13c6. Latest SDK (which, confusingly, we get from the dotnet/installer repo) includes msbuild commits from 6 days ago but we want a commit from 4 days ago. |
Tratcher
added
the
blocked
|
Updated to |
Just did a trace and I believe |
😞 unfortunately not. Source build still failing. |
|
Looks like we have a new breaking change now:
@JamesNK @BrennanConroy would you happen to have context on this? |
I just discovered this in source-build and logged an issue for it - #43253 |
|
I've added a possible fix for the warning - #43259 It might not fix the warning - ASP.NET Core isn't using the same SDK as source build so I can't test it - but if it's still a problem, then the follow-up change should be a minor fix. |
|
Might be easier to test your commit in this PR's branch @JamesNK |
|
@vitek-karas @sbomer What's going on with IL2121? A lot of them have shown up in the aspnetcore build. I tried to remove some of the suppressions, but they were correctly suppressing warnings. I see you disabled this warning in dotnet/runtime@ Should the incorrect suppression warning be suppressed because it's incorrect? 😆 |
|
IL2121 is off by default, but that "off by default" part is in SDK - which runtime or asp.net don't use to run the linker 😢 That said I would be interested to see the cases where the suppression is necessary but linker produces IL2121 still. The only place where we've seen that happening was either around feature switches, or in code which uses ifdefs. /cc @jkurdek |
|
@captainsafia you're right and I think the latest build hit an outage or two. |
|
Build w/ @wtgodbe's latest SDK update had already failed w/ usual source-build problem when I pushed my newest fixup. Earlier placement of the job variables and even-earlier attempt to control the source-build job timeout wasn't quite right. |
|
And … source-build finally worked 🎉 |
|
@javiercn @TanayParikh @MackinnonBuck any ideas? rerunning for now. |
|
As @captainsafia commented earlier in response to my "Who has a good understanding of these issues or the specific tests❔" comment, it appears our old @dsplaisted @marcpopMSFT who are the best contacts for |
ghost
added this to the
|
Note we need to track fixes for the "exit code 139" and |
|
Put another way, #43581 tracks one problem we're going to see sporadically in main until something makes updates Separately, dotnet/runtime#68393 (or some of the issues it references) has regressed again or we've hit a new variant of the problem. @jkotas @rainersigwald @radical @lewing @bartonjs @vcsjones @karelz you were all involved in some of those We may need a new issue to track addressing this. Thoughts❔ |
This is common failure mode with many different root causes. We need either crash dump or repro steps to make progress on it. |
|
Hi @jkotas. It looks like you just commented on a closed PR. The team will most probably miss it. If you'd like to bring something important up to their attention, consider filing a new issue and add enough details to build context. |
|
@jkotas there's a dump at https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-aspnetcore-refs-pull-43028-merge-9a8cf838a9654554a1/Templates.Mvc.Tests--net7.0/1/core.1000.19678?helixlogtype=result and likely one in the artifacts for the test(s) that failed during the first attempt at the last build of this PR. |
|
The stacktrace of the crash is: This is dup of dotnet/runtime#68393. It is bug in Ubuntu https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1983100 that is in the process of being fixed. |
|
Hi @jkotas. It looks like you just commented on a closed PR. The team will most probably miss it. If you'd like to bring something important up to their attention, consider filing a new issue and add enough details to build context. |
|
Will the problem automatically go away @jkotas if we migrate away from Ubuntu 18.04 build / test agents❔ I doubt that's viable in general (yet) but we at least plan to stop using that version for our |
|
The only persistent SIGSEGV that I personally know about is unique to Ubuntu 18.04, and is caused by tools having background TLS connections still running when the process exits. Ubuntu has a bug on it (https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1983100), but it sadly hasn't hit an acceptable final release yet. So... yes, moving off of Ubuntu 18.04 (or their fix finally rolling out) would probably take the segfaults away. |
|
Hi @bartonjs. It looks like you just commented on a closed PR. The team will most probably miss it. If you'd like to bring something important up to their attention, consider filing a new issue and add enough details to build context. |
|
In a separate bit of follow-up, I'm looking at including @rainersigwald @Forgind and @baronfel, environment variables are still logged in full in recent release/7.0 builds. Is that likely to change as new |
|
Yes; the change should flow with the new MSBuild, so if it's causing trouble for you, you should target release/7.0 with that. We're also discussing the best way to communicate clearly if we make certain environment variables always logged. If you're interested, I'd be happy to add you to that conversation. |
|
Hi @Forgind. It looks like you just commented on a closed PR. The team will most probably miss it. If you'd like to bring something important up to their attention, consider filing a new issue and add enough details to build context. |
I probably don't have much more to add beyond my "regression" thoughts above but sure… |
- should affect most of our pipelines though not a few post-build jobs - as we saw in dotnet#43028, binary logs no longer include all environment variables by default - this change captures the environment variables not visibly referenced in our projects etc. - fix avoids a need to add the override later to debug an issue, at the cost of larger .binlog files
|
#43644 opened, targeting release/7.0. I'll flow automatically (modulo an expected merge conflict) into main. |
…#43644) - should affect most of our pipelines though not a few post-build jobs - as we saw in #43028, binary logs no longer include all environment variables by default - this change captures the environment variables not visibly referenced in our projects etc. - fix avoids a need to add the override later to debug an issue, at the cost of larger .binlog files
No description provided.