Test Failure:System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated

[VincentBu]

Run runtime-libraries-coreclr outerloop 20210711.1
Failed Test::

net6.0-OSX-Release-x64-CoreCLR_release-OSX.1013.Amd64.Open

- System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated(useRootGroups: True)
- System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated(useRootGroups: False)

Error message:

Half-way through waiting for remote process.
Timed out at 7/11/2021 6:14:21 AM after 60000ms waiting for remote process.
Process ID: 48358
Handle: 984
Name:
MainModule: /usr/bin/sudo


Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 57
   at System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated(Boolean useRootGroups) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 605

[ghost]

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run runtime-libraries-coreclr outerloop 20210711.1
Failed Test::

net6.0-OSX-Release-x64-CoreCLR_release-OSX.1013.Amd64.Open

- System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated(useRootGroups: True)
- System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated(useRootGroups: False)

Error message:

Half-way through waiting for remote process.
Timed out at 7/11/2021 6:14:21 AM after 60000ms waiting for remote process.
Process ID: 48358
Handle: 984
Name:
MainModule: /usr/bin/sudo


Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 57
   at System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated(Boolean useRootGroups) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 605

Author:	VincentBu
Assignees:	-
Labels:	arch-x64, area-System.Diagnostics.Process, untriaged
Milestone:	-

[danmoseley]

this is failing 1/4 of the time on mac. I'll disable it or something.

TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
| where Finished >= now(-30d)
| where Type == "System.Diagnostics.Tests.ProcessTests"
| where Branch !startswith "refs/pull"
| where Method == "TestCheckChildProcessUserAndGroupIdsElevated"
| summarize count() by Method, Result, QueueName

https://engsrvprod.kusto.windows.net/engineeringdata?query=H4sIAAAAAAAEAH2PTU%2fDMAyG75P2H0xPmzTaTXDgUqQxGCoSiI8hzmliSGiaVHFCVcSPp6SDCglxsiy%2ffvx4h%2bTvkYL2NJ18wKtVBiplRK6MQQdP1lWFx5rAmp%2bmEH9Fr2wZU30dAq1Eh7BVRpFEAac5GNvODo%2bWYj6Od12DkOeQPHTUo9NzxV6MJa84pbvejdJbZzkSxSaZTrLse3XQjstbpnQyQs8cM1zCAXnmPLXKS0gcPlPWBK33DAp1zZx6R%2bA2GD%2bbQ9nBNXppxQLuAgZca8VohA6zeO7LZSORVxuptNgLPhK6tRGXzoamEHSh8Y15FL%2bUI%2feG1biKnFD2l0O6Olkep8z1OlymtkETX%2flXcPh9MQI%2fAQDWMcvIAQAA&web=0

Method	Result	QueueName	count_
TestCheckChildProcessUserAndGroupIdsElevated	Fail	osx.1013.amd64.open	26
TestCheckChildProcessUserAndGroupIdsElevated	Fail	osx.1014.amd64.open	26
TestCheckChildProcessUserAndGroupIdsElevated	Pass	osx.1013.amd64.open	72
TestCheckChildProcessUserAndGroupIdsElevated	Pass	osx.1014.amd64.open	72
TestCheckChildProcessUserAndGroupIdsElevated	Pass	ubuntu.1804.armarch.open	128
TestCheckChildProcessUserAndGroupIdsElevated	Pass	redhat.7.amd64.open.svc	128
TestCheckChildProcessUserAndGroupIdsElevated	Pass	sles.15.amd64.open.svc	128
TestCheckChildProcessUserAndGroupIdsElevated	Pass	centos.7.amd64.open.svc	128
TestCheckChildProcessUserAndGroupIdsElevated	Pass	ubuntu.1804.amd64.open.svc	128
TestCheckChildProcessUserAndGroupIdsElevated	Pass	debian.9.amd64.open.svc	128
TestCheckChildProcessUserAndGroupIdsElevated	Pass	sles.12.amd64.open.svc	128
TestCheckChildProcessUserAndGroupIdsElevated	Pass	ubuntu.1604.amd64.open.svc	772

[danmoseley]

Noticing that when TestCheckChildProcessUserAndGroupIdsElevated(useRootGroups: True) times out this way, TestCheckChildProcessUserAndGroupIdsElevated(useRootGroups: False) always does, and vice versa.
It's not machine specific - same machines pass sometimes as fail sometimes.

If it was that we were running this test even when not root, or sudo was not present or broken, then I would expect the named pipe test Connection_UnderDifferentUsers_BehavesAsExpected to fail as well. (Having said that I don't see any failures ever for that one. We don't store passes, so I can't say for certain it's running.)
If sudo doesn't exist I would also assume the process launch would fail.
@tmds any thoughts? I wonder if there is child console output we are not capturing.

[danmoseley]

Noticed it only failed between 7/8 and 7/15.
Searched for commits by @tmds since it was his test and found #55645 which was introduced and fixed on those dates.

[danmoseley]

@safern, do our runs in PR validation always define [Trait(XunitConstants.Category, XunitConstants.RequiresElevation)] (ie run as root)? I'm not sure how that trait "works". Eg., in a test run that ran this test, I see only

/tmp/helix/working/9F500890/p/dotnet exec --runtimeconfig System.Diagnostics.Process.Tests.runtimeconfig.json --depsfile System.Diagnostics.Process.Tests.deps.json xunit.console.dll System.Diagnostics.Process.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing

Some but not all of our tests that have RequiresElevation and run on Unix have [OuterLoop] // Depends on sudo but I do'nt know what that has to do with sudo. They should only need RequiresElevation. If our PR runs always run as root, and I can remove [OuterLoop], then this test would never have gotten broken.

[safern]

Not really. It seems like that trait is for when we want to execute tests that require elevation and make sure that they run (and if we are not elevated fail, like in this case).

For other tests it seems like we detect if we are elevated and if so then run them, otherwise skip them. We seem to be using ConditionalFact.

https://github.com/dotnet/runtime/search?q=Elevated

[danmoseley]

I see so [Trait(XunitConstants.Category, XunitConstants.RequiresElevation)] is just a convenience, it does nothing except make it possible to manually pass -trait category=requireselevation so that you run only these tests. My guess is that nobody does this and it's gotten pasted around because it looks like it causes the test to skip unless elevated, which is not the case. It's basically useless.

Tests that need sudo/elevation should use [ConditionalFact(nameof(IsProcessElevated))] although many choose to run either way and catch and test for rootness.

So [OuterLoop("Needs sudo access")] [OuterLoop] // Depends on sudo etc doesn't make a lot of sense, needing sudo shouldn't affect whether it's outer loop or not. Unless outer loop tests always run as root? That would explain why this test is now passing even thought it doesn't use ConditionalFact.

How can I tell which configurations run as root and which don't?