[6.0] HostActivation Tests windows x86 build failure

[carlossanlop]

• Found in a release/6.0 PR: https://github.com/dotnet/runtime/pull/81238/checks?check_run_id=11208849014
• Queue: Installer Build and Test coreclr windows_x86 Debug
• Step: Build
• Job result: https://dev.azure.com/dnceng-public/public/_build/results?buildId=163519&view=logs&j=f827f174-7a88-54e3-54c2-8dcd28f39edd&t=5bc77b52-54d3-5da9-c79e-6dacacea45ab&l=750

Error message:

XUnit : error : Tests failed: D:\a\_work\1\s\artifacts\TestResults\Debug\HostActivation.Tests_net6.0_x86.html [net6.0|x86] [D:\a\_work\1\s\src\installer\tests\HostActivation.Tests\HostActivation.Tests.csproj]
##[error]XUnit(0,0): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Tests failed: D:\a\_work\1\s\artifacts\TestResults\Debug\HostActivation.Tests_net6.0_x86.html [net6.0|x86]

Known Issue Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "System.UnauthorizedAccessException : Access to the path 'dotnet.exe' is denied",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

• Found in a release/6.0 PR: https://github.com/dotnet/runtime/pull/81238/checks?check_run_id=11208849014
• Queue: Installer Build and Test coreclr windows_x86 Debug
• Step: Build
• Job result: https://dev.azure.com/dnceng-public/public/_build/results?buildId=163519&view=logs&j=f827f174-7a88-54e3-54c2-8dcd28f39edd&t=5bc77b52-54d3-5da9-c79e-6dacacea45ab&l=750

Error message:

XUnit : error : Tests failed: D:\a\_work\1\s\artifacts\TestResults\Debug\HostActivation.Tests_net6.0_x86.html [net6.0|x86] [D:\a\_work\1\s\src\installer\tests\HostActivation.Tests\HostActivation.Tests.csproj]
##[error]XUnit(0,0): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Tests failed: D:\a\_work\1\s\artifacts\TestResults\Debug\HostActivation.Tests_net6.0_x86.html [net6.0|x86]

{
  "ErrorMessage": "",
  "BuildRetry": false,
  "ErrorPattern": "Tests failed: .+\\HostActivation\.Tests_net6\.0_x86\.html",
  "ExcludeConsoleLog": false
}

Author:	carlossanlop
Assignees:	-
Labels:	area-Host
Milestone:	-

[vitek-karas]

The actual failure is:

System.UnauthorizedAccessException : Access to the path 'dotnet.exe' is denied.
   at System.IO.FileSystem.RemoveDirectoryRecursive(String fullPath, WIN32_FIND_DATA& findData, Boolean topLevel)
   at System.IO.FileSystem.RemoveDirectory(String fullPath, Boolean recursive)
   at Microsoft.DotNet.CoreSetup.Test.HostActivation.MultilevelSDKLookup.Dispose() in /_/src/installer/tests/HostActivation.Tests/MultilevelSDKLookup.cs:line 87
   at ReflectionAbstractionExtensions.DisposeTestClass(ITest test, Object testClass, IMessageBus messageBus, ExecutionTimer timer, CancellationTokenSource cancellationTokenSource) in /_/src/xunit.execution/Extensions/ReflectionAbstractionExtensions.cs:line 87

This looks a lot like robustness things we've been working on for a while and sort of solved in 7/8 where I don't remember seeing failures like this at all recently. I don't know if it's worth the effort to try to backport all of these test infra fixes.

Short version: I'm pretty sure this is not a product issue, but it's a test infra issue.

@elinor-fung who also worked on the test infra robustness for another opinion.

[elinor-fung]

Yeah, this looks like test robustness issues we were chasing (and haven't seen recently) - not anything in the product. I'm also not sure if it is worth backporting the various fixes we've made, particularly since the failures are on cleanup, so by the time they are hit, the actual validation has been done/passed.

@carlossanlop if this starts happening in a lot in 6.0 PRs (hoping not based on the current hit count) and it slows you down too much, I can investigate what we would need to backport.

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

• Found in a release/6.0 PR: https://github.com/dotnet/runtime/pull/81238/checks?check_run_id=11208849014
• Queue: Installer Build and Test coreclr windows_x86 Debug
• Step: Build
• Job result: https://dev.azure.com/dnceng-public/public/_build/results?buildId=163519&view=logs&j=f827f174-7a88-54e3-54c2-8dcd28f39edd&t=5bc77b52-54d3-5da9-c79e-6dacacea45ab&l=750

Error message:

XUnit : error : Tests failed: D:\a\_work\1\s\artifacts\TestResults\Debug\HostActivation.Tests_net6.0_x86.html [net6.0|x86] [D:\a\_work\1\s\src\installer\tests\HostActivation.Tests\HostActivation.Tests.csproj]
##[error]XUnit(0,0): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Tests failed: D:\a\_work\1\s\artifacts\TestResults\Debug\HostActivation.Tests_net6.0_x86.html [net6.0|x86]

Known Issue Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "",
  "BuildRetry": false,
  "ErrorPattern": "Tests failed: .+\\HostActivation\.Tests_net6\.0_x86\.html",
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Author:	carlossanlop
Assignees:	-
Labels:	arch-x86, os-windows, area-Infrastructure-coreclr, Known Build Error
Milestone:	-

[hoyosjs]

I didn't update the error blob, it's a bit too broad and will catch any error in the assembly. But the correct encoding was:

{
  "BuildRetry": false,
  "ErrorPattern": "Tests failed: .+\\\\HostActivation\\.Tests_net\\d\\.0_(x64|x86|arm64)+\\.html",
  "ExcludeConsoleLog": false
}

[carlossanlop]

@hoyosjs I changed the error blob to what you suggested.

Still seeing this failure. It was hit in this 6.0 PR: #86295

[akoeplinger]

The error pattern seems to match all failures in the installer tests, e.g. this main PR https://github.com/dotnet/runtime/pull/85913/checks?check_run_id=13522623704 even though that fails with a different failure.

[hoyosjs]

Sadly, multiline regex isn't supported to glob that it's hosting and comprises activation issues...

[carlossanlop]

Ok let me revert the pattern and use x86 specifically.
I'll create a new issue for the x64 one happening in main.

[elinor-fung]

I updated the error pattern to reflect the specific UnauthorizedAccessException that was being hit in 6.0.
The table in the description still has all the ones that were unintentionally linked before, but I think it should be filtered to what we want going forward.

[carlossanlop]

@elinor-fung there's another hit in 6.0, specifically in the branding PR for 6.0.26: #94482