Suggestion: split dotnet-test-cloud.ps1 functionality into test and publishing

[SteveBush]

The dotnet test and publishing of test results are combined in dotnet-test-cloud.ps1. I have a set of tests that currently hang on the MacOS Sequoia build runner. My working theory is several of the Xunit v3 network discovery tests trigger a Local Network Policy dialog because they access the local network. The first test to access the network triggers the hang, but the hanging test isn't consistent. Also, the --blame-hang-timeout 120s parameter to dotnet test isn't being respected for some reason.

So, I added a TestingTimeout parameter to dotnet.yml and a JobTimeout to build.yml. The JobTimeout controls the overall timeout for each job. The TestingTimeout parameter controls the timeout for just the test portion. This prevents an infinite job, and having the JobTimeout > the TestingTimeout allows the rest of the failed pipeline to run before the JobTimeout is reached.

- powershell: tools/dotnet-test-cloud.ps1 -Configuration $(BuildConfiguration) -Agent $(Agent.JobName) -PublishResults -ProductionBuild ${{ parameters.ProductionBuild }}
  displayName: 🧪 dotnet test
  condition: and(succeeded(), ${{ parameters.RunTests }})
  timeoutInMinutes: ${{ parameters.TestingTimeout }}

Unfortunately, the test execution and publishing are combined in dotnet-test-cloud.ps1, so none of the test results get published even if just one test hangs. Separating the test and publication functionality would allow me to set separate timeout and error condition behavior for testing and publication.

I was going to just split dotnet-test-cloud.ps1 into dotnet-test-cloud.ps1 and dotnet-test-publish-cloud.ps1.

[AArnott]

That sounds reasonable. Wanna send a PR?

Please name the new file dotnet-test-cloud-publish.ps1 so that the names share a longer common prefix.

[SteveBush]

Sounds good. There are a couple of ways I could do this. I'm leaning towards options 2 or 3.

Option 1: Full separation but not backward compatible.

• Add dotnet-test-cloud-publish.ps1
• Remove -PublishResults switch and test publishing functionality from dotnet-test-cloud.ps1
• Add just the test publishing code to dotnet-test-cloud-publish.ps1

Option 2: Backward compatible, but test publishing duplicate code.

• No changes dotnet-test-cloud.ps1 identical for backward compatibility. Just don't pass the -PublishResults switch if you want to have separate test and publish behaviors.
• Add dotnet-test-cloud-publish.ps1
• Add just the test publishing code to dotnet-test-cloud-publish.ps1

Option 3: dotnet-test-cloud.ps1 calls dotnet-test-cloud.-publish.ps1

• Add dotnet-test-cloud-publish.ps1
• Add test publishing to dotnet-test-cloud-publish.ps1 from
• Modify dotnet-test-cloud.ps1 to call dotnet-test-cloud-publish.ps1 when the -PublishResults switch is set

[AArnott]

Let's go with option 1. Update the AzP caller to call both scripts (as a single shell script task). Do not update the GitHub workflow to call the new script since the publish step only applied to azp anyway.
For that matter, the new script should go into the azure-pipelines folder since it does nothing for github workflows.