[Merge-on-Red] - Accurately log catastrophic test failures and freezes in source-gened test infrastructure

[ivdiazsa]

Merge-on-Red Subproject

Tests should have failures logged in a format that Build Analysis can easily reasoned about and surfaced to the check tab.

Goals

• Be able to identify and record catastrophic failures to Build Analysis:
  • Native Crashes
  • Hangs and Timeouts

Work Items

• Look into the other format (other than trx) used by dotnet test and understand how it works.
• A Yaml version was decided, so it has to be implemented in the test source generators. Helix compatibility with older OS's would be broken by adding new stuff, so this work item was changed to a XML log fixer.
• Implement accordingly so the Helix script can easily understand and surface it.
• Implement a watchdog to be able to easily identify when a test has taken too long and/or frozen. Suggested ideas:
  * [ ] Add it as a new feature in corerun: Create a new flag like --timeout. Have corerun exit and report the failure if this value is exceeded.
  • Add it as a new mini-app: This mini-app will receive the test's corerun command, and the timeout value as parameters. It will spawn a separate process to run the test in, and monitor it itself. If it exceeds the time limit, then i
    t will kill the process and report the failure accordingly.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Merge-on-Red Subproject

Tests should have failures logged in a format that Build Analysis can easily reasoned about and surfaced to the check tab.

Goals

• Be able to identify and record catastrophic failures to Build Analysis:
  • Native Crashes
  • Hangs and Timeouts

Work Items

• Look into the other format (other than trx) used by dotnet test and understand how it works.
• Implement accordingly so the Helix script can easily understand and surface it.
• Implement a watchdog to be able to easily identify when a test has taken too long and/or frozen. Suggested ideas:
  • Add it as a new feature in corerun: Create a new flag like --timeout. Have corerun exit and report the failure if this value is exceeded.
  • Add it as a new mini-app: This mini-app will receive the test's corerun command, and the timeout value as parameters. It will spawn a separate process to run the test in, and monitor it itself. If it exceeds the time limit, then i
    t will kill the process and report the failure accordingly.

Author:	ivdiazsa
Assignees:	-
Labels:	area-Infrastructure-coreclr, untriaged
Milestone:	-