[ui, ci] retain artifacts from test runs including test timing

[philrenaud]

Description

Our ember-test-audit action is currently failing, and I'm using that as an excuse to improve our front-end testing story generally.

This will be a two-step approach, and this PR is the first step:

1. Save artifacts, including test timing, from test-ui test results on merge to main
2. Compare to a rolling window of these test results on subsequent PRs

This PR adds artifact uploads on the back of test-ui. Because those are partitioned/run concurrently 4x, part of this is combining those.

The artifact generated looks like this:

{
  "timestamp": "2024-12-02T18:20:59.531Z",
  "sha": "7fd10531ef3e12672a19a7a9aa7a6e4f4f874614",
  "summary": {
    "total": 1606,
    "passed": 1606,
    "failed": 0
  },
  "duration": 841403,
  "tests": [
    {
      "name": "Acceptance | allocation fs: it passes an accessibility audit",
      "partition": 1,
      "browser": "Chrome 131.0",
      "passed": true,
      "duration": 1331,
      "error": null,
      "logs": []
    },
    {
      "name": "Acceptance | allocation fs: visiting filesystem root",
      "partition": 1,
      "browser": "Chrome 131.0",
      "passed": true,
      "duration": 505,
      "error": null,
      "logs": []
    },
    {
      "name": "Acceptance | allocation fs: visiting filesystem paths",
      "partition": 1,
      "browser": "Chrome 131.0",
      "passed": true,
      "duration": 542,
      "error": null,
      "logs": []
    },
...

In phase 2 of this, we'll be able to do test-by-test duration averages and comparisons, and tease out those tests with the greatest deltas, find flakes, etc.

[philrenaud]

no-op to trigger the

  pull_request:
    paths:
      - "ui/**"

part of the action.

[github-actions]

Ember Test Audit comparison

	main	41c9725	change
passes	1580	1580	0
failures	5	5	0
flaky	0	0	0
duration	000ms	000ms	-000ms

[tgross]

LGTM

[tgross]

LGTM!

[DingoEatingFuzz]

Super cool to see this work get picked up again 🤩

[DingoEatingFuzz]

Will there be anything else in the test-results directory other than subdirectories for each partition?

I wonder if you can walk the dir tree instead of relying on synchronizing NUM_PARTITIONS and the GHA matrix config.

[philrenaud]

Yeah, I left a pretty verbose dir trail here with the hope that I'd populate this with further artifacts / test-run metadata. No explicit plans yet, though!

[DingoEatingFuzz]

Doing a combine step that takes json and emits json is really nice. Should prove to be more flexible than trying to do all the collating, aggregating, and analysis in one script.