fix(test runner): failed + skipped = flaky

[dgozman]

Fixes #17652.

[github-actions]

Test results for "tests 1"

7 failed
❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry
❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry
❌ [playwright-test] › expect-poll.spec.ts:222:5 › should show intermediate result for poll that spills over test time
❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry
❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry
❌ [playwright-test] › reporter-html.spec.ts:1452:7 › merged › labels › filter should update stats
❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry

8 flaky
⚠️ [chromium] › library/tracing.spec.ts:239:5 › should not include trace resources from the previous chunks
⚠️ [firefox] › page/page-goto.spec.ts:482:3 › should succeed on url bar navigation when there is pending navigation
⚠️ [playwright-test] › ui-mode-trace.spec.ts:53:5 › should merge web assertion events
⚠️ [playwright-test] › ui-mode-trace.spec.ts:126:5 › should show snapshots for sync assertions
⚠️ [chromium] › library/tracing.spec.ts:239:5 › should not include trace resources from the previous chunks
⚠️ [webkit] › library/browsercontext-reuse.spec.ts:50:1 › should reset serviceworker
⚠️ [playwright-test] › ui-mode-trace.spec.ts:53:5 › should merge web assertion events
⚠️ [playwright-test] › ui-mode-trace.spec.ts:126:5 › should show snapshots for sync assertions

25044 passed, 583 skipped
✔️✔️✔️

❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry

Error: expect(received).toBe(expected) // Object.is equality

Expected: 1
Received: 0

  125 |     `,
  126 |   }, { retries: 1 });
> 127 |   expect(result.exitCode).toBe(1);
      |                           ^
  128 |   expect(result.passed).toBe(2);
  129 |   expect(result.flaky).toBe(1);
  130 |   expect(result.failed).toBe(1);

    at /Users/runner/work/playwright/playwright/tests/playwright-test/test-serial.spec.ts:127:27

❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry

Error: expect(received).toBe(expected) // Object.is equality

Expected: 1
Received: 0

  125 |     `,
  126 |   }, { retries: 1 });
> 127 |   expect(result.exitCode).toBe(1);
      |                           ^
  128 |   expect(result.passed).toBe(2);
  129 |   expect(result.flaky).toBe(1);
  130 |   expect(result.failed).toBe(1);

    at /home/runner/work/playwright/playwright/tests/playwright-test/test-serial.spec.ts:127:27

❌ [playwright-test] › expect-poll.spec.ts:222:5 › should show intermediate result for poll that spills over test time

Error: expect(received).toContain(expected) // indexOf

Expected substring: "Expected: 2"
Received string:    "
Running 1 test using 1 worker·
  ✘  1 a.spec.ts:3:11 › should fail (2.0s)··
  1) a.spec.ts:3:11 › should fail ──────────────────────────────────────────────────────────────────·
    Test timeout of 2000ms exceeded.·
  1 failed
    a.spec.ts:3:11 › should fail ───────────────────────────────────────────────────────────────────
"

  230 |   }, { timeout: 2000 });
  231 |   expect(result.exitCode).toBe(1);
> 232 |   expect(result.output).toContain('Expected: 2');
      |                         ^
  233 |   expect(result.output).toContain('Received: 3');
  234 | });
  235 |

    at /home/runner/work/playwright/playwright/tests/playwright-test/expect-poll.spec.ts:232:25

❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry

Error: expect(received).toBe(expected) // Object.is equality

Expected: 1
Received: 0

  125 |     `,
  126 |   }, { retries: 1 });
> 127 |   expect(result.exitCode).toBe(1);
      |                           ^
  128 |   expect(result.passed).toBe(2);
  129 |   expect(result.flaky).toBe(1);
  130 |   expect(result.failed).toBe(1);

    at /home/runner/work/playwright/playwright/tests/playwright-test/test-serial.spec.ts:127:27

❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry

Error: expect(received).toBe(expected) // Object.is equality

Expected: 1
Received: 0

  125 |     `,
  126 |   }, { retries: 1 });
> 127 |   expect(result.exitCode).toBe(1);
      |                           ^
  128 |   expect(result.passed).toBe(2);
  129 |   expect(result.flaky).toBe(1);
  130 |   expect(result.failed).toBe(1);

    at /home/runner/work/playwright/playwright/tests/playwright-test/test-serial.spec.ts:127:27

❌ [playwright-test] › reporter-html.spec.ts:1452:7 › merged › labels › filter should update stats

Error: expect(received).toBe(expected) // Object.is equality

Expected: "Total time: 1270ms"
Received: "Total time: 1.3s"

  1774 |         await expect(page.locator('.chip', { hasText: 'c.test.js' })).toHaveCount(1);
  1775 |
> 1776 |         await flakyButton.click();
       |                                 ^
  1777 |
  1778 |         await expect(searchInput).toHaveValue('@regression @flaky');
  1779 |         await expect(page).toHaveURL(/@regression%20@flaky/);

    at checkTotalDuration (D:\a\playwright\playwright\tests\playwright-test\reporter-html.spec.ts:1776:33)
    at D:\a\playwright\playwright\tests\playwright-test\reporter-html.spec.ts:1792:9

❌ [playwright-test] › test-serial.spec.ts:98:5 › test.describe.serial should work with retry

Error: expect(received).toBe(expected) // Object.is equality

Expected: 1
Received: 0

  125 |     `,
  126 |   }, { retries: 1 });
> 127 |   expect(result.exitCode).toBe(1);
      |                           ^
  128 |   expect(result.passed).toBe(2);
  129 |   expect(result.flaky).toBe(1);
  130 |   expect(result.failed).toBe(1);

    at D:\a\playwright\playwright\tests\playwright-test\test-serial.spec.ts:127:27

Merge workflow run.

[github-actions]

Test results for "tests 1"

13 flaky
⚠️ [chromium] › page/page-event-request.spec.ts:101:3 › should report navigation requests and responses handled by service worker
⚠️ [chromium] › library/tracing.spec.ts:239:5 › should not include trace resources from the previous chunks
⚠️ [chromium] › library/tracing.spec.ts:395:14 › should produce screencast frames fit
⚠️ [chromium] › page/page-event-request.spec.ts:101:3 › should report navigation requests and responses handled by service worker
⚠️ [chromium] › page/page-event-request.spec.ts:130:3 › should report navigation requests and responses handled by service worker with routing
⚠️ [chromium] › library/tracing.spec.ts:239:5 › should not include trace resources from the previous chunks
⚠️ [firefox] › page/page-event-request.spec.ts:162:3 › should return response body when Cross-Origin-Opener-Policy is set
⚠️ [playwright-test] › ui-mode-trace.spec.ts:22:5 › should merge trace events
⚠️ [playwright-test] › ui-mode-trace.spec.ts:53:5 › should merge web assertion events
⚠️ [playwright-test] › ui-mode-trace.spec.ts:126:5 › should show snapshots for sync assertions
⚠️ [chromium] › library/tracing.spec.ts:239:5 › should not include trace resources from the previous chunks
⚠️ [webkit] › library/browsercontext-reuse.spec.ts:50:1 › should reset serviceworker
⚠️ [playwright-test] › ui-mode-trace.spec.ts:80:5 › should merge screenshot assertions

25046 passed, 583 skipped
✔️✔️✔️

Merge workflow run.