-
Notifications
You must be signed in to change notification settings - Fork 29.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: report ERROR wpt test results #50429
Conversation
780117a
to
111c759
Compare
@panva It would be great if you could take a look at this when you get a chance since you originally created wpt.fyi report workflow. Thank you! |
I believe it should just be the harness that has an error and then no tests in it, like so https://wpt.fyi/results/streams/readable-byte-streams/construct-byob-request.any.html?run_id=5105883752693760&run_id=5146271209226240&run_id=5855453021470720&run_id=5148705381941248&run_id=6261279079006208 Same for timeout in our case. |
Right. I've removed the synthetic sub-tests from the report to wpt.fyi. Thanks for the suggestion. |
I'll re-run the upload tonight to check what it looks like now. |
The evaluation in WPTRunner.runJsTests() looks alright now, it's gone. But why is there a newly added for entry for all |
When a wpt test file is exited for uncaught error, its result should be recorded in the `wptreport.json` and uploaded to wpt.fyi. For instance, `html/webappapis/timers/evil-spec-example.any.js` is exited for uncaught error in Node.js but it shows as "MISSING" at https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.
9e65978
to
e842497
Compare
Well, it seems a completion with error status was not recognized as a failed test and it was not reported to wpt.fyi. Fixed with e842497. |
Commit Queue failed- Loading data for nodejs/node/pull/50429 ✔ Done loading data for nodejs/node/pull/50429 ----------------------------------- PR info ------------------------------------ Title test: report ERROR wpt test results (#50429) ⚠ Could not retrieve the email or name of the PR author's from user's GitHub profile! Branch legendecas:wpt/report -> nodejs:main Labels test, author ready, needs-ci, commit-queue-rebase Commits 2 - test: report error wpt test results - test: recognize wpt completion error Committers 1 - legendecas PR-URL: https://github.com/nodejs/node/pull/50429 Reviewed-By: Filip Skokan ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/50429 Reviewed-By: Filip Skokan -------------------------------------------------------------------------------- ℹ This PR was created on Fri, 27 Oct 2023 15:23:01 GMT ✔ Approvals: 1 ✔ - Filip Skokan (@panva): https://github.com/nodejs/node/pull/50429#pullrequestreview-1704170475 ✘ This PR needs to wait 0 more hours to land (or 0 hours if there is one more approval) ✔ Last GitHub CI successful ℹ Last Full PR CI on 2023-11-01T06:12:59Z: https://ci.nodejs.org/job/node-test-pull-request/55378/ - Querying data for job/node-test-pull-request/55378/ ✔ Last Jenkins CI successful -------------------------------------------------------------------------------- ✔ Aborted `git node land` session in /home/runner/work/node/node/.ncuhttps://github.com/nodejs/node/actions/runs/6746717439 |
Landed in fd2368e...a00f0b1 |
When a wpt test file is exited for uncaught error, its result should be recorded in the `wptreport.json` and uploaded to wpt.fyi. For instance, `html/webappapis/timers/evil-spec-example.any.js` is exited for uncaught error in Node.js but it shows as "MISSING" at https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned. PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
When a wpt test file is exited for uncaught error, its result should be recorded in the `wptreport.json` and uploaded to wpt.fyi. For instance, `html/webappapis/timers/evil-spec-example.any.js` is exited for uncaught error in Node.js but it shows as "MISSING" at https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned. PR-URL: nodejs#50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
PR-URL: nodejs#50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
When a wpt test file is exited for uncaught error, its result should be recorded in the `wptreport.json` and uploaded to wpt.fyi. For instance, `html/webappapis/timers/evil-spec-example.any.js` is exited for uncaught error in Node.js but it shows as "MISSING" at https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned. PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
When a wpt test file is exited for uncaught error, its result should be recorded in the `wptreport.json` and uploaded to wpt.fyi. For instance, `html/webappapis/timers/evil-spec-example.any.js` is exited for uncaught error in Node.js but it shows as "MISSING" at https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned. PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
When a wpt test file is exited for uncaught error, its result should be recorded in the `wptreport.json` and uploaded to wpt.fyi. For instance, `html/webappapis/timers/evil-spec-example.any.js` is exited for uncaught error in Node.js but it shows as "MISSING" at https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned. PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
PR-URL: #50429 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
When a wpt test file is exited for uncaught error, its result should be
recorded in the
wptreport.json
and uploaded to wpt.fyi.For instance,
html/webappapis/timers/evil-spec-example.any.js
isexited for uncaught error in Node.js but it shows as "MISSING" at
https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.