POC for basic web platform test integration #2585
Conversation
jasnell
force-pushed
the
jsnell/wip-wpt
branch
from
2a42bf6
to
de0330d
Compare
| allow_empty = True, | ||
| ), | ||
| visibility = ["//visibility:public"], | ||
| ) |
There was a problem hiding this comment.
We would need to export filegroups for each of the web platform test groups we want... we don't need to export filegroups for everything since most of the wpts will never be relevant
src/workerd/api/wpt/BUILD.bazel
Outdated
| ], | ||
| ) for f in glob( | ||
| ["**/*.wd-test"], | ||
| )] |
There was a problem hiding this comment.
Most likely should list each individually rather than using the glob here so that we can selectively import the appropriate wpt filegroups.
src/workerd/api/wpt/url-test.js
Outdated
| globalThis.fetch = async(url) => { | ||
| const { default: data } = await import(url); | ||
| return { | ||
| async json() { return data; }, | ||
| }; | ||
| }; | ||
|
|
||
| globalThis.promise_test = (callback) => { | ||
| callback(); | ||
| }; | ||
|
|
||
| globalThis.assert_equals = (a, b, c) => { | ||
| strictEqual(a,b,c); | ||
| }; | ||
|
|
||
| globalThis.test = (callback, message) => { | ||
| try { | ||
| callback(); | ||
| } catch (err) { | ||
| const aerr = new AggregateError([err], message); | ||
| globalThis.errors.push(aerr); | ||
| } | ||
| } | ||
|
|
||
| globalThis.errors = []; |
There was a problem hiding this comment.
In order to be able to run the tests we need to define the test harness methods along with a way of collecting the results. This is a bit wonky here.
src/workerd/api/wpt/url-test.js
Outdated
|
|
||
| export const test = { | ||
| async test(_, env) { | ||
| const foo = await import('url-origin.any.js'); |
There was a problem hiding this comment.
The tests end up running when the module is evaluated. We have to set the global test harness methods before the dynamic import, then check the globalThis.errors when we're done. The globalThis.errors is only specific to this test file, it is not actually part of the WPT test harness. We have to do this because the tests themselves may not actually propagate the error (in the case of url-origin.any.js we end up with a swallowed rejected promise) if we don't have this.
There was a problem hiding this comment.
We also need a way to ignore certain errors. Possibly from a module_name.json file with a JSON schema that conforms to our needs
There was a problem hiding this comment.
Yeah, I'm thinking that can be a separate json module that is imported and used by the harness.
jasnell
force-pushed
the
jsnell/wip-wpt
branch
2 times, most recently
from
7c954b5
to
e5803d7
Compare
| load("//:build/wd_test.bzl", "wd_test") | ||
| load("//src/workerd/api/wpt:generate-tests.bzl", "gen_wpt_tests") | ||
|
|
||
| gen_wpt_tests(glob(["**/*.js"])) |
There was a problem hiding this comment.
Some documentation comments around this would be helpful.
| async test() { | ||
| harness.prepare(); | ||
| await import('url-constructor.any.js'); | ||
| harness.validate(); |
There was a problem hiding this comment.
Side note: Our workerd tests could benefit from having a before() { ... } and after() { ... } mechanism.
There was a problem hiding this comment.
I suppose we could have a pattern like...
import * as harness from 'harness';
export const urlConstructor = {
test: harness.test('url-constructor.any.js');
};
// or
export const urlConstructor = harness.test('url-constructor.any.js');
Where the harness.test(...) handles any of the necessary boilerplate for the most typical cases?
| @@ -0,0 +1,41 @@ | |||
| import { strictEqual } from 'node:assert'; | |||
There was a problem hiding this comment.
Side note... let's try not to forget adding the copyright headers to these (speaking to myself here mostly)
anonrig
force-pushed
the
jsnell/wip-wpt
branch
from
4cf5ddd
to
b474e8c
Compare
anonrig
force-pushed
the
jsnell/wip-wpt
branch
from
9d9af02
to
37c95b4
Compare
This is just a proof of concept to start working through what is necessary to integrate web platform tests into wd-tests.
There are some limitations to this approach...
ShadowRealm... otherwise there's always a possibility of the harness conflicting with the global scope, tho that is unlikely in practice