Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cleanup tester deps and patterns #3792

Merged
merged 2 commits into from
Apr 5, 2024
Merged

Cleanup tester deps and patterns #3792

merged 2 commits into from
Apr 5, 2024

Conversation

jedel1043
Copy link
Member

Just some refactorings to our tester. Also removes the regex and once_cell dependencies from it.

Additionally, this also reexports our most important crates from boa_engine, to avoid having to manually import the other crates if someone's already using boa_engine.

@jedel1043 jedel1043 added enhancement New feature or request test Issues and PRs related to the tests. labels Apr 5, 2024
@jedel1043 jedel1043 added this to the v0.18.1 milestone Apr 5, 2024
@jedel1043 jedel1043 requested a review from a team April 5, 2024 15:22
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 5, 2024 •
edited
Loading

Test262 conformance changes

Test result main count PR count difference
Total 50,268 50,731 +463
Passed 42,599 42,971 +372
Ignored 1,391 1,395 +4
Failed 6,278 6,365 +87
Panics 18 18 0
Conformance 84.74% 84.70% -0.04%
Fixed tests (178): 
test/built-ins/Temporal/PlainYearMonth/prototype/since/argument-propertybag-calendar-number.js (previously Failed)
test/built-ins/Temporal/PlainYearMonth/prototype/until/argument-propertybag-calendar-number.js (previously Failed)
test/built-ins/Temporal/PlainDateTime/prototype/since/argument-propertybag-calendar-number.js (previously Failed)
test/built-ins/Temporal/PlainDateTime/prototype/until/argument-propertybag-calendar-number.js (previously Failed)
test/built-ins/Proxy/preventExtensions/trap-is-undefined-target-is-proxy.js (previously Failed)
test/language/expressions/dynamic-import/eval-self-once-module.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-expr-in.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-cls-name-meth.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-cls-anon.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-expr-fn-anon.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-expr-gen-anon.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-expr-cls-anon.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-expr-cls-named.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-expr-gen-named.js (previously Failed)
test/language/expressions/dynamic-import/reuse-namespace-object-from-import.js (previously Failed)
test/language/expressions/dynamic-import/imported-self-update.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-expr-fn-named.js (previously Failed)
test/language/expressions/dynamic-import/eval-export-dflt-cls-named.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-function-import-catch-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-block-import-catch-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-else-import-catch-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-while-import-catch-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-if-import-catch-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/top-level-import-catch-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-async-gen-return-await-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-async-function-return-await-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-async-arrow-function-await-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-async-function-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-block-labeled-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-async-arrow-function-return-await-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-do-while-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-async-gen-await-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-arrow-import-catch-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/catch/nested-async-function-await-eval-script-code-target.js (previously Failed)
test/language/expressions/dynamic-import/namespace/await-ns-get-nested-namespace-dflt-indirect.js (previously Failed)
test/language/expressions/dynamic-import/namespace/promise-then-ns-get-nested-namespace-dflt-indirect.js (previously Failed)
test/language/expressions/import.meta/distinct-for-each-module.js (previously Failed)
test/language/module-code/instn-resolve-err-syntax-1.js (previously Failed)
test/language/module-code/instn-iee-err-dflt-thru-star.js (previously Failed)
test/language/module-code/instn-named-err-ambiguous.js (previously Failed)
test/language/module-code/instn-iee-err-circular-as.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-fun-named.js (previously Failed)
test/language/module-code/instn-uniq-env-rec.js (previously Failed)
test/language/module-code/instn-named-err-not-found.js (previously Failed)
test/language/module-code/instn-star-id-name.js (previously Failed)
test/language/module-code/instn-named-err-not-found-as.js (previously Failed)
test/language/module-code/instn-resolve-empty-import.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-in.js (previously Failed)
test/language/module-code/instn-iee-bndng-const.js (previously Failed)
test/language/module-code/instn-named-iee-cycle.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-name-meth.js (previously Failed)
test/language/module-code/instn-named-err-dflt-thru-star-as.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-update-dflt.js (previously Failed)
test/language/module-code/instn-iee-err-not-found-as.js (previously Failed)
test/language/module-code/export-expname-import-string-binding.js (previously Failed)
test/language/module-code/instn-iee-bndng-let.js (previously Failed)
test/language/module-code/export-expname-binding-string.js (previously Failed)
test/language/module-code/instn-named-bndng-cls.js (previously Failed)
test/language/module-code/instn-star-props-dflt-keep-indirect.js (previously Failed)
test/language/module-code/instn-iee-bndng-cls.js (previously Failed)
test/language/module-code/instn-iee-star-cycle.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-anon.js (previously Failed)
test/language/module-code/verify-dfs.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-fun-anon.js (previously Failed)
test/language/module-code/instn-resolve-err-syntax-2.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-update.js (previously Failed)
test/language/module-code/instn-star-as-props-dflt-skip.js (previously Failed)
test/language/module-code/instn-star-iee-cycle.js (previously Failed)
test/language/module-code/instn-iee-err-not-found.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-update-as.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-fn-anon.js (previously Failed)
test/language/module-code/export-expname-from-binding-string.js (previously Failed)
test/language/module-code/instn-named-err-dflt-thru-star-dflt.js (previously Failed)
test/language/module-code/instn-star-ambiguous.js (previously Failed)
test/language/module-code/instn-resolve-order-depth.js (previously Failed)
test/language/module-code/instn-named-bndng-gen.js (previously Failed)
test/language/module-code/instn-resolve-empty-export.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-gen-anon.js (previously Failed)
test/language/module-code/export-expname-from-star-string.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-cls-anon.js (previously Failed)
test/language/module-code/instn-named-err-not-found-dflt.js (previously Failed)
test/language/module-code/instn-star-star-cycle.js (previously Failed)
test/language/module-code/instn-named-bndng-fun.js (previously Failed)
test/language/module-code/instn-same-global.js (previously Failed)
test/language/module-code/eval-rqstd-order.js (previously Failed)
test/language/module-code/instn-named-bndng-const.js (previously Failed)
test/language/module-code/instn-resolve-order-src.js (previously Failed)
test/language/module-code/instn-named-star-cycle.js (previously Failed)
test/language/module-code/export-expname-from-string.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-cls.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-cls-named.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-expr.js (previously Failed)
test/language/module-code/instn-star-props-dflt-keep-local.js (previously Failed)
test/language/module-code/instn-named-id-name.js (previously Failed)
test/language/module-code/instn-iee-trlng-comma.js (previously Failed)
test/language/module-code/instn-named-bndng-var.js (previously Failed)
test/language/module-code/instn-named-bndng-let.js (previously Failed)
test/language/module-code/instn-star-err-not-found.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-star.js (previously Failed)
test/language/module-code/export-expname-from-string-string.js (previously Failed)
test/language/module-code/instn-star-props-circular.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-named.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-gen-named.js (previously Failed)
test/language/module-code/instn-named-err-ambiguous-as.js (previously Failed)
test/language/module-code/instn-iee-bndng-var.js (previously Failed)
test/language/module-code/instn-iee-err-ambiguous-as.js (previously Failed)
test/language/module-code/instn-star-props-nrml.js (previously Failed)
test/language/module-code/instn-star-equality.js (previously Failed)
test/language/module-code/instn-once.js (previously Failed)
test/language/module-code/instn-star-props-dflt-skip.js (previously Failed)
test/language/module-code/instn-star-binding.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-gen-anon.js (previously Failed)
test/language/module-code/eval-self-once.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-trlng-comma.js (previously Failed)
test/language/module-code/instn-iee-err-ambiguous.js (previously Failed)
test/language/module-code/instn-named-bndng-trlng-comma.js (previously Failed)
test/language/module-code/instn-iee-bndng-fun.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-fn-named.js (previously Failed)
test/language/module-code/instn-iee-bndng-gen.js (previously Failed)
test/language/module-code/eval-rqstd-once.js (previously Failed)
test/language/module-code/export-expname-from-star.js (previously Failed)
test/language/module-code/instn-iee-iee-cycle.js (previously Failed)
test/language/module-code/instn-iee-err-dflt-thru-star-as.js (previously Failed)
test/language/module-code/eval-rqstd-abrupt.js (previously Failed)
test/language/module-code/export-expname-binding-index.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-gen-named.js (previously Failed)
test/language/module-code/instn-iee-err-circular.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-named.js (previously Failed)
test/language/module-code/export-star-as-dflt.js (previously Failed)
test/language/module-code/export-expname-from-string-binding.js (previously Failed)
test/language/module-code/top-level-await/dynamic-import-rejection.js (previously Failed)
test/language/module-code/top-level-await/module-import-resolution.js (previously Failed)
test/language/module-code/top-level-await/await-dynamic-import-resolution.js (previously Failed)
test/language/module-code/top-level-await/module-import-rejection-body.js (previously Failed)
test/language/module-code/top-level-await/dynamic-import-resolution.js (previously Failed)
test/language/module-code/top-level-await/module-import-rejection-tick.js (previously Failed)
test/language/module-code/top-level-await/module-import-rejection.js (previously Failed)
test/language/module-code/top-level-await/module-import-unwrapped.js (previously Failed)
test/language/module-code/top-level-await/module-sync-import-async-resolution-ticks.js (previously Failed)
test/language/module-code/top-level-await/module-async-import-async-resolution-ticks.js (previously Failed)
test/language/module-code/top-level-await/dfs-invariant.js (previously Failed)
test/language/module-code/top-level-await/module-self-import-async-resolution-ticks.js (previously Failed)
test/language/module-code/namespace/Symbol.toStringTag.js (previously Failed)
test/language/module-code/namespace/Symbol.iterator.js (previously Failed)
test/language/module-code/namespace/internals/get-own-property-str-found-uninit.js (previously Failed)
test/language/module-code/namespace/internals/is-extensible.js (previously Failed)
test/language/module-code/namespace/internals/has-property-sym-not-found.js (previously Failed)
test/language/module-code/namespace/internals/get-own-property-str-found-init.js (previously Failed)
test/language/module-code/namespace/internals/delete-exported-uninit.js (previously Failed)
test/language/module-code/namespace/internals/get-own-property-str-not-found.js (previously Failed)
test/language/module-code/namespace/internals/delete-non-exported.js (previously Failed)
test/language/module-code/namespace/internals/get-str-found-uninit.js (previously Failed)
test/language/module-code/namespace/internals/set-prototype-of.js (previously Failed)
test/language/module-code/namespace/internals/get-str-update.js (previously Failed)
test/language/module-code/namespace/internals/get-nested-namespace-props-nrml.js (previously Failed)
test/language/module-code/namespace/internals/own-property-keys-sort.js (previously Failed)
test/language/module-code/namespace/internals/get-str-found-init.js (previously Failed)
test/language/module-code/namespace/internals/has-property-str-not-found.js (previously Failed)
test/language/module-code/namespace/internals/object-hasOwnProperty-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/has-property-str-found-uninit.js (previously Failed)
test/language/module-code/namespace/internals/get-sym-found.js (previously Failed)
test/language/module-code/namespace/internals/own-property-keys-binding-types.js (previously Failed)
test/language/module-code/namespace/internals/get-sym-not-found.js (previously Failed)
test/language/module-code/namespace/internals/object-keys-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/get-str-not-found.js (previously Failed)
test/language/module-code/namespace/internals/delete-exported-init.js (previously Failed)
test/language/module-code/namespace/internals/object-propertyIsEnumerable-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/set-prototype-of-null.js (previously Failed)
test/language/module-code/namespace/internals/prevent-extensions.js (previously Failed)
test/language/module-code/namespace/internals/set.js (previously Failed)
test/language/module-code/namespace/internals/has-property-sym-found.js (previously Failed)
test/language/module-code/namespace/internals/define-own-property.js (previously Failed)
test/language/module-code/namespace/internals/enumerate-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/get-str-initialize.js (previously Failed)
test/language/module-code/namespace/internals/get-nested-namespace-dflt-skip.js (previously Failed)
test/language/module-code/namespace/internals/get-prototype-of.js (previously Failed)
test/language/module-code/namespace/internals/get-own-property-sym.js (previously Failed)
test/language/module-code/namespace/internals/has-property-str-found-init.js (previously Failed)
Broken tests (18): 
test/intl402/Temporal/Calendar/prototype/era/argument-propertybag-calendar-year-zero.js (previously Passed)
test/intl402/Temporal/Calendar/prototype/eraYear/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/year/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/monthsInYear/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/inLeapYear/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/daysInMonth/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/month/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/daysInYear/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/day/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/daysInWeek/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/dayOfYear/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/monthCode/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/dateAdd/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/dateAdd/argument-propertybag-calendar-number.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/dayOfWeek/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/yearOfWeek/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Calendar/prototype/weekOfYear/argument-propertybag-calendar-year-zero.js (previously Passed)
test/built-ins/Temporal/Duration/prototype/total/duration-out-of-range-added-to-relativeto.js (previously Passed)

HalidOdat
HalidOdat approved these changes Apr 5, 2024
View reviewed changes
Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! :)

There is a small clippy error that needs to be fixed, besides that this looks good to me!

raskad
raskad approved these changes Apr 5, 2024
View reviewed changes
Copy link
Member

@raskad raskad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@jedel1043 jedel1043 enabled auto-merge April 5, 2024 20:13
@jedel1043 jedel1043 added this pull request to the merge queue Apr 5, 2024
Merged via the queue into main with commit c9d1401 Apr 5, 2024
13 checks passed
@HalidOdat HalidOdat deleted the tester-cleanup branch April 5, 2024 21:21
@raskad raskad modified the milestones: v0.18.1, v0.19.0 Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raskad raskad raskad approved these changes

@HalidOdat HalidOdat HalidOdat approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request test Issues and PRs related to the tests.
Projects
None yet
Milestone
v0.19.0
Development

Successfully merging this pull request may close these issues.
3 participants
@jedel1043 @HalidOdat @raskad