Lazily download test262 repository

[HalidOdat]

This Pull Request fixes/closes #3095 .

This PR makes the downloads the test262 repository, if we run the boa_tester with run command and we do not specify the test262 directory.

There is a con with this approach, dependabot can't suggest a updates. We could implement our own checker though.

TODO:

• The commit is hard coded, add a config file for test262 default commit (maybe merge test_ignore.toml into it).
  • Add ability to specify commit on test262 clone, via CLI commands
  • Warn user that the current commit is older than test262 HEAD
• Better error reporting

[jasonwilliams]

Good idea, I don't see there being an issue with this. It should make the whole git submodules issue easier

[gakonst]

Love it!! Thank you for taking this on, big devex upgrade for anyone using boa as a dep and doesn't want their clone to take minutes ^^

[codecov]

Codecov Report

Patch coverage has no change and project coverage change: -0.09% ⚠️

Comparison is base (8a78061) 50.31% compared to head (55215e5) 50.23%.
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3214      +/-   ##
==========================================
- Coverage   50.31%   50.23%   -0.09%     
==========================================
  Files         436      436              
  Lines       42604    42677      +73     
==========================================
  Hits        21438    21438              
- Misses      21166    21239      +73     

Files Changed	Coverage Δ
boa_tester/src/main.rs	0.00% <0.00%> (ø)
boa_tester/src/results.rs	0.00% <0.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Test262 conformance changes

Test result	main count	PR count	difference
Total	95,574	95,574	0
Passed	75,084	75,084	0
Ignored	19,494	19,494	0
Failed	996	996	0
Panics	0	0	0
Conformance	78.56%	78.56%	0.00%

[HalidOdat]

This is now complete, and ready for review/merge :)

Added --test262-update for automatically updating the local repository and --test262-commit for specifying a commit.

Now tester warns if local test262 clone is out of sync with origin/main:

Warning Test262 repository is not in sync, use '--test262-update' to automatically update it:
    Current commit: bdddd9e2d286600462958c7bac6af10a2946134b Update test for monthDayFromFields()
    Latest commit:  04a84fc5fddbc2498d344cc5b85dd6bf6d76c96e Fix test/intl402/DurationFormat/prototype/format/style-digital-en.js

[jedel1043]

Did you consider using git2 instead of calling manual commands? Maybe that could further simplify the repo handling logic.

EDIT: There's also gix which is implemented in Rust only.

[HalidOdat]

Did you consider using git2 instead of calling manual commands?

Yeah, thought about it, but decided calling manually would be fine, also I didn't want to add another dependency on test262 without heavily using git features.

The git command should be available on pretty much any development environment.

[gakonst]

Hi folks - wonder if we (we = Reth dev team) can help with this PR in any way? PR seems good / harmless to me - but understand you want to keep the bar high / ensure it gets adequately reviewed.

[jedel1043]

This looks great implementation-wise, but I'd argue we should have a way to pin the commit downloaded by the tool for reproducibility in our CI. Maybe a file that lives in the root of the repo?

[HalidOdat]

This looks great implementation-wise, but I'd argue we should have a way to pin the commit downloaded by the tool for reproducibility in our CI. Maybe a file that lives in the root of the repo?

Maybe merge test_ignore.toml file into something like test262_config.toml, to avoid having more than one config file boa_tester

[jedel1043]

Maybe merge test_ignore.toml file into something like test262_config.toml, to avoid having more than one config file boa_tester

Great idea! We just have to create a new ignore section and we can put a commit property on the root

[jasonwilliams]

Hey @gakonst what does Reth use Boa for? I’m curious

[HalidOdat]

Maybe merge test_ignore.toml file into something like test262_config.toml, to avoid having more than one config file boa_tester

Great idea! We just have to create a new ignore section and we can put a commit property on the root

Implemented in the latest commit 3a0d75c :)

[gakonst]

Hey @gakonst what does Reth use Boa for? I’m curious

Hi Jason, Reth (w/o getting in too many of the details around the whole thing) exposes a debugging JSON-RPC service to provide "execution traces" of the request, showing step by step what happened. These traces can be configured to run with certain "tracers", some of which are part of the node. However, there's benefit in allowing users to specify custom tracers. The way this was implemented originally in go-ethereum was by embedding a Javascript sandbox and on various parts of the tracer lifecycle to call to some client-side provided JS that augments the tracer output.

In Reth, we use boa to provide that JS sandbox to the tracer. See Go-ethereum's docs for more on this.

Hope that's helpful, happy to provide other context if needed but do not want to derail this issue.

[jedel1043]

Nice work! I just have a small nitpick that doesn't block merging this :)

[nekevss]

looks good to me!