Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added the $262 object to Test262 test runner #1296

Merged
merged 4 commits into from
Jun 7, 2021
Merged

Added the $262 object to Test262 test runner #1296

merged 4 commits into from
Jun 7, 2021

Conversation

Razican
Copy link
Member

@Razican Razican commented May 30, 2021 •
edited
Loading

This PR improves the Test262 runner by adding a preliminary version of the $262 global property.

After some trial and error, I saw we still need to implement #893, #1300 and #1301 to be able to implement the rest of the functions, and the agent record is even trickier, since it requires being able to run stuff concurrently (but in the same thread). For this we need an executor and an event queue capable of sleeping, stopping execution flow and continuing where we were, which is currently not possible.

In any case, It already brings improvements in testing: the cross-realm symbols and some extra tests now work as expected.

@Razican Razican added enhancement New feature or request test Issues and PRs related to the tests. labels May 30, 2021
@github-actions
Copy link

github-actions bot commented May 30, 2021 •
edited
Loading

Test262 conformance changes:

Test result master count PR count difference
Total 78,897 78,897 0
Passed 26,719 26,783 +64
Ignored 15,628 15,628 0
Failed 36,550 36,486 -64
Panics 14 14 0
Conformance 33.87% 33.95% +0.08%
Fixed tests: 
test/language/global-code/script-decl-func-dups.js [strict mode] (previously Failed)
test/language/global-code/script-decl-func-dups.js (previously Failed)
test/language/expressions/tagged-template/cache-realm.js [strict mode] (previously Failed)
test/language/expressions/tagged-template/cache-realm.js (previously Failed)
test/annexB/language/expressions/logical-assignment/emulates-undefined-or.js [strict mode] (previously Failed)
test/annexB/language/expressions/logical-assignment/emulates-undefined-or.js (previously Failed)
test/annexB/language/expressions/logical-assignment/emulates-undefined-and.js [strict mode] (previously Failed)
test/annexB/language/expressions/logical-assignment/emulates-undefined-and.js (previously Failed)
test/annexB/language/expressions/logical-not/emulates-undefined.js [strict mode] (previously Failed)
test/annexB/language/expressions/logical-not/emulates-undefined.js (previously Failed)
test/annexB/language/expressions/typeof/emulates-undefined.js [strict mode] (previously Failed)
test/annexB/language/expressions/typeof/emulates-undefined.js (previously Failed)
test/annexB/language/expressions/logical-or/emulates-undefined.js [strict mode] (previously Failed)
test/annexB/language/expressions/logical-or/emulates-undefined.js (previously Failed)
test/annexB/language/expressions/does-not-equals/emulates-undefined.js [strict mode] (previously Failed)
test/annexB/language/expressions/does-not-equals/emulates-undefined.js (previously Failed)
test/annexB/language/expressions/equals/emulates-undefined.js [strict mode] (previously Failed)
test/annexB/language/expressions/equals/emulates-undefined.js (previously Failed)
test/annexB/language/expressions/logical-and/emulates-undefined.js [strict mode] (previously Failed)
test/annexB/language/expressions/logical-and/emulates-undefined.js (previously Failed)
test/annexB/language/expressions/conditional/emulates-undefined.js [strict mode] (previously Failed)
test/annexB/language/expressions/conditional/emulates-undefined.js (previously Failed)
test/annexB/language/statements/if/emulated-undefined.js [strict mode] (previously Failed)
test/annexB/language/statements/if/emulated-undefined.js (previously Failed)
test/annexB/built-ins/Array/from/iterator-method-emulates-undefined.js [strict mode] (previously Failed)
test/annexB/built-ins/Array/from/iterator-method-emulates-undefined.js (previously Failed)
test/built-ins/BigInt/prototype/valueOf/cross-realm.js [strict mode] (previously Failed)
test/built-ins/BigInt/prototype/valueOf/cross-realm.js (previously Failed)
test/built-ins/Symbol/isConcatSpreadable/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/isConcatSpreadable/cross-realm.js (previously Failed)
test/built-ins/Symbol/toPrimitive/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/toPrimitive/cross-realm.js (previously Failed)
test/built-ins/Symbol/asyncIterator/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/asyncIterator/cross-realm.js (previously Failed)
test/built-ins/Symbol/matchAll/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/matchAll/cross-realm.js (previously Failed)
test/built-ins/Symbol/match/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/match/cross-realm.js (previously Failed)
test/built-ins/Symbol/toStringTag/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/toStringTag/cross-realm.js (previously Failed)
test/built-ins/Symbol/iterator/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/iterator/cross-realm.js (previously Failed)
test/built-ins/Symbol/split/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/split/cross-realm.js (previously Failed)
test/built-ins/Symbol/replace/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/replace/cross-realm.js (previously Failed)
test/built-ins/Symbol/unscopables/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/unscopables/cross-realm.js (previously Failed)
test/built-ins/Symbol/species/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/species/cross-realm.js (previously Failed)
test/built-ins/Symbol/hasInstance/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/hasInstance/cross-realm.js (previously Failed)
test/built-ins/Symbol/search/cross-realm.js [strict mode] (previously Failed)
test/built-ins/Symbol/search/cross-realm.js (previously Failed)
test/built-ins/Array/length/define-own-prop-length-overflow-realm.js [strict mode] (previously Failed)
test/built-ins/Array/length/define-own-prop-length-overflow-realm.js (previously Failed)
test/built-ins/Array/prototype/filter/create-proto-from-ctor-realm-non-array.js [strict mode] (previously Failed)
test/built-ins/Array/prototype/filter/create-proto-from-ctor-realm-non-array.js (previously Failed)
test/built-ins/Array/prototype/map/create-proto-from-ctor-realm-array.js [strict mode] (previously Failed)
test/built-ins/Array/prototype/map/create-proto-from-ctor-realm-array.js (previously Failed)
test/built-ins/Array/prototype/slice/create-proto-from-ctor-realm-array.js [strict mode] (previously Failed)
test/built-ins/Array/prototype/slice/create-proto-from-ctor-realm-array.js (previously Failed)
test/built-ins/Array/prototype/concat/create-proto-from-ctor-realm-array.js [strict mode] (previously Failed)
test/built-ins/Array/prototype/concat/create-proto-from-ctor-realm-array.js (previously Failed)

@github-actions
Copy link

Benchmark for 19e8280

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 377.2±0.35ns 391.8±14.98ns -3.73%
Arithmetic operations (Full) 300.8±15.43µs 301.3±12.99µs -0.17%
Array access (Execution) 7.3±0.87µs 7.4±0.64µs -1.35%
Array access (Full) 307.3±0.80µs 325.7±27.62µs -5.65%
Array creation (Execution) 3.2±0.01ms 3.2±0.01ms 0.00%
Array creation (Full) 3.6±0.16ms 3.5±0.11ms +2.86%
Array pop (Execution) 1013.9±3.30µs 1012.9±3.74µs +0.10%
Array pop (Full) 1537.1±47.14µs 1595.8±79.58µs -3.68%
Boolean Object Access (Execution) 5.7±0.01µs 5.7±0.01µs 0.00%
Boolean Object Access (Full) 299.0±0.59µs 310.3±0.41µs -3.64%
Clean js (Execution) 663.1±2.73µs 659.9±3.26µs +0.48%
Clean js (Full) 1049.5±41.75µs 1045.6±31.75µs +0.37%
Clean js (Parser) 42.6±0.22µs 42.9±0.08µs -0.70%
Create Realm 457.7±21.80ns 443.9±0.51ns +3.11%
Dynamic Object Property Access (Execution) 5.6±0.01µs 5.6±0.02µs 0.00%
Dynamic Object Property Access (Full) 305.5±0.47µs 311.9±0.41µs -2.05%
Expression (Parser) 7.9±0.26µs 7.6±0.01µs +3.95%
Fibonacci (Execution) 932.9±59.34µs 904.7±70.65µs +3.12%
Fibonacci (Full) 1250.7±53.19µs 1225.4±74.84µs +2.06%
For loop (Execution) 23.7±1.71µs 23.2±0.07µs +2.16%
For loop (Full) 333.3±24.30µs 328.4±0.63µs +1.49%
For loop (Parser) 20.6±0.02µs 20.6±0.03µs 0.00%
Goal Symbols (Parser) 15.4±0.52µs 14.7±0.10µs +4.76%
Hello World (Parser) 4.4±0.49µs 4.3±0.18µs +2.33%
Long file (Parser) 838.5±19.04ns 826.0±0.60ns +1.51%
Mini js (Execution) 620.3±32.46µs 596.8±6.67µs +3.94%
Mini js (Full) 926.9±2.84µs 972.7±20.76µs -4.71%
Mini js (Parser) 39.1±1.39µs 37.3±0.06µs +4.83%
Number Object Access (Execution) 4.6±0.42µs 4.4±0.01µs +4.55%
Number Object Access (Full) 295.7±0.46µs 301.5±0.59µs -1.92%
Object Creation (Execution) 4.8±0.01µs 5.1±0.31µs -5.88%
Object Creation (Full) 296.6±0.46µs 303.9±0.35µs -2.40%
RegExp (Execution) 13.0±0.17µs 12.8±0.07µs +1.56%
RegExp (Full) 315.2±0.38µs 321.2±0.62µs -1.87%
RegExp Literal (Execution) 12.9±0.51µs 12.8±0.06µs +0.78%
RegExp Literal (Full) 310.0±1.46µs 322.5±0.79µs -3.88%
RegExp Literal Creation (Execution) 11.4±0.04µs 11.8±0.36µs -3.39%
RegExp Literal Creation (Full) 323.4±24.41µs 312.6±0.75µs +3.45%
Static Object Property Access (Execution) 5.2±0.09µs 5.0±0.02µs +4.00%
Static Object Property Access (Full) 315.3±16.44µs 305.7±0.85µs +3.14%
String Object Access (Execution) 7.8±0.02µs 7.7±0.03µs +1.30%
String Object Access (Full) 306.8±0.35µs 311.5±0.36µs -1.51%
String comparison (Execution) 7.0±0.03µs 7.2±0.25µs -2.78%
String comparison (Full) 304.9±0.46µs 309.2±0.32µs -1.39%
String concatenation (Execution) 5.8±0.02µs 5.8±0.02µs 0.00%
String concatenation (Full) 297.6±1.70µs 320.1±14.85µs -7.03%
String copy (Execution) 4.3±0.01µs 4.7±0.22µs -8.51%
String copy (Full) 291.2±0.37µs 297.6±0.34µs -2.15%
Symbols (Execution) 3.7±0.01µs 3.9±0.15µs -5.13%
Symbols (Full) 295.2±13.31µs 301.9±12.21µs -2.22%

@Razican Razican added this to the v0.13.0 milestone May 31, 2021
@github-actions
Copy link

Benchmark for 0995cee

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 334.7±16.54ns 325.6±12.38ns +2.79%
Arithmetic operations (Full) 274.4±13.11µs 274.1±14.84µs +0.11%
Array access (Execution) 6.8±0.40µs 6.8±0.37µs 0.00%
Array access (Full) 306.9±15.01µs 294.9±11.75µs +4.07%
Array creation (Execution) 2.7±0.14ms 2.5±0.13ms +8.00%
Array creation (Full) 3.1±0.20ms 2.9±0.13ms +6.90%
Array pop (Execution) 877.6±42.68µs 822.6±34.18µs +6.69%
Array pop (Full) 1318.3±52.05µs 1316.6±40.05µs +0.13%
Boolean Object Access (Execution) 5.5±0.29µs 5.5±0.28µs 0.00%
Boolean Object Access (Full) 292.9±13.50µs 287.5±12.05µs +1.88%
Clean js (Execution) 617.3±25.71µs 642.4±28.53µs -3.91%
Clean js (Full) 912.3±48.41µs 906.7±32.75µs +0.62%
Clean js (Parser) 40.8±2.00µs 42.4±2.19µs -3.77%
Create Realm 394.7±18.49ns 398.2±26.51ns -0.88%
Dynamic Object Property Access (Execution) 5.5±0.26µs 5.6±0.32µs -1.79%
Dynamic Object Property Access (Full) 298.3±17.52µs 299.5±16.17µs -0.40%
Expression (Parser) 6.6±0.25µs 6.8±0.53µs -2.94%
Fibonacci (Execution) 862.0±38.89µs 909.8±34.16µs -5.25%
Fibonacci (Full) 1168.9±63.94µs 1231.3±49.73µs -5.07%
For loop (Execution) 22.2±1.38µs 23.5±0.95µs -5.53%
For loop (Full) 306.1±10.49µs 308.6±11.81µs -0.81%
For loop (Parser) 19.1±0.89µs 18.3±0.79µs +4.37%
Goal Symbols (Parser) 13.0±0.60µs 12.8±0.61µs +1.56%
Hello World (Parser) 3.7±0.24µs 3.7±0.22µs 0.00%
Long file (Parser) 746.4±36.75ns 751.4±52.47ns -0.67%
Mini js (Execution) 543.6±20.28µs 534.8±26.97µs +1.65%
Mini js (Full) 839.2±28.57µs 863.4±58.01µs -2.80%
Mini js (Parser) 34.2±1.63µs 34.3±1.59µs -0.29%
Number Object Access (Execution) 4.2±0.23µs 4.3±0.20µs -2.33%
Number Object Access (Full) 292.7±14.02µs 282.6±10.99µs +3.57%
Object Creation (Execution) 4.7±0.17µs 4.8±0.22µs -2.08%
Object Creation (Full) 300.3±15.48µs 295.6±16.47µs +1.59%
RegExp (Execution) 11.8±0.91µs 12.1±1.31µs -2.48%
RegExp (Full) 309.5±17.57µs 314.6±18.68µs -1.62%
RegExp Literal (Execution) 11.5±0.47µs 12.6±0.51µs -8.73%
RegExp Literal (Full) 309.7±14.53µs 297.5±12.20µs +4.10%
RegExp Literal Creation (Execution) 10.0±0.60µs 11.0±0.42µs -9.09%
RegExp Literal Creation (Full) 307.9±21.16µs 292.6±9.69µs +5.23%
Static Object Property Access (Execution) 5.0±0.27µs 5.1±0.26µs -1.96%
Static Object Property Access (Full) 298.1±13.91µs 290.5±14.20µs +2.62%
String Object Access (Execution) 7.7±0.40µs 7.4±0.43µs +4.05%
String Object Access (Full) 293.9±12.34µs 294.9±14.54µs -0.34%
String comparison (Execution) 6.9±0.37µs 6.8±0.27µs +1.47%
String comparison (Full) 299.6±15.20µs 297.0±18.89µs +0.88%
String concatenation (Execution) 5.5±0.25µs 6.1±0.87µs -9.84%
String concatenation (Full) 289.7±17.46µs 288.2±16.21µs +0.52%
String copy (Execution) 4.2±0.19µs 4.3±0.29µs -2.33%
String copy (Full) 280.2±9.80µs 288.0±15.70µs -2.71%
Symbols (Execution) 3.6±0.15µs 3.8±0.32µs -5.26%
Symbols (Full) 285.5±26.54µs 279.8±28.64µs +2.04%

@Razican Razican marked this pull request as ready for review June 1, 2021 14:13
@Razican Razican requested review from RageKnify, HalidOdat, jasonwilliams, 0x7D2B, Lan2u and tofpie and removed request for HalidOdat June 1, 2021 14:13
@Razican Razican modified the milestones: v0.13.0, v0.12.0 Jun 1, 2021
@github-actions
Copy link

github-actions bot commented Jun 1, 2021

Benchmark for 0730b5d

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 301.5±0.14ns 316.6±4.51ns -4.77%
Arithmetic operations (Full) 224.3±0.89µs 227.6±0.31µs -1.45%
Array access (Execution) 5.4±0.02µs 5.6±0.01µs -3.57%
Array access (Full) 246.6±0.52µs 252.9±0.32µs -2.49%
Array creation (Execution) 2.4±0.00ms 2.3±0.01ms +4.35%
Array creation (Full) 2.8±0.00ms 2.8±0.00ms 0.00%
Array pop (Execution) 788.9±1.89µs 741.5±2.06µs +6.39%
Array pop (Full) 1238.9±2.58µs 1219.8±5.82µs +1.57%
Boolean Object Access (Execution) 4.6±0.02µs 4.6±0.01µs 0.00%
Boolean Object Access (Full) 241.4±0.26µs 243.9±0.37µs -1.03%
Clean js (Execution) 533.3±2.57µs 525.2±1.85µs +1.54%
Clean js (Full) 802.8±2.27µs 816.5±3.27µs -1.68%
Clean js (Parser) 36.4±0.21µs 36.0±0.06µs +1.11%
Create Realm 357.0±0.86ns 358.7±0.26ns -0.47%
Dynamic Object Property Access (Execution) 4.5±0.01µs 4.5±0.03µs 0.00%
Dynamic Object Property Access (Full) 243.1±0.47µs 246.9±0.75µs -1.54%
Expression (Parser) 6.2±0.03µs 6.0±0.08µs +3.33%
Fibonacci (Execution) 686.7±0.94µs 699.7±3.36µs -1.86%
Fibonacci (Full) 946.8±1.23µs 972.0±5.46µs -2.59%
For loop (Execution) 18.7±0.10µs 18.7±0.10µs 0.00%
For loop (Full) 255.7±0.42µs 260.7±0.34µs -1.92%
For loop (Parser) 17.2±0.05µs 17.2±0.02µs 0.00%
Goal Symbols (Parser) 12.4±0.03µs 12.3±0.02µs +0.81%
Hello World (Parser) 3.4±0.01µs 3.4±0.01µs 0.00%
Long file (Parser) 690.0±3.80ns 694.3±5.53ns -0.62%
Mini js (Execution) 479.0±2.25µs 472.1±2.02µs +1.46%
Mini js (Full) 747.2±3.33µs 761.6±3.24µs -1.89%
Mini js (Parser) 31.6±0.07µs 31.5±0.04µs +0.32%
Number Object Access (Execution) 3.6±0.01µs 3.6±0.00µs 0.00%
Number Object Access (Full) 238.0±0.36µs 239.7±0.76µs -0.71%
Object Creation (Execution) 3.9±0.02µs 3.9±0.02µs 0.00%
Object Creation (Full) 238.5±0.30µs 245.7±0.74µs -2.93%
RegExp (Execution) 9.8±0.03µs 9.7±0.03µs +1.03%
RegExp (Full) 255.5±0.49µs 258.2±0.43µs -1.05%
RegExp Literal (Execution) 9.7±0.08µs 9.7±0.04µs 0.00%
RegExp Literal (Full) 251.3±0.46µs 253.3±0.76µs -0.79%
RegExp Literal Creation (Execution) 8.3±0.03µs 8.5±0.03µs -2.35%
RegExp Literal Creation (Full) 244.9±0.50µs 246.9±0.79µs -0.81%
Static Object Property Access (Execution) 4.1±0.01µs 4.1±0.01µs 0.00%
Static Object Property Access (Full) 243.2±0.28µs 247.8±0.71µs -1.86%
String Object Access (Execution) 6.4±0.11µs 6.3±0.01µs +1.59%
String Object Access (Full) 245.1±1.26µs 250.3±0.38µs -2.08%
String comparison (Execution) 5.8±0.10µs 5.8±0.01µs 0.00%
String comparison (Full) 248.9±4.37µs 248.5±0.49µs +0.16%
String concatenation (Execution) 4.6±0.01µs 4.6±0.02µs 0.00%
String concatenation (Full) 239.1±1.00µs 238.8±0.27µs +0.13%
String copy (Execution) 3.6±0.01µs 3.5±0.03µs +2.86%
String copy (Full) 232.6±0.46µs 235.6±5.18µs -1.27%
Symbols (Execution) 3.0±0.02µs 3.0±0.01µs 0.00%
Symbols (Full) 228.6±0.66µs 230.1±0.38µs -0.65%

@Razican Razican merged commit e38a841 into master Jun 7, 2021
@Razican Razican deleted the feature/tes262_obj branch June 7, 2021 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RageKnify RageKnify RageKnify approved these changes

@jasonwilliams jasonwilliams Awaiting requested review from jasonwilliams

@HalidOdat HalidOdat Awaiting requested review from HalidOdat

@0x7D2B 0x7D2B Awaiting requested review from 0x7D2B

@Lan2u Lan2u Awaiting requested review from Lan2u

@tofpie tofpie Awaiting requested review from tofpie

Assignees
No one assigned
Labels
enhancement New feature or request test Issues and PRs related to the tests.
Projects
None yet
Milestone
v0.12.0
Development

Successfully merging this pull request may close these issues.
2 participants
@Razican @RageKnify