Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using the upstream version of the benchmark action #768

Merged
merged 2 commits into from
Oct 2, 2020

Conversation

Razican
Copy link
Member

@Razican Razican commented Oct 2, 2020

It seems that some benchmarks are not showing properly in the graphs. This might be because we are not using the upstream version of the GitHub action, which includes some fixes. This could potentially fix that, and if it does, I can retroactivelly fix the benchmarks, since they need a re-adjustment anyways.

@Razican Razican added the benchmark Issues and PRs related to the benchmark subsystem. label Oct 2, 2020
@Razican Razican added this to the v0.11.0 milestone Oct 2, 2020
@codecov
Copy link

codecov bot commented Oct 2, 2020

Codecov Report

Merging #768 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #768   +/-   ##
=======================================
  Coverage   58.51%   58.51%           
=======================================
  Files         152      152           
  Lines        9476     9476           
=======================================
  Hits         5545     5545           
  Misses       3931     3931           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac85ee0...b1dac88. Read the comment docs.

@Razican Razican added the enhancement New feature or request label Oct 2, 2020
@github-actions
Copy link

github-actions bot commented Oct 2, 2020

Benchmark for a9eb96d

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 371.4±0.32ns 371.3±0.34ns +0.03%
Arithmetic operations (Full) 247.8±0.23µs 246.8±0.19µs +0.41%
Array access (Execution) 8.1±0.03µs 8.2±0.03µs -1.22%
Array access (Full) 269.8±0.40µs 271.5±0.61µs -0.63%
Array creation (Execution) 3.4±0.00ms 3.4±0.00ms 0.00%
Array creation (Full) 3.7±0.01ms 3.7±0.03ms 0.00%
Array pop (Execution) 1218.6±8.80µs 1239.1±31.25µs -1.65%
Array pop (Full) 1694.0±4.33µs 1689.4±2.61µs +0.27%
Boolean Object Access (Execution) 4.8±0.01µs 4.8±0.01µs 0.00%
Boolean Object Access (Full) 259.9±0.54µs 260.6±0.48µs -0.27%
Clean js (Execution) 741.7±5.32µs 741.6±7.19µs +0.01%
Clean js (Full) 1045.6±9.17µs 1045.0±12.35µs +0.06%
Clean js (Parser) 35.8±0.05µs 35.9±0.52µs -0.28%
Create Realm 462.8±0.31ns 480.9±4.93ns -3.76%
Dynamic Object Property Access (Execution) 5.3±0.02µs 5.4±0.08µs -1.85%
Dynamic Object Property Access (Full) 267.9±0.52µs 262.4±0.75µs +2.10%
Expression (Parser) 6.7±0.01µs 6.7±0.01µs 0.00%
Fibonacci (Execution) 803.5±1.61µs 801.2±1.74µs +0.29%
Fibonacci (Full) 1078.5±1.48µs 1090.0±1.82µs -1.06%
For loop (Execution) 22.1±0.09µs 22.2±0.11µs -0.45%
For loop (Full) 289.8±17.83µs 285.4±0.47µs +1.54%
For loop (Parser) 17.8±0.03µs 17.6±0.03µs +1.14%
Goal Symbols (Parser) 12.0±0.09µs 11.9±0.02µs +0.84%
Hello World (Parser) 3.1±0.03µs 3.0±0.02µs +3.33%
Long file (Parser) 761.4±0.49ns 762.1±0.62ns -0.09%
Mini js (Execution) 673.3±3.45µs 674.6±10.91µs -0.19%
Mini js (Full) 956.4±3.38µs 954.8±3.64µs +0.17%
Mini js (Parser) 31.7±0.05µs 31.6±0.03µs +0.32%
Number Object Access (Execution) 3.8±0.01µs 3.8±0.01µs 0.00%
Number Object Access (Full) 256.7±0.76µs 255.8±0.32µs +0.35%
Object Creation (Execution) 4.5±0.01µs 4.6±0.01µs -2.17%
Object Creation (Full) 263.4±0.42µs 260.4±0.78µs +1.15%
RegExp (Execution) 65.0±0.94µs 65.4±0.87µs -0.61%
RegExp (Full) 341.7±1.01µs 337.9±0.63µs +1.12%
RegExp Literal (Execution) 68.2±1.23µs 68.6±1.07µs -0.58%
RegExp Literal (Full) 340.5±0.57µs 336.1±10.02µs +1.31%
RegExp Literal Creation (Execution) 64.8±0.94µs 68.2±4.61µs -4.99%
RegExp Literal Creation (Full) 331.9±0.85µs 325.4±0.29µs +2.00%
Static Object Property Access (Execution) 4.8±0.02µs 4.8±0.02µs 0.00%
Static Object Property Access (Full) 264.8±0.54µs 259.3±0.57µs +2.12%
String Object Access (Execution) 7.0±0.01µs 7.0±0.02µs 0.00%
String Object Access (Full) 263.7±0.59µs 265.2±0.46µs -0.57%
String comparison (Execution) 6.4±0.01µs 6.2±0.01µs +3.23%
String comparison (Full) 265.5±0.66µs 264.7±0.43µs +0.30%
String concatenation (Execution) 5.1±0.01µs 5.1±0.01µs 0.00%
String concatenation (Full) 258.0±1.04µs 256.0±3.25µs +0.78%
String copy (Execution) 3.8±0.01µs 3.8±0.01µs 0.00%
String copy (Full) 251.9±0.29µs 254.7±0.52µs -1.10%
Symbols (Execution) 3.2±0.01µs 3.2±0.00µs 0.00%
Symbols (Full) 238.4±0.45µs 235.9±0.31µs +1.06%

@github-actions
Copy link

github-actions bot commented Oct 2, 2020

Benchmark for ab16e67

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 376.2±30.39ns 365.7±29.53ns +2.87%
Arithmetic operations (Full) 267.2±24.49µs 277.7±32.16µs -3.78%
Array access (Execution) 8.6±0.77µs 8.8±0.76µs -2.27%
Array access (Full) 292.9±58.27µs 285.2±22.00µs +2.70%
Array creation (Execution) 3.1±0.31ms 3.1±0.27ms 0.00%
Array creation (Full) 3.7±0.35ms 3.8±0.37ms -2.63%
Array pop (Execution) 1111.4±82.88µs 1151.6±96.83µs -3.49%
Array pop (Full) 1691.5±133.19µs 1637.0±126.86µs +3.33%
Boolean Object Access (Execution) 5.0±0.55µs 5.2±0.47µs -3.85%
Boolean Object Access (Full) 278.2±22.09µs 269.5±34.06µs +3.23%
Clean js (Execution) 766.7±64.22µs 714.9±53.23µs +7.25%
Clean js (Full) 1059.0±61.45µs 1114.6±109.31µs -4.99%
Clean js (Parser) 39.4±4.00µs 38.5±3.65µs +2.34%
Create Realm 489.1±49.20ns 453.5±31.38ns +7.85%
Dynamic Object Property Access (Execution) 5.9±0.58µs 5.8±0.54µs +1.72%
Dynamic Object Property Access (Full) 308.3±30.76µs 274.3±23.13µs +12.40%
Expression (Parser) 7.2±0.70µs 6.9±0.57µs +4.35%
Fibonacci (Execution) 885.7±69.18µs 892.2±66.07µs -0.73%
Fibonacci (Full) 1185.7±114.83µs 1186.4±99.51µs -0.06%
For loop (Execution) 22.8±1.69µs 24.3±2.54µs -6.17%
For loop (Full) 300.5±21.90µs 308.5±23.85µs -2.59%
For loop (Parser) 18.2±1.64µs 18.8±3.60µs -3.19%
Goal Symbols (Parser) 12.2±1.10µs 12.0±0.91µs +1.67%
Hello World (Parser) 3.3±0.32µs 3.3±0.31µs 0.00%
Long file (Parser) 805.1±65.44ns 807.2±77.31ns -0.26%
Mini js (Execution) 672.4±59.62µs 642.5±43.17µs +4.65%
Mini js (Full) 1034.4±110.80µs 966.6±92.19µs +7.01%
Mini js (Parser) 33.8±2.93µs 33.5±3.02µs +0.90%
Number Object Access (Execution) 4.1±0.40µs 3.9±0.31µs +5.13%
Number Object Access (Full) 286.9±29.99µs 263.4±14.80µs +8.92%
Object Creation (Execution) 5.0±0.56µs 5.1±0.38µs -1.96%
Object Creation (Full) 280.3±25.29µs 277.3±30.39µs +1.08%
RegExp (Execution) 71.1±7.69µs 72.3±8.04µs -1.66%
RegExp (Full) 371.5±48.93µs 369.6±37.71µs +0.51%
RegExp Literal (Execution) 74.1±6.11µs 71.7±4.59µs +3.35%
RegExp Literal (Full) 361.8±34.84µs 358.1±30.80µs +1.03%
RegExp Literal Creation (Execution) 73.1±8.60µs 72.6±7.78µs +0.69%
RegExp Literal Creation (Full) 382.9±42.44µs 346.0±28.50µs +10.66%
Static Object Property Access (Execution) 5.2±0.57µs 5.3±0.59µs -1.89%
Static Object Property Access (Full) 293.2±30.38µs 273.5±25.79µs +7.20%
String Object Access (Execution) 7.2±0.57µs 7.2±0.64µs 0.00%
String Object Access (Full) 280.6±24.11µs 283.9±28.84µs -1.16%
String comparison (Execution) 6.7±0.83µs 6.6±0.52µs +1.52%
String comparison (Full) 290.3±33.38µs 277.2±22.83µs +4.73%
String concatenation (Execution) 5.7±0.53µs 5.2±0.47µs +9.62%
String concatenation (Full) 283.3±27.51µs 276.5±27.59µs +2.46%
String copy (Execution) 4.0±0.41µs 4.0±0.38µs 0.00%
String copy (Full) 270.2±25.88µs 267.0±23.17µs +1.20%
Symbols (Execution) 3.4±0.36µs 3.4±0.29µs 0.00%
Symbols (Full) 251.6±23.90µs 270.1±24.45µs -6.85%

@Razican Razican merged commit 3cb8c94 into master Oct 2, 2020
@Razican Razican deleted the fix_master_benches branch October 2, 2020 13:29
Razican added a commit that referenced this pull request Oct 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark Issues and PRs related to the benchmark subsystem. enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants