Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental: Use weights machines for CI based benchmarks #499

Closed
wants to merge 7 commits into from

Conversation

Robbepop
Copy link
Member

@Robbepop Robbepop commented Oct 4, 2022

No description provided.

@paritytech-cicd-pr
Copy link

paritytech-cicd-pr commented Oct 5, 2022

BENCHMARKS

NATIVEWASMTIME
BENCHMARKMASTERPRDIFFMASTERPRDIFFWASMTIME OVERHEAD
execute/
count_until
974.31µs 971.29µs ⚪ -0.52% 2.14ms 2.13ms ⚪ 0.02% 🔴 119%
execute/
factorial_iterative
383.16ns 380.44ns ⚪ -0.64% 871.30ns 871.69ns ⚪ 0.09% 🔴 129%
execute/
factorial_recursive
634.03ns 627.18ns ⚪ -0.56% 1.25µs 1.25µs ⚪ 0.10% 🟡 99%
execute/
fib_iterative
1.83ms 1.83ms ⚪ 0.18% 4.82ms 4.67ms 🟢 -3.15% 🔴 155%
execute/
fib_recursive
6.38ms 6.43ms ⚪ 0.75% 11.90ms 12.11ms 🔴 1.70% 🟡 88%
execute/
global_bump
1.39ms 1.38ms ⚪ -1.09% 3.22ms 3.21ms ⚪ -0.09% 🔴 133%
execute/
host_calls
25.03µs 24.73µs ⚪ 0.28% 37.46µs 37.36µs ⚪ -0.17% 🟡 51%
execute/
memory_fill
1.68ms 1.68ms ⚪ 0.57% 4.12ms 4.15ms ⚪ 0.79% 🔴 148%
execute/
memory_sum
1.66ms 1.67ms ⚪ 0.45% 4.10ms 4.10ms ⚪ 0.00% 🔴 146%
execute/
memory_vec_add
3.57ms 3.55ms ⚪ -0.44% 8.51ms 8.64ms 🔴 1.53% 🔴 143%
execute/
recursive_is_even
1.02ms 1.03ms ⚪ 1.02% 2.03ms 2.02ms ⚪ -0.52% 🟡 96%
execute/
recursive_ok
141.38µs 140.73µs ⚪ -0.04% 286.46µs 286.66µs ⚪ 0.11% 🔴 104%
execute/
recursive_scan
180.50µs 180.52µs ⚪ 0.00% 379.60µs 380.99µs ⚪ 0.29% 🔴 111%
execute/
recursive_trap
13.55µs 13.56µs ⚪ -0.26% 29.03µs 29.15µs ⚪ 0.12% 🔴 115%
execute/
regex_redux
657.21µs 665.20µs ⚪ 0.91% 1.48ms 1.50ms ⚪ 1.03% 🔴 126%
execute/
rev_complement
616.91µs 618.93µs ⚪ 0.42% 1.51ms 1.52ms ⚪ 0.36% 🔴 145%
execute/
tiny_keccak
630.21µs 623.79µs 🟢 -1.50% 1.29ms 1.29ms ⚪ 0.28% 🔴 107%
execute/
trunc_f2i
1.00ms 997.65µs ⚪ -0.36% 2.33ms 2.34ms ⚪ 0.12% 🔴 134%
instantiate/
wasm_kernel
39.78µs 41.35µs 🔴 3.83% 90.96µs 70.96µs 🟢 -21.90% 🟡 72%
translate/
wasm_kernel
3.52ms 3.51ms ⚪ -0.35% 7.43ms 7.47ms ⚪ 0.40% 🔴 113%

Link to pipeline

@codecov-commenter
Copy link

Codecov Report

Merging #499 (f651f91) into master (bdac479) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #499   +/-   ##
=======================================
  Coverage   79.20%   79.20%           
=======================================
  Files          74       74           
  Lines        6314     6314           
=======================================
  Hits         5001     5001           
  Misses       1313     1313           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@sergejparity
Copy link
Contributor

Closing in favor of #501

@Robbepop Robbepop deleted the rf-use-weights-machines branch March 11, 2024 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants