Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: change ci runners type #501

Merged
merged 9 commits into from
Oct 11, 2022
Merged

CI: change ci runners type #501

merged 9 commits into from
Oct 11, 2022

Conversation

sergejparity
Copy link
Contributor

@sergejparity sergejparity commented Oct 7, 2022

Closes #495

@paritytech-cicd-pr
Copy link

paritytech-cicd-pr commented Oct 7, 2022

BENCHMARKS

NATIVEWASMTIME
BENCHMARKMASTERPRDIFFMASTERPRDIFFWASMTIME OVERHEAD
execute/
count_until
886.74µs 886.50µs ⚪ -0.07% 2.38ms 2.38ms ⚪ 0.09% 🔴 169%
execute/
factorial_iterative
400.14ns 408.24ns 🔴 1.68% 992.43ns 997.05ns ⚪ 0.57% 🔴 144%
execute/
factorial_recursive
762.18ns 759.61ns ⚪ -0.70% 1.55µs 1.56µs ⚪ 0.55% 🔴 106%
execute/
fib_iterative
1.79ms 1.79ms ⚪ -0.04% 5.12ms 5.11ms ⚪ -0.14% 🔴 185%
execute/
fib_recursive
6.79ms 6.89ms 🔴 1.46% 14.04ms 14.03ms ⚪ -0.11% 🔴 104%
execute/
global_bump
1.44ms 1.43ms ⚪ -0.14% 3.63ms 3.66ms ⚪ 0.69% 🔴 155%
execute/
host_calls
32.84µs 32.02µs ⚪ -4.56% 45.08µs 45.06µs ⚪ -0.06% 🟢 41%
execute/
memory_fill
1.53ms 1.54ms ⚪ 0.16% 4.33ms 4.33ms ⚪ 0.00% 🔴 182%
execute/
memory_sum
1.55ms 1.52ms ⚪ -2.12% 4.36ms 4.53ms 🔴 4.06% 🔴 199%
execute/
memory_vec_add
3.02ms 3.02ms ⚪ 0.20% 9.03ms 8.94ms ⚪ -0.90% 🔴 196%
execute/
recursive_is_even
1.28ms 1.29ms ⚪ 0.27% 2.53ms 2.60ms 🔴 2.43% 🔴 101%
execute/
recursive_ok
172.83µs 172.56µs ⚪ 0.12% 358.31µs 364.81µs ⚪ 1.82% 🔴 111%
execute/
recursive_scan
211.72µs 213.04µs ⚪ 0.62% 452.80µs 460.70µs 🔴 1.59% 🔴 116%
execute/
recursive_trap
17.29µs 17.31µs ⚪ -0.11% 34.89µs 34.76µs ⚪ -0.39% 🔴 101%
execute/
regex_redux
654.17µs 650.18µs ⚪ -0.49% 1.60ms 1.60ms ⚪ -0.38% 🔴 145%
execute/
rev_complement
586.95µs 595.73µs ⚪ 1.38% 1.56ms 1.55ms ⚪ -0.41% 🔴 160%
execute/
tiny_keccak
473.40µs 446.09µs 🟢 -5.72% 1.41ms 1.44ms 🔴 1.97% 🔴 223%
execute/
trunc_f2i
1.07ms 1.07ms ⚪ -0.45% 2.72ms 2.72ms ⚪ -0.02% 🔴 155%
instantiate/
wasm_kernel
69.67µs 68.91µs ⚪ -2.48% 126.64µs 95.40µs 🟢 -24.76% 🟢 38%
translate/
wasm_kernel
4.84ms 4.83ms ⚪ -0.20% 9.41ms 9.47ms ⚪ 0.74% 🟡 96%

Link to pipeline

@sergejparity sergejparity changed the title Experimental: tune up ci runners CI: change ci runners type Oct 11, 2022
Copy link
Member

@Robbepop Robbepop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @sergejparity for the research work behind this PR!
The new numbers look good, especially with respect to low variability.

@Robbepop Robbepop merged commit d31ca3f into master Oct 11, 2022
@Robbepop Robbepop deleted the sk-docker-perf-tuneup branch October 11, 2022 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: Stabilize continuous benchmarking runs
3 participants