Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(levm): Add BubbleSort benchmark #1839

Merged
merged 5 commits into from
Jan 31, 2025
Merged

Conversation

dsocolobsky
Copy link
Contributor

Motivation
Adding more benchmarks to have an idea of how we fare vs other implementations and find possible improvements.

Description
We add a new benchmark:

  • BubbleSort: Creates an array of n random elements, sorts it via bubble-sort and then checks the array is sorted.

@dsocolobsky dsocolobsky requested a review from a team as a code owner January 29, 2025 19:46
Copy link

github-actions bot commented Jan 29, 2025

| File                                                                                | Lines | Diff |
+-------------------------------------------------------------------------------------+-------+------+
| /home/runner/work/ethrex/ethrex/crates/vm/levm/bench/revm_comparison/src/compile.rs | 69    | +6   |
+-------------------------------------------------------------------------------------+-------+------+

Total lines added: +6
Total lines removed: 0
Total lines changed: 6

Copy link

Summary: 6475/6475 (100.00%)

Prague: 2373/2373 (100.00%)
Cancun: 3579/3579 (100.00%)
Shanghai: 221/221 (100.00%)
Paris: 62/62 (100.00%)
London: 39/39 (100.00%)
Berlin: 35/35 (100.00%)
Istanbul: 34/34 (100.00%)
Constantinople: 66/66 (100.00%)
Byzantium: 33/33 (100.00%)
Homestead: 17/17 (100.00%)
Frontier: 16/16 (100.00%)

Copy link

github-actions bot commented Jan 29, 2025

Benchmark Results Comparison

PR Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 234.7 ± 0.8 233.3 235.7 1.00
levm_Factorial 902.4 ± 4.0 897.8 910.7 3.84 ± 0.02

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.448 ± 0.106 1.324 1.585 1.00
levm_FactorialRecursive 15.837 ± 0.068 15.755 15.927 10.94 ± 0.80

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 211.5 ± 2.0 207.6 215.6 1.00
levm_Fibonacci 910.9 ± 7.8 903.0 924.4 4.31 ± 0.06

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.7 ± 0.1 8.5 8.9 1.00
levm_ManyHashes 18.0 ± 0.1 17.9 18.1 2.08 ± 0.03

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
revm_BubbleSort 3.204 ± 0.022 3.176 3.234 1.00
levm_BubbleSort 6.189 ± 0.133 6.120 6.564 1.93 ± 0.04

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 251.5 ± 7.2 247.1 271.1 1.00
levm_ERC20Transfer 3214.8 ± 23.5 3187.6 3246.3 12.78 ± 0.38

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 142.0 ± 1.0 140.2 143.5 1.00
levm_ERC20Mint 1699.0 ± 7.9 1687.7 1710.3 11.97 ± 0.10

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.057 ± 0.022 1.039 1.100 1.00
levm_ERC20Approval 11.386 ± 0.060 11.291 11.458 10.77 ± 0.23

Main Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 237.7 ± 2.2 235.3 242.3 1.00
levm_Factorial 906.1 ± 11.3 897.1 935.3 3.81 ± 0.06

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.417 ± 0.081 1.324 1.546 1.00
levm_FactorialRecursive 15.767 ± 0.029 15.730 15.816 11.12 ± 0.64

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 213.5 ± 3.1 211.1 220.3 1.00
levm_Fibonacci 912.6 ± 9.1 901.6 929.6 4.27 ± 0.08

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.6 ± 0.0 8.5 8.7 1.00
levm_ManyHashes 18.0 ± 0.1 17.9 18.2 2.09 ± 0.02

Benchmark Results: BubbleSort

No results

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 251.8 ± 1.0 251.0 254.3 1.00
levm_ERC20Transfer 3225.6 ± 44.4 3186.4 3329.8 12.81 ± 0.18

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 141.4 ± 2.0 139.9 146.0 1.00
levm_ERC20Mint 1696.0 ± 10.5 1677.0 1713.8 11.99 ± 0.18

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.046 ± 0.005 1.035 1.051 1.00
levm_ERC20Approval 11.374 ± 0.144 11.272 11.730 10.88 ± 0.15

Copy link
Contributor

@fborello-lambda fborello-lambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@fborello-lambda fborello-lambda added this pull request to the merge queue Jan 31, 2025
Merged via the queue into main with commit 0ad6473 Jan 31, 2025
28 checks passed
@fborello-lambda fborello-lambda deleted the levm/benchs/bubblesort branch January 31, 2025 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants