Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

criterion: initial benchmarks #352

Closed
wants to merge 48 commits into from
Closed

Conversation

hinto-janai
Copy link
Contributor

What

Adds initial Criterion benchmarks for:

  • cuprate-helper
  • cuprate-database
  • cuprate-cryptonight
  • cuprate-rpc-types
  • cuprate-blockchain

The location of these is benches/criterion/$CRATE e.g. benches/criterion/cuprate-helper.

This PR combines #322 #324 #327 #328 #332 to prevent merge conflicts.

In-memory disk ops

cuprate-{database,blockchain} are setup to use tempfile::tempdir() which uses std::env::temp_dir() which should be in-memory (/tmp) on most Linuxes, not sure about Windows/macOS. This should remove disk-related variables and provide more accurate function runtime data.

When broader benches/benchmarks are created, those could use disk to provide more realistic real-world data.

cuprate-helper

cargo bench -p cuprate-criterion-helper
benchmark average time per call
integer 1.2848 ns
unsigned 1.2876 ns
combine_low_high_bits_to_u128 846.72 ps
split_u128_into_low_high_bits 647.49 ps
timelock_to_u64 1.6886 ns
u64_to_timelock 1.7356 ns
cmp_float 1.0841 ns
cmp_float_nan 659.60 ps
get_mid 3.9615 ns
median 55.030 ns
tx_fee 6.9573 ns

cuprate-database

cargo bench -p cuprate-criterion-database
  • redb values are only shown for >3% diffs
  • code formatting is faster
benchmark heed redb diff
ro_get 24.193 ns 21.809 ns -9.98%
ro_len 4.2254 ns
ro_first 29.797 ns
ro_last 31.675 ns 33.186 ns +4.39%
ro_is_empty 7.6418 ns
ro_contains 22.448 ns 21.696 ns -3.96%
rw_get 23.204 ns 22.107 ns -4.63%
rw_len 4.6742 ns 4.4666 ns -4.63%
rw_first 31.337 ns
rw_last 33.221 ns 34.402 ns +3.90%
rw_is_empty 7.7049 ns
rw_contains 22.874 ns 22.017 ns -3.72%
get_range 1.6075 µs
iter 1.9882 µs 1.9254 µs -3.54%
keys 766.35 ns
values 1.5660 µs
put 3.2995 µs 3.5970 µs +8.76%
delete 422.90 ns
pop_first 313.42 ns 286.33 ns -8.70%
pop_last 329.65 ns 305.60 ns -7.11%
take 4.8301 µs
env_inner 3.7474 ns
tx_ro 77.387 ns 65.183 ns -15.73%
tx_rw 33.163 ns 35.133 ns +6.00%
open_db_ro 18.273 ns
open_db_rw 17.722 ns
create_db 18.222 ns
resize 15.658 µs
current_map_size 4.2498 ns
disk_size_bytes 2.2181 µs
pre_rct_output_id_as_bytes 649.28 ps
pre_rct_output_id_from_bytes 782.82 ps
output_as_bytes 650.27 ps
output_from_bytes 5.8138 ns

cuprate-cryptonight

Benchmark naming scheme: format!("{hash_fn_name}_{}", input.len())

E.g. cuprate_cryptonight::cryptonight_hash_v0(&[0; 8]) is v0_8.

benchmark average time per call
r_8 57.091 ms
r_64 58.804 ms
r_512 58.892 ms
r_4096 58.645 ms
r_65536 58.925 ms
v0_8 25.941 ms
v0_64 26.790 ms
v0_512 26.750 ms
v0_4096 26.300 ms
v0_65536 26.787 ms
v1_8 3.4884 ns
v1_64 26.920 ms
v1_512 26.406 ms
v1_4096 26.896 ms
v1_65536 27.092 ms
v2_64 28.900 ms
v2_512 29.331 ms
v2_4096 28.683 ms
v2_65536 29.345 ms

cuprate-rpc-types

benchmark average time per call
epee_from_bytes_get_blocks_request 86.675 ns
epee_to_bytes_get_blocks_request 108.94 ns
serde_from_str_tx_entry 928.48 ns
serde_to_string_tx_entry 190.51 ns
serde_from_str_get_last_block_header_response 651.84 ns
serde_to_string_get_last_block_header_response 507.25 ns
serde_from_str_calc_pow_request 183.96 ns
serde_to_string_calc_pow_request 60.611 ns
serde_from_str_sync_info_response 1.5147 µs
serde_to_string_sync_info_response 180.23 ns
serde_from_str_get_info_response 997.70 ns
serde_to_string_get_info_response 843.08 ns
serde_from_str_get_block_response 1.7308 µs
serde_to_string_get_block_response 576.69 ns
serde_from_str_get_connections_response 1.0853 µs
serde_to_string_get_connections_response 46.413 ns
serde_from_str_get_block_template_response 469.21 ns
serde_to_string_get_block_template_response 267.68 ns
serde_from_str_get_block_headers_range_response 1.1637 µs
serde_to_string_get_block_headers_range_response 74.125 ns

cuprate-blockchain

benchmark average time per call
add_block_v1_tx2 244.05 µs
add_block_v9_tx3 54.324 µs
add_block_v16_tx0 5.4705 µs
add_alt_block_v1_tx2 1.5027 µs
add_alt_block_v9_tx3 1.7813 µs
add_alt_block_v16_tx0 1.4159 µs

@github-actions github-actions bot added A-dependency Related to dependencies, or changes to a Cargo.{toml,lock} file. A-workspace Changes to a root workspace file or general repo file. A-docs Related to documentation. A-storage Related to storage. A-benches Related to benchmarks. labels Nov 28, 2024
@hinto-janai hinto-janai mentioned this pull request Nov 28, 2024
28 tasks
@github-actions github-actions bot added the A-pruning Related to pruning. label Nov 28, 2024
@github-actions github-actions bot removed the A-pruning Related to pruning. label Nov 28, 2024
hinto-janai added a commit to Cuprate/benches that referenced this pull request Dec 3, 2024
@hinto-janai
Copy link
Contributor Author

See #354.

@hinto-janai hinto-janai closed this Dec 3, 2024
@hinto-janai hinto-janai deleted the criterion branch December 4, 2024 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-benches Related to benchmarks. A-dependency Related to dependencies, or changes to a Cargo.{toml,lock} file. A-docs Related to documentation. A-storage Related to storage. A-workspace Changes to a root workspace file or general repo file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant