Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Rewrite benchmark scripts in ts #2765

Merged
merged 22 commits into from
Oct 11, 2023
Merged

chore: Rewrite benchmark scripts in ts #2765

merged 22 commits into from
Oct 11, 2023

Conversation

spalladino
Copy link
Collaborator

@spalladino spalladino commented Oct 10, 2023

Adds types for all tracked metrics and stats, so the compiler will warn if we break the data we're logging on the app and expecting on the processing scripts. Introduces a new package scripts in the yarn-project workspace for typescript-based CI scripts. Also adds comparison with base benchmark on master, and shortens the bench processing job until we get better times out of it.

@spalladino spalladino changed the base branch from palla/measure-block-build-time to master October 10, 2023 15:12
@AztecBot
Copy link
Collaborator

AztecBot commented Oct 10, 2023

Benchmark results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit c7945337 and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 8 txs 32 txs 128 txs
l1_rollup_calldata_size_in_bytes 45,444 179,588 716,132
l1_rollup_calldata_gas 222,984 868,172 3,447,968
l1_rollup_execution_gas 842,071 3,595,280 22,203,337
l2_block_processing_time_in_ms 1,020 (-4%) 3,916 15,313 (-2%)
note_successful_decrypting_time_in_ms 323 (-2%) 1,107 (+8%) 3,633 (-4%)
note_trial_decrypting_time_in_ms ⚠️ 26.0 (-19%) 107 (-1%) 136
l2_block_building_time_in_ms 8,985 (-1%) 35,796 (-2%) 149,307 (-2%)
l2_block_rollup_simulation_time_in_ms 6,683 (-1%) 26,634 (-1%) 105,964 (-1%)
l2_block_public_tx_process_time_in_ms 2,263 (-2%) 9,037 (-2%) 42,780 (-3%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric 5 blocks 10 blocks
node_history_sync_time_in_ms 14,569 31,372 (+2%)
note_history_successful_decrypting_time_in_ms 2,352 4,751 (+1%)
note_history_trial_decrypting_time_in_ms 122 147 (-1%)
node_database_size_in_bytes 1,649,253 1,194,505
pxe_database_size_in_bytes 27,188 54,187

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit circuit_simulation_time_in_ms circuit_input_size_in_bytes circuit_output_size_in_bytes
private-kernel-init ⚠️ 44.0 (-23%) 56,577 14,745
private-kernel-ordering ⚠️ 21.9 (-28%) 20,137 8,089
base-rollup 851 (-2%) 631,605 811
root-rollup 40.0 (+6%) 4,072 1,097
private-kernel-inner ⚠️ 37.5 (-28%) 72,288 14,745
public-kernel-private-input 46.9 (-9%) 37,359 14,745
public-kernel-non-first-iteration 28.4 (-9%) 37,401 14,745
merge-rollup ⚠️ 1.12 (+13%) 2,592 873

@spalladino spalladino merged commit 8efa374 into master Oct 11, 2023
4 checks passed
@spalladino spalladino deleted the palla/bench-in-ts branch October 11, 2023 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants