Beefy: add benchmarks for `report_fork_voting()` #5188

serban300 · 2024-07-30T13:13:31Z

Related to #4523

This PR adds benchmarks for report_fork_voting().

Important: Even though the benchmarks are now available, we still use Weight::MAX. That's because I realized while working on this PR that there's still one missing piece. We should also check that the ancestry proof is optimal. I plan to do this in a future PR, hopefully the last one related to #4523.

Otherwise the tests will fail with errors related to acessing the offchain DB. That's related to the changes in pre previous commits. I couldn't find another way to fix this.

substrate/frame/beefy/src/lib.rs

Co-authored-by: Branislav Kontur <bkontur@gmail.com>

serban300 · 2024-08-13T08:01:05Z

bot bench substrate-pallet --pallet=beefy_mmr

command-bot · 2024-08-13T08:01:11Z

@serban300 "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=beefy_mmr (https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981779) was cancelled in #5188 (comment)

serban300 · 2024-08-13T08:07:06Z

bot cancel 1-3315d102-6309-4f16-98bd-90ba72271042

command-bot · 2024-08-13T08:07:25Z

@serban300 Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=beefy_mmr has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981779 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981779/artifacts/download.

serban300 · 2024-08-13T08:09:59Z

bot bench polkadot-pallet --pallet=pallet_beefy_mmr --runtime=rococo
bot bench polkadot-pallet --pallet=pallet_beefy_mmr --runtime=westend

command-bot · 2024-08-13T08:10:04Z

@serban300 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981937 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=rococo --target_dir=polkadot --pallet=pallet_beefy_mmr. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 2-34fa7219-8a87-4c39-b971-8be6f3ad18c3 to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot · 2024-08-13T08:10:09Z

@serban300 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981938 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=westend --target_dir=polkadot --pallet=pallet_beefy_mmr. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 3-35692c93-9f08-4b64-9f6f-714703d53885 to cancel this command or bot cancel to cancel all commands in this pull request.

serban300 · 2024-08-13T08:16:24Z

bot bench substrate-pallet --pallet=pallet_beefy_mmr

command-bot · 2024-08-13T08:16:30Z

@serban300 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981942 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_beefy_mmr. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 4-48f7d192-e508-4b8e-b84f-6a2a2884ad97 to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot · 2024-08-13T08:52:23Z

@serban300 Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=rococo --target_dir=polkadot --pallet=pallet_beefy_mmr has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981937 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981937/artifacts/download.

command-bot · 2024-08-13T08:56:59Z

@serban300 Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=westend --target_dir=polkadot --pallet=pallet_beefy_mmr has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981938 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981938/artifacts/download.

command-bot · 2024-08-13T09:01:51Z

@serban300 Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_beefy_mmr has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981942 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6981942/artifacts/download.

serban300 · 2024-08-13T09:39:08Z

bot bench substrate-pallet --pallet=pallet_beefy_mmr
bot bench polkadot-pallet --pallet=pallet_beefy_mmr --runtime=rococo
bot bench polkadot-pallet --pallet=pallet_beefy_mmr --runtime=westend

command-bot · 2024-08-13T12:03:20Z

@serban300 option '--runtime ' argument 'rococo' is invalid. Allowed choices are dev.

serban300 · 2024-08-13T12:29:43Z

bot bench substrate-pallet --pallet=pallet_beefy_mmr

command-bot · 2024-08-13T12:29:48Z

@serban300 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6988941 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_beefy_mmr. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 11-577ae898-98ea-48d2-ad18-b7c1443b5e49 to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot · 2024-08-13T13:15:46Z

@serban300 Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_beefy_mmr has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6988941 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6988941/artifacts/download.

This reverts commit 4a71e72.

serban300 · 2024-08-13T17:45:35Z

@acatangiu , @bkontur ,

Please, could you take another look ? I just want to check if there are any concerns related to this commit: 9fab84a . I needed a way to know whether we're in offchain context or not.

maybe also if someone from @paritytech/frame-coders could take a look it would be great

acatangiu · 2024-08-14T09:20:40Z

substrate/frame/merkle-mountain-range/src/mmr/storage.rs

+ if sp_io::offchain::is_offchain_db_ext_available() {
+ sp_io::offchain::local_storage_set(StorageKind::PERSISTENT, key, value);
+ } else {
+ sp_io::offchain_index::set(key, value);


Doesn't sp_io::offchain_index::set(key, value); check if extension is available internally?

What does it even do when offchain_db ext is not available?

AFAIU sp_io::offchain_index::set(key, value); writes to a buffer, which is commited to offchain storage if the transaction succeeds and if the offchain_db ext is available.

The problem is that when running benchmarks, the values are not commited to offchain storage. But good point, I'll check why.

I think it's because the benchmark initialization code is outside of the extrinsic, and the changes that happen there are not commited anywhere. Anyway, rolled back 9fab84a and added a helper storage variable, only for runtime-benchmarks instead.

…n-runtime

Related to #4523 This PR adds benchmarks for `report_fork_voting()`. **Important: Even though the benchmarks are now available, we still use `Weight::MAX`. That's because I realized while working on this PR that there's still one missing piece. We should also check that the ancestry proof is optimal. I plan to do this in a future PR, hopefully the last one related to #4523.** --------- Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: command-bot <>

* master: (167 commits) Upgrade accidentally downgraded deps (#5365) [Pools] Fix issues with member migration to `DelegateStake` (#4822) Unify `no_genesis` check (#5360) [CI] Fix prdoc command (#5358) Beefy: add benchmarks for `report_fork_voting()` (#5188) Fix OurViewChange small race (#5356) Make ticket non-optional and add ensure_successful method to Consideration trait (#5359) [tests] dedup test code, add more tests, improve naming and docs (#5338) Stop running the wishlist workflow on forks (#5297) Migrate foreign assets v3::Location to v4::Location (#4129) Minor clean up (#5284) [Pools] Ensure members can always exit the pool gracefully (#4998) StorageWeightReclaim: set to node pov size if higher (#5281) [Bot] Add prdoc generation (#5331) Small nits found accidentally along the way (#5341) Create subsystem-benchmarks.yml (#5325) Bump libp2p-identity from 0.2.8 to 0.2.9 (#5232) Bump authoring duration for async backing to 2s. (#5195) Fix spelling issues (#5206) Bump the known_good_semver group across 1 directory with 3 updates (#5315) ...

* master: Remove redundant minimal template workspace (#5330) approval-distribution: Fix handling of conclude (#5375) More logs in `is_potential_spam` from `dispute-coordinator` (#5252) Fix zombienet bridges test (#5373) Update Readme of the `polkadot` crate (#5326) allow for `u8` to be used as hold/freeze reason (#5348) Moving cargo check for runtimes to GHA (#5340) Update links in the documentation (#5175) fix visibility for `pallet_nfts` types used as call arguments (#3634) Correct some typos in crates' descriptions (#5262) Aura: Ensure we are building on each relay chain fork (#5352) Update Identity pallet README.md (#5183) Bump trie-db from 0.29.0 to 0.29.1 (#5231) [Coretime] Always include UnpaidExecution, not just when revenue is > 0 (#5369) [Pools] fix derivation of pool account (#4999) Upgrade accidentally downgraded deps (#5365) [Pools] Fix issues with member migration to `DelegateStake` (#4822) Unify `no_genesis` check (#5360) [CI] Fix prdoc command (#5358) Beefy: add benchmarks for `report_fork_voting()` (#5188)

…ct-candidate-weight * ao-fix-parainclusion-weight-overestimation: Remove redundant minimal template workspace (#5330) approval-distribution: Fix handling of conclude (#5375) More logs in `is_potential_spam` from `dispute-coordinator` (#5252) Fix zombienet bridges test (#5373) Update Readme of the `polkadot` crate (#5326) allow for `u8` to be used as hold/freeze reason (#5348) Moving cargo check for runtimes to GHA (#5340) Update links in the documentation (#5175) fix visibility for `pallet_nfts` types used as call arguments (#3634) Correct some typos in crates' descriptions (#5262) Aura: Ensure we are building on each relay chain fork (#5352) Update Identity pallet README.md (#5183) Bump trie-db from 0.29.0 to 0.29.1 (#5231) [Coretime] Always include UnpaidExecution, not just when revenue is > 0 (#5369) [Pools] fix derivation of pool account (#4999) Upgrade accidentally downgraded deps (#5365) [Pools] Fix issues with member migration to `DelegateStake` (#4822) Unify `no_genesis` check (#5360) [CI] Fix prdoc command (#5358) Beefy: add benchmarks for `report_fork_voting()` (#5188)

Fix report_future_block_voting weight

0ae1e63

serban300 added the T15-bridges This PR/Issue is related to bridges. label Jul 30, 2024

serban300 self-assigned this Jul 30, 2024

serban300 requested a review from acatangiu as a code owner July 30, 2024 13:13

serban300 mentioned this pull request Jul 30, 2024

Beefy: add runtime support for reporting fork voting #4523

Open

3 tasks

serban300 added 4 commits July 30, 2024 16:24

Add empty benchmarks for pallet-beefy-mmr

15a18ec

[MMR] Make offchain set/get work in benchmarks

91b34da

Ancestry proof benchmarks

8e0ea6f

Compute report_fork_voting() weight

e6599ea

serban300 force-pushed the beefy-equivocation-runtime branch from 81c805f to e6599ea Compare July 30, 2024 13:24

acatangiu requested a review from bkontur July 30, 2024 13:31

Remove impl_benchmark_test_suite!() for the MMR pallet

3f8272e

Otherwise the tests will fail with errors related to acessing the offchain DB. That's related to the changes in pre previous commits. I couldn't find another way to fix this.

acatangiu approved these changes Aug 7, 2024

View reviewed changes

acatangiu added R0-silent Changes should not be mentioned in any release notes and removed R0-silent Changes should not be mentioned in any release notes labels Aug 7, 2024

bkontur approved these changes Aug 13, 2024

View reviewed changes

substrate/frame/beefy/src/lib.rs Outdated Show resolved Hide resolved

Update TODO comment

928201f

Co-authored-by: Branislav Kontur <bkontur@gmail.com>

Add pallet_beefy_mmr benchmarks to kitchensink

4a71e72

bot bench substrate-pallet --pallet=pallet_beefy_mmr

d1fcb6e

serban300 enabled auto-merge August 13, 2024 13:22

serban300 disabled auto-merge August 13, 2024 13:37

Revert "Add pallet_beefy_mmr benchmarks to kitchensink"

6b17188

This reverts commit 4a71e72.

serban300 enabled auto-merge August 13, 2024 14:07

serban300 added 2 commits August 13, 2024 19:45

Call local_storage_set only in offline context

9fab84a

improvements

1454f87

serban300 disabled auto-merge August 13, 2024 16:48

serban300 added 2 commits August 14, 2024 08:24

Update prdoc

4d525a1

Merge branch 'master' into beefy-equivocation-runtime

e044ae4

acatangiu reviewed Aug 14, 2024

View reviewed changes

serban300 added 2 commits August 14, 2024 17:40

Remove is_offchain_db_ext_available()

c7149cc

Merge remote-tracking branch 'upstream/master' into beefy-equivocatio…

87e474e

…n-runtime

serban300 added this pull request to the merge queue Aug 14, 2024

github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Aug 14, 2024

serban300 added this pull request to the merge queue Aug 14, 2024

Merged via the queue into paritytech:master with commit 81d8f0c Aug 14, 2024
172 of 175 checks passed

serban300 deleted the beefy-equivocation-runtime branch August 14, 2024 19:00

serban300 mentioned this pull request Aug 16, 2024

Compute expected ancestry proof size paritytech/merkle-mountain-range#6

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Beefy: add benchmarks for `report_fork_voting()` #5188

Beefy: add benchmarks for `report_fork_voting()` #5188

serban300 commented Jul 30, 2024 •

edited

Loading

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 •

edited

Loading

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024 •

edited

Loading

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

acatangiu Aug 14, 2024 •

edited

Loading

serban300 Aug 14, 2024

serban300 Aug 14, 2024

Beefy: add benchmarks for report_fork_voting() #5188

Beefy: add benchmarks for report_fork_voting() #5188

Conversation

serban300 commented Jul 30, 2024 • edited Loading

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 • edited Loading

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 • edited Loading

command-bot bot commented Aug 13, 2024 • edited Loading

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 • edited Loading

command-bot bot commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

command-bot bot commented Aug 13, 2024 • edited Loading

command-bot bot commented Aug 13, 2024

serban300 commented Aug 13, 2024

acatangiu Aug 14, 2024 • edited Loading

Choose a reason for hiding this comment

serban300 Aug 14, 2024

Choose a reason for hiding this comment

serban300 Aug 14, 2024

Choose a reason for hiding this comment

Beefy: add benchmarks for `report_fork_voting()` #5188

Beefy: add benchmarks for `report_fork_voting()` #5188

serban300 commented Jul 30, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024 •

edited

Loading

command-bot bot commented Aug 13, 2024 •

edited

Loading

acatangiu Aug 14, 2024 •

edited

Loading