[bridges] Prune messages from confirmation tx body, not from the on_idle #5006

bkontur · 2024-07-11T14:07:04Z

(Please, do not merge until SA, reverted and restored of #4944)

Original PR with more context: paritytech/parity-bridges-common#2211
Relates to: paritytech/parity-bridges-common#2210

TODO

fresh weighs for pallet_bridge_messages
add try_state for pallet_bridge_messages which checks for unpruned messages - relates to the comment
~~prepare migration, that prunes leftovers, which would be pruned eventually from on_idle the comment~~ can be done also by set_storage / kill_storage or with OnRuntimeUpgrade implementatino when do_try_state_for_outbound_lanes detects problem.

Open question

Do we really need oldest_unpruned_nonce afterwards?
- after the runtime upgrade and when do_try_state_for_outbound_lanes pass, we won't need any migrations here
  - we won't even need do_try_state_for_outbound_lanes
- please check comments bellow: [bridges] Prune messages from confirmation tx body, not from the on_idle #4944 (comment)

* Define Chain::STATE_VERSION * Add vec_db module * Use VecDb instead of StorageProof for message delivery * Make sure that the TrustedVecDb is sorted * Address review comments * Run benchmarks on parent commit * Run benchmarks with new code * Fix test * Fix code review comments

* moved FromBridgedChainMessagesDeliveryProof to bp-messages * spelling * fix benchmarks compilation

* moved FromBridgedChainMessagesProof to bp-messages * fmt

* moved message files to separate folder * ...and fix paths in code * fmt rebase nit

* Adjust messages pallet benchmarks * Address comment

* moved files around * and fix compilation Move Chain::ID from relay-level Chain to primitives-level Chain (#2181) * move Chain::ID from relay-level Chain to primitives-level Chain * removed chain IDs from bp-runtime * add missing file header Adjust weights (#2185) fix clippy (#2186) Remove source header chain (#2183) * moved messages proof verification to messages pallet * removed SourceHeaderChain * cleanup * benchmarks compilation * fix benchmark tests compilation * clippy * fmt Use compact proofs for messages delivery confirmation (#2187) * Use compact proofs for messages delivery confirmation * Fix benchmarks Move messages delivery proof verification to pallet (#2189) * rename messages_proof.rs to proofs.rs * moved delivery proof verification to the messages pallet * removed TargetHeaderChain * cleaning up * fixed benchmarks compilation * Update modules/messages/README.md Co-authored-by: Adrian Catangiu <adrian@parity.io> * uncommented test and removed printlns * vec![].into_iter().collect() -> vec![].into() --------- Co-authored-by: Adrian Catangiu <adrian@parity.io> removed MaxUnrewardedRelayerEntriesAtInboundLane and MaxUnconfirmedMessagesAtInboundLane from messages pallet config (#2190) * removed MaxUnrewardedRelayerEntriesAtInboundLane and MaxUnconfirmedMessagesAtInboundLane from messages pallet config * fixed doc * fixed benchmarking code * more fixes remove MaximalOutboundPayloadSize and use BridgedChain::maximal_incomging_message_size instead (#2191) remove MessageBridge trait (#2192)

* Use compact proofs for parachains * Remove StorageProofChecker * Cleanup Reuse messages proof generation from messages pallet (#2195) * reuse messages proof generation from messages pallet * incorrect merge fixed remaining TODOs after messages pallet Config refactoring (#2196) Return UntrustedVecDb from prove_storage() (#2197) Storage proofs related renamings (#2198) * Storage proofs related renamings * Leftovers * StorageSize -> StorageProofSize

* add test for maximal message dispatch weight * fixed weight computations - use kb for both size-related benchmarks * fix typo in weight formula * decreased maximal message size to 64Kb

* Box messages proof argument in receive_messages_proof call * fix benchmark tests

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

* update async-trait to 0.1.69 * more clippy fixes

* some code grooming: enable warn(missing_docs) for all piblic crates + added missing documentation + removed obsolete clippy/deny workarounds * removed strange allow + added comment related to other allow * removed incorrect_clone_impl_on_copy_type which is unknown to CI clippy

…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages ".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages

…` with compact proofs

…mpact-proofs

…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa

…the on_idle" (#5005) Reverts #4944

Original PR with more context: paritytech/parity-bridges-common#2211 Signed-off-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>

…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages

…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages

paritytech-cicd-pr · 2024-07-11T14:11:23Z

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: check-core-crypto-features
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6677710

…port-pruning

acatangiu · 2024-08-23T09:11:53Z

Please, do not merge until SA

@bkontur What is SA? 🙈
What is blocking this?

serban300 · 2024-08-23T12:02:21Z

@bkontur What is SA? 🙈

Security Audit maybe ?

* master: (36 commits) Bump the ci_dependencies group across 1 directory with 2 updates (#5401) Remove deprecated calls in cumulus-parachain-system (#5439) Make the PR template a default for new PRs (#5462) Only log the propagating transactions when they are not empty (#5424) [CI] Fix SemVer check base commit (#5361) Sync status refactoring (#5450) Add build options to the srtool build step (#4956) `MaybeConsideration` extension trait for `Consideration` (#5384) Skip slot before creating inherent data providers during major sync (#5344) Add symlinks for code of conduct and contribution guidelines (#5447) pallet-collator-selection: correctly register weight in `new_session` (#5430) Derive `Clone` on `EncodableOpaqueLeaf` (#5442) Moving `Find FAIL-CI` check to GHA (#5377) Remove panic, as proof is invalid. (#5427) Reactive syncing metrics (#5410) [bridges] Prune messages from confirmation tx body, not from the on_idle (#5006) Change the chain to Rococo in the parachain template Zombienet config (#5279) Improve the appearance of crates on `crates.io` (#5243) Add initial version of `pallet_revive` (#5293) Update OpenZeppelin template documentation (#5398) ...

serban300 and others added 30 commits June 26, 2024 21:08

Moved FromBridgedChainMessagesDeliveryProof to bp-messages (#2169)

18b5b22

* moved FromBridgedChainMessagesDeliveryProof to bp-messages * spelling * fix benchmarks compilation

Moved FromBridgedChainMessagesProof to bp-messages (#2170)

379a600

* moved FromBridgedChainMessagesProof to bp-messages * fmt

Separate folder for messages in relay lib (#2173)

7b5b01c

* moved message files to separate folder * ...and fix paths in code * fmt rebase nit

Messages pallet benchmarks - cosmetics (#2175)

0278a54

Improve receive_messages_proof benchmarks accuracy (#2176)

9197d85

* Adjust messages pallet benchmarks * Address comment

use BridgedChain::Account instead of InbundRelayer (#2193)

acb15cc

prune messages from confirmation tx, not from the on_idle (#2211)

11cd08c

Fix maximal message dispatch weight (#2219)

e5acef8

* add test for maximal message dispatch weight * fixed weight computations - use kb for both size-related benchmarks * fix typo in weight formula * decreased maximal message size to 64Kb

Box messages proof argument in receive_messages_proof call (#2222)

5303b2c

* Box messages proof argument in receive_messages_proof call * fix benchmark tests

bump Substrate, Polkadot and Cumulus (#2223)

6019c51

Add test method for growing storage proof size (#2224)

4cbee73

fix some nightly errors (#2225)

1265806

Renamings (#2226)

8a1d006

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

Fix clippy (#2240)

ccf2919

* update async-trait to 0.1.69 * more clippy fixes

Set test-helpers for bp-runtime

f06c39f

Adjusted BH runtimes to the latest vogue

e2e7be3

Removed unused OtherBridgedChain and TargetClientBase

948e4ab

zepter+taplo

b1b6ccf

Revert wrong value + feature propagation

096aca1

Getter for raw proof and fix features

7e6b6b2

Added tests for macro compatibility with pallet_bridge_messages + fix

c8a40ef

Updated new substrate-relay version `v2.0.0-rc1-compact-proof-ro-we…

cdd1280

…` with compact proofs

Merge remote-tracking branch 'upstream/master' into bko-bridges-v2-co…

bcf8050

…mpact-proofs

Revert compact-proofs-related changes (runtimes)

ded4e86

Remove compact proofs so they could be restored in different branch

4c94e43

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…

953e4bd

…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa

bkontur added the T15-bridges This PR/Issue is related to bridges. label Jul 11, 2024

Revert "[bridges] Prune messages from confirmation tx body, not from …

397dd61

…the on_idle" (#5005) Reverts #4944

Base automatically changed from revert-4944-bko-bridges-v2-backport-pruning to bko-bridges-v2-backport-refactoring July 11, 2024 14:08

bkontur marked this pull request as ready for review July 11, 2024 14:08

paritytech-review-bot bot requested a review from a team July 11, 2024 14:09

bkontur force-pushed the bko-bridges-v2-backport-pruning branch from fb8d0ba to a14c7c9 Compare July 11, 2024 14:10

bkontur and others added 5 commits July 11, 2024 16:11

prune messages from confirmation tx, not from the on_idle (#2211)

9b90f4b

Original PR with more context: paritytech/parity-bridges-common#2211 Signed-off-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>

Added do_try_state_for_outbound_lanes which checks unpruned messages

92515b8

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…

885ffbc

…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…

63d1074

…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages

Aviod iteration all the time to the beginning

59003a5

bkontur force-pushed the bko-bridges-v2-backport-pruning branch from a14c7c9 to 59003a5 Compare July 11, 2024 14:11

bkontur added the R0-silent Changes should not be mentioned in any release notes label Jul 11, 2024

serban300 approved these changes Jul 11, 2024

View reviewed changes

acatangiu approved these changes Jul 11, 2024

View reviewed changes

Base automatically changed from bko-bridges-v2-backport-refactoring to master July 12, 2024 08:48

bkontur added 2 commits July 12, 2024 12:57

Merge remote-tracking branch 'origin/master' into bko-bridges-v2-back…

7be729a

…port-pruning

Merge remote-tracking branch 'origin/master' into bko-bridges-v2-back…

7ec9b12

…port-pruning

bkontur self-assigned this Jul 16, 2024

acatangiu added this pull request to the merge queue Aug 23, 2024

Merged via the queue into master with commit e4ffba6 Aug 23, 2024
155 of 159 checks passed

acatangiu deleted the bko-bridges-v2-backport-pruning branch August 23, 2024 12:52

github-actions bot mentioned this pull request Oct 8, 2024

Update polkadot-sdk from stable2407 to stable2409 moonbeam-foundation/moonbeam#2994

Closed

github-actions bot mentioned this pull request Dec 5, 2024

Update polkadot-sdk from stable2407 to stable2409-2 moonbeam-foundation/moonbeam#3075

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bridges] Prune messages from confirmation tx body, not from the on_idle #5006

[bridges] Prune messages from confirmation tx body, not from the on_idle #5006

bkontur commented Jul 11, 2024 •

edited

Loading

paritytech-cicd-pr commented Jul 11, 2024

acatangiu commented Aug 23, 2024

serban300 commented Aug 23, 2024

[bridges] Prune messages from confirmation tx body, not from the on_idle #5006

[bridges] Prune messages from confirmation tx body, not from the on_idle #5006

Conversation

bkontur commented Jul 11, 2024 • edited Loading

TODO

Open question

paritytech-cicd-pr commented Jul 11, 2024

acatangiu commented Aug 23, 2024

serban300 commented Aug 23, 2024

bkontur commented Jul 11, 2024 •

edited

Loading