[FRAME Core] tracking issue for full unleash of `DefaultConfig` for FRAME #171

kianenigma · 2023-05-26T11:37:20Z

paritytech/substrate#13454 will bring in the basic functionality needed to start using this feature. See this example to learn more.

This will mark the first version of this work complete, but a lot has to be done in order for it to be fully complete.

Improve Error Handling, Bugs

Improve error handling. There seems to be an error in the parse/config.rs file that allows #[pallet::whatever] to be parsed and ignored in #[pallet::config]. This has to be fixed.
Properly hide __export_tokens_tt_test_default_config and similar from docs.
Handle pallet instancing.

Full usage in Substrate Pallets

migrate the docs for derive_impl to use real code. As all of this functionality can work on normal traits, and as per add new documentation guidelines substrate#14115, we should strive to make the docs of derive_impl be real rust code. We might even want to remove ./frame/example/default-config in favor this.
Provide testing defaults for common pallets
- System: already provided
- Balances add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453
- Assets Adds default config for assets pallet #3637
Then, we should start migrating existing pallets and their test setup to use the unified testing prelude of system, balances and so on. Here's the tracking issue: [FRAME Core] remove unnecessary overrides while using derive_impl for frame_system #3237

Full Usage in `node-template-runtime` and `parachain-template-runtime`

We should provide runtime templates that utilize this, evangelising its usage. This requires us to carefully consider what is a reasonable default for a node template in order to use. We should provide possibly two new preludes:

Full usage in `westend` -> `kusama` -> `polkadot` runtimes.

Once we have are more and more confident in both the core functionality, and our skills to define sensible defaults, we can slowly start migrating perhaps some of the configs define in Polkadot to use this feature.

For security reasons we should be super caution here. Adding #[derive_impl] will open the door for upstream changes manipulating the configurations used in the Polkadot runtime. Perhaps we should only ever do this once we have more tooling that could notify us of unexpected upstream changes to config_preludes.

I would personally find it reasonable to never use this feature in production runtimes for this reason.

V2

At the moment, the system that is introduced is has some limitations. Most notably:

it cannot provide default for types that are reliant on types that are themselves reliant on frame_system::Config. For example, a type Admin: Get<Self::AccountId> cannot have a default. Is is mainly because trait DefaultConfig is not in itself trait DefaultConfig: frame_system::DefaultConfig. Moreover, sadly, because the definition of BalanceOf<T> (somewhat needlessly) also relies on frame_system::Config::AccountId, anything akin to Get<BalanceOf<Self>> can also not have a sensible default. add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453

I believe this is father easy to fix, by standardizing frame_system::DefaultConfig and making trait DefaultConfig: frame_system::DefaultConfig.

If the above is done, we are only left with types that cannot really have a default in any pallet. Such types are: PalletInfo, RuntimeEvent, RuntimeCall and such. Such types can also be mocked but they require more coordination with construct_runtime!. A new attribute can mark these types simply as #[derive_from_runtime] or similar, which would mean they are expected to be found in the existing scope with the same name. Adds ability to provide defaults for types provided by construct_runtime substrate#14682

Other approaches could be explored for the last step, as I am not quite sure what is the best way forward.

Improvements

Remove as frame_system::DefaultConfig from the required syntax Remove as frame_system::DefaultConfig from the required syntax in derive_impl #3505

The text was updated successfully, but these errors were encountered:

bkchr · 2023-05-29T11:50:24Z

it cannot provide default for types that are reliant on types that are themselves reliant on frame_system::Config. For example, a type Admin: Get<Self::AccountId> cannot have a default. Is is mainly because trait DefaultConfig is not in itself trait DefaultConfig: frame_system::DefaultConfig. Moreover, sadly, because the definition of BalanceOf<T> (somewhat needlessly) also relies on frame_system::Config::AccountId, anything akin to Get<BalanceOf<Self>> can also not have a sensible default.

Personally I would say that we do this before we do all the other things, otherwise you will start changing all this code twice. I already left a comment here on how it could work: paritytech/substrate#13454 (comment)

In general to this feature, I had some idea recently which could improve the versioning around our crates. Currently when we are adding a new type to a Config trait it requires that we bump the major version of the crate. If we have the system outlined here working and it is being used everywhere. (I would even go that far and say it is fine to use it in production runtimes, at least having the attribute above each Config implementation) Then we could introduce some kind of "versioning" of these Config traits. Then every time you would add a new config type, you would bump this version in your pallet. We would then only require to bump the minor version. Downstream users would then at some point bump the pallet minor version. It would still compile, because the attribute above the impl Config would put there some safe/secure default value. But it would also generate a warning that tells the user that it put there some new type. It could even show some more context to the new type added. The user can accept the new safe default value by bumping the "trait version to use" or they just add the type on their own and bump the trait version in use.
Not sure if this idea is something we should really do, but I wanted to leave it here :P

sam0x17 · 2023-06-25T12:57:41Z

is the parse issue still present with #[pallet::whatever]?

Checked off some other things in here that are now fixed as of the recent macro_magic overhaul

sam0x17 · 2023-07-04T10:55:12Z

potential vast improvement: paritytech/substrate#14437 (comment)

Moving a few pallets to the latest and greatest `derive_impl` to give it a try. Part of #171 --------- Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

Step in #171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in #3505

Step in paritytech/polkadot-sdk#171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in paritytech/polkadot-sdk#3505 (cherry picked from commit 7099f6e)

Step in paritytech#171

…ch#3652) Step in paritytech#171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in paritytech#3505

* Add two new zombienet tests for bridges (manual run) (#3072) extracted useful code from #2982 This PR: - adds test 2 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers while there are no any messages; - adds test 3 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers when there are messages; - fixes most of comments from #2439 (like: log names, ability to run specify test number when calling `run-tests.sh`). Right now of all our tests, only test 2 is working (until BHs will be upgraded to use async backing), so you can test it with `./bridges/zombienet/run-tests.sh --test 2` locally. (cherry picked from commit 2e6067d) * [cumulus] Improved check for sane bridge fees calculations (#3175) - [x] change constants when CI fails (should fail :) ) On the AssetHubRococo: 1701175800126 -> 1700929825257 = 0.15 % decreased. ``` Feb 02 12:59:05.520 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1701175800126 for runtime: statemine-1006000 (statemine-0.tx14.au1) Feb 02 13:02:40.647 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1700929825257 for runtime: statemine-1006000 (statemine-0.tx14.au1) ``` On the AssetHubWestend: 2116038876326 -> 1641718372993 = 22.4 % decreased. ``` Feb 02 12:56:00.880 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 2116038876326 for runtime: westmint-1006000 (westmint-0.tx14.au1) Feb 02 13:04:42.515 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1641718372993 for runtime: westmint-1006000 (westmint-0.tx14.au1) ``` (cherry picked from commit 74b597f) * Enable async backing on all testnet system chains (#2949) Built on top of paritytech/polkadot-sdk#2826 which was a trial run. Guide: https://github.com/w3f/polkadot-wiki/blob/master/docs/maintain/maintain-guides-async-backing.md --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> (cherry picked from commit 700d5f8) * Introduce submit_finality_proof_ex call to bridges GRANDPA pallet (#3225) backport of #2821 (see detailed description there) (cherry picked from commit a462207) * Bridge zombienet tests refactoring (#3260) Related to paritytech/polkadot-sdk#3242 Reorganizing the bridge zombienet tests in order to: - separate the environment spawning from the actual tests - offer better control over the tests and some possibility to orchestrate them as opposed to running everything from the zndsl file Only rewrote the asset transfer test using this new "framework". The old logic and old tests weren't functionally modified or deleted. The plan is to get feedback on this approach first and if this is agreed upon, migrate the other 2 tests later in separate PRs and also do other improvements later. (cherry picked from commit dfc8e46) * Bridges: add test 0002 to CI (#3310) Bridges: add test 0002 to CI (cherry picked from commit 1b66bb5) * Bridge zombienet tests - move all test scripts to the same folder (#3333) Related to paritytech/polkadot-sdk#3242 (cherry picked from commit 5fc7622) * Lift dependencies to the workspace (Part 2/x) (#3366) Lifting some more dependencies to the workspace. Just using the most-often updated ones for now. It can be reproduced locally. ```sh $ zepter transpose dependency lift-to-workspace --ignore-errors syn quote thiserror "regex:^serde.*" $ zepter transpose dependency lift-to-workspace --version-resolver=highest syn quote thiserror "regex:^serde.*" --fix $ taplo format --config .config/taplo.toml ``` --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> (cherry picked from commit e89d0fc) * Add support for BHP local and BHK local (#3443) Related to paritytech/polkadot-sdk#3400 Extracting small parts of paritytech/polkadot-sdk#3429 into separate PR: - Add support for BHP local and BHK local - Increase the timeout for the bridge zomienet tests (cherry picked from commit e4b6b8c) * Bridge zombienet tests: move all "framework" files under one folder (#3462) Related to paritytech/polkadot-sdk#3400 Moving all bridges testing "framework" files under one folder in order to be able to download the entire folder when we want to add tests in other repos No significant functional changes (cherry picked from commit 6fc1d41) * Bridge zombienet tests: Check amount received at destination (#3490) Related to paritytech/polkadot-sdk#3475 (cherry picked from commit 2cdda0e) * FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280) Closes #2160 First part of [Extrinsic Horizon](paritytech/polkadot-sdk#2415) Introduces a new trait `TransactionExtension` to replace `SignedExtension`. Introduce the idea of transactions which obey the runtime's extensions and have according Extension data (né Extra data) yet do not have hard-coded signatures. Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have __*General*__ for the former and __*Bare*__ for the latter. (Ultimately, the latter will be phased out as a type of transaction, and Bare will only be used for Inherents.) Types of extrinsic are now therefore: - Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned") - Bare transactions (deprecated): Gossiped, validated with `ValidateUnsigned` (deprecated) and the `_bare_compat` bits of `TransactionExtension` (deprecated). - Inherents: Not gossiped, validated with `ProvideInherent`. - Extended (Extra data): Gossiped, validated via `TransactionExtension`. - Signed transactions (with a hardcoded signature). - General transactions (without a hardcoded signature). `TransactionExtension` differs from `SignedExtension` because: - A signature on the underlying transaction may validly not be present. - It may alter the origin during validation. - `pre_dispatch` is renamed to `prepare` and need not contain the checks present in `validate`. - `validate` and `prepare` is passed an `Origin` rather than a `AccountId`. - `validate` may pass arbitrary information into `prepare` via a new user-specifiable type `Val`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. It is encoded *for the entire transaction* and passed in to each extension as a new argument to `validate`. This facilitates the ability of extensions to acts as underlying crypto. There is a new `DispatchTransaction` trait which contains only default function impls and is impl'ed for any `TransactionExtension` impler. It provides several utility functions which reduce some of the tedium from using `TransactionExtension` (indeed, none of its regular functions should now need to be called directly). Three transaction version discriminator ("versions") are now permissible: - 0b000000100: Bare (used to be called "Unsigned"): contains Signature or Extra (extension data). After bare transactions are no longer supported, this will strictly identify an Inherents only. - 0b100000100: Old-school "Signed" Transaction: contains Signature and Extra (extension data). - 0b010000100: New-school "General" Transaction: contains Extra (extension data), but no Signature. For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above. Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be accompanied by renaming your aggregate type in line with the new terminology. E.g. Before: ```rust /// The SignedExtension to the basic transaction logic. pub type SignedExtra = ( /* snip */ MySpecialSignedExtension, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>; ``` After: ```rust /// The extension to the basic transaction logic. pub type TxExtension = ( /* snip */ AsTransactionExtension<MySpecialSignedExtension>, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>; ``` You'll also need to alter any transaction building logic to add a `.into()` to make the conversion happen. E.g. Before: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let extra: SignedExtra = ( /* snip */ MySpecialSignedExtension::new(/* snip */), ); let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), extra, ) } ``` After: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let tx_ext: TxExtension = ( /* snip */ MySpecialSignedExtension::new(/* snip */).into(), ); let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), tx_ext, ) } ``` Most `SignedExtension`s can be trivially converted to become a `TransactionExtension`. There are a few things to know. - Instead of a single trait like `SignedExtension`, you should now implement two traits individually: `TransactionExtensionBase` and `TransactionExtension`. - Weights are now a thing and must be provided via the new function `fn weight`. This trait takes care of anything which is not dependent on types specific to your runtime, most notably `Call`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. - Weight must be returned by implementing the `weight` function. If your extension is associated with a pallet, you'll probably want to do this via the pallet's existing benchmarking infrastructure. Generally: - `pre_dispatch` is now `prepare` and you *should not reexecute the `validate` functionality in there*! - You don't get an account ID any more; you get an origin instead. If you need to presume an account ID, then you can use the trait function `AsSystemOriginSigner::as_system_origin_signer`. - You get an additional ticket, similar to `Pre`, called `Val`. This defines data which is passed from `validate` into `prepare`. This is important since you should not be duplicating logic from `validate` to `prepare`, you need a way of passing your working from the former into the latter. This is it. - This trait takes two type parameters: `Call` and `Context`. `Call` is the runtime call type which used to be an associated type; you can just move it to become a type parameter for your trait impl. `Context` is not currently used and you can safely implement over it as an unbounded type. - There's no `AccountId` associated type any more. Just remove it. Regarding `validate`: - You get three new parameters in `validate`; all can be ignored when migrating from `SignedExtension`. - `validate` returns a tuple on success; the second item in the tuple is the new ticket type `Self::Val` which gets passed in to `prepare`. If you use any information extracted during `validate` (off-chain and on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can pass it through with this. For the tuple's last item, just return the `origin` argument. Regarding `prepare`: - This is renamed from `pre_dispatch`, but there is one change: - FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM `validate`!! - (This is different to `SignedExtension` which was required to run the same checks in `pre_dispatch` as in `validate`.) Regarding `post_dispatch`: - Since there are no unsigned transactions handled by `TransactionExtension`, `Pre` is always defined, so the first parameter is `Self::Pre` rather than `Option<Self::Pre>`. If you make use of `SignedExtension::validate_unsigned` or `SignedExtension::pre_dispatch_unsigned`, then: - Just use the regular versions of these functions instead. - Have your logic execute in the case that the `origin` is `None`. - Ensure your transaction creation logic creates a General Transaction rather than a Bare Transaction; this means having to include all `TransactionExtension`s' data. - `ValidateUnsigned` can still be used (for now) if you need to be able to construct transactions which contain none of the extension data, however these will be phased out in stage 2 of the Transactions Horizon, so you should consider moving to an extension-centric design. - [x] Introduce `CheckSignature` impl of `TransactionExtension` to ensure it's possible to have crypto be done wholly in a `TransactionExtension`. - [x] Deprecate `SignedExtension` and move all uses in codebase to `TransactionExtension`. - [x] `ChargeTransactionPayment` - [x] `DummyExtension` - [x] `ChargeAssetTxPayment` (asset-tx-payment) - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment) - [x] `CheckWeight` - [x] `CheckTxVersion` - [x] `CheckSpecVersion` - [x] `CheckNonce` - [x] `CheckNonZeroSender` - [x] `CheckMortality` - [x] `CheckGenesis` - [x] `CheckOnlySudoAccount` - [x] `WatchDummy` - [x] `PrevalidateAttests` - [x] `GenericSignedExtension` - [x] `SignedExtension` (chain-polkadot-bulletin) - [x] `RefundSignedExtensionAdapter` - [x] Implement `fn weight` across the board. - [ ] Go through all pre-existing extensions which assume an account signer and explicitly handle the possibility of another kind of origin. - [x] `CheckNonce` should probably succeed in the case of a non-account origin. - [x] `CheckNonZeroSender` should succeed in the case of a non-account origin. - [x] `ChargeTransactionPayment` and family should fail in the case of a non-account origin. - [ ] - [x] Fix any broken tests. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Chevdor <chevdor@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io> Co-authored-by: Javier Viola <javier@parity.io> Co-authored-by: Marcin S. <marcin@realemail.net> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Javier Bullrich <javier@bullrich.dev> Co-authored-by: Koute <koute@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: Vladimir Istyufeev <vladimir@parity.io> Co-authored-by: Ross Bulat <ross@parity.io> Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Sebastian Kunert <skunert49@gmail.com> Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com> Co-authored-by: Julian Eager <eagr@tutanota.com> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Davide Galassi <davxy@datawok.net> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: yjh <yjh465402634@gmail.com> Co-authored-by: Tom Mi <tommi@niemi.lol> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com> Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com> Co-authored-by: PG Herveou <pgherveou@gmail.com> Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Juan Girini <juangirini@gmail.com> Co-authored-by: bader y <ibnbassem@gmail.com> Co-authored-by: James Wilson <james@jsdw.me> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Parth <desaiparth08@gmail.com> Co-authored-by: Andrew Jones <ascjones@gmail.com> Co-authored-by: Jonathan Udd <jonathan@dwellir.com> Co-authored-by: Serban Iorga <serban@parity.io> Co-authored-by: Egor_P <egor@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Evgeny Snitko <evgeny@parity.io> Co-authored-by: Just van Stam <vstam1@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro> Co-authored-by: zhiqiangxu <652732310@qq.com> Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com> Co-authored-by: Anwesh <anweshknayak@gmail.com> Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> Co-authored-by: Muharem <ismailov.m.h@gmail.com> Co-authored-by: joepetrowski <joe@parity.io> Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com> Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com> Co-authored-by: Squirrel <gilescope@gmail.com> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: command-bot <> (cherry picked from commit fd5f929) * Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665) This PR reverts #2280 which introduced `TransactionExtension` to replace `SignedExtension`. As a result of the discussion [here](paritytech/polkadot-sdk#3623 (comment)), the changes will be reverted for now with plans to reintroduce the concept in the future. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> (cherry picked from commit bbd51ce) * Increase timeout for assertions (#3680) Prevents timeouts in ci like https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5516019 (cherry picked from commit c4c9257) * Removes `as [disambiguation_path]` from `derive_impl` usage (#3652) Step in paritytech/polkadot-sdk#171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in paritytech/polkadot-sdk#3505 (cherry picked from commit 7099f6e) * Fix typo (#3691) (cherry picked from commit 6b1179f) * Bridge zombienet tests: remove unneeded accounts (#3700) Bridge zombienet tests: remove unneeded accounts (cherry picked from commit 0c6c837) * Fix typos (#3753) (cherry picked from commit 7241a8d) * Update polkadot-sdk refs * Fix dependency conflicts * Fix build * cargo fmt * Fix spellcheck test --------- Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: jokess123 <163112061+jokess123@users.noreply.github.com> Co-authored-by: slicejoke <163888128+slicejoke@users.noreply.github.com>

…tytech#1636) Moving a few pallets to the latest and greatest `derive_impl` to give it a try. Part of paritytech#171 --------- Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

Moving a babe and authorship pallets to the latest and greatest derive_impl. Part of paritytech#171 --------- Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Part of paritytech#171

Step in paritytech#171 This PR adds `derive_impl` on all `frame_system` config impls for mock runtimes. The overridden configs are maintained as of now to ensure minimal changes. --------- Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Step in paritytech#171 This PR fixes a bug in `derive_impl` causing `#[inject_runtime_type]` attribute to be parsed incorrectly when docs are added.

…ytech#1801) ## Summary This PR consolidates `pallet-state-trie-migration` as a part of paritytech#226 / paritytech#171: `pallet-state-trie-migration`: - [x] replace `Currency` with `fungible` traits - [x] run benchmarks - [x] refactor to `DefaultConfig` `pallet_nicks`: - [x] remove others: - [x] remove `as Fn*` or `asFun*` stuff based on discussion [here](paritytech#226 (comment)) --------- Co-authored-by: Richard Melkonian <35300528+0xmovses@users.noreply.github.com> Co-authored-by: command-bot <>

…derive_impl` (paritytech#3505) Step in paritytech#171 This PR removes the need to specify `as [disambiguation_path]` for cases where the trait definition resides within the same scope as default impl path. For example, in the following macro invocation ```rust #[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)] impl frame_system::Config for Runtime { ... } ``` the trait `DefaultConfig` lies within the `frame_system` scope and `TestDefaultConfig` impls the `DefaultConfig` trait. Using this information, we can compute the disambiguation path internally, thus removing the need of an explicit specification. In cases where the trait lies outside this scope, we would still need to specify it explicitly, but this should take care of most (if not all) uses of `derive_impl` within FRAME's context.

Step in paritytech#171

* Add two new zombienet tests for bridges (manual run) (paritytech#3072) extracted useful code from paritytech#2982 This PR: - adds test 2 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers while there are no any messages; - adds test 3 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers when there are messages; - fixes most of comments from paritytech#2439 (like: log names, ability to run specify test number when calling `run-tests.sh`). Right now of all our tests, only test 2 is working (until BHs will be upgraded to use async backing), so you can test it with `./bridges/zombienet/run-tests.sh --test 2` locally. (cherry picked from commit 2e6067d) * [cumulus] Improved check for sane bridge fees calculations (#3175) - [x] change constants when CI fails (should fail :) ) On the AssetHubRococo: 1701175800126 -> 1700929825257 = 0.15 % decreased. ``` Feb 02 12:59:05.520 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1701175800126 for runtime: statemine-1006000 (statemine-0.tx14.au1) Feb 02 13:02:40.647 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1700929825257 for runtime: statemine-1006000 (statemine-0.tx14.au1) ``` On the AssetHubWestend: 2116038876326 -> 1641718372993 = 22.4 % decreased. ``` Feb 02 12:56:00.880 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 2116038876326 for runtime: westmint-1006000 (westmint-0.tx14.au1) Feb 02 13:04:42.515 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1641718372993 for runtime: westmint-1006000 (westmint-0.tx14.au1) ``` (cherry picked from commit 74b597f) * Enable async backing on all testnet system chains (paritytech#2949) Built on top of paritytech/polkadot-sdk#2826 which was a trial run. Guide: https://github.com/w3f/polkadot-wiki/blob/master/docs/maintain/maintain-guides-async-backing.md --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> (cherry picked from commit 700d5f8) * Introduce submit_finality_proof_ex call to bridges GRANDPA pallet (#3225) backport of paritytech#2821 (see detailed description there) (cherry picked from commit a462207) * Bridge zombienet tests refactoring (#3260) Related to paritytech/polkadot-sdk#3242 Reorganizing the bridge zombienet tests in order to: - separate the environment spawning from the actual tests - offer better control over the tests and some possibility to orchestrate them as opposed to running everything from the zndsl file Only rewrote the asset transfer test using this new "framework". The old logic and old tests weren't functionally modified or deleted. The plan is to get feedback on this approach first and if this is agreed upon, migrate the other 2 tests later in separate PRs and also do other improvements later. (cherry picked from commit dfc8e46) * Bridges: add test 0002 to CI (#3310) Bridges: add test 0002 to CI (cherry picked from commit 1b66bb5) * Bridge zombienet tests - move all test scripts to the same folder (#3333) Related to paritytech/polkadot-sdk#3242 (cherry picked from commit 5fc7622) * Lift dependencies to the workspace (Part 2/x) (#3366) Lifting some more dependencies to the workspace. Just using the most-often updated ones for now. It can be reproduced locally. ```sh $ zepter transpose dependency lift-to-workspace --ignore-errors syn quote thiserror "regex:^serde.*" $ zepter transpose dependency lift-to-workspace --version-resolver=highest syn quote thiserror "regex:^serde.*" --fix $ taplo format --config .config/taplo.toml ``` --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> (cherry picked from commit e89d0fc) * Add support for BHP local and BHK local (#3443) Related to paritytech/polkadot-sdk#3400 Extracting small parts of paritytech/polkadot-sdk#3429 into separate PR: - Add support for BHP local and BHK local - Increase the timeout for the bridge zomienet tests (cherry picked from commit e4b6b8c) * Bridge zombienet tests: move all "framework" files under one folder (#3462) Related to paritytech/polkadot-sdk#3400 Moving all bridges testing "framework" files under one folder in order to be able to download the entire folder when we want to add tests in other repos No significant functional changes (cherry picked from commit 6fc1d41) * Bridge zombienet tests: Check amount received at destination (#3490) Related to paritytech/polkadot-sdk#3475 (cherry picked from commit 2cdda0e) * FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280) Closes paritytech#2160 First part of [Extrinsic Horizon](paritytech/polkadot-sdk#2415) Introduces a new trait `TransactionExtension` to replace `SignedExtension`. Introduce the idea of transactions which obey the runtime's extensions and have according Extension data (né Extra data) yet do not have hard-coded signatures. Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have __*General*__ for the former and __*Bare*__ for the latter. (Ultimately, the latter will be phased out as a type of transaction, and Bare will only be used for Inherents.) Types of extrinsic are now therefore: - Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned") - Bare transactions (deprecated): Gossiped, validated with `ValidateUnsigned` (deprecated) and the `_bare_compat` bits of `TransactionExtension` (deprecated). - Inherents: Not gossiped, validated with `ProvideInherent`. - Extended (Extra data): Gossiped, validated via `TransactionExtension`. - Signed transactions (with a hardcoded signature). - General transactions (without a hardcoded signature). `TransactionExtension` differs from `SignedExtension` because: - A signature on the underlying transaction may validly not be present. - It may alter the origin during validation. - `pre_dispatch` is renamed to `prepare` and need not contain the checks present in `validate`. - `validate` and `prepare` is passed an `Origin` rather than a `AccountId`. - `validate` may pass arbitrary information into `prepare` via a new user-specifiable type `Val`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. It is encoded *for the entire transaction* and passed in to each extension as a new argument to `validate`. This facilitates the ability of extensions to acts as underlying crypto. There is a new `DispatchTransaction` trait which contains only default function impls and is impl'ed for any `TransactionExtension` impler. It provides several utility functions which reduce some of the tedium from using `TransactionExtension` (indeed, none of its regular functions should now need to be called directly). Three transaction version discriminator ("versions") are now permissible: - 0b000000100: Bare (used to be called "Unsigned"): contains Signature or Extra (extension data). After bare transactions are no longer supported, this will strictly identify an Inherents only. - 0b100000100: Old-school "Signed" Transaction: contains Signature and Extra (extension data). - 0b010000100: New-school "General" Transaction: contains Extra (extension data), but no Signature. For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above. Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be accompanied by renaming your aggregate type in line with the new terminology. E.g. Before: ```rust /// The SignedExtension to the basic transaction logic. pub type SignedExtra = ( /* snip */ MySpecialSignedExtension, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>; ``` After: ```rust /// The extension to the basic transaction logic. pub type TxExtension = ( /* snip */ AsTransactionExtension<MySpecialSignedExtension>, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>; ``` You'll also need to alter any transaction building logic to add a `.into()` to make the conversion happen. E.g. Before: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let extra: SignedExtra = ( /* snip */ MySpecialSignedExtension::new(/* snip */), ); let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), extra, ) } ``` After: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let tx_ext: TxExtension = ( /* snip */ MySpecialSignedExtension::new(/* snip */).into(), ); let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), tx_ext, ) } ``` Most `SignedExtension`s can be trivially converted to become a `TransactionExtension`. There are a few things to know. - Instead of a single trait like `SignedExtension`, you should now implement two traits individually: `TransactionExtensionBase` and `TransactionExtension`. - Weights are now a thing and must be provided via the new function `fn weight`. This trait takes care of anything which is not dependent on types specific to your runtime, most notably `Call`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. - Weight must be returned by implementing the `weight` function. If your extension is associated with a pallet, you'll probably want to do this via the pallet's existing benchmarking infrastructure. Generally: - `pre_dispatch` is now `prepare` and you *should not reexecute the `validate` functionality in there*! - You don't get an account ID any more; you get an origin instead. If you need to presume an account ID, then you can use the trait function `AsSystemOriginSigner::as_system_origin_signer`. - You get an additional ticket, similar to `Pre`, called `Val`. This defines data which is passed from `validate` into `prepare`. This is important since you should not be duplicating logic from `validate` to `prepare`, you need a way of passing your working from the former into the latter. This is it. - This trait takes two type parameters: `Call` and `Context`. `Call` is the runtime call type which used to be an associated type; you can just move it to become a type parameter for your trait impl. `Context` is not currently used and you can safely implement over it as an unbounded type. - There's no `AccountId` associated type any more. Just remove it. Regarding `validate`: - You get three new parameters in `validate`; all can be ignored when migrating from `SignedExtension`. - `validate` returns a tuple on success; the second item in the tuple is the new ticket type `Self::Val` which gets passed in to `prepare`. If you use any information extracted during `validate` (off-chain and on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can pass it through with this. For the tuple's last item, just return the `origin` argument. Regarding `prepare`: - This is renamed from `pre_dispatch`, but there is one change: - FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM `validate`!! - (This is different to `SignedExtension` which was required to run the same checks in `pre_dispatch` as in `validate`.) Regarding `post_dispatch`: - Since there are no unsigned transactions handled by `TransactionExtension`, `Pre` is always defined, so the first parameter is `Self::Pre` rather than `Option<Self::Pre>`. If you make use of `SignedExtension::validate_unsigned` or `SignedExtension::pre_dispatch_unsigned`, then: - Just use the regular versions of these functions instead. - Have your logic execute in the case that the `origin` is `None`. - Ensure your transaction creation logic creates a General Transaction rather than a Bare Transaction; this means having to include all `TransactionExtension`s' data. - `ValidateUnsigned` can still be used (for now) if you need to be able to construct transactions which contain none of the extension data, however these will be phased out in stage 2 of the Transactions Horizon, so you should consider moving to an extension-centric design. - [x] Introduce `CheckSignature` impl of `TransactionExtension` to ensure it's possible to have crypto be done wholly in a `TransactionExtension`. - [x] Deprecate `SignedExtension` and move all uses in codebase to `TransactionExtension`. - [x] `ChargeTransactionPayment` - [x] `DummyExtension` - [x] `ChargeAssetTxPayment` (asset-tx-payment) - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment) - [x] `CheckWeight` - [x] `CheckTxVersion` - [x] `CheckSpecVersion` - [x] `CheckNonce` - [x] `CheckNonZeroSender` - [x] `CheckMortality` - [x] `CheckGenesis` - [x] `CheckOnlySudoAccount` - [x] `WatchDummy` - [x] `PrevalidateAttests` - [x] `GenericSignedExtension` - [x] `SignedExtension` (chain-polkadot-bulletin) - [x] `RefundSignedExtensionAdapter` - [x] Implement `fn weight` across the board. - [ ] Go through all pre-existing extensions which assume an account signer and explicitly handle the possibility of another kind of origin. - [x] `CheckNonce` should probably succeed in the case of a non-account origin. - [x] `CheckNonZeroSender` should succeed in the case of a non-account origin. - [x] `ChargeTransactionPayment` and family should fail in the case of a non-account origin. - [ ] - [x] Fix any broken tests. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Chevdor <chevdor@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io> Co-authored-by: Javier Viola <javier@parity.io> Co-authored-by: Marcin S. <marcin@realemail.net> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Javier Bullrich <javier@bullrich.dev> Co-authored-by: Koute <koute@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: Vladimir Istyufeev <vladimir@parity.io> Co-authored-by: Ross Bulat <ross@parity.io> Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Sebastian Kunert <skunert49@gmail.com> Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com> Co-authored-by: Julian Eager <eagr@tutanota.com> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Davide Galassi <davxy@datawok.net> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: yjh <yjh465402634@gmail.com> Co-authored-by: Tom Mi <tommi@niemi.lol> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com> Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com> Co-authored-by: PG Herveou <pgherveou@gmail.com> Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Juan Girini <juangirini@gmail.com> Co-authored-by: bader y <ibnbassem@gmail.com> Co-authored-by: James Wilson <james@jsdw.me> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Parth <desaiparth08@gmail.com> Co-authored-by: Andrew Jones <ascjones@gmail.com> Co-authored-by: Jonathan Udd <jonathan@dwellir.com> Co-authored-by: Serban Iorga <serban@parity.io> Co-authored-by: Egor_P <egor@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Evgeny Snitko <evgeny@parity.io> Co-authored-by: Just van Stam <vstam1@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro> Co-authored-by: zhiqiangxu <652732310@qq.com> Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com> Co-authored-by: Anwesh <anweshknayak@gmail.com> Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> Co-authored-by: Muharem <ismailov.m.h@gmail.com> Co-authored-by: joepetrowski <joe@parity.io> Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com> Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com> Co-authored-by: Squirrel <gilescope@gmail.com> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: command-bot <> (cherry picked from commit fd5f929) * Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665) This PR reverts #2280 which introduced `TransactionExtension` to replace `SignedExtension`. As a result of the discussion [here](paritytech/polkadot-sdk#3623 (comment)), the changes will be reverted for now with plans to reintroduce the concept in the future. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> (cherry picked from commit bbd51ce) * Increase timeout for assertions (#3680) Prevents timeouts in ci like https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5516019 (cherry picked from commit c4c9257) * Removes `as [disambiguation_path]` from `derive_impl` usage (#3652) Step in paritytech/polkadot-sdk#171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in paritytech/polkadot-sdk#3505 (cherry picked from commit 7099f6e) * Fix typo (#3691) (cherry picked from commit 6b1179f) * Bridge zombienet tests: remove unneeded accounts (#3700) Bridge zombienet tests: remove unneeded accounts (cherry picked from commit 0c6c837) * Fix typos (#3753) (cherry picked from commit 7241a8d) * Update polkadot-sdk refs * Fix dependency conflicts * Fix build * cargo fmt * Fix spellcheck test --------- Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: jokess123 <163112061+jokess123@users.noreply.github.com> Co-authored-by: slicejoke <163888128+slicejoke@users.noreply.github.com>

* Add two new zombienet tests for bridges (manual run) (paritytech#3072) extracted useful code from paritytech#2982 This PR: - adds test 2 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers while there are no any messages; - adds test 3 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers when there are messages; - fixes most of comments from paritytech#2439 (like: log names, ability to run specify test number when calling `run-tests.sh`). Right now of all our tests, only test 2 is working (until BHs will be upgraded to use async backing), so you can test it with `./bridges/zombienet/run-tests.sh --test 2` locally. (cherry picked from commit 2e6067d) * [cumulus] Improved check for sane bridge fees calculations (paritytech#3175) - [x] change constants when CI fails (should fail :) ) On the AssetHubRococo: 1701175800126 -> 1700929825257 = 0.15 % decreased. ``` Feb 02 12:59:05.520 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1701175800126 for runtime: statemine-1006000 (statemine-0.tx14.au1) Feb 02 13:02:40.647 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1700929825257 for runtime: statemine-1006000 (statemine-0.tx14.au1) ``` On the AssetHubWestend: 2116038876326 -> 1641718372993 = 22.4 % decreased. ``` Feb 02 12:56:00.880 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 2116038876326 for runtime: westmint-1006000 (westmint-0.tx14.au1) Feb 02 13:04:42.515 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1641718372993 for runtime: westmint-1006000 (westmint-0.tx14.au1) ``` (cherry picked from commit 74b597f) * Enable async backing on all testnet system chains (paritytech#2949) Built on top of paritytech#2826 which was a trial run. Guide: https://github.com/w3f/polkadot-wiki/blob/master/docs/maintain/maintain-guides-async-backing.md --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> (cherry picked from commit 700d5f8) * Introduce submit_finality_proof_ex call to bridges GRANDPA pallet (paritytech#3225) backport of paritytech/parity-bridges-common#2821 (see detailed description there) (cherry picked from commit a462207) * Bridge zombienet tests refactoring (paritytech#3260) Related to paritytech#3242 Reorganizing the bridge zombienet tests in order to: - separate the environment spawning from the actual tests - offer better control over the tests and some possibility to orchestrate them as opposed to running everything from the zndsl file Only rewrote the asset transfer test using this new "framework". The old logic and old tests weren't functionally modified or deleted. The plan is to get feedback on this approach first and if this is agreed upon, migrate the other 2 tests later in separate PRs and also do other improvements later. (cherry picked from commit dfc8e46) * Bridges: add test 0002 to CI (paritytech#3310) Bridges: add test 0002 to CI (cherry picked from commit 1b66bb5) * Bridge zombienet tests - move all test scripts to the same folder (paritytech#3333) Related to paritytech#3242 (cherry picked from commit 5fc7622) * Lift dependencies to the workspace (Part 2/x) (paritytech#3366) Lifting some more dependencies to the workspace. Just using the most-often updated ones for now. It can be reproduced locally. ```sh $ zepter transpose dependency lift-to-workspace --ignore-errors syn quote thiserror "regex:^serde.*" $ zepter transpose dependency lift-to-workspace --version-resolver=highest syn quote thiserror "regex:^serde.*" --fix $ taplo format --config .config/taplo.toml ``` --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> (cherry picked from commit e89d0fc) * Add support for BHP local and BHK local (paritytech#3443) Related to paritytech#3400 Extracting small parts of paritytech#3429 into separate PR: - Add support for BHP local and BHK local - Increase the timeout for the bridge zomienet tests (cherry picked from commit e4b6b8c) * Bridge zombienet tests: move all "framework" files under one folder (paritytech#3462) Related to paritytech#3400 Moving all bridges testing "framework" files under one folder in order to be able to download the entire folder when we want to add tests in other repos No significant functional changes (cherry picked from commit 6fc1d41) * Bridge zombienet tests: Check amount received at destination (paritytech#3490) Related to paritytech#3475 (cherry picked from commit 2cdda0e) * FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (paritytech#2280) Closes paritytech#2160 First part of [Extrinsic Horizon](paritytech#2415) Introduces a new trait `TransactionExtension` to replace `SignedExtension`. Introduce the idea of transactions which obey the runtime's extensions and have according Extension data (né Extra data) yet do not have hard-coded signatures. Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have __*General*__ for the former and __*Bare*__ for the latter. (Ultimately, the latter will be phased out as a type of transaction, and Bare will only be used for Inherents.) Types of extrinsic are now therefore: - Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned") - Bare transactions (deprecated): Gossiped, validated with `ValidateUnsigned` (deprecated) and the `_bare_compat` bits of `TransactionExtension` (deprecated). - Inherents: Not gossiped, validated with `ProvideInherent`. - Extended (Extra data): Gossiped, validated via `TransactionExtension`. - Signed transactions (with a hardcoded signature). - General transactions (without a hardcoded signature). `TransactionExtension` differs from `SignedExtension` because: - A signature on the underlying transaction may validly not be present. - It may alter the origin during validation. - `pre_dispatch` is renamed to `prepare` and need not contain the checks present in `validate`. - `validate` and `prepare` is passed an `Origin` rather than a `AccountId`. - `validate` may pass arbitrary information into `prepare` via a new user-specifiable type `Val`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. It is encoded *for the entire transaction* and passed in to each extension as a new argument to `validate`. This facilitates the ability of extensions to acts as underlying crypto. There is a new `DispatchTransaction` trait which contains only default function impls and is impl'ed for any `TransactionExtension` impler. It provides several utility functions which reduce some of the tedium from using `TransactionExtension` (indeed, none of its regular functions should now need to be called directly). Three transaction version discriminator ("versions") are now permissible: - 0b000000100: Bare (used to be called "Unsigned"): contains Signature or Extra (extension data). After bare transactions are no longer supported, this will strictly identify an Inherents only. - 0b100000100: Old-school "Signed" Transaction: contains Signature and Extra (extension data). - 0b010000100: New-school "General" Transaction: contains Extra (extension data), but no Signature. For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above. Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be accompanied by renaming your aggregate type in line with the new terminology. E.g. Before: ```rust /// The SignedExtension to the basic transaction logic. pub type SignedExtra = ( /* snip */ MySpecialSignedExtension, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>; ``` After: ```rust /// The extension to the basic transaction logic. pub type TxExtension = ( /* snip */ AsTransactionExtension<MySpecialSignedExtension>, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>; ``` You'll also need to alter any transaction building logic to add a `.into()` to make the conversion happen. E.g. Before: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let extra: SignedExtra = ( /* snip */ MySpecialSignedExtension::new(/* snip */), ); let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), extra, ) } ``` After: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let tx_ext: TxExtension = ( /* snip */ MySpecialSignedExtension::new(/* snip */).into(), ); let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), tx_ext, ) } ``` Most `SignedExtension`s can be trivially converted to become a `TransactionExtension`. There are a few things to know. - Instead of a single trait like `SignedExtension`, you should now implement two traits individually: `TransactionExtensionBase` and `TransactionExtension`. - Weights are now a thing and must be provided via the new function `fn weight`. This trait takes care of anything which is not dependent on types specific to your runtime, most notably `Call`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. - Weight must be returned by implementing the `weight` function. If your extension is associated with a pallet, you'll probably want to do this via the pallet's existing benchmarking infrastructure. Generally: - `pre_dispatch` is now `prepare` and you *should not reexecute the `validate` functionality in there*! - You don't get an account ID any more; you get an origin instead. If you need to presume an account ID, then you can use the trait function `AsSystemOriginSigner::as_system_origin_signer`. - You get an additional ticket, similar to `Pre`, called `Val`. This defines data which is passed from `validate` into `prepare`. This is important since you should not be duplicating logic from `validate` to `prepare`, you need a way of passing your working from the former into the latter. This is it. - This trait takes two type parameters: `Call` and `Context`. `Call` is the runtime call type which used to be an associated type; you can just move it to become a type parameter for your trait impl. `Context` is not currently used and you can safely implement over it as an unbounded type. - There's no `AccountId` associated type any more. Just remove it. Regarding `validate`: - You get three new parameters in `validate`; all can be ignored when migrating from `SignedExtension`. - `validate` returns a tuple on success; the second item in the tuple is the new ticket type `Self::Val` which gets passed in to `prepare`. If you use any information extracted during `validate` (off-chain and on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can pass it through with this. For the tuple's last item, just return the `origin` argument. Regarding `prepare`: - This is renamed from `pre_dispatch`, but there is one change: - FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM `validate`!! - (This is different to `SignedExtension` which was required to run the same checks in `pre_dispatch` as in `validate`.) Regarding `post_dispatch`: - Since there are no unsigned transactions handled by `TransactionExtension`, `Pre` is always defined, so the first parameter is `Self::Pre` rather than `Option<Self::Pre>`. If you make use of `SignedExtension::validate_unsigned` or `SignedExtension::pre_dispatch_unsigned`, then: - Just use the regular versions of these functions instead. - Have your logic execute in the case that the `origin` is `None`. - Ensure your transaction creation logic creates a General Transaction rather than a Bare Transaction; this means having to include all `TransactionExtension`s' data. - `ValidateUnsigned` can still be used (for now) if you need to be able to construct transactions which contain none of the extension data, however these will be phased out in stage 2 of the Transactions Horizon, so you should consider moving to an extension-centric design. - [x] Introduce `CheckSignature` impl of `TransactionExtension` to ensure it's possible to have crypto be done wholly in a `TransactionExtension`. - [x] Deprecate `SignedExtension` and move all uses in codebase to `TransactionExtension`. - [x] `ChargeTransactionPayment` - [x] `DummyExtension` - [x] `ChargeAssetTxPayment` (asset-tx-payment) - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment) - [x] `CheckWeight` - [x] `CheckTxVersion` - [x] `CheckSpecVersion` - [x] `CheckNonce` - [x] `CheckNonZeroSender` - [x] `CheckMortality` - [x] `CheckGenesis` - [x] `CheckOnlySudoAccount` - [x] `WatchDummy` - [x] `PrevalidateAttests` - [x] `GenericSignedExtension` - [x] `SignedExtension` (chain-polkadot-bulletin) - [x] `RefundSignedExtensionAdapter` - [x] Implement `fn weight` across the board. - [ ] Go through all pre-existing extensions which assume an account signer and explicitly handle the possibility of another kind of origin. - [x] `CheckNonce` should probably succeed in the case of a non-account origin. - [x] `CheckNonZeroSender` should succeed in the case of a non-account origin. - [x] `ChargeTransactionPayment` and family should fail in the case of a non-account origin. - [ ] - [x] Fix any broken tests. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Chevdor <chevdor@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io> Co-authored-by: Javier Viola <javier@parity.io> Co-authored-by: Marcin S. <marcin@realemail.net> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Javier Bullrich <javier@bullrich.dev> Co-authored-by: Koute <koute@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: Vladimir Istyufeev <vladimir@parity.io> Co-authored-by: Ross Bulat <ross@parity.io> Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Sebastian Kunert <skunert49@gmail.com> Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com> Co-authored-by: Julian Eager <eagr@tutanota.com> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Davide Galassi <davxy@datawok.net> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: yjh <yjh465402634@gmail.com> Co-authored-by: Tom Mi <tommi@niemi.lol> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com> Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com> Co-authored-by: PG Herveou <pgherveou@gmail.com> Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Juan Girini <juangirini@gmail.com> Co-authored-by: bader y <ibnbassem@gmail.com> Co-authored-by: James Wilson <james@jsdw.me> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Parth <desaiparth08@gmail.com> Co-authored-by: Andrew Jones <ascjones@gmail.com> Co-authored-by: Jonathan Udd <jonathan@dwellir.com> Co-authored-by: Serban Iorga <serban@parity.io> Co-authored-by: Egor_P <egor@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Evgeny Snitko <evgeny@parity.io> Co-authored-by: Just van Stam <vstam1@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro> Co-authored-by: zhiqiangxu <652732310@qq.com> Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com> Co-authored-by: Anwesh <anweshknayak@gmail.com> Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> Co-authored-by: Muharem <ismailov.m.h@gmail.com> Co-authored-by: joepetrowski <joe@parity.io> Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com> Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com> Co-authored-by: Squirrel <gilescope@gmail.com> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: command-bot <> (cherry picked from commit fd5f929) * Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (paritytech#2280)" (paritytech#3665) This PR reverts paritytech#2280 which introduced `TransactionExtension` to replace `SignedExtension`. As a result of the discussion [here](paritytech#3623 (comment)), the changes will be reverted for now with plans to reintroduce the concept in the future. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> (cherry picked from commit bbd51ce) * Increase timeout for assertions (paritytech#3680) Prevents timeouts in ci like https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5516019 (cherry picked from commit c4c9257) * Removes `as [disambiguation_path]` from `derive_impl` usage (paritytech#3652) Step in paritytech#171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in paritytech#3505 (cherry picked from commit 7099f6e) * Fix typo (paritytech#3691) (cherry picked from commit 6b1179f) * Bridge zombienet tests: remove unneeded accounts (paritytech#3700) Bridge zombienet tests: remove unneeded accounts (cherry picked from commit 0c6c837) * Fix typos (paritytech#3753) (cherry picked from commit 7241a8d) * Update polkadot-sdk refs * Fix dependency conflicts * Fix build * cargo fmt * Fix spellcheck test --------- Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: jokess123 <163112061+jokess123@users.noreply.github.com> Co-authored-by: slicejoke <163888128+slicejoke@users.noreply.github.com>

* Add two new zombienet tests for bridges (manual run) (#3072) extracted useful code from #2982 This PR: - adds test 2 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers while there are no any messages; - adds test 3 for Rococo <> Westend bridge: checks that relayer doesn't submit any extra headers when there are messages; - fixes most of comments from #2439 (like: log names, ability to run specify test number when calling `run-tests.sh`). Right now of all our tests, only test 2 is working (until BHs will be upgraded to use async backing), so you can test it with `./bridges/zombienet/run-tests.sh --test 2` locally. (cherry picked from commit 2e6067d) * [cumulus] Improved check for sane bridge fees calculations (#3175) - [x] change constants when CI fails (should fail :) ) On the AssetHubRococo: 1701175800126 -> 1700929825257 = 0.15 % decreased. ``` Feb 02 12:59:05.520 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1701175800126 for runtime: statemine-1006000 (statemine-0.tx14.au1) Feb 02 13:02:40.647 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1700929825257 for runtime: statemine-1006000 (statemine-0.tx14.au1) ``` On the AssetHubWestend: 2116038876326 -> 1641718372993 = 22.4 % decreased. ``` Feb 02 12:56:00.880 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 2116038876326 for runtime: westmint-1006000 (westmint-0.tx14.au1) Feb 02 13:04:42.515 ERROR bridges::estimate: `bridging::XcmBridgeHubRouterBaseFee` actual value: 1641718372993 for runtime: westmint-1006000 (westmint-0.tx14.au1) ``` (cherry picked from commit 74b597f) * Enable async backing on all testnet system chains (#2949) Built on top of #2826 which was a trial run. Guide: https://github.com/w3f/polkadot-wiki/blob/master/docs/maintain/maintain-guides-async-backing.md --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> (cherry picked from commit 700d5f8) * Introduce submit_finality_proof_ex call to bridges GRANDPA pallet (#3225) backport of paritytech/parity-bridges-common#2821 (see detailed description there) (cherry picked from commit a462207) * Bridge zombienet tests refactoring (#3260) Related to #3242 Reorganizing the bridge zombienet tests in order to: - separate the environment spawning from the actual tests - offer better control over the tests and some possibility to orchestrate them as opposed to running everything from the zndsl file Only rewrote the asset transfer test using this new "framework". The old logic and old tests weren't functionally modified or deleted. The plan is to get feedback on this approach first and if this is agreed upon, migrate the other 2 tests later in separate PRs and also do other improvements later. (cherry picked from commit dfc8e46) * Bridges: add test 0002 to CI (#3310) Bridges: add test 0002 to CI (cherry picked from commit 1b66bb5) * Bridge zombienet tests - move all test scripts to the same folder (#3333) Related to #3242 (cherry picked from commit 5fc7622) * Lift dependencies to the workspace (Part 2/x) (#3366) Lifting some more dependencies to the workspace. Just using the most-often updated ones for now. It can be reproduced locally. ```sh $ zepter transpose dependency lift-to-workspace --ignore-errors syn quote thiserror "regex:^serde.*" $ zepter transpose dependency lift-to-workspace --version-resolver=highest syn quote thiserror "regex:^serde.*" --fix $ taplo format --config .config/taplo.toml ``` --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> (cherry picked from commit e89d0fc) * Add support for BHP local and BHK local (#3443) Related to #3400 Extracting small parts of #3429 into separate PR: - Add support for BHP local and BHK local - Increase the timeout for the bridge zomienet tests (cherry picked from commit e4b6b8c) * Bridge zombienet tests: move all "framework" files under one folder (#3462) Related to #3400 Moving all bridges testing "framework" files under one folder in order to be able to download the entire folder when we want to add tests in other repos No significant functional changes (cherry picked from commit 6fc1d41) * Bridge zombienet tests: Check amount received at destination (#3490) Related to #3475 (cherry picked from commit 2cdda0e) * FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280) Closes #2160 First part of [Extrinsic Horizon](#2415) Introduces a new trait `TransactionExtension` to replace `SignedExtension`. Introduce the idea of transactions which obey the runtime's extensions and have according Extension data (né Extra data) yet do not have hard-coded signatures. Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have __*General*__ for the former and __*Bare*__ for the latter. (Ultimately, the latter will be phased out as a type of transaction, and Bare will only be used for Inherents.) Types of extrinsic are now therefore: - Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned") - Bare transactions (deprecated): Gossiped, validated with `ValidateUnsigned` (deprecated) and the `_bare_compat` bits of `TransactionExtension` (deprecated). - Inherents: Not gossiped, validated with `ProvideInherent`. - Extended (Extra data): Gossiped, validated via `TransactionExtension`. - Signed transactions (with a hardcoded signature). - General transactions (without a hardcoded signature). `TransactionExtension` differs from `SignedExtension` because: - A signature on the underlying transaction may validly not be present. - It may alter the origin during validation. - `pre_dispatch` is renamed to `prepare` and need not contain the checks present in `validate`. - `validate` and `prepare` is passed an `Origin` rather than a `AccountId`. - `validate` may pass arbitrary information into `prepare` via a new user-specifiable type `Val`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. It is encoded *for the entire transaction* and passed in to each extension as a new argument to `validate`. This facilitates the ability of extensions to acts as underlying crypto. There is a new `DispatchTransaction` trait which contains only default function impls and is impl'ed for any `TransactionExtension` impler. It provides several utility functions which reduce some of the tedium from using `TransactionExtension` (indeed, none of its regular functions should now need to be called directly). Three transaction version discriminator ("versions") are now permissible: - 0b000000100: Bare (used to be called "Unsigned"): contains Signature or Extra (extension data). After bare transactions are no longer supported, this will strictly identify an Inherents only. - 0b100000100: Old-school "Signed" Transaction: contains Signature and Extra (extension data). - 0b010000100: New-school "General" Transaction: contains Extra (extension data), but no Signature. For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above. Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be accompanied by renaming your aggregate type in line with the new terminology. E.g. Before: ```rust /// The SignedExtension to the basic transaction logic. pub type SignedExtra = ( /* snip */ MySpecialSignedExtension, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>; ``` After: ```rust /// The extension to the basic transaction logic. pub type TxExtension = ( /* snip */ AsTransactionExtension<MySpecialSignedExtension>, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>; ``` You'll also need to alter any transaction building logic to add a `.into()` to make the conversion happen. E.g. Before: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let extra: SignedExtra = ( /* snip */ MySpecialSignedExtension::new(/* snip */), ); let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), extra, ) } ``` After: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let tx_ext: TxExtension = ( /* snip */ MySpecialSignedExtension::new(/* snip */).into(), ); let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), tx_ext, ) } ``` Most `SignedExtension`s can be trivially converted to become a `TransactionExtension`. There are a few things to know. - Instead of a single trait like `SignedExtension`, you should now implement two traits individually: `TransactionExtensionBase` and `TransactionExtension`. - Weights are now a thing and must be provided via the new function `fn weight`. This trait takes care of anything which is not dependent on types specific to your runtime, most notably `Call`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. - Weight must be returned by implementing the `weight` function. If your extension is associated with a pallet, you'll probably want to do this via the pallet's existing benchmarking infrastructure. Generally: - `pre_dispatch` is now `prepare` and you *should not reexecute the `validate` functionality in there*! - You don't get an account ID any more; you get an origin instead. If you need to presume an account ID, then you can use the trait function `AsSystemOriginSigner::as_system_origin_signer`. - You get an additional ticket, similar to `Pre`, called `Val`. This defines data which is passed from `validate` into `prepare`. This is important since you should not be duplicating logic from `validate` to `prepare`, you need a way of passing your working from the former into the latter. This is it. - This trait takes two type parameters: `Call` and `Context`. `Call` is the runtime call type which used to be an associated type; you can just move it to become a type parameter for your trait impl. `Context` is not currently used and you can safely implement over it as an unbounded type. - There's no `AccountId` associated type any more. Just remove it. Regarding `validate`: - You get three new parameters in `validate`; all can be ignored when migrating from `SignedExtension`. - `validate` returns a tuple on success; the second item in the tuple is the new ticket type `Self::Val` which gets passed in to `prepare`. If you use any information extracted during `validate` (off-chain and on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can pass it through with this. For the tuple's last item, just return the `origin` argument. Regarding `prepare`: - This is renamed from `pre_dispatch`, but there is one change: - FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM `validate`!! - (This is different to `SignedExtension` which was required to run the same checks in `pre_dispatch` as in `validate`.) Regarding `post_dispatch`: - Since there are no unsigned transactions handled by `TransactionExtension`, `Pre` is always defined, so the first parameter is `Self::Pre` rather than `Option<Self::Pre>`. If you make use of `SignedExtension::validate_unsigned` or `SignedExtension::pre_dispatch_unsigned`, then: - Just use the regular versions of these functions instead. - Have your logic execute in the case that the `origin` is `None`. - Ensure your transaction creation logic creates a General Transaction rather than a Bare Transaction; this means having to include all `TransactionExtension`s' data. - `ValidateUnsigned` can still be used (for now) if you need to be able to construct transactions which contain none of the extension data, however these will be phased out in stage 2 of the Transactions Horizon, so you should consider moving to an extension-centric design. - [x] Introduce `CheckSignature` impl of `TransactionExtension` to ensure it's possible to have crypto be done wholly in a `TransactionExtension`. - [x] Deprecate `SignedExtension` and move all uses in codebase to `TransactionExtension`. - [x] `ChargeTransactionPayment` - [x] `DummyExtension` - [x] `ChargeAssetTxPayment` (asset-tx-payment) - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment) - [x] `CheckWeight` - [x] `CheckTxVersion` - [x] `CheckSpecVersion` - [x] `CheckNonce` - [x] `CheckNonZeroSender` - [x] `CheckMortality` - [x] `CheckGenesis` - [x] `CheckOnlySudoAccount` - [x] `WatchDummy` - [x] `PrevalidateAttests` - [x] `GenericSignedExtension` - [x] `SignedExtension` (chain-polkadot-bulletin) - [x] `RefundSignedExtensionAdapter` - [x] Implement `fn weight` across the board. - [ ] Go through all pre-existing extensions which assume an account signer and explicitly handle the possibility of another kind of origin. - [x] `CheckNonce` should probably succeed in the case of a non-account origin. - [x] `CheckNonZeroSender` should succeed in the case of a non-account origin. - [x] `ChargeTransactionPayment` and family should fail in the case of a non-account origin. - [ ] - [x] Fix any broken tests. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Chevdor <chevdor@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io> Co-authored-by: Javier Viola <javier@parity.io> Co-authored-by: Marcin S. <marcin@realemail.net> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Javier Bullrich <javier@bullrich.dev> Co-authored-by: Koute <koute@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: Vladimir Istyufeev <vladimir@parity.io> Co-authored-by: Ross Bulat <ross@parity.io> Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Sebastian Kunert <skunert49@gmail.com> Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com> Co-authored-by: Julian Eager <eagr@tutanota.com> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Davide Galassi <davxy@datawok.net> Co-authored-by: Dónal Murray <donal.murray@parity.io> Co-authored-by: yjh <yjh465402634@gmail.com> Co-authored-by: Tom Mi <tommi@niemi.lol> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com> Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com> Co-authored-by: PG Herveou <pgherveou@gmail.com> Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Juan Girini <juangirini@gmail.com> Co-authored-by: bader y <ibnbassem@gmail.com> Co-authored-by: James Wilson <james@jsdw.me> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Parth <desaiparth08@gmail.com> Co-authored-by: Andrew Jones <ascjones@gmail.com> Co-authored-by: Jonathan Udd <jonathan@dwellir.com> Co-authored-by: Serban Iorga <serban@parity.io> Co-authored-by: Egor_P <egor@parity.io> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Evgeny Snitko <evgeny@parity.io> Co-authored-by: Just van Stam <vstam1@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro> Co-authored-by: zhiqiangxu <652732310@qq.com> Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com> Co-authored-by: Anwesh <anweshknayak@gmail.com> Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com> Co-authored-by: Muharem <ismailov.m.h@gmail.com> Co-authored-by: joepetrowski <joe@parity.io> Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com> Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com> Co-authored-by: Squirrel <gilescope@gmail.com> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: georgepisaltu <george.pisaltu@parity.io> Co-authored-by: command-bot <> (cherry picked from commit fd5f929) * Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665) This PR reverts #2280 which introduced `TransactionExtension` to replace `SignedExtension`. As a result of the discussion [here](#3623 (comment)), the changes will be reverted for now with plans to reintroduce the concept in the future. --------- Signed-off-by: georgepisaltu <george.pisaltu@parity.io> (cherry picked from commit bbd51ce) * Increase timeout for assertions (#3680) Prevents timeouts in ci like https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5516019 (cherry picked from commit c4c9257) * Removes `as [disambiguation_path]` from `derive_impl` usage (#3652) Step in #171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in #3505 (cherry picked from commit 7099f6e) * Fix typo (#3691) (cherry picked from commit 6b1179f) * Bridge zombienet tests: remove unneeded accounts (#3700) Bridge zombienet tests: remove unneeded accounts (cherry picked from commit 0c6c837) * Fix typos (#3753) (cherry picked from commit 7241a8d) * Update polkadot-sdk refs * Fix dependency conflicts * Fix build * cargo fmt * Fix spellcheck test --------- Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com> Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com> Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com> Co-authored-by: jokess123 <163112061+jokess123@users.noreply.github.com> Co-authored-by: slicejoke <163888128+slicejoke@users.noreply.github.com>

Polkadot-Forum · 2024-04-26T05:13:00Z

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/seeking-feedback-on-derive-impl-for-default-configurations-in-pallet-templates/7584/1

kianenigma added the J1-meta label May 26, 2023

kianenigma mentioned this issue May 26, 2023

[FRAME Core] Default Pallet Config Trait / derive_impl paritytech/substrate#13454

Merged

kianenigma self-assigned this May 30, 2023

kianenigma added the Z6-mentor label Jun 1, 2023

juangirini added the T1-runtime label Jun 7, 2023

This was referenced Jun 12, 2023

Use default config trait in mock runtimes paritytech/substrate#14336

Closed

[FRAME/Meta/PM] "experimental" FRAME features paritytech/substrate#14345

Closed

kianenigma mentioned this issue Jun 24, 2023

add frame_system::DefaultConfig to individual pallet DefaultConfigs paritytech/substrate#14453

Merged

kianenigma changed the title ~~tracking issue for full unleash of DefaultConfig for FRAME~~ [FRAME Core] tracking issue for full unleash of DefaultConfig for FRAME Jun 26, 2023

bkchr mentioned this issue Jun 28, 2023

Default Config is not able to provide defaults for types that access other associated types paritytech/substrate#14473

Closed

gupnik mentioned this issue Jul 30, 2023

Adds ability to provide defaults for types provided by construct_runtime paritytech/substrate#14682

Merged

2 tasks

kianenigma assigned gupnik and unassigned kianenigma Aug 5, 2023

juangirini transferred this issue from paritytech/substrate Aug 24, 2023

This was referenced Sep 19, 2023

migrate alliance, fast-unstake and bags list to use derive-impl #1636

Merged

Tasks: general system for recognizing and executing service work #1343

Merged

Default Config for Staking #1599

Closed

gupnik mentioned this issue Sep 21, 2023

Move all tests to use u32 as BlockNumber #1657

Closed

2 tasks

Dmitry-Borodin mentioned this issue Oct 4, 2023

migrate babe and authorship to use derive-impl #1790

Merged

bkontur mentioned this issue Oct 5, 2023

Refactor pallet-state-trie-migration to fungible::* traits #1801

Merged

5 tasks

github-merge-queue bot pushed a commit that referenced this issue Mar 15, 2024

Removes as [disambiguation_path] from derive_impl usage (#3652)

7099f6e

Step in #171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in #3505

dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this issue Mar 24, 2024

Adds default config for assets pallet (paritytech#3637)

be36f1e

Step in paritytech#171

bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this issue Mar 25, 2024

frame: use derive-impl for beefy and mmr pallets (paritytech#1867)

d82a3c0

Part of paritytech#171

bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this issue Mar 25, 2024

Adds default config for assets pallet (paritytech#3637)

fdcade6

Step in paritytech#171

liamaharon mentioned this issue Apr 24, 2024

Improvements in minimal template #4119

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FRAME Core] tracking issue for full unleash of `DefaultConfig` for FRAME #171

[FRAME Core] tracking issue for full unleash of `DefaultConfig` for FRAME #171

kianenigma commented May 26, 2023 •

edited by gupnik

Loading

bkchr commented May 29, 2023

sam0x17 commented Jun 25, 2023 •

edited

Loading

sam0x17 commented Jul 4, 2023

Polkadot-Forum commented Apr 26, 2024

[FRAME Core] tracking issue for full unleash of DefaultConfig for FRAME #171

[FRAME Core] tracking issue for full unleash of DefaultConfig for FRAME #171

Comments

kianenigma commented May 26, 2023 • edited by gupnik Loading

Improve Error Handling, Bugs

Full usage in Substrate Pallets

Full Usage in node-template-runtime and parachain-template-runtime

Full usage in westend -> kusama -> polkadot runtimes.

V2

Improvements

bkchr commented May 29, 2023

sam0x17 commented Jun 25, 2023 • edited Loading

sam0x17 commented Jul 4, 2023

Polkadot-Forum commented Apr 26, 2024

[FRAME Core] tracking issue for full unleash of `DefaultConfig` for FRAME #171

[FRAME Core] tracking issue for full unleash of `DefaultConfig` for FRAME #171

kianenigma commented May 26, 2023 •

edited by gupnik

Loading

Full Usage in `node-template-runtime` and `parachain-template-runtime`

Full usage in `westend` -> `kusama` -> `polkadot` runtimes.

sam0x17 commented Jun 25, 2023 •

edited

Loading