Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

contracts: Lazy storage removal #7740

Merged
19 commits merged into from
Jan 4, 2021
Merged

contracts: Lazy storage removal #7740

19 commits merged into from
Jan 4, 2021

Conversation

athei
Copy link
Member

@athei athei commented Dec 16, 2020

This PR adds a mechanism that moves the removal of contract storage to the on_initialize function. What that means is that when removing a contract due to eviction or deliberate termination its metadata is removed and is no longer accessible but the potentially heavy weight deletion of the child trie holding all the data is delayed. Because the deletion will happen earliest at the next block we need to store all the tries that should be deleted in on-chain storage as a queue. The maximum size of this queue is configurable and prevents the storage from growing indefinitely:

/// The maximum number of tries that can be queued for deletion.
type DeletionQueueDepth: Get<u32>;

This also means that seal_terminate and claim_surcharge are able to fail when this queue is full. There are no other implicit ways to delete a contract anymore (See the first commit in this PR).

The amount of weight that is dedicated to storage removal per block can be configured, too:

/// The maximum amount of weight that can be consumed per block for lazy trie removal.
type DeletionWeightLimit: Get<Weight>;

Tries that are too big to be removed within the configured limit will be split up and removed over the course of multiple blocks.

fixes #6594

We don't want to trigger contract eviction automatically when
a contract is called. This is because those changes can be
reverted due to how storage transactions are used at the moment.
More Information:
#6439 (comment)

It can be re-introduced once the linked issue is resolved. In the meantime
`claim_surcharge` must be called to evict a contract.
@athei athei added A0-please_review Pull request needs code review. B3-apinoteworthy C1-low PR touches the given topic and has a low impact on builders. labels Dec 16, 2020
It being the only dispatchable that is private is an oversight.
@@ -714,6 +715,12 @@ parameter_types! {
pub const MaxDepth: u32 = 32;
pub const StorageSizeOffset: u32 = 8;
pub const MaxValueSize: u32 = 16 * 1024;
pub DeletionWeightLimit: Weight = Perbill::from_percent(20) *
RuntimeBlockWeights::get().max_block;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder how this can collide with staking implementation of OffchainSolutionWeightLimit.

Basically staking offchain worker will use the maximum weight available for normal extrinsic. This maximum is computed using AVERAGE_ON_INITIALIZE_RATIO. If the on_initialize weight is more than AVERAGE_ON_INITIALIZE_RATIO for all the time where offchain worker needs to submit election result. Then a less good on-chain election will happen (I'm not sure how much this should be avoided).

So having underestimated AVERAGE_ON_INITIALIZE_RATIO under the offchain election is open can be a problem maybe, WDYT @kianenigma

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having multiple on_initialize functions that want to take some non-substantial amounts of weight from the block are a problem. They need to be coordinated somehow. Moving the lazy removal from on_initialize to some potential on_idle could be a solution.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said, I am open to change those numbers here to whatever works.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess one way could be to have DeletionWeightLimitRatio < AVERAGE_ON_INITIALIZE_RATIO. so that even if the chain is busy removing lots of tries, election is still fine.

But I agree on_idle could be helpful, maybe staking could notify on_idle that it needs some space somehow

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess one way could be to have DeletionWeightLimitRatio < AVERAGE_ON_INITIALIZE_RATIO. so that even if the chain is busy removing lots of tries, election is still fine.

But this assumes that this would be the only on_initialize in the block which it isn't. The AVERAGE_ON_INITIALIZE_RATIO shared between all of them.

Don't we already have this problem because of this code already:

parameter_types! {
pub OffencesWeightSoftLimit: Weight = Perbill::from_percent(60) *
RuntimeBlockWeights::get().max_block;
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tuned it down to AVERAGE_ON_INITIALIZE_RATIO. With that setting we get the following throughput of the lazy removal:

Lazy deletion throughput per block (empty queue, full queue): 1954, 1564

In order to increase this throughput in the future we should look into on_idle or off chain workers.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current OffchainSolutionWeightLimit is a bit tricky because it essential to the system to work properly, yet it is an unsigned transaction, so it is always likely that sum of on_initialize causes it to fail, in which case we do the notorious on-chain election. This will be fixed with #7319 which is far from being shipped yet, but almost ready. Once this is merged, we do hope to do most of the solution processing in on_initialize as well. In which case, the competition becomes fair. Something like staking, because it is critical to the system, would stay in on_initialize, and this can move to lazy removal.

A little brain dump about that: Let's say that a block has 100 weight units. Usually, 10 is consumed in on_initialize, and 40 in extrinsic, leaving 50 unused. Spreading this into the on_idle cases resembleas a thread/process scheduling situation. Two important notes come to mind:

  1. Can we enforce the weight of each on_idle {} call? No, the runtime should have trust within it and the on_idle call of a module should simply return the weight that it has consumed, similar to on_initialize.
  2. How can we ensure that the idle-scheduling is fair? This can be tricky but I guess a number of policies can be used, and ideally we can have it configurable. Naively, we can start by running the on_idle of each pallet until there is no weight left. Else, we can do it in a ring fashion, where if pallet at indices {0..3} got to execute their on_idle on this block, on the next block we start from {4..}.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we discuss this in the on_idle issue? Or do you think the lazy deletion would essentially break elections and therefore we need to implement on_idle first?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the following issue to keep track of this issue: #7817

That said, I will merge in the current form because:

  • This configuration is only for the dev node and neither kusama nor polkadot have it deployed
  • This is only an issue when contracts are heavily utilized the time an offchain election happens
  • Having an on-chain election will not break the chain
  • Decouple Staking and Election #7319 will most likely be merged before it gets important
  • on_idle will most likely be available before it gets important

Copy link
Contributor

@kianenigma kianenigma Jan 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have any objections with merging this as it is. Thanks for making the issue!

About

Or do you think the lazy deletion would essentially break elections and therefore we need to implement on_idle first?

Yes, it could break it, but I don't think it is a showstopper because afaik there are not mainnets with both pallets. Just once you are setting the value of maximum weight that you can consume for this operation in node/runtime/src/lib, for the sake of prosperity, document that fact that the value shoudl be chosen with care and with care.

bin/node/runtime/src/lib.rs Show resolved Hide resolved
Copy link
Contributor

@gui1117 gui1117 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me, with the additional test suggested by Shawn, and not sure how to fix AVERAGE_ON_INITIALIZE usage.

frame/contracts/src/lib.rs Outdated Show resolved Hide resolved
@kianenigma
Copy link
Contributor

Having just read the description, I guess maybe this can use something like #4064?

@athei
Copy link
Member Author

athei commented Jan 4, 2021

Yes we already discussed that it could profit from it. Until then I tuned down the weight consumption so it should be OK. The on_idle would be very useful to improve on deletion throughput.

@kianenigma
Copy link
Contributor

ahh right is is mentioned multiple times :D 🤦‍♂️

@athei
Copy link
Member Author

athei commented Jan 4, 2021

bot merge

@ghost
Copy link

ghost commented Jan 4, 2021

Waiting for commit status.

@ghost
Copy link

ghost commented Jan 4, 2021

Merge failed: "Required status check \"continuous-integration/gitlab-check-labels\" is expected."

@athei
Copy link
Member Author

athei commented Jan 4, 2021

bot merge

@ghost
Copy link

ghost commented Jan 4, 2021

Trying merge.

@ghost ghost merged commit fcdba68 into master Jan 4, 2021
@ghost ghost deleted the at-lazy-contract-removal branch January 4, 2021 12:35
darkfriend77 pushed a commit to mogwaicoin/substrate that referenced this pull request Jan 11, 2021
* Do not evict a contract from within a call stack

We don't want to trigger contract eviction automatically when
a contract is called. This is because those changes can be
reverted due to how storage transactions are used at the moment.
More Information:
paritytech#6439 (comment)

It can be re-introduced once the linked issue is resolved. In the meantime
`claim_surcharge` must be called to evict a contract.

* Lazily delete storage in on_initialize instead of when removing the contract

* Add missing documentation of new error

* Make Module::claim_surcharge public

It being the only dispatchable that is private is an oversight.

* review: Add final newline

* review: Simplify assert statement

* Add test that checks that partial remove of a contract works

* Premote warning to error

* Added missing docs for seal_terminate

* Lazy deletion should only take AVERAGE_ON_INITIALIZE_RATIO of the block

* Added informational about the lazy deletion throughput

* Avoid lazy deletion in case the block is already full

* Prevent queue decoding in case of an already full block

* Add test that checks that on_initialize honors block limits
darkfriend77 added a commit to mogwaicoin/substrate that referenced this pull request Feb 11, 2021
* Features needed for reserve-backed stablecoins (#7152)

* Features needed for reserve-backed stablecoins

* Builds & tests.

* Double map for an efficient destroy.

* Update frame/assets/src/lib.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* ED/zombie-count/refs

Feature: ED/minimum balance enforcement
Feature: enforce zombie count
Feature: allow system-alive accounts to exist, but add reference

* Update frame/assets/src/lib.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Update frame/assets/Cargo.toml

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Docs

* Some tests

* More tests

* Allow for max_zombies to be adjusted

* Test for set_max_zombies

* Tests and a couple of fixes

* First few benchmarks

* Benchmarks.

* Fix error message in test

* Fixes

* Fixes

* Fixes

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_assets

* Update frame/assets/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Fixes

* Fixes

* Fixes

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_assets

* Fixes

* Update default weight

* Add proper verification to benchmarks

* minor improvements to tests

* Update frame/assets/src/benchmarking.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix

* New weights system

* fix compile

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add Standard Error to Weight Template (#7652)

* Add standard error to weight template

* fix test compile

* Adds support for tuples in runtime-interface (#7672)

* Fix two potential connection poisonings (#7677)

* Fix #7629 (#7680)

* Fix #7629

* Update client/network/src/service.rs

Co-authored-by: Max Inden <mail@max-inden.de>

Co-authored-by: Max Inden <mail@max-inden.de>

* Update Cargo.toml (#7627)

On behalf of dependabot, because we use 1.4.x in it clashes with this.

* Make sure the companion CI test fails on any error (#7683)

* implement more convertion on NumberOrHex (#7682)

* Streamline frame_system weight parametrization (#6629)

* Basic weights builder.

* Fixing WiP

* Make the tests work.

* Fix weights in node/runtime.

* WiP.

* Update pallets with new weights parameters.

* Validate returns a Result now.

* Count mandatory weight separately.

* DRY

* BREAKING: Updating state root, because of the left-over weight-tracking stuff

* Update tests affected by Mandatory tracking.

* Fixing tests.

* Fix defaults for simple_max

* Update frame/system/src/weights.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Rework the API a bit.

* Fix compilation & tests.

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add extra docs & rename few things.

* Fix whitespace in ASCII art.

* Update frame/system/src/limits.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix max_extrinsic calculations.

* Fix conflicts.

* Fix compilation.

* Fix new code.

* re-remove generic asset

* Fix usage.

* Update state root.

* Update proxy.

* Fix tests.

* Move weights validity to integrity_test

* Remove redundant BlockWeights.

* Add all/non_mandatory comment

* Add test.

* Remove fn block_weights

* Make the macro prettier.

* Fix some docs.

* Make max_total behave more predictabily.

* Add BlockWeights to metadata.

* fix balances test

* Fix utility test.

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>

* client: remove duplicate implementation of block_status (#7694)

* Pin md link checker's version (#7697)

* CI: pin md link checker's version to the previous release

* CI: add a whitelist for allowed GH actions pins

* CI: try master tip [skip ci]

* CI: return to a previous markdown-link-check dependency version [skip ci]

* CI: substitute %20 for a literal space [skip ci]

* CI: substitute %20 for a literal space

* Remove waterfall CI trigger (#7519)

* fix (CI): fix the triggered project name

* fix (CI): debug run

* fix (CI): debug run 2

* fix (CI): revert debug2

* Revert "fix (CI): debug run"

This reverts commit 96866a953da797ef6e0689374eecc0d8cba4627d.

* CI: remove the trigger

* add an upgrade_keys method for pallet-session (#7688)

* add an upgrade_keys method for pallet-session

* test the upgrade_keys function

* Allow capping the amount of work performed when deleting a child trie (#7671)

* Allow Backend::for_keys_in_child_storage to be aborted by the closure

* Ext::kill_child_storage now takes an upper limit for backend deletion

* Add Storage::storage_kill_limited() runtime interface

* review: Use a new version of kill_storage instead of a new interface

* review: Simplify boolean expression

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* review: Rename for_keys_in_child_storage

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* benchmarks: Fix panic in case of a missing model (#7698)

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Expand remote keystore interface to allow for hybrid mode (#7628)

* update to latest master

* updates on docs, license, meta

* hide ssrs behind feature flag

* implement remaining functions on the server

* sign server line length fix

* fix tests

* fixup in-memory-keystore

* adding failsafe

* skipping ecdsa test for now

* remote keystore param

* remote sign urls made available

* integrating keystore remotes features

* don't forget the dependency

* remove old cruft

* reset local keystore

* applying suggestions

* Switch to single remote, minor grumbles

* minor grumbles, docs

* fork-tree: expose silent failures of is_descendent_of (#7695)

* fork-tree: expose silent failures of is_descendent_of

* fork-tree: use mem::take instead of mem::swap

* Merkle Mountain Range pallet (#7312)

* Add MMR pallet.

* WiP

* Working on testing.

* WiP - test

* Tests passing.

* Add proof generation.

* Generate and verify proofs.

* Allow verification of older proofs.

* Move stuff to a module.

* Split MMR stuff to it's own module.

* Add docs.

* Make parent hash optional.

* LeafData failed approach.

* Finally implement Compact stuff.

* Compact encoding WiP

* Implement remaining pieces.

* Fix tests

* Add docs to compact.

* Implement for tuples.

* Fix documentation.

* Fix warnings and address review suggestion.

* Update frame/merkle-mountain-range/src/primitives.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Address review grumbles.

* Removing missing crate.

* Fix test.

* Add some docs and test.

* Add multiple instances.

* Cargo.toml sync.

* Fix no_std compilation.

* More no_std stuff.

* Rename MMR struct.

* Addressing other grumbles.

* Fix test.

* Remove format for no_std compat.

* Add test for MMR pallet.

* Fix std feature.

* Update versions.

* Add to node/runtime.

* Add hook to insert digest.

* Make primitives public.

* Update lib.rs

tech spec/typos etc

* Use WeightInfo and benchmarks.

* Fix test.

* Fix benchmarks.

* Trait -> Config.

* Fix typo.

* Fix tests.

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Addie Wagenknecht <addie@nortd.com>

* frame/staking/README: Fix broken link (#7703)

* *: Update to libp2p v0.32.0 (#7696)

* *: Update to libp2p v0.32.0

* Cargo.lock: Update async-tls to 0.10.2

* client/network/request_response: Adjust to new request response events

* client/network/request_response.rs: Clean up silently failing responses

* client/network/discovery: Lazily instantiate mdns

* client/network/discovery: Exclude MdnsWrapper for target_os unknown

* client/network/discovery: Fix indentation

* client/network/request-response: Use LruCache to track pending resp time

* client/network/request_responses: Fix early connection closed error

* client/network/request-response: Replace debug_assert with debug

* client/network/request-response: Fix typo

* client/network/request-response: Don't emit event on send_response fail

* client/network/request-response: Revert waker.wake_by_ref()

* client/network/request-resp: Make duration in InboundRequest optional

* client/network/req-resp: Don't emit two events for busy builder

When a response builder is busy incoming requests are dropped.
Previously this was reported both via a `ResponseFailure::Busy` and a
`ReponseFailure::Network(InboundFailure::Omisssion)` event.

With this commit the former is removed, leaving only the latter in
place.

* *: Update to libp2p v0.32.2 (#7708)

Version v0.32.2 pins async-tls to the semver compliant v0.11.0 instead
of the yanked async-tls v0.10.2.

* kilt&polimec ss58 prefix (#7547)

* kilt ss58 address

* polimec ss58 prefix

* Fix docs (#7710)

* fix docs

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update md link checker (#7709)

* CI: ignore %20 for md link checker

* CI: update to a new action release that fixes %20 issue

* Fix PerThing::from_percent. (#7701)

* Fix overflow in per_things::from_percent.

* Fix test

* Fix the whole thing.. :|

* Support Multiple Instances with Benchmarks (#7669)

* Support multiple instances with benchmarks

* fix tests

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* docs

* fix output

* Update lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add keccak-512 to host functions. (#7531)

* Update debug.rs (#7721)

* Avoid too high values. (#7716)

* Bump hyper from 0.12.35 to 0.13.9 (#7569)

* Bump hyper from 0.12.35 to 0.13.9

Bumps [hyper](https://github.com/hyperium/hyper) from 0.12.35 to 0.13.9.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.12.35...v0.13.9)

Signed-off-by: dependabot[bot] <support@github.com>

* Update bin/node/rpc-client/Cargo.toml

* Update Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Prevent dust in staking by disallowing cheap bond_extra (#7718)

* prevent bond_extra to cause staking actve lower than ed

* prevent bond_extra to cause staking actve lower than ed

* Check in post conditions.

* check rebond as well.

* also change withdraw_unbonded.

* Fix build

* change check format.

* Apply suggestions from code review

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Default block announce validation should reject additional data (#7726)

If we are using the default block announce validator and receive
additional data alongside the block announcement, we should disconnect
this peer.

* Fix wrong order of values in log line (#7728)

* Fix polkadot companion test (#7732)

* remove duplicate accounts in chain-spec. (#7725)

* remove duplicate accounts in chain-spec.

* Fix build

* Enforce in balances modules.

* Apply suggestions from code review

* Update frame/balances/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

* Participating in Council Governance is Free for First Time Voters and Successful Closing (#7661)

* wk2049 | D5 | addition-motions first free vote for members | p1

* Update frame/collective/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* wk2049 | D6 | addition-motions first free vote for members | p2

* Update frame/collective/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update frame/collective/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* wk2049 | D7 | addition-motions first free vote for members | p3

* Update frame/collective/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* wk2049 | D7 | addition-motions first free vote for members | p4

* wk2049 | D7 | addition-motions first free vote for members | p6

* Update frame/collective/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/collective/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/collective/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/collective/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/collective/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/collective/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/collective/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* wk2050 | D3 | addition-motions first free vote for members | p7

* wk2050 | D3 | addition-motions first free vote for members | p8

* update comment

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Bump zeroize from 1.1.1 to 1.2.0 (#7731)

Bumps [zeroize](https://github.com/iqlusioninc/crates) from 1.1.1 to 1.2.0.
- [Release notes](https://github.com/iqlusioninc/crates/releases)
- [Commits](https://github.com/iqlusioninc/crates/compare/zeroize/v1.1.1...zeroize/v1.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Leave some header related info for inspection (#7727)

* Leave some system data for inspection

There is not much benefit in being active when removing this data. It's
actively harmful when one tries to read the block number in runtime
APIs in the context of a block.

* Update the expected root hash

This is excepted since now we persist new members.

* Revert extrinsics_root to `take`

It's going away in one of the following PRs anyway

* Update the state root once again

* Update the comment on the storage items that are left in the storage

Excluding ExtrinsicsRoot since it's going away

* Issue 7143 | Refactor Treasury Pallet into Bounties, Tips, and Proposals (#7536)

* wk2046 | D5 | treasury refactor to bounties and tips

* wk2046 | D5 | treasury refactor to bounties and tips | p2

* fix test compilation, ignoring events

* initialize treasury in genesis

* wk2046 | D7 | treasury refactor | fix bounties test build issues

* wk2047 | D1 | treasury refactor | tips pallet bringup

* wk2047 | D2 | treasury refactor | bounties  pallet | unit test bringup

* wk2047 | D2 | treasury refactor | bounties  pallet | unit test bringup | p2

* wk2047 | D2 | treasury refactor | pallet-tips| test_last_reward_migration | test failure - analysis

* wk2047 | D3 | treasury refactor | pallet-tips| test_last_reward_migration | test failure - fix

* wk2047 | D3 | treasury refactor | pallet-bounties | on_initialize() fix

* wk2047 | D3 | treasury refactor | pallet-bounties | on_initialize() fix | p2

* wk2047 | D4 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks

* wk2047 | D4 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks | p2

* wk2047 | D4 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks | p3

* wk2047 | D5 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks | p4

* wk2047 | D6 | treasury refactor | review comments fix

* some fixes

* fix bounties instantiable

* remove instantiable from tips and bounties

* fix compile for benchmarks

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_treasury --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/treasury/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update lib.rs

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_tips --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/tips/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bounties --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bounties/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* add back `on_initialize_bounties`

* patch up bounties benchmarks

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bounties --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bounties/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Remove development TODO from public doc comment (#7500)

* refactor subtrait/elevated trait as not needed (#7497)

* Fix comments of indices pallet events (#7511)

Arguments for IndexAssigned and IndexFrozen were inverted in comments.

* Skip slot lenience on first block in BABE (#7515)

The genesis header doesn't have the BABE pre-digest and we insert `0` as
slot number. The slot lenience calculation will return the maximum in
this situation. Besides returning the maximum which is not bad at all,
it also prints some a debug message that can be confusing in the first
moment. To prevent printing this debug message, we now just return early
when we see that the parent block is the genesis block.

* slots: incrementally backoff claiming slots if finality lags behind (#7186)

* babe: backoff authoring blocks when finality lags

* babe: move backoff authoring params to default constructor

* babe: deduplicate the test a bit

* babe: set backoff constants in service

* babe: use better names for backoff authoring block parameters

* babe: remove last unwrap

* babe: slight style tweak

* babe: fix comment

* slots: move backoff block authorship logic to SimpleSlotWorker

* aura: append SlotInfo in on_slot

* slots: use the correct types for parameters

* slots: fix review comments

* aura: add missing backoff authoring blocks parameters

* slots: add comments for default values

* slots: add additional checks in test

* slots: update implementation for new master

* slots: revert the change to SlotInfo

* Fix review comments

* slots: rework unit tests for backing off claiming slots

* slots: add test for asymptotic behaviour for slot claims

* slots: address review comments

* slots: add test for max_interval

* slots: add assertion for intervals between between claimed slots

* slots: remove rustfmt directive

* slots: another attempt at explaining authoring_rate

* slots: up unfinalized_slack to 50 by default

* slots: add tests for time to reach max_interval

* slots: fix typo in comments

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* slots: additional tweaks to comments and info calls

* slots: rename to BackoffAuthoringOnFinalizedHeadLagging

* slots: make the backing off strategy generic

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* slots: implement backoff trait for () for simplicity

* slots: move logging inside backing off function to make it more specific

* aura: add missing function parameter

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Export app-crypto specific keystore functions (#7489)

* Export app-crypto specific keystore functions

* Also add back the insert function

* Switch KeystoreContainer to an enum

* Only export the bare minimal for LocalKeystore and fix service compile

* fix: should return Arc

* Add docs stating that functions only available in local keystore

* Remove insert and generate functions

* fix: generate function should be available in test

* Add keypair function to trait

* Revert "Add keypair function to trait"

This reverts commit ad921b09ca73d3c09298e3a51b562ef8e0067781.

* Add note for local_keystore function in service

* Update doc for the --chain flag (#7520)

* contracts: Add missing instruction to the `Schedule` (#7527)

* Don't log with colors when we are writing to a tty (#7525)

* Don't log with colors when we are writing to a tty

This fixes a regression that was introduced by the switch to tracing.
Before we killed all colors before writing to a tty, this pr brings the
behaviour back.

* Remove accidentally added crate

* Review feedback

* More feedback

* Update client/cli/src/logging.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/cli/src/logging.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* MemoryId -> u32 (#7534)

* Enable local addresses in DHT when chain type == `Local` | `Development` (#7538)

* Enable local addresses in DHT when chain type == `Local` | `Development`

This pr changes when to add local addresses to DHT. Instead of only
checking if `--discover-local` and `--dev` are present, we now also
check if the chain type is `Local` or `Development`.

* Update the docs!

* Update tiny-bip39 to v0.8 (#7539)

It would improve secret zeroization due to
https://github.com/maciejhirsz/tiny-bip39/pull/22, and would also remove
one of the points where we depend on `failure` crate, which is
deprecated (see https://github.com/rust-lang-nursery/failure/pull/347)

* make LocalCallExecutor public (#7528)

* Fix some weirdness in `offchain_worker` (#7541)

We call `offchain_worker` with the state of the imported block and pass
the header of this block. However in the runtime we call all
`offchain_worker` functions with the number of the parent block. Besides
that we also pass all digests and not only the pre runtime digests. In
the context where the offchain worker is executed we have all digests, so
there is no real reason to only pass pre runtime digests. Another fix is
that we also insert the hash of the current header into the block hash map.

* Use inbound peerslot slots when a substream is received, rather than a connection (#7464)

* Use inbound peerslot slots when a substream is received, rather than a connection

* Refactor PeerState

* Some bugfixes

* Fix warnings so that CI runs, gmlrlblbl

* Bugfixes

* Update docs

* Apply suggestions from code review

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Clean up Banned state

* Refactor connections state

* Fix possibility of Enabled with no Opening or Open connection

* Line width

* Add some debug_asserts! and fix TODO

* Refactor legacy handler

* Rewrite group.rs entirely [part 1]

* Rewrite group.rs entirely [part 2]

* Remove faulty assertion

Because of the asynchronous nature of the behaviour <-> handler communications, it is possible to receive notifications while in the Closing state

* Don't poll the legacy substream is not Open

* Tolerate when not all substreams are accepted

* Remove TODOs

* Dummy commit to make CI log interesting things

* Try race condition fix

* Revert "Try race condition fix"

This reverts commit 0675c659d06195c30f8c5bc13e2d88141d57a3ba.

* Correctly rebuild pending_opening

* Minor tweaks

* Printlns for CI debugging

* Revert "Printlns for CI debugging"

This reverts commit e7852a231f4fc418898767aaa27c9a4358e12e8b.

* Revert "Dummy commit to make CI log interesting things"

This reverts commit 259ddd74088e53e7c6a9b0a62a8d1573a0063ce3.

* mv group.rs ../handler.rs

* Apply suggestions from code review

Co-authored-by: Max Inden <mail@max-inden.de>

* Banned => Backoff

* Mention the actual PeerStates

* OpenDesired -> OpenDesiredByRemote

* OpeningThenClosing

* Add doc links to PeerState

* Simplify increment logic

* One more debug_assert

* debug_assert!

* OpenDesiredByRemote

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>

* *: Update to libp2p v0.30.0 (#7508)

* *: Update to libp2p v0.30.0

* Cargo.lock: Update

* *: Update to libp2p v0.30.1

* make ClientConfig public (#7544)

* sc-basic-authorship: remove useless dependencies (#7550)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Add slashing events to elections-phragmen. (#7543)

* Add slashing events to elections-phragmen.

* Fix build

* Apply suggestions from code review

* Update frame/elections-phragmen/src/lib.rs

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Remove necessity to pass ConsensusEngineId when registering notifications protocol (#7549)

* Remove necessity to pass ConsensusEngineId when registering notifications protocol

* Line width

* Fix tests protocol name

* Other renames

* Doc update

* Change issue in TODO

* sc-cli: replace bip39 with tiny-bip39 (#7551)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Add extra docs to on_initialize (#7552)

* Add some extra on_initialize docs.

* Address review comments.

* More Extensible Multiaddress Format (#7380)

* More extensible multiaddress format

* update name

* Don't depend on indices to define multiaddress type

* Use MultiAddress in Node Template too!

* reduce traits, fix build

* support multiple `StaticLookup`

* bump tx version

* feedback

* Fix weight template to remove ugliness in rust doc (#7565)

fixed weight template

* Cargo.lock: Run cargo update (#7553)

* Cargo.lock: Run cargo update

* Cargo.lock: Downgrade cc to v1.0.62

* Cargo.lock: Revert wasm-* updates

* .github: Add dependabot config and thus enable dependabot (#7509)

* .github: Add dependabot config and thus enable dependabot

* Update .github/dependabot.yml

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Thread-local parameter_types for testing. (#7542)

* Thread-local parameter_types for testing.

* Better docs.

* Some minors

* Merge'em

* Update frame/support/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Align more to basti's trick

* Update frame/support/src/lib.rs

* Update frame/support/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

* Bump wasm-bindgen-test from 0.3.12 to 0.3.17 (#7567)

* Bump wasm-bindgen-test from 0.3.12 to 0.3.17

Bumps [wasm-bindgen-test](https://github.com/rustwasm/wasm-bindgen) from 0.3.12 to 0.3.17.
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

Signed-off-by: dependabot[bot] <support@github.com>

* Update wasm-bindgen pin to 0.2.68

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* wk2047 | D6 | treasury refactor | review comments fix | p2

* wk2048 | D1 | treasury refactor | review comments fix | p3

* Update bin/node/runtime/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update bin/node/runtime/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* wk2048 | D3 | treasury refactor | review comments fix | p4

* wk2048 | D3 | treasury refactor | review comments fix | p5

* wk2048 | D4 | treasury refactor | review comments fix |  removal of deadcode | p6

* remove broken link

* wk2048 | D5 | treasury refactor | review comments fix | bountise doc string | p7

* wk2048 | D5 | treasury refactor | review comments fix | p8

* docs and formatting

* Update frame/tips/src/benchmarking.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* formatting nits

* Trait -> Config

* trait -> config in benchmarks

* clean up weight docs

* Trait -> Config in Runtime

* fix test build

* try to fix polkadot build check

* fix traits

* Update lib.rs

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* fix trait location

* nits

* uncomment on_initialize for bounties benchmarks

* update weights

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Caio <c410.f3r@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Antoine Le Calvez <alecalve@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com>
Co-authored-by: Wei Tang <wei@that.world>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Kirill Pimenov <kirill@parity.io>
Co-authored-by: Andrew Plaza <aplaza@liquidthink.net>
Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Qinxuan Chen <koushiro.cqx@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Addie Wagenknecht <addie@nortd.com>

* CI: buildah to build images (#7741)

* CI: buildah to build images

* CI: simplify variables

* CI: whitespaces

* CI: secure login

* optimize arithmetic cargo path (#7745)

* optimize runtime-interface cargo path (#7729)

* delete primitives in Cargo in primitives (#7751)

* delete client in Cargo in client (#7752)

* prevent too many genesis council members (#7749)

* Prevent too many genesis members in elections-phragmen.

* Fix test.

* reformat.'

* Update common block in sync after importing blocks of a peer, please read UPDATE (#7733)

* Update common block in sync after importing blocks of a peer

This updates the sync code to update the common block of a peer, after
we have imported blocks from this peer. This fixes a bug for when we are
connected to one or more nodes that are doing a full sync as our node.
Nodes in full sync will not announce new blocks, as we don't send import
notifications on full sync. The problem as now that we were connected to
some peer that reported some low number as its best and we tried to sync
these blocks. But, as we did not update the common block of this peer,
we would sync these blocks over and over again. Being captured in some
time warp.
The solution to this problem is that we increase the common number as we
import blocks from this peer.

* Test

* Test name..

* Fix test

* Cleanup some code and write some new regression test

* Implement the ancestor search

* Check that the common number is smaller than the last finalized block

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Change the way we build the status messages

* Start some new test...

* Finish test

* Rename test

* Update client/network/src/protocol.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Remove unnecessary parameter from sync (#7761)

* Improve `InvalidTransaction::AncientBirthBlock` doc comment (#7743)

* Improve `InvalidTransaction::AncientBirthBlock` doc comment

Update

Remove stray parentheses

Tabs not spaces

Improve

whoops

* Update primitives/runtime/src/transaction_validity.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Possible causes section; Expand example

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Switch back to the crates.io release of trybuild (#7764)

* Switch back to the crates.io release of trybuild

My fix was merged on upstream and this release contains it. So, no
more reason to keep the git dependency.

* The lock file...

* No longer put keys back manually after a failed restoration (#7747)

This is no longer necessary with storage transactions.

* Fix UI Tests (#7722)

* Fix UI tests

* make updates.

* Revert "make updates."

This reverts commit 3f41979c12e86acb75fd086675ac09659dde533e.

* Try to fix companion build

* Use some depth

* tests: fix UI tests on the latest nightly

* tests: fix polkadot companion test, depth

* chore: update deps

* chore: update deps

* chore: sp-io

Co-authored-by: kianenigma <kian.peymani@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Denis P <denis.pisarev@parity.io>

* update substrate to sc in docs (#7767)

* Allow Staking tests to run with session length other than 1 (#7719)

* fix periodic session

* Allow staking tests to run with session lengths other than 1.

* Update frame/staking/src/mock.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Fix all tests with session length 5.

* Test for active != current

* Better doc

* Update frame/staking/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* also set the timestamp properly.

* trigger CI

* Revert "trigger CI"

This reverts commit 0f254944cdad848aa6e63bd8a618db95447a8e68.

* Update frame/staking/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* optimize biguint div closure (#7754)

* optimize biguint div closure

* optimize biguint sub and fix note

* change and add biguint split test

* add biguint div_unit test

* update biguint sub v to v1

* add biguint shift_check

* Make it possible to calculate the storage root as often as you want (#7714)

* Make it possible to calculate the storage as often as you want

So, until now each Substrate based blockchain has calculated the storage
root once, at the end of the block. Now there is Frontier that wants to
calculate some intermediate storage root. However this failed on block
import. The problem with that was the extrinsics root. When building the
block we stored `Default::default()` as extrinsics root, because yeah,
we don't know the extrinsics root before finishing the block. At the end
this extrinsics root was then calculated. But on block import we passed
the already known extrinsics root. This was no problem, as we removed
this value at the end of the block. However when you all the storage
root in between, that changes the storage root between block building
and block import.

This pr changes this behavior. It removes the `ExtrinsicsRoot` storage
entry and also doesn't pass it anymore to `System::initialize`. By doing
it, we remove the difference in the storage and fix the storage root mismatch.

* Fix bug with incorrectly calculating the extrinscs root

* Review feedback

* Typo fix: eror => error (#7773)

* Adds `try_get` for `StorageMap` and `StorageDoubleMap` (#7774)

* Adds `try_get` for `StorageMap` and `StorageDoubleMap`

* Switch to Value as return type

* Add account prefix for Valiu (#7776)

* Feat support weight test (#7769)

* update support weights_are_correct

* add support polynomial_does_not_underflow

* Add `pallet` attribute macro to declare pallets (#6877)

* rename system Config to system Trait.

command used:
```
find frame/ bin/ test-utils/ utils/ -name *.rs -exec sed -i 's/system::Trait>::/system::Config>::/g' {} \;
find frame/ bin/ test-utils/ utils/ -name *.rs -exec sed -i 's/impl frame_system::Trait for /impl frame_system::Config for /g' {} \;
find frame/ bin/ test-utils/ utils/ -name *.rs -exec sed -i 's/impl system::Trait for /impl system::Config for /g' {} \;
```
plus some manual ones especially for frame-support tests and frame-system

* make construct_runtime handle Pallet and Module

pallets can now be implemented on struct named Pallet or Module, both
definition are valid.
This is because next macro will generate only Pallet placeholder.

* introduce pallet attribute macro

currently just with tests, frame_system and other example hasn't been
upgraded

* allow to print some upgrade helper from decl_storage

* Improved error msg, typo.

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Improved error msg, typo.

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Improved error message on unexpected attributes + ui test

* add test for transactional

* various typo

* some tips when spans are lost

* allow pallet to depend on other pallet instances

* make event type metadata consistent with call and constant

* error messages

* ignore doc example

* fix pallet upgrade template

* fixup

* fix doc

* fix indentation

* Apply suggestions code formatting

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* some renames + fix compilation

* remove unsupported genesis config type alias

* merge fixup

* fix ui tests

* additional doc

* implement StorageInstance with new syntax

* fix line width

* fix doc: because pallet doc goes below reexport doc

* Update frame/support/procedural/src/pallet/parse/event.rs

Co-authored-by: Andrew Jones <ascjones@gmail.com>

* Update frame/system/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update frame/support/test/tests/pallet_ui.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* improve doc as suggested

* revert construct_runtime Pallet part.

This revert the changes on construct_runtime. Now construct_runtime is
unchanged and instead pallet macro create a type alias
`type Module<..> = Pallet<..>` to be used by construct_runtime

* refactor with less intricated code

* fix ui test with new image

* fix ui tests

* add minor tests

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* add a `current_epoch` to BabeApi (#7789)

* add a `current_epoch` to BabeApi

* add current_epoch_start back again

* fix node-runtime

* bump spec version

* Update tests.rs (#7784)

Fix testing for number of slashing spans.

* add a slot-duration getter to babe config (#7793)

* Fix base-path handling in `key insert` (#7775)

This fixes the handling of base-path when using `key insert`. Before
the base-path wasn't setup correctly, as done when starting a node. This
resulted in putting the keys into the wrong directory. This pr fixes
this by creating the correct base-path/config dir for the keystore.
Besides that it also removes the insert command from `subkey` as it
doesn't make that much sense. If requested, we could bring it back later.

* Lazily evaluate error string (#7802)

* contracts: Change `define_env!` to expect a `Result<T, DispatchError>` for every function (#7762)

* Make host functions return TrapReason

This avoids the need to manually store any trap reasons
to the `Runtime` from the host function. This adds the following
benefits:

* It properly composes with the upcoming chain extensions
* Missing to set a trap value is now a compile error

* review: Remove superflous .into()

* Hex Balance deserialize for contracts_call RPC (#7807)

* Hex Balance deserialize for contracts_call RPC

* Avoid temporary conversion into u128

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Don't allow self proxies (#7803)

* Allow council to slash treasury tip (#7753)

* wk2051 | D4 |Allow council to slash treasury tip | p1

* Update frame/tips/src/lib.rs

Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>

* wk2051 | D5 |Allow council to slash treasury tip | p2

* wk2051 | D5 |Allow council to slash treasury tip | p3

* wk2051 | D5 |Allow council to slash treasury tip | p4

* wk2051 | D5 |Allow council to slash treasury tip | p5

* random change

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_tips --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/tips/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix typo

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/tests.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* wk2052 | D1 | Allow council to slash treasury tip | p6

Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add CLI argument to disable log color output (#7795)

* Use path instead of ident (#7809)

* Add proper `commit_all` to `TestExternalities` (#7808)

* Add proper `commit_all` to `TestExternalities`

This pr adds a propoer `commit_all` function to `TestExternalities` to
commit all changes from the overlay to the internal backend. Besides
that it fixes some bugs with handling empty dbs when calculating a delta
storage root. It also changes the way data is added to the in memory
backend.

* Update primitives/state-machine/src/testing.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Don't allow self proxies (#7803)

* Allow council to slash treasury tip (#7753)

* wk2051 | D4 |Allow council to slash treasury tip | p1

* Update frame/tips/src/lib.rs

Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>

* wk2051 | D5 |Allow council to slash treasury tip | p2

* wk2051 | D5 |Allow council to slash treasury tip | p3

* wk2051 | D5 |Allow council to slash treasury tip | p4

* wk2051 | D5 |Allow council to slash treasury tip | p5

* random change

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_tips --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/tips/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix typo

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/tests.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* wk2052 | D1 | Allow council to slash treasury tip | p6

Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Review feedback

* Review feedback

* Update docs

* More docs

* Make it private

* Use `None`

* Use apply transaction

* Update primitives/state-machine/src/testing.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: RK <r.raajey@gmail.com>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Define ss58 prefix inside the runtime (#7810)

* Add SS58Prefix type to the frame_system config trait

* Remove unused chain_id runtime interface

* remove some unecessary bound (#7813)

* Fix ss58check test when executed with other tests (#7815)

There was a bug that could make other ss58 tests fail when being
executed with this one in parallel. This test changes the default ss58
version and if other tests are run at the time the default version is
changed, they would fail. To fix this problem, we now run the actual
test as a new process.

* Cleanup some warnings (#7816)

* client: cleanup redundant semicolon warnings

* grandpa: remove usage of deprecated compare_and_swap

* Happy new year (#7814)

* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL

* contracts: Allow runtime authors to define a chain extension (#7548)

* Make host functions return TrapReason

This avoids the need to manually store any trap reasons
to the `Runtime` from the host function. This adds the following
benefits:

* It properly composes with the upcoming chain extensions
* Missing to set a trap value is now a compile error

* Add chain extension

The chain extension is a way for the contract author to add new
host functions for contracts to call.

* Add tests for chain extensions

* Fix regression in set_rent.wat fixture

Not all offsets where properly updated when changing the fixtures
for the new salt on instantiate.

* Pre-charge a weight amount based off the specified length

* Improve fn write docs

* Renamed state to phantom

* Fix typo

* *: Update to libp2p v0.33.0 (#7759)

* *: Update to libp2p v0.33.0

* client/network: Consistently track request arrival time

With https://github.com/libp2p/rust-libp2p/pull/1886/ one is guaranteed
to receive either a `ResponseSent` or a `InboundFailure` event for each
received inbound request via `RequestResponseEvent::Message`. Given this
guarantee there is no need to track arrival times in a best-effort
manner and thus there is no need to use a LRU cache for arrival times.

* client/offchain: Adjust to PeerId API changes

* contracts: Lazy storage removal (#7740)

* Do not evict a contract from within a call stack

We don't want to trigger contract eviction automatically when
a contract is called. This is because those changes can be
reverted due to how storage transactions are used at the moment.
More Information:
https://github.com/paritytech/substrate/issues/6439#issuecomment-648754324

It can be re-introduced once the linked issue is resolved. In the meantime
`claim_surcharge` must be called to evict a contract.

* Lazily delete storage in on_initialize instead of when removing the contract

* Add missing documentation of new error

* Make Module::claim_surcharge public

It being the only dispatchable that is private is an oversight.

* review: Add final newline

* review: Simplify assert statement

* Add test that checks that partial remove of a contract works

* Premote warning to error

* Added missing docs for seal_terminate

* Lazy deletion should only take AVERAGE_ON_INITIALIZE_RATIO of the block

* Added informational about the lazy deletion throughput

* Avoid lazy deletion in case the block is already full

* Prevent queue decoding in case of an already full block

* Add test that checks that on_initialize honors block limits

* fix template (#7823)

* rename HEADER files so that they are consistent with LICENSE filenames (#7825)

* contracts: Prevent contracts from allocating a too large buffer (#7818)

* Prevent contracts from allocating a too large buffer

* Fix possible integer overflow

* Improve error message on where clause on pallet error (#7821)

* improve error message on where clause on pallet error

* Revert "improve error message on where clause on pallet error"

This reverts commit 5a3cc38976813fccef3357833553ce30f5b988ea.

* Revert "Revert "improve error message on where clause on pallet error""

This reverts commit e3b3fca6bc4fa89816f80dbcb82dc4536a9b2549.

* Feat sp keystore (#7826)

* delete not used VRFTranscriptValue

* specification variable naming

* minor fix (#7828)

* Participation Lottery Pallet (#7221)

* Basic design

* start adding tests

* finish tests

* clean up crates

* use call index for match

* finish benchmarks

* add to runtime

* fix

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* more efficient storage

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update lib.rs

* Update bin/node/runtime/src/lib.rs

* trait -> config

* add repeating lottery

* new benchmarks

* fix build

* move trait for warning

* feedback from @xlc

* add stop_repeat

* fix

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Support static calls

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix test

* add loop to mitigate modulo bias

* Update weights for worst case scenario loop

* Initialize pot with ED

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* client/network: Use request response for block requests (#7478)

* client/network: Add scaffolding for finality req to use req resp
	#sc

* client/network/src/finality_requests: Remove

* client/network/src/behaviour: Pass request id down to sync

* client/network: Use request response for block requests

* client/network: Move handler logic into *_*_handler.rs

* client/network: Track ongoing finality requests in protocol.rs

* client/network: Remove commented out finalization initialization

* client/network: Add docs for request handlers

* client/network/finality_request_handler: Log errors

* client/network/block_request_handler: Log errors

* client/network: Format

* client/network: Handle block request failure

* protocols/network: Fix tests

* client/network/src/behaviour: Handle request sending errors

* client/network: Move response handling into custom method

* client/network/protocol: Handle block response errors

* client/network/protocol: Remove tracking of obsolete requests

* client/network/protocol: Remove block request start time tracking

This will be handled generically via request-responses.

* client/network/protocol: Refactor on_*_request_started

* client/network: Pass protocol config instead of protocol name

* client/network: Pass protocol config in tests

* client/network/config: Document request response configs

* client/network/src/_request_handler: Document protocol config gen

* client/network/src/protocol: Document Peer request values

* client/network: Rework request response to always use oneshot

* client/network: Unified metric reporting for all request protocols

* client/network: Move protobuf parsing into protocol.rs

* client/network/src/protocol: Return pending events after poll

* client/network: Improve error handling and documentation

* client/network/behaviour: Remove outdated error types

* Update client/network/src/block_request_handler.rs

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* Update client/network/src/finality_request_handler.rs

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* client/network/protocol: Reduce reputation on timeout

* client/network/protocol: Refine reputation changes

* client/network/block_request_handler: Set and explain queue length

* client/service: Deny block requests when light client

* client/service: Fix role matching

* client: Enforce line width

* client/network/request_responses: Fix unit tests

* client/network: Expose time to build response via metrics

* client/network/request_responses: Fix early connection closed error

* client/network/protocol: Fix line length

* client/network/protocol: Disconnect on most request failures

* client/network/protocol: Disconnect peer when oneshot is canceled

* client/network/protocol: Disconnect peer even when connection closed

* client/network/protocol: Remove debugging log line

* client/network/request_response: Use Clone::clone for error

* client/network/request_response: Remove outdated comment

With libp2p v0.33.0 libp2p-request-response properly sends inbound
failures on connections being closed.

Co-authored-by: Addie Wagenknecht <addie@nortd.com>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* fix : remove `_{ }` syntax from benchmark macro (#7822)

* commented use of common

* hack to pass tests

* another hack

* remove all commented code

* fix the easy tests

* temp hack

* follow through comma hack until better solution

* patch macro

* missed one

* update benchmarks

* update docs

* fix docs

* removed too much

* fix changes

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Improve spans of pallet macro (#7830)

* fix spans

* convert name to snake case

* Fix master build (#7837)

* Fix master build

* Use correct copyright year

* babe: expose next epoch data (#7829)

* babe: expose next epoch data

* babe: add runtime api for next_epoch

* babe: avoid reading next authorities from storage unnecessarily

* babe: add notes about epoch duration constraints

* babe: guard against overflow

* babe: add test for fetching current and next epoch data

* contracts: Add configurable per-storage item cost (#7819)

* Rework rent parameters

* No need for empty_pair_count any longer

* Parameterize runtime

* upgrade a few dependencies (#7831)

* upgrade a few dependencies

* make it compile at the expense of duplicate deps

* fix web-wasm and a warning

* introduce activate-wasm-bindgen-features crate

* Revert "introduce activate-wasm-bindgen-features crate"

This reverts commit 5a6e41e683f8a4844c0a735dcd08caabb2313f11.

* add getrandom feature to sc-consensus-aura

* CI: remove squash and fix buildah push (#7841)

* Fix incorrect use of syn::exports (#7838)

* Fix incorrect use of syn::exports

Instead of using `syn::exports` we should import the trait from the
quote crate directly.

* Use own macro for test cases to fix compilation with latest syn

* Fix test

* Subkey should not import the entire world. (#7845)

There is no reason for subkey to import the default Substrate node to
support a feature that would only be usable for the Substrate node.
Subkey itself should be more the default key management binary for
Substrate related chains. If certain chains require some special
functionality, they can easily stick together their own "my-chain-key".

* Rework priority groups, take 2 (#7700)

* Rework priority groups

* Broken tests fix

* Fix warning causing CI to fail

* [Hack] Try restore backwards-compatibility

* Fix peerset bug

* Doc fixes and clean up

* Error on state mismatch

* Try debug CI

* CI debugging

* [CI debug] Can I please see this line

* Revert "[CI debug] Can I please see this line"

This reverts commit 4b7cf7c1511f579cd818b21d46bd11642dfac5cb.

* Revert "CI debugging"

This reverts commit 9011f1f564b860386dc7dd6ffa9fc34ea7107623.

* Fix error! which isn't actually an error

* Fix Ok() returned when actually Err()

* Tweaks and fixes

* Fix build

* Peerset bugfix

* [Debug] Try outbound GrandPa slots

* Another bugfix

* Revert "[Debug] Try outbound GrandPa slots"

This reverts commit d175b9208c088faad77d9f0ce36ff6f48bd92dd3.

* [Debug] Try outbound GrandPa slots

* Apply suggestions from code review

Co-authored-by: Max Inden <mail@max-inden.de>

* Use consts for hardcoded peersets

* Revert "Try debug CI"

This reverts commit 62c4ad5e79c03d561c714a008022ecac463a597e.

* Renames

* Line widths

* Add doc

Co-authored-by: Max Inden <mail@max-inden.de>

* Better Handle Dead Accounts in Balances (#7843)

* Don't mutate storage when account is dead and should stay dead

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* more concrete storage noop

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* bump fs-swap (#7834)

* UniArts reserve SS58 address id 38 (#7651)

* UniArts reserve SS58 address id 45

* Update ss58-registry.json

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

Co-authored-by: Xiang Li <bigxiang@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update to futures 0.3.9 (#7854)

* Store dispatch info of calls locally in weight calculation (#7849)

* utility

* sudo

* more

* recovery

* better formatting

* client/network: Re-enable light_client_handler.rs unit tests (#7853)

* Fix max log level (#7851)

With the switch to tracing we did not set the `max_log_level` anymore.
This resulted in a performance degradation as logging did not early
exited and thus, `trace` logs were at least resolved every time.

This pr fixes it by ensuring that we set the correct max log level.

* Fix missing network for uniarts (#7859)

* contracts: Collect rent for the first block during deployment (#7847)

* Pay first rent during instantiation

* Fix and add new tests

* Do not increment trie id counter on failure

* Merge 2.0.1 backport branch into mainline master (#7842)

* Backport paritytech/substrate#7381

* Bring back genesis storage build in aura/timestamp

To not change spec version, see
https://github.com/paritytech/substrate/pull/7686#discussion_r540032743

* Backport paritytech/substrate#7238

* Backport paritytech/substrate#7395

* Bump impl_version

* Fix UI tests and bump trybuild dep

See https://github.com/rust-lang/rust/pull/73996

Backports:
https://github.com/paritytech/substrate/pull/7764
https://github.com/paritytech/substrate/pull/7656

* Partially backport paritytech/substrate#7838

* Release frame-support with a dep compilation fix

* Bump patch level for remaining crates

This is done because at the time of writing cargo-unleash does not fully
support partial workspace publishing and mixes both local and crates.io
versions of the packages, leading to errors in the release check workflow.

* Backport paritytech/substrate#7854

...to fix compilation error when using futures-* v0.3.9.

* Adding Changelog  entry for patch release

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Benjamin Kampmann <ben@parity.io>

* Bump cargo-unleash to latest alpha release (#7867)

* Bump sha2 from 0.8.2 to 0.9.2 (#7643)

* Bump sha2 from 0.8.2 to 0.9.2

Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.8.2 to 0.9.2.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.8.2...streebog-v0.9.2)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix compilation error

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

* bumpd minor version (#7873)

* Add Prometheus alerts if unbounded channels are too large (#7866)

* Add Prometheus alerts if unbounded channels are too large

* Tweaks

* Bump retain_mut from 0.1.1 to 0.1.2 (#7869)

Bumps [retain_mut](https://github.com/upsuper/retain_mut) from 0.1.1 to 0.1.2.
- [Release notes](https://github.com/upsuper/retain_mut/releases)
- [Commits](https://github.com/upsuper/retain_mut/compare/v0.1.1...v0.1.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* babe: initialize next authorities on genesis (#7872)

* babe: initialize next authorities on genesis

* babe: add test for genesis authorities

* Update serde and parity-multiaddr, to fix master CI (#7877)

* Add ss58 version prefix for CORD (from Dhiway) (#7862)

* Add ss58 version prefix for CORD

* Add ss58 version pref…
@athei athei mentioned this pull request Mar 22, 2021
This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

contracts: Lazy contract removal
5 participants