Rework priority groups, take 2 #7700

tomaka · 2020-12-09T11:51:44Z

(this PR is the finished version of #7374)
polkadot companion: paritytech/polkadot#2095
close #6087

cc @rphmeier @ordian @eskimor
While you don't have to review the PR, I would appreciate opinions on the design described below.

This PR refactors the slots attribution system of the networking.

Before this PR

A Substrate or Polkadot node establishes TCP connections with other libp2p-compatible nodes.
The set of connected peers is divided in two: the ones that have a slot, and the ones that don't.

The ones that don't have a slot are able to perform requests using request-response protocols, but are disconnected after 10 seconds of inactivity. Performing a Kademlia iterative query, for example, requires establishing connections to many nodes, and these connections don't need a slot.

The ones that have a slot are the ones notifications substreams are opened with. Multiple notifications protocols exist (block announces, transactions, grandpa, and, in Polkadot, collation and validity), and when a slot is attributed to a peer, one substream for each notifications protocol is opened.
As long as a substream is maintained using the block announces notification protocol (special-cased for this purpose), the peer keeps its slot, even if this peer doesn't support or has refused the protocols other than the block announces. As such, it is for example possible for all slots to be filled only with peers that only support block announces.

Due to backwards-compatibility issue, once a peer has performed (successfully or not) its handshake on all notifications protocols substreams, an Event::NotificationStreamOpened event is reported to the event subscribers. One such event is emitted for all notifications protocols, even if the substream of said protocol hasn't been accepted or isn't supported! This will lead the rest of the codebase to try send notifications using this protocol to a peer even though there is no substream for it.

Slots are attributed in the following way:

25 incoming slots, for peers that initiate opening the substreams to the local node.
25 outgoing slots, assigned randomly and based on reputation.
The API of sc-network makes it possible to define a list of priority groups containing peers that automatically have a slot attributed to them.
Reserved nodes, similar to priority groups, but defined by the end user with the --reserved-node CLI option.

Priority groups are currently used for two purposes: the authority-discovery module maintains a priority group containing 10 randomly-chosen validators, and Polkadot maintains a priority group containing the validators assigned to the current parachain.

When the user passes the --reserved-only CLI flag, all slots are closed except for nodes passed through --reserved-node.

Problems with the current situation

The problems with the current situation are:

We need to guarantee connectivity between collators and validators. Validators don't know in advance who is a collator and who isn't, and it is possible, at the moment, for a validator to have all its incoming slots filled with non-collators, in which case collators will be unable to collate.
It is not desirable to make the code base believe that notifications protocols substreams are open, even when it is not the case.
It is not desirable to open the collation notifications protocol substream with non-validators/non-collators, and it is not desirable to open the validity notifications protocol substream with non-validators.

After this PR

The networking is now using peer sets. A peer set is associated to each notifications protocol. As such, in a Polkadot node, there exists five peer sets: block announces, transactions, grandpa, collation, validity.

Rather than being global, slots are now specific to a peer set. Each TCP connection can have, or not, a slot open with regards to each peer set. For example, a connection can have a slot for block announces but not for grandpa.

The way slots are attributed depends on the set.

The block announces peer set keeps the same slots attribution mechanism as the previously-global slots-attribution mechanism: 25 incoming, 25 outgoing, and reserved nodes. Priority groups have been removed from the codebase.

The transactions and grandpa slots attribution is configured to be permanently in "reserved only" mode, meaning that only a specific list of nodes are allowed to have a slot. This list of nodes is constantly kept synchronized with the nodes that occupy a block announces slot.
In other words, the transactions and grandpa notifications substreams keep the same behaviour as before this PR: they follow the block announces slots. The one exception is that no Event::NotificationStreamOpened event is reported anymore if there isn't actually a substream open.

Similarly, the validity peers sets slots attribution is configured to be permanently in "reserved only" mode, and its list of reserved nodes is meant to follow the list of validators discovered by the authority-discovery.
The mechanism where the authority-discovery module maintains a priority group of 10 randomly-chosen validators has been removed.

The collation peers set slots attribution is configured to have 25 inbound slots. Its list of reserved nodes is meant to follow the list of validators assigned to a specific parachain. As explained above, the links established for collation are asymmetrical: the collators know which validators to be connected to, and uses reserved nodes for that purpose, but validators don't know who the collators are, hence the 25 inbound slots.

Please let me know whether these rules regarding the validity and collation peer sets make sense.

These five peer sets are how the generic_proto/behaviour.rs and generic_proto/handler.rs files function, plus everything in sc_peerset.
However, the API of sc_network exposes the concepts of "default set" and "extra sets".
The default sets refers to the block announces and transactions, which are handled internally, and the extra sets refers to the peers sets/notifications protocols registered by API users.
Two events have been added: SyncConnected and SyncDisconnected, to report to users slots being attributed on the default set. This is necessary in order to implement the grandpa reserved nodes tracking of the block announces slots.

The protocol.rs files acts as the translation layer, as it is the file that handles the block announces and transactions in its internals. The behaviour.rs and service.rs files are only aware of the concepts of default set and extra sets, and expose APIs where extra sets are referred to by their notifications protocol names.
The --reserved-nodes and --reserved-only CLI options only refer to the default set.

Concerns

Additional round-trip

A round-trip is added when it comes to opening transactions and grandpa substreams. Previously, all substreams would immediately open simultaneously. Now, the block announces slot is first attributed, and it is only once the block announces handshake has been performed that transaction and grandpa slots are attributed to the peer.
I don't think that this really has big consequences, as connections for block-announces/transactions/grandpa are mostly stable and are used only for gossiping. But I admit that I did this in order to make the implementation easier to write, and to not expose complicated APIs. If that turns out to be a problem, it should be possible to expose APIs that instruct the implementation to "group" multiple protocols together.

--reserved-only might be confusing

The --reserved-only flag now only acts on the default peers set (block announces, transactions, and, by extension, grandpa).
This means that validators and nodes configured for collating will still establish connections and substreams and gossip messages to non-reserved nodes.

This behaviour could cause some confusions. However, the alternative, which would be to apply --reserved-only to everything, would cause the networking to not work.

Might contain bugs

This is a big PR that refactors a lot of code, and it is likely that bugs have been introduced. It should be thoroughly tested before reviewing, especially considering that I'm been lenient on the reviewing of #7464.

tomaka · 2020-12-09T13:23:54Z

Testing is showing that, when connecting to Polkadot, we don't receive any notification on sets 1 and 2 (transactions and grandpa).

I believe that there is a compatibility problem with the fact that we only later add peers to these sets.
When the remote tries to open their transaction and grandpa substreams, they face a rejection from the local node. Later, when the local node attempts to connect, the remote still remembers this rejection and doesn't try to re-open its own substream.

I'm pushing a commit that changes the behaviour in order to test this hypothesis.
EDIT: latest commit (0e91d3d) indeed fixes the problem.

mxinden

I did a first pass over this pull request. I don't yet fully understand the bigger picture, thus some suggestions might be wrong due to misunderstandings.

client/peerset/tests/fuzz.rs

client/network/src/service.rs

client/peerset/src/lib.rs

mxinden · 2020-12-09T14:44:00Z

client/network-gossip/src/lib.rs

+	fn add_set_reserved(&self, who: PeerId, protocol: Cow<'static, str>) {
+		let addr = iter::once(multiaddr::Protocol::P2p(who.into()))
+			.collect::<multiaddr::Multiaddr>();
+		let result = NetworkService::add_to_peers_set(self, protocol, iter::once(addr).collect());


Suggested change

let result = NetworkService::add_to_peers_set(self, protocol, iter::once(addr).collect());

let result = NetworkService::add_peers_set_reservedself, protocol, iter::once(addr).collect());

Should the node not be added as a reserved node?

Indeed. I changed that here: #7700 (comment)
I'm a bit unsure whether to keep this.

mxinden · 2020-12-09T14:45:21Z

client/network-gossip/src/lib.rs

+	fn remove_set_reserved(&self, who: PeerId, protocol: Cow<'static, str>) {
+		let addr = iter::once(multiaddr::Protocol::P2p(who.into()))
+			.collect::<multiaddr::Multiaddr>();
+		let result = NetworkService::remove_from_peers_set(self, protocol, iter::once(addr).collect());


Same here, should this not call remove_peers_set_reserved?

Suggested change

let result = NetworkService::remove_from_peers_set(self, protocol, iter::once(addr).collect());

let result = NetworkService::remove_peers_set_reserved(self, protocol, iter::once(addr).collect());

client/network-gossip/src/bridge.rs

rphmeier · 2020-12-09T19:51:49Z

@tomaka

Thanks. The high-level description of before-and-after is really helpful to understand the context.

I have a few questions about the content, but it generally seems clear.

The collation peers set slots attribution is configured to have 25 inbound slots. Its list of reserved nodes is meant to follow the list of validators assigned to a specific parachain. As explained above, the links established for collation are asymmetrical: the collators know which validators to be connected to, and uses reserved nodes for that purpose, but validators don't know who the collators are, hence the 25 inbound slots.

If I understand correctly, in the previous implementation, inbound slots were cleared when block announcements weren't made on the slot. What is the automated criterion for clearing the inbound slots for the collator protocol now? What input does the higher-level code have on this criterion ?

These five peer sets are how the generic_proto/behaviour.rs and generic_proto/handler.rs files function, plus everything in sc_peerset.

Do you mean that the validation & collation peer-sets are actually defined in Substrate? That seems a little leaky, as it is a higher-level concern. Not something I would block this on, but worth knowing about. And if so, do you have any plan, however vague, to remove that from Substrate per-se?

The transactions and grandpa slots attribution is configured to be permanently in "reserved only" mode, meaning that only a specific list of nodes are allowed to have a slot. This list of nodes is constantly kept synchronized with the nodes that occupy a block announces slot.

Testing my understanding: transactions and grandpa slots are kept synced with the block announce slots by assigning the reserved peers to be equal to whoever happens to be in the block announce slots at any given moment?

The last questions I have are about number of slots & dormant connections as we move towards 1000 validators. Is the number of slots cli-configurable or at least in network configuration? What guidelines should we use to determine the number of slots we use? And is there any notion of peers we have 'dormant' connections to so we can speak with them on-demand but potentially very intermittently without doing a handshake?

burdges · 2020-12-10T05:22:12Z

I'm slightly confused by the "peer sets: block announces, transactions, grandpa, collation, validity" bit, but this sounds like progress. :)

At a high level, we have three connection categories: validator-validator, validator-collator, and validator-others (in future hierarchical relay chains shall complicate this, but they're way way off).

A priori, we'd want all validator-validator connections open fairly permanently, so if there are 1000 validators then they all open direct connections to one another. We should push low-level networking to support ala these connections, ala QUIC, node shutdown if ulimit set wrong, say validators cannot run on OSX or Windows, or anything else this requires. We can relax this slightly like num_validators/3 connections or so, but anything like 25 kills our scalability and soundness.

We've three or four usages for validator-collator connections: (a) collators informing validators about candidates aka collations, (b) collators fetching incoming XCMP messages from the sending parachain validators, (c) validators informing collators about relay chain's progress, and (d) validators providing parachains with censorship resistance.

(a) I'd suggest that collators open direct connections with their assigned validators when they have a collation ready. If (n-k+1)-of-n validators reject this connection attempt then that parachain shall not make progress, where availability requires k backers out of n parachain validators. This makes DoSing the parachain access to the relay chain easy, but that could be solved by adding flexibility in core usage.

(b) We've discussed many XCMP transports but only one pattern covers all cases: collators fetch incoming XCMP messages from the sending parachain validators, which requires short lived connections. We'll explore other patterns in future if we noticed bottlenecks or whatever.

(c) We need collators to track the relay chain so that they can build correct state references for the relay parent they select for the candidate they build. At present, collators should be full nodes of the relay chain, which dictates many other networking decisions. We've numerous unrealized optimizations here, ala contextual execution and XCMP state handling, which should enable more direct and optimized networking choice eventually, but not today.

(d) We do not provide parachains with censorship resistance currently because not all parachain designs benefit, so this does not fit within the MVP. We should support this eventually however because we'll need this for our own internal efforts. We'd grant collators with upcoming block production slots some preference for either fetching the block from an approval checker or doing a full reconstruction.

We need validator-others connections so that non-validators can become validators and relay chain transactions.

What does this all give us? We've (1) semi-permenent complete graph on validators, (2) collators opening ephemeral connections to validators, both theirs (a) and random others (b), (3) validators asisting non-validator full nodes with tracking the chain for (c) and other usages, and (4) the future work of validators ensuring parachain block availability for anyone besides validators (d).

We depend upon validator-validator connections to communicate approval checker assignments for parachain soundness, so any network level, DoS, etc. attack should impact validator-validator connections last.

If I understand, we're currently doing (2) not using direct connections but instead using the mechanisms designed for (3), largely out of fear of demanding more connections of validators. This if fine for now, but it should be reevaluated next year. If we ever do nQUIC with noise IK then then validators could prove themselves to other validators inside one UDP packet, which helps making (1) sacrosanct. Afaik, we cannot prove upcoming collator status in only one UDP packet. We thus create a DoS vector with (2) but ideally it should only DoS (2). I'm unsure about relative priorities of (2-4) but maybe (3) outranks (2) slightly.

eskimor · 2020-12-10T11:02:16Z

Sounds sensible to me. With regards to --reserved-only .. is it possible for the user to make the network work by specifying reserved nodes on the command line for example? In this case it could be better to make it do what it says and to just add a warning in the man page, that you need to add nodes to the reserved set to make this work. Having a way of enforcing a particular set of nodes and no others, e.g. for testing sounds useful to me.

In that case, if we need the less strict behavior as well, we could introduce another more descriptive option for this.

On the other hand, if there is no way ever, somebody would want that more strict behavior, it should be fine to change it and just explain the option in the man page. Problem is, if that is not the case, we will end up eventually with a --reserved-only-really option and that would be a pity.

tomaka · 2020-12-14T09:41:54Z

If I understand correctly, in the previous implementation, inbound slots were cleared when block announcements weren't made on the slot. What is the automated criterion for clearing the inbound slots for the collator protocol now? What input does the higher-level code have on this criterion ?

To clarify: we don't detect situations where no block announcement message is being sent. Inbound slots are only cleared in case of disconnect, or if the reputation of the remote drops too low.
Similarly, inbound slots for collation are cleared automatically only in case of disconnect, of if the reputation drops too low.

In terms of higher-level code, NetworkService::remove_from_peer_set can be called in order to disconnect from the node in the context of a specific set. This method has the effect that the PeerId is not longer considered as belonging to the given set, which means that the local node will also not attempt to connect back to this peer if any out slot was available.

Do you mean that the validation & collation peer-sets are actually defined in Substrate? That seems a little leaky, as it is a higher-level concern. Not something I would block this on, but worth knowing about. And if so, do you have any plan, however vague, to remove that from Substrate per-se?

No, see the Polkadot companion.

The API surface that sc-network exposes is: a "default peer set" (managed internally and used for syncing, block announces and transactions), and "extra peer sets" passed as part of the configuration and managed entirely by higher-level code. See this.

Testing my understanding: transactions and grandpa slots are kept synced with the block announce slots by assigning the reserved peers to be equal to whoever happens to be in the block announce slots at any given moment?

Yes. Transactions and grandpa slots are consequently in "reserved-only" mode.

tomaka · 2020-12-14T09:57:14Z

A priori, we'd want all validator-validator connections open fairly permanently, so if there are 1000 validators then they all open direct connections to one another. We should push low-level networking to support ala these connections, ala QUIC, node shutdown if ulimit set wrong, say validators cannot run on OSX or Windows, or anything else this requires. We can relax this slightly like num_validators/3 connections or so, but anything like 25 kills our scalability and soundness.

Yes, the idea is that Polkadot's code maintains the list of reserved node of the "validity" set to be equal to the list of on-chain validators (after being passed through the authority-discovery system). If there are 1000 on-chain validators, there will be 1000 entries in the reserved nodes, and the networking code will attempt to open a substream with all 1000.

I'd suggest that collators open direct connections with their assigned validators when they have a collation ready.

At the moment, collators try to maintain a direct connection with their assigned validators at all time, even if no collation is ready. This is done in the same way: fetching the list of assigned validators from the runtime, then adding them to a list of reserved nodes.

If I understand, we're currently doing (2) not using direct connections but instead using the mechanisms designed for (3), largely out of fear of demanding more connections of validators.

I don't really think that fear is justified. We're supposed to be rather easily able to maintain tens of thousands of TCP connections open.

We thus create a DoS vector with (2) but ideally it should only DoS (2). I'm unsure about relative priorities of (2-4) but maybe (3) outranks (2) slightly.

To me, the major risk of DoS attack is to block the TCP listening port of a validator by opening a large number of connections until the limit of the node (which right now has been arbitrarily fixed at 10,000 incoming connections) is reached.
However, the node being attacked should still be able to establish outbound connections. To prevent validator A and B from connecting to each other, you'd need to attack both A and B at the same time.

Another risk is that, since we don't prioritize the bandwidth of important connections such as validator-validator connections, an attacker can send enough traffic to use up all the node's available bandwidth, and slow down the node enough that timings are missed.

tomaka · 2020-12-14T10:00:19Z

is it possible for the user to make the network work by specifying reserved nodes on the command line for example?

Yes

In this case it could be better to make it do what it says and to just add a warning in the man page, that you need to add nodes to the reserved set to make this work. Having a way of enforcing a particular set of nodes and no others, e.g. for testing sounds useful to me.

The problem here is that collators need to be connected to validators, and validators need to be connected to other validators, for the Polkadot networking as a whole to work.
It would be really tedious to have to find out in advance who the hundreds of validators are, and this list changes dynamically after startup.

Excluding collator-validator and validator-validator connections, everything else respects --reserved-only.

burdges · 2020-12-14T16:51:32Z

However, the node being attacked should still be able to establish outbound connections. To prevent validator A and B from connecting to each other, you'd need to attack both A and B at the same time.

Interesting. There are DoS attacks at all levels, but if validator-validator connections staying live prevents most higher level DoS attacks then that's super handy. :)

Another risk is that, since we don't prioritize the bandwidth of important connections such as validator-validator connections, an attacker can send enough traffic to use up all the node's available bandwidth, and slow down the node enough that timings are missed.

We should "one day" think about prioritizing bandwidth too of course, especially so that validators could always send outgoing gossip messages.

Important question: Are routers elsewhere enforcing flow control and providing DoS protections for TCP? If so, this could provides one advantage for TCP over QUIC, assuming we use this correctly. We could consider QUIC among validators with TCP for validators-others too, although maybe even having UDP ports open creates risks. Or maybe even UDP packets get dropped elsewhere in ways that help prevent UDP DoS attacks? In that case, our problem could become "explaining" our QoS goals to the network elsewhere.

This reverts commit d175b92.

Co-authored-by: Max Inden <mail@max-inden.de>

…roups-take2

tomaka · 2021-01-06T15:57:02Z

I have finally made the CI pass. The Substrate integration test was failing because of this bug, while the Polkadot integration test was failing because I intentionally modified the behaviour (collators no longer in the validation peers set) but it turned out that this change isn't correct.

Will finish addressing review concerns.

A burn-in has been started here: paritytech/polkadot#2098

This reverts commit 62c4ad5.

…roups-take2

tomaka · 2021-01-07T10:50:02Z

Testing is showing that, when connecting to Polkadot, we don't receive any notification on sets 1 and 2 (transactions and grandpa).

I believe that there is a compatibility problem with the fact that we only later add peers to these sets.
When the remote tries to open their transaction and grandpa substreams, they face a rejection from the local node. Later, when the local node attempts to connect, the remote still remembers this rejection and doesn't try to re-open its own substream.

I'm pushing a commit that changes the behaviour in order to test this hypothesis.
EDIT: latest commit (0e91d3d) indeed fixes the problem.

To continue on this, this "hack" is necessary to maintain backwards compatibility with nodes that don't have #7464, which hasn't been released yet.

Rather than going through complicated means to maintain backwards compatibility in a clean way, I'd suggest keeping this "hack" in until the release of 0.7.27 (which is expected to happen tomorrow) then remove it.
If agreed, I'll open an issue.

tomaka · 2021-01-07T12:29:43Z

Burn-in shows that the node seems to work correctly.
In particular, the number of sync peers is higher than before.

Bandwidth has been reduced. Variations in the bandwidth seems to sometimes happen when deploying versions of PRs, and I've never been able to figure out why.

tomaka · 2021-01-07T13:52:35Z

bot merge

ghost · 2021-01-07T13:52:38Z

Trying merge.

* Rework priority groups * Broken tests fix * Fix warning causing CI to fail * [Hack] Try restore backwards-compatibility * Fix peerset bug * Doc fixes and clean up * Error on state mismatch * Try debug CI * CI debugging * [CI debug] Can I please see this line * Revert "[CI debug] Can I please see this line" This reverts commit 4b7cf7c. * Revert "CI debugging" This reverts commit 9011f1f. * Fix error! which isn't actually an error * Fix Ok() returned when actually Err() * Tweaks and fixes * Fix build * Peerset bugfix * [Debug] Try outbound GrandPa slots * Another bugfix * Revert "[Debug] Try outbound GrandPa slots" This reverts commit d175b92. * [Debug] Try outbound GrandPa slots * Apply suggestions from code review Co-authored-by: Max Inden <mail@max-inden.de> * Use consts for hardcoded peersets * Revert "Try debug CI" This reverts commit 62c4ad5. * Renames * Line widths * Add doc Co-authored-by: Max Inden <mail@max-inden.de>

* Features needed for reserve-backed stablecoins (#7152) * Features needed for reserve-backed stablecoins * Builds & tests. * Double map for an efficient destroy. * Update frame/assets/src/lib.rs Co-authored-by: Nikolay Volf <nikvolf@gmail.com> * ED/zombie-count/refs Feature: ED/minimum balance enforcement Feature: enforce zombie count Feature: allow system-alive accounts to exist, but add reference * Update frame/assets/src/lib.rs Co-authored-by: Nikolay Volf <nikvolf@gmail.com> * Update frame/assets/Cargo.toml Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> * Docs * Some tests * More tests * Allow for max_zombies to be adjusted * Test for set_max_zombies * Tests and a couple of fixes * First few benchmarks * Benchmarks. * Fix error message in test * Fixes * Fixes * Fixes * cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_assets * Update frame/assets/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Fixes * Fixes * Fixes * cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_assets * Fixes * Update default weight * Add proper verification to benchmarks * minor improvements to tests * Update frame/assets/src/benchmarking.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Fix * New weights system * fix compile * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs Co-authored-by: Nikolay Volf <nikvolf@gmail.com> Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Add Standard Error to Weight Template (#7652) * Add standard error to weight template * fix test compile * Adds support for tuples in runtime-interface (#7672) * Fix two potential connection poisonings (#7677) * Fix #7629 (#7680) * Fix #7629 * Update client/network/src/service.rs Co-authored-by: Max Inden <mail@max-inden.de> Co-authored-by: Max Inden <mail@max-inden.de> * Update Cargo.toml (#7627) On behalf of dependabot, because we use 1.4.x in it clashes with this. * Make sure the companion CI test fails on any error (#7683) * implement more convertion on NumberOrHex (#7682) * Streamline frame_system weight parametrization (#6629) * Basic weights builder. * Fixing WiP * Make the tests work. * Fix weights in node/runtime. * WiP. * Update pallets with new weights parameters. * Validate returns a Result now. * Count mandatory weight separately. * DRY * BREAKING: Updating state root, because of the left-over weight-tracking stuff * Update tests affected by Mandatory tracking. * Fixing tests. * Fix defaults for simple_max * Update frame/system/src/weights.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Rework the API a bit. * Fix compilation & tests. * Apply suggestions from code review Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Add extra docs & rename few things. * Fix whitespace in ASCII art. * Update frame/system/src/limits.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Fix max_extrinsic calculations. * Fix conflicts. * Fix compilation. * Fix new code. * re-remove generic asset * Fix usage. * Update state root. * Update proxy. * Fix tests. * Move weights validity to integrity_test * Remove redundant BlockWeights. * Add all/non_mandatory comment * Add test. * Remove fn block_weights * Make the macro prettier. * Fix some docs. * Make max_total behave more predictabily. * Add BlockWeights to metadata. * fix balances test * Fix utility test. Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org> Co-authored-by: thiolliere <gui.thiolliere@gmail.com> * client: remove duplicate implementation of block_status (#7694) * Pin md link checker's version (#7697) * CI: pin md link checker's version to the previous release * CI: add a whitelist for allowed GH actions pins * CI: try master tip [skip ci] * CI: return to a previous markdown-link-check dependency version [skip ci] * CI: substitute %20 for a literal space [skip ci] * CI: substitute %20 for a literal space * Remove waterfall CI trigger (#7519) * fix (CI): fix the triggered project name * fix (CI): debug run * fix (CI): debug run 2 * fix (CI): revert debug2 * Revert "fix (CI): debug run" This reverts commit 96866a953da797ef6e0689374eecc0d8cba4627d. * CI: remove the trigger * add an upgrade_keys method for pallet-session (#7688) * add an upgrade_keys method for pallet-session * test the upgrade_keys function * Allow capping the amount of work performed when deleting a child trie (#7671) * Allow Backend::for_keys_in_child_storage to be aborted by the closure * Ext::kill_child_storage now takes an upper limit for backend deletion * Add Storage::storage_kill_limited() runtime interface * review: Use a new version of kill_storage instead of a new interface * review: Simplify boolean expression Co-authored-by: cheme <emericchevalier.pro@gmail.com> * review: Rename for_keys_in_child_storage Co-authored-by: cheme <emericchevalier.pro@gmail.com> * benchmarks: Fix panic in case of a missing model (#7698) Co-authored-by: Alexander Theißen <alex.theissen@me.com> * Expand remote keystore interface to allow for hybrid mode (#7628) * update to latest master * updates on docs, license, meta * hide ssrs behind feature flag * implement remaining functions on the server * sign server line length fix * fix tests * fixup in-memory-keystore * adding failsafe * skipping ecdsa test for now * remote keystore param * remote sign urls made available * integrating keystore remotes features * don't forget the dependency * remove old cruft * reset local keystore * applying suggestions * Switch to single remote, minor grumbles * minor grumbles, docs * fork-tree: expose silent failures of is_descendent_of (#7695) * fork-tree: expose silent failures of is_descendent_of * fork-tree: use mem::take instead of mem::swap * Merkle Mountain Range pallet (#7312) * Add MMR pallet. * WiP * Working on testing. * WiP - test * Tests passing. * Add proof generation. * Generate and verify proofs. * Allow verification of older proofs. * Move stuff to a module. * Split MMR stuff to it's own module. * Add docs. * Make parent hash optional. * LeafData failed approach. * Finally implement Compact stuff. * Compact encoding WiP * Implement remaining pieces. * Fix tests * Add docs to compact. * Implement for tuples. * Fix documentation. * Fix warnings and address review suggestion. * Update frame/merkle-mountain-range/src/primitives.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> * Address review grumbles. * Removing missing crate. * Fix test. * Add some docs and test. * Add multiple instances. * Cargo.toml sync. * Fix no_std compilation. * More no_std stuff. * Rename MMR struct. * Addressing other grumbles. * Fix test. * Remove format for no_std compat. * Add test for MMR pallet. * Fix std feature. * Update versions. * Add to node/runtime. * Add hook to insert digest. * Make primitives public. * Update lib.rs tech spec/typos etc * Use WeightInfo and benchmarks. * Fix test. * Fix benchmarks. * Trait -> Config. * Fix typo. * Fix tests. Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Addie Wagenknecht <addie@nortd.com> * frame/staking/README: Fix broken link (#7703) * *: Update to libp2p v0.32.0 (#7696) * *: Update to libp2p v0.32.0 * Cargo.lock: Update async-tls to 0.10.2 * client/network/request_response: Adjust to new request response events * client/network/request_response.rs: Clean up silently failing responses * client/network/discovery: Lazily instantiate mdns * client/network/discovery: Exclude MdnsWrapper for target_os unknown * client/network/discovery: Fix indentation * client/network/request-response: Use LruCache to track pending resp time * client/network/request_responses: Fix early connection closed error * client/network/request-response: Replace debug_assert with debug * client/network/request-response: Fix typo * client/network/request-response: Don't emit event on send_response fail * client/network/request-response: Revert waker.wake_by_ref() * client/network/request-resp: Make duration in InboundRequest optional * client/network/req-resp: Don't emit two events for busy builder When a response builder is busy incoming requests are dropped. Previously this was reported both via a `ResponseFailure::Busy` and a `ReponseFailure::Network(InboundFailure::Omisssion)` event. With this commit the former is removed, leaving only the latter in place. * *: Update to libp2p v0.32.2 (#7708) Version v0.32.2 pins async-tls to the semver compliant v0.11.0 instead of the yanked async-tls v0.10.2. * kilt&polimec ss58 prefix (#7547) * kilt ss58 address * polimec ss58 prefix * Fix docs (#7710) * fix docs * Update frame/merkle-mountain-range/src/lib.rs Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: Alexander Theißen <alex.theissen@me.com> * Update md link checker (#7709) * CI: ignore %20 for md link checker * CI: update to a new action release that fixes %20 issue * Fix PerThing::from_percent. (#7701) * Fix overflow in per_things::from_percent. * Fix test * Fix the whole thing.. :| * Support Multiple Instances with Benchmarks (#7669) * Support multiple instances with benchmarks * fix tests * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * docs * fix output * Update lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Add keccak-512 to host functions. (#7531) * Update debug.rs (#7721) * Avoid too high values. (#7716) * Bump hyper from 0.12.35 to 0.13.9 (#7569) * Bump hyper from 0.12.35 to 0.13.9 Bumps [hyper](https://github.com/hyperium/hyper) from 0.12.35 to 0.13.9. - [Release notes](https://github.com/hyperium/hyper/releases) - [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md) - [Commits](https://github.com/hyperium/hyper/compare/v0.12.35...v0.13.9) Signed-off-by: dependabot[bot] <support@github.com> * Update bin/node/rpc-client/Cargo.toml * Update Cargo.lock Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Prevent dust in staking by disallowing cheap bond_extra (#7718) * prevent bond_extra to cause staking actve lower than ed * prevent bond_extra to cause staking actve lower than ed * Check in post conditions. * check rebond as well. * also change withdraw_unbonded. * Fix build * change check format. * Apply suggestions from code review Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Default block announce validation should reject additional data (#7726) If we are using the default block announce validator and receive additional data alongside the block announcement, we should disconnect this peer. * Fix wrong order of values in log line (#7728) * Fix polkadot companion test (#7732) * remove duplicate accounts in chain-spec. (#7725) * remove duplicate accounts in chain-spec. * Fix build * Enforce in balances modules. * Apply suggestions from code review * Update frame/balances/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> * Participating in Council Governance is Free for First Time Voters and Successful Closing (#7661) * wk2049 | D5 | addition-motions first free vote for members | p1 * Update frame/collective/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * wk2049 | D6 | addition-motions first free vote for members | p2 * Update frame/collective/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Update frame/collective/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * wk2049 | D7 | addition-motions first free vote for members | p3 * Update frame/collective/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * wk2049 | D7 | addition-motions first free vote for members | p4 * wk2049 | D7 | addition-motions first free vote for members | p6 * Update frame/collective/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/collective/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/collective/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/collective/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/collective/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/collective/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/collective/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * wk2050 | D3 | addition-motions first free vote for members | p7 * wk2050 | D3 | addition-motions first free vote for members | p8 * update comment Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Bump zeroize from 1.1.1 to 1.2.0 (#7731) Bumps [zeroize](https://github.com/iqlusioninc/crates) from 1.1.1 to 1.2.0. - [Release notes](https://github.com/iqlusioninc/crates/releases) - [Commits](https://github.com/iqlusioninc/crates/compare/zeroize/v1.1.1...zeroize/v1.2.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Leave some header related info for inspection (#7727) * Leave some system data for inspection There is not much benefit in being active when removing this data. It's actively harmful when one tries to read the block number in runtime APIs in the context of a block. * Update the expected root hash This is excepted since now we persist new members. * Revert extrinsics_root to `take` It's going away in one of the following PRs anyway * Update the state root once again * Update the comment on the storage items that are left in the storage Excluding ExtrinsicsRoot since it's going away * Issue 7143 | Refactor Treasury Pallet into Bounties, Tips, and Proposals (#7536) * wk2046 | D5 | treasury refactor to bounties and tips * wk2046 | D5 | treasury refactor to bounties and tips | p2 * fix test compilation, ignoring events * initialize treasury in genesis * wk2046 | D7 | treasury refactor | fix bounties test build issues * wk2047 | D1 | treasury refactor | tips pallet bringup * wk2047 | D2 | treasury refactor | bounties pallet | unit test bringup * wk2047 | D2 | treasury refactor | bounties pallet | unit test bringup | p2 * wk2047 | D2 | treasury refactor | pallet-tips| test_last_reward_migration | test failure - analysis * wk2047 | D3 | treasury refactor | pallet-tips| test_last_reward_migration | test failure - fix * wk2047 | D3 | treasury refactor | pallet-bounties | on_initialize() fix * wk2047 | D3 | treasury refactor | pallet-bounties | on_initialize() fix | p2 * wk2047 | D4 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks * wk2047 | D4 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks | p2 * wk2047 | D4 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks | p3 * wk2047 | D5 | treasury refactor | pallet-bounties + pallet-treasury | spend_fund runtime hooks | p4 * wk2047 | D6 | treasury refactor | review comments fix * some fixes * fix bounties instantiable * remove instantiable from tips and bounties * fix compile for benchmarks * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_treasury --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/treasury/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Update lib.rs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_tips --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/tips/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bounties --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bounties/src/weights.rs --template=./.maintain/frame-weight-template.hbs * add back `on_initialize_bounties` * patch up bounties benchmarks * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_bounties --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/bounties/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Remove development TODO from public doc comment (#7500) * refactor subtrait/elevated trait as not needed (#7497) * Fix comments of indices pallet events (#7511) Arguments for IndexAssigned and IndexFrozen were inverted in comments. * Skip slot lenience on first block in BABE (#7515) The genesis header doesn't have the BABE pre-digest and we insert `0` as slot number. The slot lenience calculation will return the maximum in this situation. Besides returning the maximum which is not bad at all, it also prints some a debug message that can be confusing in the first moment. To prevent printing this debug message, we now just return early when we see that the parent block is the genesis block. * slots: incrementally backoff claiming slots if finality lags behind (#7186) * babe: backoff authoring blocks when finality lags * babe: move backoff authoring params to default constructor * babe: deduplicate the test a bit * babe: set backoff constants in service * babe: use better names for backoff authoring block parameters * babe: remove last unwrap * babe: slight style tweak * babe: fix comment * slots: move backoff block authorship logic to SimpleSlotWorker * aura: append SlotInfo in on_slot * slots: use the correct types for parameters * slots: fix review comments * aura: add missing backoff authoring blocks parameters * slots: add comments for default values * slots: add additional checks in test * slots: update implementation for new master * slots: revert the change to SlotInfo * Fix review comments * slots: rework unit tests for backing off claiming slots * slots: add test for asymptotic behaviour for slot claims * slots: address review comments * slots: add test for max_interval * slots: add assertion for intervals between between claimed slots * slots: remove rustfmt directive * slots: another attempt at explaining authoring_rate * slots: up unfinalized_slack to 50 by default * slots: add tests for time to reach max_interval * slots: fix typo in comments * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * slots: additional tweaks to comments and info calls * slots: rename to BackoffAuthoringOnFinalizedHeadLagging * slots: make the backing off strategy generic * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * slots: implement backoff trait for () for simplicity * slots: move logging inside backing off function to make it more specific * aura: add missing function parameter Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Export app-crypto specific keystore functions (#7489) * Export app-crypto specific keystore functions * Also add back the insert function * Switch KeystoreContainer to an enum * Only export the bare minimal for LocalKeystore and fix service compile * fix: should return Arc * Add docs stating that functions only available in local keystore * Remove insert and generate functions * fix: generate function should be available in test * Add keypair function to trait * Revert "Add keypair function to trait" This reverts commit ad921b09ca73d3c09298e3a51b562ef8e0067781. * Add note for local_keystore function in service * Update doc for the --chain flag (#7520) * contracts: Add missing instruction to the `Schedule` (#7527) * Don't log with colors when we are writing to a tty (#7525) * Don't log with colors when we are writing to a tty This fixes a regression that was introduced by the switch to tracing. Before we killed all colors before writing to a tty, this pr brings the behaviour back. * Remove accidentally added crate * Review feedback * More feedback * Update client/cli/src/logging.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update client/cli/src/logging.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * MemoryId -> u32 (#7534) * Enable local addresses in DHT when chain type == `Local` | `Development` (#7538) * Enable local addresses in DHT when chain type == `Local` | `Development` This pr changes when to add local addresses to DHT. Instead of only checking if `--discover-local` and `--dev` are present, we now also check if the chain type is `Local` or `Development`. * Update the docs! * Update tiny-bip39 to v0.8 (#7539) It would improve secret zeroization due to https://github.com/maciejhirsz/tiny-bip39/pull/22, and would also remove one of the points where we depend on `failure` crate, which is deprecated (see https://github.com/rust-lang-nursery/failure/pull/347) * make LocalCallExecutor public (#7528) * Fix some weirdness in `offchain_worker` (#7541) We call `offchain_worker` with the state of the imported block and pass the header of this block. However in the runtime we call all `offchain_worker` functions with the number of the parent block. Besides that we also pass all digests and not only the pre runtime digests. In the context where the offchain worker is executed we have all digests, so there is no real reason to only pass pre runtime digests. Another fix is that we also insert the hash of the current header into the block hash map. * Use inbound peerslot slots when a substream is received, rather than a connection (#7464) * Use inbound peerslot slots when a substream is received, rather than a connection * Refactor PeerState * Some bugfixes * Fix warnings so that CI runs, gmlrlblbl * Bugfixes * Update docs * Apply suggestions from code review Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> * Clean up Banned state * Refactor connections state * Fix possibility of Enabled with no Opening or Open connection * Line width * Add some debug_asserts! and fix TODO * Refactor legacy handler * Rewrite group.rs entirely [part 1] * Rewrite group.rs entirely [part 2] * Remove faulty assertion Because of the asynchronous nature of the behaviour <-> handler communications, it is possible to receive notifications while in the Closing state * Don't poll the legacy substream is not Open * Tolerate when not all substreams are accepted * Remove TODOs * Dummy commit to make CI log interesting things * Try race condition fix * Revert "Try race condition fix" This reverts commit 0675c659d06195c30f8c5bc13e2d88141d57a3ba. * Correctly rebuild pending_opening * Minor tweaks * Printlns for CI debugging * Revert "Printlns for CI debugging" This reverts commit e7852a231f4fc418898767aaa27c9a4358e12e8b. * Revert "Dummy commit to make CI log interesting things" This reverts commit 259ddd74088e53e7c6a9b0a62a8d1573a0063ce3. * mv group.rs ../handler.rs * Apply suggestions from code review Co-authored-by: Max Inden <mail@max-inden.de> * Banned => Backoff * Mention the actual PeerStates * OpenDesired -> OpenDesiredByRemote * OpeningThenClosing * Add doc links to PeerState * Simplify increment logic * One more debug_assert * debug_assert! * OpenDesiredByRemote * Update client/network/src/protocol/generic_proto/behaviour.rs Co-authored-by: Max Inden <mail@max-inden.de> Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> Co-authored-by: Max Inden <mail@max-inden.de> * *: Update to libp2p v0.30.0 (#7508) * *: Update to libp2p v0.30.0 * Cargo.lock: Update * *: Update to libp2p v0.30.1 * make ClientConfig public (#7544) * sc-basic-authorship: remove useless dependencies (#7550) Signed-off-by: koushiro <koushiro.cqx@gmail.com> * Add slashing events to elections-phragmen. (#7543) * Add slashing events to elections-phragmen. * Fix build * Apply suggestions from code review * Update frame/elections-phragmen/src/lib.rs * Update frame/elections-phragmen/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Remove necessity to pass ConsensusEngineId when registering notifications protocol (#7549) * Remove necessity to pass ConsensusEngineId when registering notifications protocol * Line width * Fix tests protocol name * Other renames * Doc update * Change issue in TODO * sc-cli: replace bip39 with tiny-bip39 (#7551) Signed-off-by: koushiro <koushiro.cqx@gmail.com> * Add extra docs to on_initialize (#7552) * Add some extra on_initialize docs. * Address review comments. * More Extensible Multiaddress Format (#7380) * More extensible multiaddress format * update name * Don't depend on indices to define multiaddress type * Use MultiAddress in Node Template too! * reduce traits, fix build * support multiple `StaticLookup` * bump tx version * feedback * Fix weight template to remove ugliness in rust doc (#7565) fixed weight template * Cargo.lock: Run cargo update (#7553) * Cargo.lock: Run cargo update * Cargo.lock: Downgrade cc to v1.0.62 * Cargo.lock: Revert wasm-* updates * .github: Add dependabot config and thus enable dependabot (#7509) * .github: Add dependabot config and thus enable dependabot * Update .github/dependabot.yml Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Thread-local parameter_types for testing. (#7542) * Thread-local parameter_types for testing. * Better docs. * Some minors * Merge'em * Update frame/support/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Align more to basti's trick * Update frame/support/src/lib.rs * Update frame/support/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> * Bump wasm-bindgen-test from 0.3.12 to 0.3.17 (#7567) * Bump wasm-bindgen-test from 0.3.12 to 0.3.17 Bumps [wasm-bindgen-test](https://github.com/rustwasm/wasm-bindgen) from 0.3.12 to 0.3.17. - [Release notes](https://github.com/rustwasm/wasm-bindgen/releases) - [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/master/CHANGELOG.md) - [Commits](https://github.com/rustwasm/wasm-bindgen/commits) Signed-off-by: dependabot[bot] <support@github.com> * Update wasm-bindgen pin to 0.2.68 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * wk2047 | D6 | treasury refactor | review comments fix | p2 * wk2048 | D1 | treasury refactor | review comments fix | p3 * Update bin/node/runtime/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Update bin/node/runtime/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * wk2048 | D3 | treasury refactor | review comments fix | p4 * wk2048 | D3 | treasury refactor | review comments fix | p5 * wk2048 | D4 | treasury refactor | review comments fix | removal of deadcode | p6 * remove broken link * wk2048 | D5 | treasury refactor | review comments fix | bountise doc string | p7 * wk2048 | D5 | treasury refactor | review comments fix | p8 * docs and formatting * Update frame/tips/src/benchmarking.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * formatting nits * Trait -> Config * trait -> config in benchmarks * clean up weight docs * Trait -> Config in Runtime * fix test build * try to fix polkadot build check * fix traits * Update lib.rs * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * fix trait location * nits * uncomment on_initialize for bounties benchmarks * update weights Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Caio <c410.f3r@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Antoine Le Calvez <alecalve@users.noreply.github.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com> Co-authored-by: Wei Tang <wei@that.world> Co-authored-by: Sergei Shulepov <sergei@parity.io> Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> Co-authored-by: Kirill Pimenov <kirill@parity.io> Co-authored-by: Andrew Plaza <aplaza@liquidthink.net> Co-authored-by: Roman Borschel <romanb@users.noreply.github.com> Co-authored-by: Max Inden <mail@max-inden.de> Co-authored-by: Qinxuan Chen <koushiro.cqx@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Addie Wagenknecht <addie@nortd.com> * CI: buildah to build images (#7741) * CI: buildah to build images * CI: simplify variables * CI: whitespaces * CI: secure login * optimize arithmetic cargo path (#7745) * optimize runtime-interface cargo path (#7729) * delete primitives in Cargo in primitives (#7751) * delete client in Cargo in client (#7752) * prevent too many genesis council members (#7749) * Prevent too many genesis members in elections-phragmen. * Fix test. * reformat.' * Update common block in sync after importing blocks of a peer, please read UPDATE (#7733) * Update common block in sync after importing blocks of a peer This updates the sync code to update the common block of a peer, after we have imported blocks from this peer. This fixes a bug for when we are connected to one or more nodes that are doing a full sync as our node. Nodes in full sync will not announce new blocks, as we don't send import notifications on full sync. The problem as now that we were connected to some peer that reported some low number as its best and we tried to sync these blocks. But, as we did not update the common block of this peer, we would sync these blocks over and over again. Being captured in some time warp. The solution to this problem is that we increase the common number as we import blocks from this peer. * Test * Test name.. * Fix test * Cleanup some code and write some new regression test * Implement the ancestor search * Check that the common number is smaller than the last finalized block * Update client/network/src/protocol/sync.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update client/network/src/protocol/sync.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update client/network/src/protocol/sync.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Change the way we build the status messages * Start some new test... * Finish test * Rename test * Update client/network/src/protocol.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Remove unnecessary parameter from sync (#7761) * Improve `InvalidTransaction::AncientBirthBlock` doc comment (#7743) * Improve `InvalidTransaction::AncientBirthBlock` doc comment Update Remove stray parentheses Tabs not spaces Improve whoops * Update primitives/runtime/src/transaction_validity.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Possible causes section; Expand example Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Switch back to the crates.io release of trybuild (#7764) * Switch back to the crates.io release of trybuild My fix was merged on upstream and this release contains it. So, no more reason to keep the git dependency. * The lock file... * No longer put keys back manually after a failed restoration (#7747) This is no longer necessary with storage transactions. * Fix UI Tests (#7722) * Fix UI tests * make updates. * Revert "make updates." This reverts commit 3f41979c12e86acb75fd086675ac09659dde533e. * Try to fix companion build * Use some depth * tests: fix UI tests on the latest nightly * tests: fix polkadot companion test, depth * chore: update deps * chore: update deps * chore: sp-io Co-authored-by: kianenigma <kian.peymani@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Denis P <denis.pisarev@parity.io> * update substrate to sc in docs (#7767) * Allow Staking tests to run with session length other than 1 (#7719) * fix periodic session * Allow staking tests to run with session lengths other than 1. * Update frame/staking/src/mock.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Fix all tests with session length 5. * Test for active != current * Better doc * Update frame/staking/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * also set the timestamp properly. * trigger CI * Revert "trigger CI" This reverts commit 0f254944cdad848aa6e63bd8a618db95447a8e68. * Update frame/staking/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * optimize biguint div closure (#7754) * optimize biguint div closure * optimize biguint sub and fix note * change and add biguint split test * add biguint div_unit test * update biguint sub v to v1 * add biguint shift_check * Make it possible to calculate the storage root as often as you want (#7714) * Make it possible to calculate the storage as often as you want So, until now each Substrate based blockchain has calculated the storage root once, at the end of the block. Now there is Frontier that wants to calculate some intermediate storage root. However this failed on block import. The problem with that was the extrinsics root. When building the block we stored `Default::default()` as extrinsics root, because yeah, we don't know the extrinsics root before finishing the block. At the end this extrinsics root was then calculated. But on block import we passed the already known extrinsics root. This was no problem, as we removed this value at the end of the block. However when you all the storage root in between, that changes the storage root between block building and block import. This pr changes this behavior. It removes the `ExtrinsicsRoot` storage entry and also doesn't pass it anymore to `System::initialize`. By doing it, we remove the difference in the storage and fix the storage root mismatch. * Fix bug with incorrectly calculating the extrinscs root * Review feedback * Typo fix: eror => error (#7773) * Adds `try_get` for `StorageMap` and `StorageDoubleMap` (#7774) * Adds `try_get` for `StorageMap` and `StorageDoubleMap` * Switch to Value as return type * Add account prefix for Valiu (#7776) * Feat support weight test (#7769) * update support weights_are_correct * add support polynomial_does_not_underflow * Add `pallet` attribute macro to declare pallets (#6877) * rename system Config to system Trait. command used: ``` find frame/ bin/ test-utils/ utils/ -name *.rs -exec sed -i 's/system::Trait>::/system::Config>::/g' {} \; find frame/ bin/ test-utils/ utils/ -name *.rs -exec sed -i 's/impl frame_system::Trait for /impl frame_system::Config for /g' {} \; find frame/ bin/ test-utils/ utils/ -name *.rs -exec sed -i 's/impl system::Trait for /impl system::Config for /g' {} \; ``` plus some manual ones especially for frame-support tests and frame-system * make construct_runtime handle Pallet and Module pallets can now be implemented on struct named Pallet or Module, both definition are valid. This is because next macro will generate only Pallet placeholder. * introduce pallet attribute macro currently just with tests, frame_system and other example hasn't been upgraded * allow to print some upgrade helper from decl_storage * Improved error msg, typo. Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Improved error msg, typo. Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Improved error message on unexpected attributes + ui test * add test for transactional * various typo * some tips when spans are lost * allow pallet to depend on other pallet instances * make event type metadata consistent with call and constant * error messages * ignore doc example * fix pallet upgrade template * fixup * fix doc * fix indentation * Apply suggestions code formatting Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * some renames + fix compilation * remove unsupported genesis config type alias * merge fixup * fix ui tests * additional doc * implement StorageInstance with new syntax * fix line width * fix doc: because pallet doc goes below reexport doc * Update frame/support/procedural/src/pallet/parse/event.rs Co-authored-by: Andrew Jones <ascjones@gmail.com> * Update frame/system/src/lib.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Update frame/support/test/tests/pallet_ui.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * improve doc as suggested * revert construct_runtime Pallet part. This revert the changes on construct_runtime. Now construct_runtime is unchanged and instead pallet macro create a type alias `type Module<..> = Pallet<..>` to be used by construct_runtime * refactor with less intricated code * fix ui test with new image * fix ui tests * add minor tests Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Andrew Jones <ascjones@gmail.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * add a `current_epoch` to BabeApi (#7789) * add a `current_epoch` to BabeApi * add current_epoch_start back again * fix node-runtime * bump spec version * Update tests.rs (#7784) Fix testing for number of slashing spans. * add a slot-duration getter to babe config (#7793) * Fix base-path handling in `key insert` (#7775) This fixes the handling of base-path when using `key insert`. Before the base-path wasn't setup correctly, as done when starting a node. This resulted in putting the keys into the wrong directory. This pr fixes this by creating the correct base-path/config dir for the keystore. Besides that it also removes the insert command from `subkey` as it doesn't make that much sense. If requested, we could bring it back later. * Lazily evaluate error string (#7802) * contracts: Change `define_env!` to expect a `Result<T, DispatchError>` for every function (#7762) * Make host functions return TrapReason This avoids the need to manually store any trap reasons to the `Runtime` from the host function. This adds the following benefits: * It properly composes with the upcoming chain extensions * Missing to set a trap value is now a compile error * review: Remove superflous .into() * Hex Balance deserialize for contracts_call RPC (#7807) * Hex Balance deserialize for contracts_call RPC * Avoid temporary conversion into u128 Co-authored-by: Alexander Theißen <alex.theissen@me.com> * Don't allow self proxies (#7803) * Allow council to slash treasury tip (#7753) * wk2051 | D4 |Allow council to slash treasury tip | p1 * Update frame/tips/src/lib.rs Co-authored-by: Xiliang Chen <xlchen1291@gmail.com> * wk2051 | D5 |Allow council to slash treasury tip | p2 * wk2051 | D5 |Allow council to slash treasury tip | p3 * wk2051 | D5 |Allow council to slash treasury tip | p4 * wk2051 | D5 |Allow council to slash treasury tip | p5 * random change * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_tips --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/tips/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix typo * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/tests.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * wk2052 | D1 | Allow council to slash treasury tip | p6 Co-authored-by: Xiliang Chen <xlchen1291@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Add CLI argument to disable log color output (#7795) * Use path instead of ident (#7809) * Add proper `commit_all` to `TestExternalities` (#7808) * Add proper `commit_all` to `TestExternalities` This pr adds a propoer `commit_all` function to `TestExternalities` to commit all changes from the overlay to the internal backend. Besides that it fixes some bugs with handling empty dbs when calculating a delta storage root. It also changes the way data is added to the in memory backend. * Update primitives/state-machine/src/testing.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> * Don't allow self proxies (#7803) * Allow council to slash treasury tip (#7753) * wk2051 | D4 |Allow council to slash treasury tip | p1 * Update frame/tips/src/lib.rs Co-authored-by: Xiliang Chen <xlchen1291@gmail.com> * wk2051 | D5 |Allow council to slash treasury tip | p2 * wk2051 | D5 |Allow council to slash treasury tip | p3 * wk2051 | D5 |Allow council to slash treasury tip | p4 * wk2051 | D5 |Allow council to slash treasury tip | p5 * random change * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_tips --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/tips/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix typo * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/tips/src/tests.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * wk2052 | D1 | Allow council to slash treasury tip | p6 Co-authored-by: Xiliang Chen <xlchen1291@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Review feedback * Review feedback * Update docs * More docs * Make it private * Use `None` * Use apply transaction * Update primitives/state-machine/src/testing.rs Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: RK <r.raajey@gmail.com> Co-authored-by: Xiliang Chen <xlchen1291@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Define ss58 prefix inside the runtime (#7810) * Add SS58Prefix type to the frame_system config trait * Remove unused chain_id runtime interface * remove some unecessary bound (#7813) * Fix ss58check test when executed with other tests (#7815) There was a bug that could make other ss58 tests fail when being executed with this one in parallel. This test changes the default ss58 version and if other tests are run at the time the default version is changed, they would fail. To fix this problem, we now run the actual test as a new process. * Cleanup some warnings (#7816) * client: cleanup redundant semicolon warnings * grandpa: remove usage of deprecated compare_and_swap * Happy new year (#7814) * Happy new year Updates the copyright years and fixes wrong license headers. * Fix the template * Split HEADER into HEADER-APACHE & HEADER-GPL * contracts: Allow runtime authors to define a chain extension (#7548) * Make host functions return TrapReason This avoids the need to manually store any trap reasons to the `Runtime` from the host function. This adds the following benefits: * It properly composes with the upcoming chain extensions * Missing to set a trap value is now a compile error * Add chain extension The chain extension is a way for the contract author to add new host functions for contracts to call. * Add tests for chain extensions * Fix regression in set_rent.wat fixture Not all offsets where properly updated when changing the fixtures for the new salt on instantiate. * Pre-charge a weight amount based off the specified length * Improve fn write docs * Renamed state to phantom * Fix typo * *: Update to libp2p v0.33.0 (#7759) * *: Update to libp2p v0.33.0 * client/network: Consistently track request arrival time With https://github.com/libp2p/rust-libp2p/pull/1886/ one is guaranteed to receive either a `ResponseSent` or a `InboundFailure` event for each received inbound request via `RequestResponseEvent::Message`. Given this guarantee there is no need to track arrival times in a best-effort manner and thus there is no need to use a LRU cache for arrival times. * client/offchain: Adjust to PeerId API changes * contracts: Lazy storage removal (#7740) * Do not evict a contract from within a call stack We don't want to trigger contract eviction automatically when a contract is called. This is because those changes can be reverted due to how storage transactions are used at the moment. More Information: https://github.com/paritytech/substrate/issues/6439#issuecomment-648754324 It can be re-introduced once the linked issue is resolved. In the meantime `claim_surcharge` must be called to evict a contract. * Lazily delete storage in on_initialize instead of when removing the contract * Add missing documentation of new error * Make Module::claim_surcharge public It being the only dispatchable that is private is an oversight. * review: Add final newline * review: Simplify assert statement * Add test that checks that partial remove of a contract works * Premote warning to error * Added missing docs for seal_terminate * Lazy deletion should only take AVERAGE_ON_INITIALIZE_RATIO of the block * Added informational about the lazy deletion throughput * Avoid lazy deletion in case the block is already full * Prevent queue decoding in case of an already full block * Add test that checks that on_initialize honors block limits * fix template (#7823) * rename HEADER files so that they are consistent with LICENSE filenames (#7825) * contracts: Prevent contracts from allocating a too large buffer (#7818) * Prevent contracts from allocating a too large buffer * Fix possible integer overflow * Improve error message on where clause on pallet error (#7821) * improve error message on where clause on pallet error * Revert "improve error message on where clause on pallet error" This reverts commit 5a3cc38976813fccef3357833553ce30f5b988ea. * Revert "Revert "improve error message on where clause on pallet error"" This reverts commit e3b3fca6bc4fa89816f80dbcb82dc4536a9b2549. * Feat sp keystore (#7826) * delete not used VRFTranscriptValue * specification variable naming * minor fix (#7828) * Participation Lottery Pallet (#7221) * Basic design * start adding tests * finish tests * clean up crates * use call index for match * finish benchmarks * add to runtime * fix * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs * more efficient storage * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Update lib.rs * Update bin/node/runtime/src/lib.rs * trait -> config * add repeating lottery * new benchmarks * fix build * move trait for warning * feedback from @xlc * add stop_repeat * fix * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Support static calls * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix test * add loop to mitigate modulo bias * Update weights for worst case scenario loop * Initialize pot with ED * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_lottery --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/lottery/src/weights.rs --template=./.maintain/frame-weight-template.hbs Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * client/network: Use request response for block requests (#7478) * client/network: Add scaffolding for finality req to use req resp #sc * client/network/src/finality_requests: Remove * client/network/src/behaviour: Pass request id down to sync * client/network: Use request response for block requests * client/network: Move handler logic into *_*_handler.rs * client/network: Track ongoing finality requests in protocol.rs * client/network: Remove commented out finalization initialization * client/network: Add docs for request handlers * client/network/finality_request_handler: Log errors * client/network/block_request_handler: Log errors * client/network: Format * client/network: Handle block request failure * protocols/network: Fix tests * client/network/src/behaviour: Handle request sending errors * client/network: Move response handling into custom method * client/network/protocol: Handle block response errors * client/network/protocol: Remove tracking of obsolete requests * client/network/protocol: Remove block request start time tracking This will be handled generically via request-responses. * client/network/protocol: Refactor on_*_request_started * client/network: Pass protocol config instead of protocol name * client/network: Pass protocol config in tests * client/network/config: Document request response configs * client/network/src/_request_handler: Document protocol config gen * client/network/src/protocol: Document Peer request values * client/network: Rework request response to always use oneshot * client/network: Unified metric reporting for all request protocols * client/network: Move protobuf parsing into protocol.rs * client/network/src/protocol: Return pending events after poll * client/network: Improve error handling and documentation * client/network/behaviour: Remove outdated error types * Update client/network/src/block_request_handler.rs Co-authored-by: Ashley <ashley.ruglys@gmail.com> * Update client/network/src/finality_request_handler.rs Co-authored-by: Ashley <ashley.ruglys@gmail.com> * client/network/protocol: Reduce reputation on timeout * client/network/protocol: Refine reputation changes * client/network/block_request_handler: Set and explain queue length * client/service: Deny block requests when light client * client/service: Fix role matching * client: Enforce line width * client/network/request_responses: Fix unit tests * client/network: Expose time to build response via metrics * client/network/request_responses: Fix early connection closed error * client/network/protocol: Fix line length * client/network/protocol: Disconnect on most request failures * client/network/protocol: Disconnect peer when oneshot is canceled * client/network/protocol: Disconnect peer even when connection closed * client/network/protocol: Remove debugging log line * client/network/request_response: Use Clone::clone for error * client/network/request_response: Remove outdated comment With libp2p v0.33.0 libp2p-request-response properly sends inbound failures on connections being closed. Co-authored-by: Addie Wagenknecht <addie@nortd.com> Co-authored-by: Ashley <ashley.ruglys@gmail.com> * fix : remove `_{ }` syntax from benchmark macro (#7822) * commented use of common * hack to pass tests * another hack * remove all commented code * fix the easy tests * temp hack * follow through comma hack until better solution * patch macro * missed one * update benchmarks * update docs * fix docs * removed too much * fix changes Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Improve spans of pallet macro (#7830) * fix spans * convert name to snake case * Fix master build (#7837) * Fix master build * Use correct copyright year * babe: expose next epoch data (#7829) * babe: expose next epoch data * babe: add runtime api for next_epoch * babe: avoid reading next authorities from storage unnecessarily * babe: add notes about epoch duration constraints * babe: guard against overflow * babe: add test for fetching current and next epoch data * contracts: Add configurable per-storage item cost (#7819) * Rework rent parameters * No need for empty_pair_count any longer * Parameterize runtime * upgrade a few dependencies (#7831) * upgrade a few dependencies * make it compile at the expense of duplicate deps * fix web-wasm and a warning * introduce activate-wasm-bindgen-features crate * Revert "introduce activate-wasm-bindgen-features crate" This reverts commit 5a6e41e683f8a4844c0a735dcd08caabb2313f11. * add getrandom feature to sc-consensus-aura * CI: remove squash and fix buildah push (#7841) * Fix incorrect use of syn::exports (#7838) * Fix incorrect use of syn::exports Instead of using `syn::exports` we should import the trait from the quote crate directly. * Use own macro for test cases to fix compilation with latest syn * Fix test * Subkey should not import the entire world. (#7845) There is no reason for subkey to import the default Substrate node to support a feature that would only be usable for the Substrate node. Subkey itself should be more the default key management binary for Substrate related chains. If certain chains require some special functionality, they can easily stick together their own "my-chain-key". * Rework priority groups, take 2 (#7700) * Rework priority groups * Broken tests fix * Fix warning causing CI to fail * [Hack] Try restore backwards-compatibility * Fix peerset bug * Doc fixes and clean up * Error on state mismatch * Try debug CI * CI debugging * [CI debug] Can I please see this line * Revert "[CI debug] Can I please see this line" This reverts commit 4b7cf7c1511f579cd818b21d46bd11642dfac5cb. * Revert "CI debugging" This reverts commit 9011f1f564b860386dc7dd6ffa9fc34ea7107623. * Fix error! which isn't actually an error * Fix Ok() returned when actually Err() * Tweaks and fixes * Fix build * Peerset bugfix * [Debug] Try outbound GrandPa slots * Another bugfix * Revert "[Debug] Try outbound GrandPa slots" This reverts commit d175b9208c088faad77d9f0ce36ff6f48bd92dd3. * [Debug] Try outbound GrandPa slots * Apply suggestions from code review Co-authored-by: Max Inden <mail@max-inden.de> * Use consts for hardcoded peersets * Revert "Try debug CI" This reverts commit 62c4ad5e79c03d561c714a008022ecac463a597e. * Renames * Line widths * Add doc Co-authored-by: Max Inden <mail@max-inden.de> * Better Handle Dead Accounts in Balances (#7843) * Don't mutate storage when account is dead and should stay dead * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs * more concrete storage noop Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * bump fs-swap (#7834) * UniArts reserve SS58 address id 38 (#7651) * UniArts reserve SS58 address id 45 * Update ss58-registry.json Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: Xiang Li <bigxiang@gmail.com> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update to futures 0.3.9 (#7854) * Store dispatch info of calls locally in weight calculation (#7849) * utility * sudo * more * recovery * better formatting * client/network: Re-enable light_client_handler.rs unit tests (#7853) * Fix max log level (#7851) With the switch to tracing we did not set the `max_log_level` anymore. This resulted in a performance degradation as logging did not early exited and thus, `trace` logs were at least resolved every time. This pr fixes it by ensuring that we set the correct max log level. * Fix missing network for uniarts (#7859) * contracts: Collect rent for the first block during deployment (#7847) * Pay first rent during instantiation * Fix and add new tests * Do not increment trie id counter on failure * Merge 2.0.1 backport branch into mainline master (#7842) * Backport paritytech/substrate#7381 * Bring back genesis storage build in aura/timestamp To not change spec version, see https://github.com/paritytech/substrate/pull/7686#discussion_r540032743 * Backport paritytech/substrate#7238 * Backport paritytech/substrate#7395 * Bump impl_version * Fix UI tests and bump trybuild dep See https://github.com/rust-lang/rust/pull/73996 Backports: https://github.com/paritytech/substrate/pull/7764 https://github.com/paritytech/substrate/pull/7656 * Partially backport paritytech/substrate#7838 * Release frame-support with a dep compilation fix * Bump patch level for remaining crates This is done because at the time of writing cargo-unleash does not fully support partial workspace publishing and mixes both local and crates.io versions of the packages, leading to errors in the release check workflow. * Backport paritytech/substrate#7854 ...to fix compilation error when using futures-* v0.3.9. * Adding Changelog entry for patch release Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Benjamin Kampmann <ben@parity.io> * Bump cargo-unleash to latest alpha release (#7867) * Bump sha2 from 0.8.2 to 0.9.2 (#7643) * Bump sha2 from 0.8.2 to 0.9.2 Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.8.2 to 0.9.2. - [Release notes](https://github.com/RustCrypto/hashes/releases) - [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.8.2...streebog-v0.9.2) Signed-off-by: dependabot[bot] <support@github.com> * Fix compilation error Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de> * bumpd minor version (#7873) * Add Prometheus alerts if unbounded channels are too large (#7866) * Add Prometheus alerts if unbounded channels are too large * Tweaks * Bump retain_mut from 0.1.1 to 0.1.2 (#7869) Bumps [retain_mut](https://github.com/upsuper/retain_mut) from 0.1.1 to 0.1.2. - [Release notes](https://github.com/upsuper/retain_mut/releases) - [Commits](https://github.com/upsuper/retain_mut/compare/v0.1.1...v0.1.2) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * babe: initialize next authorities on genesis (#7872) * babe: initialize next authorities on genesis * babe: add test for genesis authorities * Update serde and parity-multiaddr, to fix master CI (#7877) * Add ss58 version prefix for CORD (from Dhiway) (#7862) * Add ss58 version prefix for CORD * Add ss58 version pref…

Rework priority groups

9c4b449

tomaka added A0-please_review Pull request needs code review. B5-clientnoteworthy C1-low PR touches the given topic and has a low impact on builders. labels Dec 9, 2020

tomaka requested review from romanb and mxinden December 9, 2020 11:51

tomaka requested a review from andresilva as a code owner December 9, 2020 11:51

This was referenced Dec 9, 2020

Rework priority groups #7374

Closed

Companion PR for refactoring priority groups paritytech/polkadot#2095

Merged

tomaka added 2 commits December 9, 2020 13:11

Broken tests fix

d673ab4

Fix warning causing CI to fail

126ba30

tomaka mentioned this pull request Dec 9, 2020

Burn-in PR for refactoring priority groups paritytech/polkadot#2098

Closed

[Hack] Try restore backwards-compatibility

0e91d3d

mxinden reviewed Dec 9, 2020

View reviewed changes

tomaka added 2 commits December 9, 2020 16:16

Fix peerset bug

7d26639

Doc fixes and clean up

c69bf7a

mxinden reviewed Dec 9, 2020

View reviewed changes

client/network-gossip/src/bridge.rs Show resolved Hide resolved

tomaka added 2 commits December 14, 2020 13:05

Error on state mismatch

a7f1a65

Try debug CI

62c4ad5

rphmeier mentioned this pull request Jan 3, 2021

Implement dormant connections/low overhead connections #7797

Closed

burdges mentioned this pull request Jan 3, 2021

Implement own peer set for availability-distribution paritytech/polkadot#2177

Closed

tomaka and others added 7 commits January 4, 2021 22:24

Revert "[Debug] Try outbound GrandPa slots"

ffad0a6

This reverts commit d175b92.

[Debug] Try outbound GrandPa slots

65cf06e

Apply suggestions from code review

31b9a0b

Co-authored-by: Max Inden <mail@max-inden.de>

Use consts for hardcoded peersets

ebaeaab

Merge remote-tracking branch 'upstream/master' into rework-priority-g…

9517536

…roups-take2

Merge remote-tracking branch 'upstream/master' into rework-priority-g…

2fb69a7

…roups-take2

Merge remote-tracking branch 'upstream/master' into rework-priority-g…

30f6573

…roups-take2

tomaka added 5 commits January 7, 2021 10:02

Revert "Try debug CI"

8535c90

This reverts commit 62c4ad5.

Renames

14b3b22

Merge remote-tracking branch 'upstream/master' into rework-priority-g…

6c709db

…roups-take2

Line widths

1117b56

Add doc

9e702d6

mxinden approved these changes Jan 7, 2021

View reviewed changes

romanb approved these changes Jan 7, 2021

View reviewed changes

ghost merged commit c9d9365 into paritytech:master Jan 7, 2021

tomaka deleted the rework-priority-groups-take2 branch January 7, 2021 13:54

tomaka mentioned this pull request Jan 8, 2021

Remove backwards-compatibility hack in networking #7852

Closed

mxinden mentioned this pull request Jan 8, 2021

client/network: Re-enable light_client_handler.rs unit tests #7853

Merged

This was referenced Jan 25, 2021

Called on_block_announce with a bad PeerId #5592

Closed

Grandpa notifications dropping to 0 #7983

Closed

This was referenced Feb 2, 2021

Missed Declare from the collator paritytech/polkadot#2362

Closed

Remove backwards-compatibility networking hack #8068

Merged

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework priority groups, take 2 #7700

Rework priority groups, take 2 #7700

tomaka commented Dec 9, 2020 •

edited

Loading

tomaka commented Dec 9, 2020 •

edited

Loading

mxinden left a comment

mxinden Dec 9, 2020

tomaka Dec 9, 2020 •

edited

Loading

mxinden Dec 9, 2020

rphmeier commented Dec 9, 2020

burdges commented Dec 10, 2020

eskimor commented Dec 10, 2020

tomaka commented Dec 14, 2020 •

edited

Loading

tomaka commented Dec 14, 2020

tomaka commented Dec 14, 2020 •

edited

Loading

burdges commented Dec 14, 2020

tomaka commented Jan 6, 2021 •

edited

Loading

tomaka commented Jan 7, 2021

tomaka commented Jan 7, 2021

tomaka commented Jan 7, 2021

ghost commented Jan 7, 2021

	let result = NetworkService::add_to_peers_set(self, protocol, iter::once(addr).collect());
	let result = NetworkService::add_peers_set_reservedself, protocol, iter::once(addr).collect());

	let result = NetworkService::remove_from_peers_set(self, protocol, iter::once(addr).collect());
	let result = NetworkService::remove_peers_set_reserved(self, protocol, iter::once(addr).collect());

Rework priority groups, take 2 #7700

Rework priority groups, take 2 #7700

Conversation

tomaka commented Dec 9, 2020 • edited Loading

Before this PR

Problems with the current situation

After this PR

Concerns

Additional round-trip

--reserved-only might be confusing

Might contain bugs

tomaka commented Dec 9, 2020 • edited Loading

mxinden left a comment

Choose a reason for hiding this comment

mxinden Dec 9, 2020

Choose a reason for hiding this comment

tomaka Dec 9, 2020 • edited Loading

Choose a reason for hiding this comment

mxinden Dec 9, 2020

Choose a reason for hiding this comment

rphmeier commented Dec 9, 2020

burdges commented Dec 10, 2020

eskimor commented Dec 10, 2020

tomaka commented Dec 14, 2020 • edited Loading

tomaka commented Dec 14, 2020

tomaka commented Dec 14, 2020 • edited Loading

burdges commented Dec 14, 2020

tomaka commented Jan 6, 2021 • edited Loading

tomaka commented Jan 7, 2021

tomaka commented Jan 7, 2021

tomaka commented Jan 7, 2021

ghost commented Jan 7, 2021

tomaka commented Dec 9, 2020 •

edited

Loading

tomaka commented Dec 9, 2020 •

edited

Loading

tomaka Dec 9, 2020 •

edited

Loading

tomaka commented Dec 14, 2020 •

edited

Loading

tomaka commented Dec 14, 2020 •

edited

Loading

tomaka commented Jan 6, 2021 •

edited

Loading