Runtime changes for Asynchronous Backing (#4786)

* inclusion: utility for allowed relay-parents

* inclusion: use prev number instead of prev hash

* track most recent context of paras

* inclusion: accept previous relay-parents

* update dmp  advancement rule for async backing

* fmt

* add a comment about validation outputs

* clean up a couple of TODOs

* weights

* fix weights

* fmt

* Resolve dmp todo

* Restore inclusion tests

* Restore paras_inherent tests

* MostRecentContext test

* Benchmark for new paras dispatchable

* Prepare check_validation_outputs for upgrade

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Implementers guide changes

* More tests for allowed relay parents

* Add a github issue link

* Compute group index based on relay parent

* Storage migration

* Move allowed parents tracker to shared

* Compile error

* Get group assigned to core at the next block

* Test group assignment

* fmt

* Error instead of panic

* Update guide

* Extend doc-comment

* Update runtime/parachains/src/shared.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

roadmap/implementers-guide/src/runtime/dmp.md

@@ -27,9 +27,9 @@ No initialization routine runs for this module.
 
 Candidate Acceptance Function:
 
-* `check_processed_downward_messages(P: ParaId, processed_downward_messages: u32)`:
+* `check_processed_downward_messages(P: ParaId, relay_parent_number: BlockNumber, processed_downward_messages: u32)`:
+    1. Checks that `processed_downward_messages` is at least 1 if `DownwardMessageQueues` for `P` is not empty at the given `relay_parent_number`.
     1. Checks that `DownwardMessageQueues` for `P` is at least `processed_downward_messages` long.
-    1. Checks that `processed_downward_messages` is at least 1 if `DownwardMessageQueues` for `P` is not empty.
 
 Candidate Enactment:
 

roadmap/implementers-guide/src/runtime/inclusion.md

@@ -68,26 +68,26 @@ All failed checks should lead to an unrecoverable error making the block invalid
   1. check that the validator bit index is not out of bounds.
   1. check the validators signature, iff `full_check=FullCheck::Yes`.
 
-* `sanitize_backed_candidates<T: crate::inclusion::Config, F: Fn(CandidateHash) -> bool>(
-    relay_parent: T::Hash,
+* `sanitize_backed_candidates<T: crate::inclusion::Config, F: FnMut(usize, &BackedCandidate<T::Hash>) -> bool>(
     mut backed_candidates: Vec<BackedCandidate<T::Hash>>,
     candidate_has_concluded_invalid_dispute: F,
     scheduled: &[CoreAssignment],
   ) `
   1. filter out any backed candidates that have concluded invalid.
-  1. filter out backed candidates that don't have a matching `relay_parent`.
   1. filters backed candidates whom's paraid was scheduled by means of the provided `scheduled` parameter.
+  1. sorts remaining candidates with respect to the core index assigned to them.
 
-* `process_candidates(parent_storage_root, BackedCandidates, scheduled: Vec<CoreAssignment>, group_validators: Fn(GroupIndex) -> Option<Vec<ValidatorIndex>>)`:
+* `process_candidates(allowed_relay_parents, BackedCandidates, scheduled: Vec<CoreAssignment>, group_validators: Fn(GroupIndex) -> Option<Vec<ValidatorIndex>>)`:
+    > For details on `AllowedRelayParentsTracker` see documentation for [Shared](./shared.md) module.
   1. check that each candidate corresponds to a scheduled core and that they are ordered in the same order the cores appear in assignments in `scheduled`.
   1. check that `scheduled` is sorted ascending by `CoreIndex`, without duplicates.
+  1. check that the relay-parent from each candidate receipt is one of the allowed relay-parents.
   1. check that there is no candidate pending availability for any scheduled `ParaId`.
-  1. check that each candidate's `validation_data_hash` corresponds to a `PersistedValidationData` computed from the current state.
-    > NOTE: With contextual execution in place, validation data will be obtained as of the state of the context block. However, only the state of the current block can be used for such a query.
+  1. check that each candidate's `validation_data_hash` corresponds to a `PersistedValidationData` computed from the state of the context block.
   1. If the core assignment includes a specific collator, ensure the backed candidate is issued by that collator.
   1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_cooldown` of `Paras::last_code_upgrade(para_id, true)`, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID.
   1. Check the collator's signature on the candidate data.
-  1. check the backing of the candidate using the signatures and the bitfields, comparing against the validators assigned to the groups, fetched with the `group_validators` lookup.
+  1. check the backing of the candidate using the signatures and the bitfields, comparing against the validators assigned to the groups, fetched with the `group_validators` lookup, while group indices are computed by `Scheduler` according to group rotation info. 
   1. call `Ump::check_upward_messages(para, commitments.upward_messages)` to check that the upward messages are valid.
   1. call `Dmp::check_processed_downward_messages(para, commitments.processed_downward_messages)` to check that the DMQ is properly drained.
   1. call `Hrmp::check_hrmp_watermark(para, commitments.hrmp_watermark)` for each candidate to check rules of processing the HRMP watermark.

roadmap/implementers-guide/src/runtime/parainherent.md

@@ -35,6 +35,7 @@ OnChainVotes: Option<ScrapedOnChainVotes>,
     1. Set `Included` as `Some`.
     1. Unpack `ParachainsInherentData` into `signed_bitfields`, `backed_candidates`, `parent_header`, and `disputes`.
     1. Hash the parent header and make sure that it corresponds to the block hash of the parent (tracked by the `frame_system` FRAME module).
+    1. Add a previous block to the `AllowedRelayParents` before anything else and read the resulting value from `shared` storage.
     1. Calculate the `candidate_weight`, `bitfields_weight`, and `disputes_weight`.
     1. If the sum of `candidate_weight`, `bitfields_weight`, and `disputes_weight` is greater than the max block weight we do the following with the goal of prioritizing the inclusion of disputes without making it game-able by block authors:
       1. clear `bitfields` and set `bitfields_weight` equal to 0.
@@ -48,10 +49,9 @@ OnChainVotes: Option<ScrapedOnChainVotes>,
     1. If `Scheduler::availability_timeout_predicate` is `Some`, invoke `Inclusion::collect_pending` using it and annotate each of those freed cores with `FreedReason::TimedOut`.
     1. Combine and sort the the bitfield-freed cores and the timed-out cores.
     1. Invoke `Scheduler::clear`
-    1. Invoke `Scheduler::schedule(freed_cores, System::current_block())`
-    1. Extract `parent_storage_root` from the parent header,
+    1. Invoke `Scheduler::schedule(freed_cores, System::current_block())` 
     1. If `Disputes::concluded_invalid(current_session, candidate)` is true for any of the `backed_candidates`, fail.
-    1. Invoke the `Inclusion::process_candidates` routine with the parameters `(parent_storage_root, backed_candidates, Scheduler::scheduled(), Scheduler::group_validators)`.
+    1. Invoke the `Inclusion::process_candidates` routine with the parameters `(allowed_relay_parents, backed_candidates, Scheduler::scheduled(), Scheduler::group_validators)`.
     1. Deconstruct the returned `ProcessedCandidates` value into `occupied` core indices, and backing validators by candidate `backing_validators_per_candidate` represented by `Vec<(CandidateReceipt, Vec<(ValidatorIndex, ValidityAttestation)>)>`.
     1. Set `OnChainVotes` to `ScrapedOnChainVotes`, based on the `current_session`, concluded `disputes`, and `backing_validators_per_candidate`.
     1. Call `Scheduler::occupied` using the `occupied` core indices of the returned  above, first sorting the list of assigned core indices.
@@ -68,6 +68,7 @@ OnChainVotes: Option<ScrapedOnChainVotes>,
 * `create_inherent_inner(data: &InherentData) -> Option<ParachainsInherentData<T::Header>>`
   1. Unpack `InherentData` into its parts, `bitfields`, `backed_candidates`, `disputes` and the `parent_header`. If data cannot be unpacked return `None`.
   1. Hash the `parent_header` and make sure that it corresponds to the block hash of the parent (tracked by the `frame_system` FRAME module).
+  1. Read `AllowedRelayParents` from `shared` storage and add a previous block to this value so that we operate with the same look-back as in `enter`.
   1. Invoke `Disputes::filter_multi_dispute_data` to remove duplicates et al from `disputes`.
   1. Run the following within a  `with_transaction` closure to avoid side effects (we are essentially replicating the logic that would otherwise happen within `enter` so we can get the filtered bitfields and the `concluded_invalid_disputes` + `scheduled` to use in filtering the `backed_candidates`.):
     1. Invoke `Disputes::provide_multi_dispute_data`.
@@ -81,7 +82,7 @@ OnChainVotes: Option<ScrapedOnChainVotes>,
     1. Invoke `scheduler::Pallet<T>>::schedule` with `freed` and the current block number to create the same schedule of the cores that `enter` will create.
     1. Read the new `<scheduler::Pallet<T>>::scheduled()` into `schedule`.
     1. From the `with_transaction` closure return `concluded_invalid_disputes`, `bitfields`, and `scheduled`.
-  1. Invoke `sanitize_backed_candidates` using the `scheduled` return from the `with_transaction` and pass the closure `|candidate_hash: CandidateHash| -> bool { DisputesHandler::concluded_invalid(current_session, candidate_hash) }` for the param `candidate_has_concluded_invalid_dispute`.
+  1. Invoke `sanitize_backed_candidates` using the `scheduled` return from the `with_transaction` and pass the closure `|candidate_idx: usize, candidate_hash: CandidateHash| -> bool` which returns `true` either if the candidate is concluded to be invalid during the dispute or it doesn't pass the verification in the context of the most recent parachain head, such as relay-parent being out-of-bounds or commitments hashes mismatch.
   1. create a `rng` from `rand_chacha::ChaChaRng::from_seed(compute_entropy::<T>(parent_hash))`.
   1. Invoke `limit_disputes` with the max block weight and `rng`, storing the returned weigh in `remaining_weight`.
   1. Fill up the remaining of the block weight with backed candidates and bitfields by invoking `apply_weight_limit` with `remaining_weigh` and `rng`.

roadmap/implementers-guide/src/runtime/paras.md

@@ -153,6 +153,8 @@ Parachains: Vec<ParaId>,
 ParaLifecycle: map ParaId => Option<ParaLifecycle>,
 /// The head-data of every registered para.
 Heads: map ParaId => Option<HeadData>;
+/// The context (relay-chain block number) of the most recent parachain head.
+MostRecentContext: map ParaId => BlockNumber;
 /// The validation code hash of every live para.
 CurrentCodeHash: map ParaId => Option<ValidationCodeHash>;
 /// Actual past code hash, indicated by the para id as well as the block number at which it became outdated.
@@ -220,14 +222,12 @@ CodeByHash: map ValidationCodeHash => Option<ValidationCode>
 
 1. Execute all queued actions for paralifecycle changes:
   1. Clean up outgoing paras.
-     1. This means removing the entries under `Heads`, `CurrentCode`, `FutureCodeUpgrades`, and
-        `FutureCode`. An according entry should be added to `PastCode`, `PastCodeMeta`, and
-        `PastCodePruning` using the outgoing `ParaId` and removed `CurrentCode` value. This is
-        because any outdated validation code must remain available on-chain for a determined amount
+     1. This means removing the entries under `Heads`, `CurrentCode`, `FutureCodeUpgrades`,
+        `FutureCode` and `MostRecentContext`. An according entry should be added to `PastCode`, `PastCodeMeta`, and `PastCodePruning` using the outgoing `ParaId` and removed `CurrentCode` value. This is because any outdated validation code must remain available on-chain for a determined amount
         of blocks, and validation code outdated by de-registering the para is still subject to that
         invariant.
   1. Apply all incoming paras by initializing the `Heads` and `CurrentCode` using the genesis
-     parameters.
+     parameters as well as `MostRecentContext` to `0`.
   1. Amend the `Parachains` list and `ParaLifecycle` to reflect changes in registered parachains.
   1. Amend the `ParaLifecycle` set to reflect changes in registered parathreads.
   1. Upgrade all parathreads that should become parachains, updating the `Parachains` list and
@@ -261,8 +261,7 @@ CodeByHash: map ValidationCodeHash => Option<ValidationCode>
   executed in the context of a relay-chain block with number >= `relay_parent + config.validation_upgrade_delay`. If the upgrade is scheduled `UpgradeRestrictionSignal` is set and it will remain set until `relay_parent + config.validation_upgrade_cooldown`.
 In case the PVF pre-checking is enabled, or the new code is not already present in the storage, then the PVF pre-checking run will be scheduled for that validation code. If the pre-checking concludes with rejection, then the upgrade is canceled. Otherwise, after pre-checking is concluded the upgrade will be scheduled and be enacted as described above.
 * `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head,
-  where the new head was executed in the context of a relay-chain block with given number. This will
-  apply pending code upgrades based on the block number provided. If an upgrade took place it will clear the `UpgradeGoAheadSignal`.
+  where the new head was executed in the context of a relay-chain block with given number, the latter value is inserted into the `MostRecentContext` mapping. This will apply pending code upgrades based on the block number provided. If an upgrade took place it will clear the `UpgradeGoAheadSignal`.
 * `lifecycle(ParaId) -> Option<ParaLifecycle>`: Return the `ParaLifecycle` of a para.
 * `is_parachain(ParaId) -> bool`: Returns true if the para ID references any live parachain,
   including those which may be transitioning to a parathread in the future.

roadmap/implementers-guide/src/runtime/scheduler.md

@@ -137,7 +137,6 @@ struct CoreAssignment {
   core: CoreIndex,
   para_id: ParaId,
   kind: AssignmentKind,
-  group_idx: GroupIndex,
 }
 // reasons a core might be freed.
 enum FreedReason {

roadmap/implementers-guide/src/runtime/shared.md

@@ -19,6 +19,27 @@ pub(crate) const SESSION_DELAY: SessionIndex = 2;
 
 ## Storage
 
+Helper structs:
+
+```rust
+struct AllowedRelayParentsTracker<Hash, BlockNumber> {
+  // The past relay parents, paired with state roots, that are viable to build upon.
+  //
+  // They are in ascending chronologic order, so the newest relay parents are at
+  // the back of the deque.
+  //
+  // (relay_parent, state_root)
+  //
+  // NOTE: the size limit of look-back is currently defined as a constant in Runtime.
+  buffer: VecDeque<(Hash, Hash)>,
+
+  // The number of the most recent relay-parent, if any.
+  latest_number: BlockNumber,
+}
+```
+
+Storage Layout:
+
 ```rust
 /// The current session index within the Parachains Runtime system.
 CurrentSessionIndex: SessionIndex;
@@ -28,6 +49,8 @@ ActiveValidatorIndices: Vec<ValidatorIndex>,
 /// The parachain attestation keys of the validators actively participating in parachain consensus.
 /// This should be the same length as `ActiveValidatorIndices`.
 ActiveValidatorKeys: Vec<ValidatorId>
+/// Relay-parents allowed to build candidates upon.
+AllowedRelayParents: AllowedRelayParentsTracker<T::Hash, T::BlockNumber>,
 ```
 
 ## Initialization
@@ -51,6 +74,8 @@ This information is used in the:
   passed.
 * Paras Module: For delaying updates to paras until at least one full session has passed.
 
+Allowed relay parents buffer, which is maintained by [ParaInherent](./parainherent.md) module, is cleared on every session change.
+
 ## Finalization
 
 The Shared Module currently has no finalization routines.

runtime/kusama/src/weights/runtime_parachains_paras.rs

@@ -16,11 +16,11 @@
 //! Autogenerated weights for `runtime_parachains::paras`
 //!
 //! THIS FILE WAS AUTO-GENERATED USING THE SUBSTRATE BENCHMARK CLI VERSION 4.0.0-dev
-//! DATE: 2021-12-28, STEPS: `50`, REPEAT: 20, LOW RANGE: `[]`, HIGH RANGE: `[]`
-//! EXECUTION: Some(Wasm), WASM-EXECUTION: Compiled, CHAIN: Some("kusama-dev"), DB CACHE: 128
+//! DATE: 2022-02-01, STEPS: `50`, REPEAT: 20, LOW RANGE: `[]`, HIGH RANGE: `[]`
+//! EXECUTION: Some(Wasm), WASM-EXECUTION: Compiled, CHAIN: Some("kusama-dev"), DB CACHE: 1024
 
 // Executed Command:
-// target/release/polkadot
+// target/production/polkadot
 // benchmark
 // --chain=kusama-dev
 // --steps=50
@@ -45,21 +45,29 @@ pub struct WeightInfo<T>(PhantomData<T>);
 impl<T: frame_system::Config> runtime_parachains::paras::WeightInfo for WeightInfo<T> {
 	// Storage: Paras CurrentCodeHash (r:1 w:1)
 	// Storage: Paras CodeByHashRefs (r:1 w:1)
+	// Storage: Paras PastCodeMeta (r:1 w:1)
+	// Storage: Paras PastCodePruning (r:1 w:1)
+	// Storage: Paras PastCodeHash (r:0 w:1)
 	// Storage: Paras CodeByHash (r:0 w:1)
 	fn force_set_current_code(c: u32, ) -> Weight {
 		(0 as Weight)
 			// Standard Error: 0
-			.saturating_add((3_000 as Weight).saturating_mul(c as Weight))
-			.saturating_add(T::DbWeight::get().reads(2 as Weight))
-			.saturating_add(T::DbWeight::get().writes(3 as Weight))
+			.saturating_add((2_000 as Weight).saturating_mul(c as Weight))
+			.saturating_add(T::DbWeight::get().reads(4 as Weight))
+			.saturating_add(T::DbWeight::get().writes(6 as Weight))
 	}
 	// Storage: Paras Heads (r:0 w:1)
 	fn force_set_current_head(s: u32, ) -> Weight {
-		(11_803_000 as Weight)
+		(9_718_000 as Weight)
 			// Standard Error: 0
 			.saturating_add((1_000 as Weight).saturating_mul(s as Weight))
 			.saturating_add(T::DbWeight::get().writes(1 as Weight))
 	}
+	// Storage: Paras MostRecentContext (r:0 w:1)
+	fn force_set_most_recent_context() -> Weight {
+		(1_170_000 as Weight)
+			.saturating_add(T::DbWeight::get().writes(1 as Weight))
+	}
 	// Storage: Configuration ActiveConfig (r:1 w:0)
 	// Storage: Paras FutureCodeHash (r:1 w:1)
 	// Storage: Paras CurrentCodeHash (r:1 w:0)
@@ -74,23 +82,25 @@ impl<T: frame_system::Config> runtime_parachains::paras::WeightInfo for WeightIn
 	fn force_schedule_code_upgrade(c: u32, ) -> Weight {
 		(0 as Weight)
 			// Standard Error: 0
-			.saturating_add((3_000 as Weight).saturating_mul(c as Weight))
+			.saturating_add((2_000 as Weight).saturating_mul(c as Weight))
 			.saturating_add(T::DbWeight::get().reads(9 as Weight))
 			.saturating_add(T::DbWeight::get().writes(8 as Weight))
 	}
 	// Storage: Paras FutureCodeUpgrades (r:1 w:0)
 	// Storage: Paras Heads (r:0 w:1)
+	// Storage: Paras UpgradeGoAheadSignal (r:0 w:1)
+	// Storage: Paras MostRecentContext (r:0 w:1)
 	fn force_note_new_head(s: u32, ) -> Weight {
-		(18_655_000 as Weight)
+		(12_348_000 as Weight)
 			// Standard Error: 0
 			.saturating_add((1_000 as Weight).saturating_mul(s as Weight))
 			.saturating_add(T::DbWeight::get().reads(1 as Weight))
-			.saturating_add(T::DbWeight::get().writes(1 as Weight))
+			.saturating_add(T::DbWeight::get().writes(3 as Weight))
 	}
 	// Storage: ParasShared CurrentSessionIndex (r:1 w:0)
 	// Storage: Paras ActionsQueue (r:1 w:1)
 	fn force_queue_action() -> Weight {
-		(23_208_000 as Weight)
+		(16_559_000 as Weight)
 			.saturating_add(T::DbWeight::get().reads(2 as Weight))
 			.saturating_add(T::DbWeight::get().writes(1 as Weight))
 	}
@@ -99,14 +109,14 @@ impl<T: frame_system::Config> runtime_parachains::paras::WeightInfo for WeightIn
 	fn add_trusted_validation_code(c: u32, ) -> Weight {
 		(0 as Weight)
 			// Standard Error: 0
-			.saturating_add((3_000 as Weight).saturating_mul(c as Weight))
+			.saturating_add((2_000 as Weight).saturating_mul(c as Weight))
 			.saturating_add(T::DbWeight::get().reads(2 as Weight))
 			.saturating_add(T::DbWeight::get().writes(1 as Weight))
 	}
 	// Storage: Paras CodeByHashRefs (r:1 w:0)
 	// Storage: Paras CodeByHash (r:0 w:1)
 	fn poke_unused_validation_code() -> Weight {
-		(4_639_000 as Weight)
+		(2_811_000 as Weight)
 			.saturating_add(T::DbWeight::get().reads(1 as Weight))
 			.saturating_add(T::DbWeight::get().writes(1 as Weight))
 	}

runtime/parachains/src/dmp.rs

@@ -175,13 +175,27 @@ impl<T: Config> Pallet<T> {
 	/// Checks if the number of processed downward messages is valid.
 	pub(crate) fn check_processed_downward_messages(
 		para: ParaId,
+		relay_parent_number: T::BlockNumber,
 		processed_downward_messages: u32,
 	) -> Result<(), ProcessedDownwardMessagesAcceptanceErr> {
 		let dmq_length = Self::dmq_length(para);
 
 		if dmq_length > 0 && processed_downward_messages == 0 {
-			return Err(ProcessedDownwardMessagesAcceptanceErr::AdvancementRule)
+			// The advancement rule is for at least one downwards message to be processed
+			// if the queue is non-empty at the relay-parent. Downwards messages are annotated
+			// with the block number, so we compare the earliest (first) against the relay parent.
+			let contents = Self::dmq_contents(para);
+
+			// sanity: if dmq_length is >0 this should always be 'Some'.
+			if contents.get(0).map_or(false, |msg| msg.sent_at <= relay_parent_number) {
+				return Err(ProcessedDownwardMessagesAcceptanceErr::AdvancementRule)
+			}
 		}
+
+		// Note that we might be allowing a parachain to signal that it's processed
+		// messages that hadn't been placed in the queue at the relay_parent.
+		// only 'stupid' parachains would do it and we don't (and can't) force anyone
+		// to act on messages, so the lenient approach is fine here.
 		if dmq_length < processed_downward_messages {
 			return Err(ProcessedDownwardMessagesAcceptanceErr::Underflow {
 				processed_downward_messages,