Parachains: Use relay chain slot for velocity measurement #6825

skunert · 2024-12-10T10:05:01Z

Changes

We now use relay chain slots to measure velocity on chain. Previously we were storing the current parachain slot. Then in on_state_proof of the ConsensusHook we were checking how many blocks were athored in the current parachain slot. This works well when the parachain slot time and relay chain slot time is the same. With elastic scaling, we can have parachain slot times lower than that of the relay chain. In these cases we want to measure velocity in relation to the relay chain. This PR adjusts that.

Migration

This PR includes a migration. Storage item SlotInfo of pallet aura-ext is renamed to RelaySlotInfo to better reflect its new content. A migration has been added that just kills the old storage item. RelaySlotInfo will be None initially but its value will be adjusted after one new relay chain slot arrives.

skunert · 2024-12-10T11:23:26Z

bot fmt

command-bot · 2024-12-10T11:23:32Z

@skunert https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7892036 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh". Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 23-6352bad6-c9e1-4490-bf52-e6b6f707a5f3 to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot · 2024-12-10T11:24:31Z

@skunert Command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh" has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7892036 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7892036/artifacts/download.

skunert · 2024-12-11T08:00:43Z

bot fmt

command-bot · 2024-12-11T08:00:48Z

@skunert https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7898854 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh". Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 26-f389b4df-c557-4572-a276-5937d6dc8d7d to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot · 2024-12-11T08:04:16Z

@skunert Command "$PIPELINE_SCRIPTS_DIR/commands/fmt/fmt.sh" has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7898854 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7898854/artifacts/download.

skunert · 2024-12-13T15:05:12Z

/cmd prdoc --bump patch --audience runtime_dev,node_dev

alindima

LGTM overall!

alindima · 2024-12-16T14:00:38Z

cumulus/client/consensus/aura/src/collators/mod.rs

+	let Ok(Some(api_version)) =
+		runtime_api.api_version::<dyn AuraUnincludedSegmentApi<Block>>(parent_hash)
+	else {
+		return Some(SlotClaim::unchecked::<P>(author_pub, para_slot, timestamp))


shouldn't we return None if we couldn't get the api version?

Yeah, if we have None, it means the api doesn't exist and we should return an error.

I changed this to be backward compatible with the previous behaviour. If we are building on top of the included block, the previous implementation would return Some. I don't think it is likely that someone runs this with a runtime without AuraUnincludedSegmentApi, but technically its possible.

alindima · 2024-12-16T14:20:14Z

cumulus/pallets/aura-ext/src/lib.rs

@@ -70,20 +74,7 @@ pub mod pallet {
 			// Fetch the authorities once to get them into the storage proof of the PoV.
 			Authorities::<T>::get();

-			let new_slot = pallet_aura::CurrentSlot::<T>::get();


it feels a bit weird that the pallet now has a RelaySlotInfo storage item that is never updated by the pallet (but only in an optional ConsensusHook, which parachains could in theory overwrite). Not a big deal though

cumulus/pallets/aura-ext/src/test.rs

cumulus/pallets/aura-ext/src/consensus_hook.rs

bkchr · 2024-12-16T21:29:56Z

cumulus/client/consensus/aura/src/collators/mod.rs

+	let Ok(Some(api_version)) =
+		runtime_api.api_version::<dyn AuraUnincludedSegmentApi<Block>>(parent_hash)
+	else {
+		return Some(SlotClaim::unchecked::<P>(author_pub, para_slot, timestamp))


Yeah, if we have None, it means the api doesn't exist and we should return an error.

cumulus/pallets/aura-ext/src/consensus_hook.rs

cumulus/pallets/aura-ext/src/migration.rs

# Conflicts: # cumulus/client/consensus/aura/src/collators/slot_based/block_builder_task.rs # cumulus/client/parachain-inherent/src/mock.rs

skunert · 2024-12-18T16:45:25Z

bot fmt

skunert · 2024-12-19T13:49:30Z

bot fmt

cumulus/client/consensus/aura/src/collators/mod.rs

bkchr · 2025-01-09T15:59:19Z

cumulus/pallets/aura-ext/src/consensus_hook.rs

+			None => (relay_chain_slot, 0),
+		};
+
+		// We need to allow one additional block to be built to fill the unincluded segment.


I don't really get this comment.

cumulus/pallets/aura-ext/src/migration.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

paritytech-workflow-stopper · 2025-01-13T17:08:36Z

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/12752246539
Failed job name: fmt

* master: (33 commits) Implement `pallet-asset-rewards` (#3926) [pallet-revive] Add host function `to_account_id` (#7091) [pallet-revive] Remove revive events (#7164) [pallet-revive] Remove debug buffer (#7163) litep2p: Provide partial results to speedup GetRecord queries (#7099) [pallet-revive] Bump asset-hub westend spec version (#7176) Remove 0 as a special case in gas/storage meters (#6890) [pallet-revive] Fix `caller_is_root` return value (#7086) req-resp/litep2p: Reject inbound requests from banned peers (#7158) Add "run to block" tools (#7109) Fix reversed error message in DispatchInfo (#7170) approval-voting: Make importing of duplicate assignment idempotent (#6971) Parachains: Use relay chain slot for velocity measurement (#6825) PRDOC: Document `validate: false` (#7117) xcm: convert properly assets in xcmpayment apis (#7134) CI: Only format umbrella crate during umbrella check (#7139) approval-voting: Fix sending of assignments after restart (#6973) Retry approval on availability failure if the check is still needed (#6807) [pallet-revive-eth-rpc] persist eth transaction hash (#6836) litep2p: Sufix litep2p to the identify agent version for visibility (#7133) ...

…d block (#7205) Follow-up to #6825, which introduced this bug. We use the `can_build_upon` method to ask the runtime if it is fine to build another block. The runtime checks this based on the [`ConsensusHook`](https://github.com/paritytech/polkadot-sdk/blob/c1b7c3025aa4423d4cf3e57309b60fb7602c2db6/cumulus/pallets/aura-ext/src/consensus_hook.rs#L110-L110) implementation, the most popular one being the `FixedConsensusHook`. In #6825 I removed a check that would always allow us to build when we are building on an included block. Turns out this check is still required when: 1. The [`UnincludedSegment` ](https://github.com/paritytech/polkadot-sdk/blob/c1b7c3025aa4423d4cf3e57309b60fb7602c2db6/cumulus/pallets/parachain-system/src/lib.rs#L758-L758) storage item in pallet-parachain-system is equal or larger than the unincluded segment. 2. We are calling the `can_build_upon` runtime API where the included block has progressed offchain to the current parent block (so last entry in the `UnincludedSegment` storage item). In this scenario the last entry in `UnincludedSegment` does not have a hash assigned yet (because it was not available in `on_finalize` of the previous block). So the unincluded segment will be reported at its maximum length which will forbid building another block. Ideally we would have a more elegant solution than to rely on the node-side here. But for now the check is reintroduced and a test is added to not break it again by accident. --------- Co-authored-by: command-bot <> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

Use relay chain slots for velocity measurement

2cea63d

skunert added the T9-cumulus This PR/Issue is related to cumulus. label Dec 10, 2024

".git/.scripts/commands/fmt/fmt.sh"

3086345

Import vec

ad5b1af

".git/.scripts/commands/fmt/fmt.sh"

2305334

skunert added 8 commits December 11, 2024 10:21

Clippy

a023784

Adjust velocity check in can_build_upon

b69df30

Improve variable name

ee45c93

Merge branch 'master' into skunert/relay_slot_velocity

665039b

Fix tests

720bd03

Fix tests

73ba8d1

Unused constant

bd1acea

Fix tests

6b8ce29

skunert added 2 commits December 13, 2024 17:09

PRDOC

db26437

Merge branch 'master' into skunert/relay_slot_velocity

74992c2

skunert marked this pull request as ready for review December 13, 2024 16:12

skunert requested review from bkchr and alindima December 13, 2024 16:13

alindima reviewed Dec 16, 2024

View reviewed changes

bkchr reviewed Dec 16, 2024

View reviewed changes

skunert added 3 commits December 18, 2024 09:43

Comments

7a7aa52

Comments

d78df61

Merge branch 'master' into skunert/relay_slot_velocity

8f03a9f

# Conflicts: # cumulus/client/consensus/aura/src/collators/slot_based/block_builder_task.rs # cumulus/client/parachain-inherent/src/mock.rs

This comment was marked as resolved.

Sign in to view

".git/.scripts/commands/fmt/fmt.sh"

ce0a28c

This comment was marked as resolved.

Sign in to view

Deps

31a91d3

This comment was marked as resolved.

Sign in to view

".git/.scripts/commands/fmt/fmt.sh"

a2d5936

This comment was marked as resolved.

Sign in to view

alindima approved these changes Dec 19, 2024

View reviewed changes

skunert requested a review from bkchr January 3, 2025 13:40

Merge branch 'master' into skunert/relay_slot_velocity

ef2a785

bkchr approved these changes Jan 9, 2025

View reviewed changes

Update cumulus/client/consensus/aura/src/collators/mod.rs

257f15b

Co-authored-by: Bastian Köcher <git@kchr.de>

skunert added 5 commits January 13, 2025 18:20

Make the migration more prominent in the changelog.

03b8056

fmt

40de356

Merge branch 'master' into skunert/relay_slot_velocity

d67e56b

Fix breakage from review suggestions

d7a6e55

Merge branch 'master' into skunert/relay_slot_velocity

1756059

skunert enabled auto-merge January 14, 2025 20:20

Taplo

e4f8c8d

skunert added this pull request to the merge queue Jan 14, 2025

Merged via the queue into master with commit d5539aa Jan 14, 2025
192 of 204 checks passed

skunert deleted the skunert/relay_slot_velocity branch January 14, 2025 23:20

skunert mentioned this pull request Jan 16, 2025

Collator: Fix can_build_upon by always allowing to build on included block #7205

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parachains: Use relay chain slot for velocity measurement #6825

Parachains: Use relay chain slot for velocity measurement #6825

skunert commented Dec 10, 2024

skunert commented Dec 10, 2024

command-bot bot commented Dec 10, 2024 •

edited

Loading

command-bot bot commented Dec 10, 2024

skunert commented Dec 11, 2024

command-bot bot commented Dec 11, 2024 •

edited

Loading

command-bot bot commented Dec 11, 2024

skunert commented Dec 13, 2024

alindima left a comment

alindima Dec 16, 2024

bkchr Dec 16, 2024

skunert Dec 18, 2024

alindima Dec 16, 2024

bkchr Dec 16, 2024

skunert commented Dec 18, 2024

This comment was marked as resolved.

This comment was marked as resolved.

skunert commented Dec 19, 2024

This comment was marked as resolved.

This comment was marked as resolved.

bkchr Jan 9, 2025

paritytech-workflow-stopper bot commented Jan 13, 2025

Parachains: Use relay chain slot for velocity measurement #6825

Parachains: Use relay chain slot for velocity measurement #6825

Conversation

skunert commented Dec 10, 2024

Changes

Migration

skunert commented Dec 10, 2024

command-bot bot commented Dec 10, 2024 • edited Loading

command-bot bot commented Dec 10, 2024

skunert commented Dec 11, 2024

command-bot bot commented Dec 11, 2024 • edited Loading

command-bot bot commented Dec 11, 2024

skunert commented Dec 13, 2024

alindima left a comment

Choose a reason for hiding this comment

alindima Dec 16, 2024

Choose a reason for hiding this comment

bkchr Dec 16, 2024

Choose a reason for hiding this comment

skunert Dec 18, 2024

Choose a reason for hiding this comment

alindima Dec 16, 2024

Choose a reason for hiding this comment

bkchr Dec 16, 2024

Choose a reason for hiding this comment

skunert commented Dec 18, 2024

This comment was marked as resolved.

This comment was marked as resolved.

skunert commented Dec 19, 2024

This comment was marked as resolved.

This comment was marked as resolved.

bkchr Jan 9, 2025

Choose a reason for hiding this comment

paritytech-workflow-stopper bot commented Jan 13, 2025

command-bot bot commented Dec 10, 2024 •

edited

Loading

command-bot bot commented Dec 11, 2024 •

edited

Loading