Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HRMP - set DefaultChannelSizeAndCapacityWithSystem with dynamic values according to the ActiveConfig #4332

Merged
merged 4 commits into from
May 1, 2024

Conversation

bkontur
Copy link
Contributor

@bkontur bkontur commented Apr 30, 2024

Summary

This PR enhances the capability to set DefaultChannelSizeAndCapacityWithSystem for HRMP. Currently, all testnets (Rococo, Westend) have a hard-coded value set as 'half of the maximum' determined by the live ActiveConfig. While this approach appears satisfactory, potential issues could arise if the live ActiveConfig are adjusted below these hard-coded values, necessitating a new runtime release with updated values. Additionally, hard-coded values have consequences, such as Rococo's benchmarks not functioning: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6082656.

Solution

The proposed solution here is to utilize ActiveConfigHrmpChannelSizeAndCapacityRatio, which reads the current ActiveConfig and calculates DefaultChannelSizeAndCapacityWithSystem, for example, "half of the maximum" based on live data. This way, whenever ActiveConfig is modified, ActiveConfigHrmpChannelSizeAndCapacityRatio automatically returns adjusted values with the appropriate ratio. Thus, manual adjustments and new runtime releases become unnecessary.

Relates to a comment/discussion: https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1541001420
Relates to a comment/discussion: https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1549291588

@bkontur bkontur added R0-silent Changes should not be mentioned in any release notes T8-polkadot This PR/Issue is related to/affects the Polkadot network. labels Apr 30, 2024
@bkontur
Copy link
Contributor Author

bkontur commented Apr 30, 2024

@xlc wdyt?

@bkontur
Copy link
Contributor Author

bkontur commented Apr 30, 2024

bot bench polkadot-pallet --runtime=rococo --pallet=runtime_parachains::hrmp
bot bench polkadot-pallet --runtime=westend --pallet=runtime_parachains::hrmp

@xlc
Copy link
Contributor

xlc commented Apr 30, 2024

lgtm

bkontur and others added 3 commits April 30, 2024 13:05
…=westend --target_dir=polkadot --pallet=runtime_parachains::hrmp
…=rococo --target_dir=polkadot --pallet=runtime_parachains::hrmp
@bkontur
Copy link
Contributor Author

bkontur commented Apr 30, 2024

bot clean

@bkontur bkontur added this pull request to the merge queue May 1, 2024
Merged via the queue into master with commit e5a93fb May 1, 2024
142 of 144 checks passed
@bkontur bkontur deleted the bko-hrmp-dynamic-default-sizes branch May 1, 2024 20:25
dcolley added a commit to metaspan/polkadot-sdk that referenced this pull request May 6, 2024
* 'master' of https://github.com/metaspan/polkadot-sdk: (65 commits)
  Introduces `TypeWithDefault<T, D: Get<T>>` (paritytech#4034)
  Publish `polkadot-sdk-frame`  crate (paritytech#4370)
  Add validate field to prdoc (paritytech#4368)
  State trie migration on asset-hub westend and collectives westend (paritytech#4185)
  Fix: dust unbonded for zero existential deposit (paritytech#4364)
  Bridge: added subcommand to relay single parachain header (paritytech#4365)
  Bridge: fix zombienet tests (paritytech#4367)
  [WIP][CI] Add more GHA jobs (paritytech#4270)
  Allow for 0 existential deposit in benchmarks for `pallet_staking`, `pallet_session`, and `pallet_balances` (paritytech#4346)
  Deprecate `NativeElseWasmExecutor` (paritytech#4329)
  More `xcm::v4` cleanup and `xcm_fee_payment_runtime_api::XcmPaymentApi` nits (paritytech#4355)
  sc-tracing: enable env-filter feature (paritytech#4357)
  deps: update jsonrpsee to v0.22.5 (paritytech#4330)
  Add PoV-reclaim enablement guide to polkadot-sdk-docs (paritytech#4244)
  cargo: Update experimental litep2p to latest version (paritytech#4344)
  Bridge: ignore client errors when calling recently added `*_free_headers_interval` methods (paritytech#4350)
  Make parachain template great again (and async backing ready) (paritytech#4295)
  [Backport] Version bumps and reorg prdocs from 1.11.0 (paritytech#4336)
  HRMP - set `DefaultChannelSizeAndCapacityWithSystem` with dynamic values according to the `ActiveConfig` (paritytech#4332)
  Statement Distribution Per Peer Rate Limit (paritytech#3444)
  ...
github-merge-queue bot pushed a commit that referenced this pull request May 7, 2024
Closes: #4003 (please
see for the problem description)

## TODO
- [x] add more tests covering `WrapVersion` corner cases (e.g. para has
lower version, ...)
- [x] regenerate benchmarks `runtime_parachains::hrmp` (fix for Rococo
is here: #4332)

## Questions / possible improvements
- [ ] A `WrapVersion` implementation for `pallet_xcm` initiates version
discovery with
[note_unknown_version](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L2527C5-L2527C25),
there is possibility to avoid this overhead in this HRMP case to create
new `WrapVersion` adapter for `pallet_xcm` which would not use
`note_unknown_version`. Is it worth to do it or not?
- [ ] There's a possibility to decouple XCM functionality from the HRMP
pallet, allowing any relay chain to generate its own notifications. This
approach wouldn't restrict notifications solely to the XCM. However,
it's uncertain whether it's worthwhile or desirable to do so? It means
making HRMP pallet more generic. E.g. hiding HRMP notifications behind
some trait:
	```
	trait HrmpNotifications {

		fn on_channel_open_request(
			sender: ParaId,
			proposed_max_capacity: u32,
			proposed_max_message_size: u32) -> primitives::DownwardMessage;

fn on_channel_accepted(recipient: ParaId) ->
primitives::DownwardMessage;

fn on_channel_closing(initiator: ParaId, sender: ParaId, recipient:
ParaId) -> primitives::DownwardMessage;
	}
	```
and then we could have whatever adapter, `impl HrmpNotifications for
VersionedXcmHrmpNotifications {...}`,
	```
	impl parachains_hrmp::Config for Runtime {
	..
		type HrmpNotifications = VersionedXcmHrmpNotifications;
	..
	}
	```

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
bkontur added a commit that referenced this pull request May 27, 2024
…ues according to the `ActiveConfig` (#4332)

This PR enhances the capability to set
`DefaultChannelSizeAndCapacityWithSystem` for HRMP. Currently, all
testnets (Rococo, Westend) have a hard-coded value set as 'half of the
maximum' determined by the live `ActiveConfig`. While this approach
appears satisfactory, potential issues could arise if the live
`ActiveConfig` are adjusted below these hard-coded values, necessitating
a new runtime release with updated values. Additionally, hard-coded
values have consequences, such as Rococo's benchmarks not functioning:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6082656.

The proposed solution here is to utilize
`ActiveConfigHrmpChannelSizeAndCapacityRatio`, which reads the current
`ActiveConfig` and calculates `DefaultChannelSizeAndCapacityWithSystem`,
for example, "half of the maximum" based on live data. This way,
whenever `ActiveConfig` is modified,
`ActiveConfigHrmpChannelSizeAndCapacityRatio` automatically returns
adjusted values with the appropriate ratio. Thus, manual adjustments and
new runtime releases become unnecessary.

Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1541001420
Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1549291588

---------

Co-authored-by: command-bot <>
EgorPopelyaev pushed a commit that referenced this pull request May 27, 2024
…ues according to the `ActiveConfig` (#4332)

This PR enhances the capability to set
`DefaultChannelSizeAndCapacityWithSystem` for HRMP. Currently, all
testnets (Rococo, Westend) have a hard-coded value set as 'half of the
maximum' determined by the live `ActiveConfig`. While this approach
appears satisfactory, potential issues could arise if the live
`ActiveConfig` are adjusted below these hard-coded values, necessitating
a new runtime release with updated values. Additionally, hard-coded
values have consequences, such as Rococo's benchmarks not functioning:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6082656.

The proposed solution here is to utilize
`ActiveConfigHrmpChannelSizeAndCapacityRatio`, which reads the current
`ActiveConfig` and calculates `DefaultChannelSizeAndCapacityWithSystem`,
for example, "half of the maximum" based on live data. This way,
whenever `ActiveConfig` is modified,
`ActiveConfigHrmpChannelSizeAndCapacityRatio` automatically returns
adjusted values with the appropriate ratio. Thus, manual adjustments and
new runtime releases become unnecessary.

Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1541001420
Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1549291588

---------

Co-authored-by: command-bot <>
fellowship-merge-bot bot pushed a commit to polkadot-fellows/runtimes that referenced this pull request May 29, 2024
Allows any parachain to have bidirectional channel with any system
parachains.

Relates to: polkadot-fellows/RFCs#82
Relates to: paritytech/polkadot-sdk#3721
Relates to: paritytech/polkadot-sdk#4332

<!-- Remember that you can run `/merge` to enable auto-merge in the PR
-->

<!-- Remember to modify the changelog. If you don't need to modify it,
you can check the following box.
Instead, if you have already modified it, simply delete the following
line. -->

- [x] regenerate weights for `runtime_parachains::hrmp`
- [ ] Does not require a CHANGELOG entry

## Polkadot weights
(the threshold moved from the default 5 to 0.1 to see some differences)
```
subweight compare commits          --path-pattern "./relay/polkadot/**/weights/**/*.rs"          --format markdown --no-color           --change added changed          --method asymptotic --ignore-errors --threshold 0.1          remotes/polkadot-fellows/main          origin/bko-hrmp-patch
```
| File | Extrinsic | Old | New | Change [%] |

|-------------------------------------------------------|-------------------------------|----------|----------|------------|
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
poke_channel_deposits | 137.20us | 140.03us | +2.06 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
hrmp_accept_open_channel | 555.89us | 563.03us | +1.28 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
hrmp_close_channel | 556.97us | 563.79us | +1.22 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
hrmp_init_open_channel | 732.03us | 740.10us | +1.10 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
force_open_hrmp_channel | 1.16ms | 1.17ms | +0.98 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
establish_system_channel | 1.15ms | 1.16ms | +0.69 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
force_process_hrmp_open | 102.05ms | 102.47ms | +0.42 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
force_clean_hrmp | 91.52ms | 91.86ms | +0.37 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
clean_open_channel_requests | 16.56ms | 16.61ms | +0.35 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
force_process_hrmp_close | 75.41ms | 75.65ms | +0.33 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
hrmp_cancel_open_request | 422.12us | 415.03us | -1.68 |
| relay/polkadot/src/weights/runtime_parachains_hrmp.rs |
establish_channel_with_system | | 1.67ms | Added |

## Kusama weights
(the threshold moved from the default 5 to 0.1 to see some differences)
```
subweight compare commits          --path-pattern "./relay/kusama/**/weights/**/*.rs"          --format markdown --no-color           --change added changed          --method asymptotic --ignore-errors --threshold 0.1          remotes/polkadot-fellows/main          origin/bko-hrmp-patch
```
| File | Extrinsic | Old | New | Change [%] |

|-----------------------------------------------------|-------------------------------|----------|----------|------------|
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
poke_channel_deposits | 137.23us | 140.25us | +2.20 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
hrmp_close_channel | 557.39us | 564.18us | +1.22 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
hrmp_accept_open_channel | 556.49us | 563.23us | +1.21 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
hrmp_init_open_channel | 735.26us | 743.59us | +1.13 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
force_open_hrmp_channel | 1.16ms | 1.17ms | +0.97 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
establish_system_channel | 1.15ms | 1.16ms | +0.84 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
hrmp_cancel_open_request | 413.79us | 416.03us | +0.54 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
force_process_hrmp_open | 101.96ms | 102.43ms | +0.46 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
clean_open_channel_requests | 16.54ms | 16.61ms | +0.45 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs | force_clean_hrmp
| 91.43ms | 91.82ms | +0.43 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
force_process_hrmp_close | 75.34ms | 75.63ms | +0.40 |
| relay/kusama/src/weights/runtime_parachains_hrmp.rs |
establish_channel_with_system | | 1.67ms | Added |
hitchhooker pushed a commit to ibp-network/polkadot-sdk that referenced this pull request Jun 5, 2024
Closes: paritytech#4003 (please
see for the problem description)

## TODO
- [x] add more tests covering `WrapVersion` corner cases (e.g. para has
lower version, ...)
- [x] regenerate benchmarks `runtime_parachains::hrmp` (fix for Rococo
is here: paritytech#4332)

## Questions / possible improvements
- [ ] A `WrapVersion` implementation for `pallet_xcm` initiates version
discovery with
[note_unknown_version](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L2527C5-L2527C25),
there is possibility to avoid this overhead in this HRMP case to create
new `WrapVersion` adapter for `pallet_xcm` which would not use
`note_unknown_version`. Is it worth to do it or not?
- [ ] There's a possibility to decouple XCM functionality from the HRMP
pallet, allowing any relay chain to generate its own notifications. This
approach wouldn't restrict notifications solely to the XCM. However,
it's uncertain whether it's worthwhile or desirable to do so? It means
making HRMP pallet more generic. E.g. hiding HRMP notifications behind
some trait:
	```
	trait HrmpNotifications {

		fn on_channel_open_request(
			sender: ParaId,
			proposed_max_capacity: u32,
			proposed_max_message_size: u32) -> primitives::DownwardMessage;

fn on_channel_accepted(recipient: ParaId) ->
primitives::DownwardMessage;

fn on_channel_closing(initiator: ParaId, sender: ParaId, recipient:
ParaId) -> primitives::DownwardMessage;
	}
	```
and then we could have whatever adapter, `impl HrmpNotifications for
VersionedXcmHrmpNotifications {...}`,
	```
	impl parachains_hrmp::Config for Runtime {
	..
		type HrmpNotifications = VersionedXcmHrmpNotifications;
	..
	}
	```

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
…ues according to the `ActiveConfig` (paritytech#4332)

## Summary
This PR enhances the capability to set
`DefaultChannelSizeAndCapacityWithSystem` for HRMP. Currently, all
testnets (Rococo, Westend) have a hard-coded value set as 'half of the
maximum' determined by the live `ActiveConfig`. While this approach
appears satisfactory, potential issues could arise if the live
`ActiveConfig` are adjusted below these hard-coded values, necessitating
a new runtime release with updated values. Additionally, hard-coded
values have consequences, such as Rococo's benchmarks not functioning:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6082656.

## Solution
The proposed solution here is to utilize
`ActiveConfigHrmpChannelSizeAndCapacityRatio`, which reads the current
`ActiveConfig` and calculates `DefaultChannelSizeAndCapacityWithSystem`,
for example, "half of the maximum" based on live data. This way,
whenever `ActiveConfig` is modified,
`ActiveConfigHrmpChannelSizeAndCapacityRatio` automatically returns
adjusted values with the appropriate ratio. Thus, manual adjustments and
new runtime releases become unnecessary.


Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1541001420
Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1549291588

---------

Co-authored-by: command-bot <>
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
Closes: paritytech#4003 (please
see for the problem description)

## TODO
- [x] add more tests covering `WrapVersion` corner cases (e.g. para has
lower version, ...)
- [x] regenerate benchmarks `runtime_parachains::hrmp` (fix for Rococo
is here: paritytech#4332)

## Questions / possible improvements
- [ ] A `WrapVersion` implementation for `pallet_xcm` initiates version
discovery with
[note_unknown_version](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L2527C5-L2527C25),
there is possibility to avoid this overhead in this HRMP case to create
new `WrapVersion` adapter for `pallet_xcm` which would not use
`note_unknown_version`. Is it worth to do it or not?
- [ ] There's a possibility to decouple XCM functionality from the HRMP
pallet, allowing any relay chain to generate its own notifications. This
approach wouldn't restrict notifications solely to the XCM. However,
it's uncertain whether it's worthwhile or desirable to do so? It means
making HRMP pallet more generic. E.g. hiding HRMP notifications behind
some trait:
	```
	trait HrmpNotifications {

		fn on_channel_open_request(
			sender: ParaId,
			proposed_max_capacity: u32,
			proposed_max_message_size: u32) -> primitives::DownwardMessage;

fn on_channel_accepted(recipient: ParaId) ->
primitives::DownwardMessage;

fn on_channel_closing(initiator: ParaId, sender: ParaId, recipient:
ParaId) -> primitives::DownwardMessage;
	}
	```
and then we could have whatever adapter, `impl HrmpNotifications for
VersionedXcmHrmpNotifications {...}`,
	```
	impl parachains_hrmp::Config for Runtime {
	..
		type HrmpNotifications = VersionedXcmHrmpNotifications;
	..
	}
	```

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
R0-silent Changes should not be mentioned in any release notes T8-polkadot This PR/Issue is related to/affects the Polkadot network.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants