Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MBM try-runtime support #4251

Merged
merged 49 commits into from
Sep 25, 2024
Merged

MBM try-runtime support #4251

merged 49 commits into from
Sep 25, 2024

Conversation

liamaharon
Copy link
Contributor

@liamaharon liamaharon commented Apr 23, 2024

MBM try-runtime support

This MR adds support to the try-runtime trait such that the try-runtime-CLI will be able to support MBM testing here. It mainly adds two feature-gated hooks to the SteppedMigration hook to facilitate testing. These hooks are named pre_upgrade and post_upgrade and have the same signature and implications as for single-block migrations.

Integration

To make use of this in your Multi-Block-Migration, just implement the two new hooks and test pre- and post-conditions in them:

#[cfg(feature = "try-runtime")]
fn pre_upgrade() -> Result<Vec<u8>, frame_support::sp_runtime::TryRuntimeError> {
	// ...
}

#[cfg(feature = "try-runtime")]
fn post_upgrade(prev: Vec<u8>) -> Result<(), frame_support::sp_runtime::TryRuntimeError> {
    // ...
}

You may return an error or panic in these functions to indicate failure. This will then show up in the try-runtime-CLI and can be used in CI for testing.

Changes:

  • Adds try-runtime gated methods pre_upgrade and post_upgrade on SteppedMigration
  • Adds try-runtime gated methods nth_pre_upgrade and nth_post_upgrade on SteppedMigrations
  • Modifies pallet_migrations implementation to run pre_upgrade and post_upgrade steps at the appropriate times, and panic in the event of migration failure.

@liamaharon liamaharon added the T1-FRAME This PR/Issue is related to core FRAME, the framework. label Apr 23, 2024
@liamaharon liamaharon requested a review from ggwpez April 23, 2024 07:47
@bkchr
Copy link
Member

bkchr commented Apr 24, 2024

I remember that you asked somewhere which approach to take. I would take the approach of building multiple blocks until the MBMs are finished. I know that this is more complicated, but this directly checks that your runtime can handle this properly and also that you stay always in the POV limits.

(Didn't check the pr yet :D)

@liamaharon
Copy link
Contributor Author

I remember that you asked somewhere which approach to take. I would take the approach of building multiple blocks until the MBMs are finished. I know that this is more complicated, but this directly checks that your runtime can handle this properly and also that you stay always in the POV limits.

(Didn't check the pr yet :D)

Here's the discussion paritytech/try-runtime-cli#17 (comment)

I missed that Kian also supports building proper blocks. Although more involved, I will proceed with that approach :)

@liamaharon liamaharon marked this pull request as ready for review May 29, 2024 16:43
@liamaharon liamaharon requested a review from a team as a code owner May 29, 2024 16:43
@liamaharon liamaharon marked this pull request as draft May 29, 2024 16:44
Copy link
Contributor

@georgepisaltu georgepisaltu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -21,6 +21,7 @@ frame-benchmarking = { optional = true, workspace = true }
log = { workspace = true }
scale-info = { workspace = true }
sp-io = { workspace = true }
sp-std = { workspace = true }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be removed, right?

let bytes = T::Migrations::nth_pre_upgrade(cursor.index)
.expect("Invalid cursor.index")
.expect("Pre-upgrade failed");
PreUpgradeBytes::<T>::insert(&bounded_id, PreUpgradeBytesWrapper(bytes));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok, it's just try-runtime storage.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
georgepisaltu and others added 4 commits September 12, 2024 15:29
Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@ggwpez ggwpez enabled auto-merge September 16, 2024 20:13
@ggwpez ggwpez added this pull request to the merge queue Sep 24, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Sep 24, 2024
@ggwpez ggwpez enabled auto-merge September 24, 2024 20:51
@ggwpez ggwpez disabled auto-merge September 24, 2024 20:52
@ggwpez
Copy link
Member

ggwpez commented Sep 24, 2024

Waiting for #5649 to merge as to not block the merge queue.

@ggwpez ggwpez enabled auto-merge September 25, 2024 10:20
@ggwpez ggwpez disabled auto-merge September 25, 2024 10:20
@ggwpez ggwpez enabled auto-merge September 25, 2024 12:26
Copy link
Contributor

@alvicsam alvicsam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to move this check to github action. If this is not urgent I can do it in this PR, if it's urgent - I can do it later.

@ggwpez
Copy link
Member

ggwpez commented Sep 25, 2024

I am trying to get it in soon since i then want to finally merge this #5676 without any conflicts. I enabled auto-merge last week, but apparently nothing happened.

I will re-try it one more time and if that does not work then i guess we have to move the check?

@ggwpez ggwpez added this pull request to the merge queue Sep 25, 2024
Merged via the queue into master with commit cc6a513 Sep 25, 2024
221 checks passed
@ggwpez ggwpez deleted the liam-mbm-try-runtime branch September 25, 2024 16:50
ggwpez added a commit to paritytech/try-runtime-cli that referenced this pull request Oct 3, 2024
Draft PR opened for whoever will be taking this over.

Next TODO is investigate why MBMs don't seem to progress between blocks
being mined.

Should be used alongside the changes in this PR:
paritytech/polkadot-sdk#4251

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: claravanstaden <claravanstaden64@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants