Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

deprecate try-runtime subcommand and direct users to standalone cli #14731

Merged
merged 22 commits into from
Aug 17, 2023
Merged

deprecate try-runtime subcommand and direct users to standalone cli #14731

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
485c9e2
add deprecation notice
liamaharon Aug 8, 2023
9a979ea
remove try-runtime-cli tests
liamaharon Aug 8, 2023
575d475
update docs
liamaharon Aug 8, 2023
a2c6c68
add estimated removal date
liamaharon Aug 8, 2023
94476d1
deprecate and remove from node-cli and node-template
liamaharon Aug 11, 2023
c758a63
Merge branch 'master' of github.com:paritytech/substrate into liam-tr…
liamaharon Aug 11, 2023
6315af7
try fix build script
liamaharon Aug 11, 2023
b703338
update comment
liamaharon Aug 11, 2023
154dd35
fix link
liamaharon Aug 11, 2023
ad34a65
typo in build script
liamaharon Aug 11, 2023
5dbd215
Move `try-runtime-cli` install step outside of `check_dependent_proje…
rcny Aug 11, 2023
e27687a
Update scripts/ci/gitlab/pipeline/build.yml
liamaharon Aug 11, 2023
05c732d
remove chain arg
liamaharon Aug 11, 2023
909b0ee
build runtime with try-runtime feature
liamaharon Aug 11, 2023
ccc5ab7
kick ci
liamaharon Aug 11, 2023
43a1d79
kick ci
liamaharon Aug 11, 2023
33a7ee7
Merge branch 'master' of github.com:paritytech/substrate into liam-tr…
liamaharon Aug 16, 2023
cc95fed
use main branch
liamaharon Aug 16, 2023
9716275
specify sha in try-runtime-cli cargo install
liamaharon Aug 16, 2023
512b61d
kick ci
liamaharon Aug 16, 2023
185c000
kick ci
liamaharon Aug 16, 2023
b550ff4
Merge branch 'master' of github.com:paritytech/substrate into liam-tr…
liamaharon Aug 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
336 changes: 5 additions & 331 deletions utils/frame/try-runtime/cli/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,338 +17,10 @@

//! # Try-runtime
//!
//! Substrate's ultimate testing framework for the power users.
//! Substrate's `try-runtime` subcommand has been migrated to a [standalone
//! CLI](https://github.com/paritytech/try-runtime-cli).
//!
//! > As the name suggests, `try-runtime` is a detailed testing framework that gives you a lot of
//! control over what is being executed in which environment. It is recommended that user's first
//! familiarize themselves with substrate in depth, particularly the execution model. It is critical
//! to deeply understand how the wasm/client/runtime interactions, and the runtime apis work in the
//! substrate runtime, before commencing to working with `try-runtime`.
//!
//! #### Resources
//!
//! Some resources about the above:
//!
//! 1. <https://docs.substrate.io/reference/command-line-tools/try-runtime/>
//! 2. <https://www.crowdcast.io/e/substrate-seminar/41>
//! 3. <https://docs.substrate.io/fundamentals/runtime-development/>
//!
//! ---
//!
//! ## Background Knowledge
//!
//! The basis of all try-runtime commands is the same: connect to a live node, scrape its *state*
//! and put it inside a `TestExternalities`, then call into a *specific runtime-api* using the given
//! state and some *runtime*.
//!
//! Alternatively, the state could come from a snapshot file.
//!
//! All of the variables in the above statement are made *italic*. Let's look at each of them:
//!
//! 1. **State** is the key-value pairs of data that comprise the canonical information that any
//! blockchain is keeping. A state can be full (all key-value pairs), or be partial (only pairs
//! related to some pallets/prefixes). Moreover, some keys are especial and are not related to
//! specific pallets, known as [`well_known_keys`] in substrate. The most important of these is
//! the `:CODE:` key, which contains the code used for execution, when wasm execution is chosen.
//!
//! 2. *A runtime-api* call is a call into a function defined in the runtime, *on top of a given
//! state*. Each subcommand of `try-runtime` utilizes a specific *runtime-api*.
//!
//! 3. Finally, the **runtime** is the actual code that is used to execute the aforementioned
//! runtime-api. Everything in this crate assumes wasm execution, which means the runtime that
//! you use is the one stored onchain, namely under the `:CODE:` key.
//!
//! To recap, a typical try-runtime command does the following:
//!
//! 1. Download the state of a live chain, and write to an `externalities`.
//! 2. Overwrite the `:CODE:` with a given wasm blob
//! 3. Test some functionality via calling a runtime-api.
//!
//! ## Usage
//!
//! To use any of the provided commands, [`SharedParams`] must be provided. The most important of
//! which being [`SharedParams::runtime`], which specifies which runtime to use. Furthermore,
//! [`SharedParams::overwrite_state_version`] can be used to alter the state-version (see
//! <https://forum.polkadot.network/t/state-trie-migration/852> for more info).
//!
//! Then, the specific command has to be specified. See [`Command`] for more information about each
//! command's specific customization flags, and assumptions regarding the runtime being used.
//!
//! Said briefly, this CLI is capable of executing:
//!
//! * [`Command::OnRuntimeUpgrade`]: execute all the `on_runtime_upgrade` hooks.
//! * [`Command::ExecuteBlock`]: re-execute the given block.
//! * [`Command::OffchainWorker`]: re-execute the given block's offchain worker code path.
//! * [`Command::FollowChain`]: continuously execute the blocks of a remote chain on top of a given
//! runtime.
//! * [`Command::CreateSnapshot`]: Create a snapshot file from a remote node.
//!
//! Finally, To make sure there are no errors regarding this, always run any `try-runtime` command
//! with `executor=trace` logging targets, which will specify which runtime is being used per api
//! call. Moreover, `remote-ext`, `try-runtime` and `runtime` logs targets will also be useful.
//!
//! ## Spec name check
//!
//! A common pitfall is that you might be running some test on top of the state of chain `x`, with
//! the runtime of chain `y`. To avoid this all commands do a spec-name check before executing
//! anything by default. This will check the, if any alterations are being made to the `:CODE:`,
//! then the spec names match. The spec versions are warned, but are not mandated to match.
//!
//! > If anything, in most cases, we expect spec-versions to NOT match, because try-runtime is all
//! > about testing unreleased runtimes.
//!
//! ## Note on signature and state-root checks
//!
//! All of the commands calling into `TryRuntime_execute_block` ([`Command::ExecuteBlock`] and
//! [`Command::FollowChain`]) disable both state root and signature checks. This is because in 99%
//! of the cases, the runtime that is being tested is different from the one that is stored in the
//! canonical chain state. This implies:
//!
//! 1. the state root will NEVER match, because `:CODE:` is different between the two.
//! 2. replaying all transactions will fail, because the spec-version is part of the transaction
//! signature.
//!
//! ## Best Practices
//!
//! Try-runtime is all about battle-testing unreleased runtime. The following list of suggestions
//! help developers maximize the testing coverage and make base use of `try-runtime`.
//!
//! #### Adding pre/post hooks
//!
//! One of the gems that come only in the `try-runtime` feature flag is the `pre_upgrade` and
//! `post_upgrade` hooks for `OnRuntimeUpgrade`. This trait is implemented either inside the pallet,
//! or manually in a runtime, to define a migration. In both cases, these functions can be added,
//! given the right flag:
//!
//! ```ignore
//! #[cfg(feature = "try-runtime")]
//! fn pre_upgrade() -> Result<Vec<u8>, TryRuntimeError> {}
//!
//! #[cfg(feature = "try-runtime")]
//! fn post_upgrade(state: Vec<u8>) -> Result<(), TryRuntimeError> {}
//! ```
//!
//! (The pallet macro syntax will support this simply as a part of `#[pallet::hooks]`).
//!
//! These hooks allow you to execute some code, only within the `on-runtime-upgrade` command, before
//! and after the migration. Moreover, `pre_upgrade` can return a `Vec<u8>` that contains arbitrary
//! encoded data (usually some pre-upgrade state) which will be passed to `post_upgrade` after
//! upgrading and used for post checking.
//!
//! ## State Consistency
//!
//! Similarly, each pallet can expose a function in `#[pallet::hooks]` section as follows:
//!
//! ```ignore
//! #[cfg(feature = "try-runtime")]
//! fn try_state(_: BlockNumber) -> Result<(), TryRuntimeError> {}
//! ```
//!
//! which is called on numerous code paths in the try-runtime tool. These checks should ensure that
//! the state of the pallet is consistent and correct. See `frame_support::try_runtime::TryState`
//! for more info.
//!
//! #### Logging
//!
//! It is super helpful to make sure your migration code uses logging (always with a `runtime` log
//! target prefix, e.g. `runtime::balance`) and state exactly at which stage it is, and what it is
//! doing.
//!
//! #### Guarding migrations
//!
//! Always make sure that any migration code is guarded either by `StorageVersion`, or by some
//! custom storage item, so that it is NEVER executed twice, even if the code lives in two
//! consecutive runtimes.
//!
//! ## Examples
//!
//! For the following examples, we assume the existence of the following:
//!
//! 1. a substrate node compiled without `--feature try-runtime`, called `substrate`. This will be
//! the running node that you connect to. then, after some changes to this node, you compile it with
//! `--features try-runtime`. This gives you:
//! 2. a substrate binary that has the try-runtime sub-command enabled.
//! 3. a wasm blob that has try-runtime functionality.
//!
//! ```bash
//! # this is like your running deployed node.
//! cargo build --release && cp target/release/substrate .
//!
//! # this is like your WIP branch.
//! cargo build --release --features try-runtime
//! cp target/release/substrate substrate-try-runtime
//! cp ./target/release/wbuild/kitchensink-runtime/kitchensink_runtime.wasm runtime-try-runtime.wasm
//! ```
//!
//! > The above example is with `substrate`'s `kitchensink-runtime`, but is applicable to any
//! > substrate-based chain that has implemented `try-runtime-cli`.
//!
//! * If you run `try-runtime` subcommand against `substrate` binary listed above, you get the
//! following error.
//!
//! ```bash
//! [substrate] ./substrate try-runtime
//! Error: Input("TryRuntime wasn't enabled when building the node. You can enable it with `--features try-runtime`.")
//! ```
//!
//! * If you run the same against `substrate-try-runtime`, it will work.
//!
//! ```bash
//! [substrate] ./substrate-try-runtime try-runtime
//! Try some command against runtime state
//!
//! Usage: substrate-try-runtime try-runtime [OPTIONS] --runtime <RUNTIME> <COMMAND>
//!
//! Commands:
//! on-runtime-upgrade Execute the migrations of the "local runtime"
//! execute-block Executes the given block against some state
//! offchain-worker Executes *the offchain worker hooks* of a given block against some state
//! follow-chain Follow the given chain's finalized blocks and apply all of its extrinsics
//! create-snapshot Create a new snapshot file
//! help Print this message or the help of the given subcommand(s)
//!
//! Options:
//! --chain <CHAIN_SPEC>
//! Specify the chain specification
//! --dev
//! Specify the development chain
//! -d, --base-path <PATH>
//! Specify custom base path
//! -l, --log <LOG_PATTERN>...
//! Sets a custom logging filter. Syntax is `<target>=<level>`, e.g. -lsync=debug
//! --detailed-log-output
//! Enable detailed log output
//! --disable-log-color
//! Disable log color output
//! --enable-log-reloading
//! Enable feature to dynamically update and reload the log filter
//! --tracing-targets <TARGETS>
//! Sets a custom profiling filter. Syntax is the same as for logging: `<target>=<level>`
//! --tracing-receiver <RECEIVER>
//! Receiver to process tracing messages [default: log] [possible values: log]
//! --runtime <RUNTIME>
//! The runtime to use
//! --wasm-execution <METHOD>
//! Type of wasm execution used [default: compiled] [possible values: interpreted-i-know-what-i-do, compiled]
//! --wasm-instantiation-strategy <STRATEGY>
//! The WASM instantiation method to use [default: pooling-copy-on-write] [possible values: pooling-copy-on-write, recreate-instance-copy-on-write, pooling, recreate-instance, legacy-instance-reuse]
//! --heap-pages <HEAP_PAGES>
//! The number of 64KB pages to allocate for Wasm execution. Defaults to [`sc_service::Configuration.default_heap_pages`]
//! --overwrite-state-version <OVERWRITE_STATE_VERSION>
//! Overwrite the `state_version`
//! -h, --help
//! Print help information (use `--help` for more detail)
//! -V, --version
//! Print version information
//! ```
//!
//! * Run the migrations of a given runtime on top of a live state.
//!
//! ```bash
//! # assuming there's `./substrate --dev --tmp --ws-port 9999` or similar running.
//! ./substrate-try-runtime \
//! try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! on-runtime-upgrade \
//! live --uri ws://localhost:9999
//! ```
//!
//! * Same as the previous one, but run it at specific block number's state. This means that this
//! block hash's state shall not yet have been pruned in `rpc.polkadot.io`.
//!
//! ```bash
//! ./substrate-try-runtime \
//! try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! on-runtime-upgrade \
//! live --uri ws://localhost:9999 \
//! # replace with your desired block hash!
//! --at 0xa1b16c1efd889a9f17375ec4dd5c1b4351a2be17fa069564fced10d23b9b3836
//! ```
//!
//! * Executing the same command with the [`Runtime::Existing`] will fail because the existing
//! runtime, stored onchain in `substrate` binary that we compiled earlier does not have
//! `try-runtime` feature!
//!
//! ```bash
//! ./substrate-try-runtime try-runtime --runtime existing -lruntime=debug on-runtime-upgrade live --uri ws://localhost:9999
//! ...
//! Error: Input("given runtime is NOT compiled with try-runtime feature!")
//! ```
//!
//! * Now, let's use a snapshot file. First, we create the snapshot:
//!
//! ```bash
//! ./substrate-try-runtime try-runtime --runtime existing -lruntime=debug create-snapshot --uri ws://localhost:9999
//! 2022-12-13 10:28:17.516 INFO main try-runtime::cli: snapshot path not provided (-s), using 'node-268@latest.snap'
//! 2022-12-13 10:28:17.516 INFO main remote-ext: since no at is provided, setting it to latest finalized head, 0xe7d0b614dfe89af65b33577aae46a6f958c974bf52f8a5e865a0f4faeb578d22
//! 2022-12-13 10:28:17.516 INFO main remote-ext: since no prefix is filtered, the data for all pallets will be downloaded
//! 2022-12-13 10:28:17.550 INFO main remote-ext: writing snapshot of 1611464 bytes to "node-268@latest.snap"
//! 2022-12-13 10:28:17.551 INFO main remote-ext: initialized state externalities with storage root 0x925e4e95de4c08474fb7f976c4472fa9b8a1091619cd7820a793bf796ee6d932 and state_version V1
//! ```
//!
//! > Note that the snapshot contains the `existing` runtime, which does not have the correct
//! > `try-runtime` feature. In the following commands, we still need to overwrite the runtime.
//!
//! Then, we can use it to have the same command as before, `on-runtime-upgrade`
//!
//! ```bash
//! ./substrate-try-runtime try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! on-runtime-upgrade \
//! snap -s node-268@latest.snap
//! ```
//!
//! * Execute the latest finalized block with the given runtime.
//!
//! ```bash
//! ./substrate-try-runtime try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! execute-block live \
//! --uri ws://localhost:9999
//! ```
//!
//! This can still be customized at a given block with `--at`. If you want to use a snapshot, you
//! can still use `--block-ws-uri` to provide a node form which the block data can be fetched.
//!
//! Moreover, this runs the `frame_support::try_runtime::TryState` hooks as well. The hooks to run
//! can be customized with the `--try-state`. For example:
//!
//! ```bash
//! ./substrate-try-runtime try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! execute-block \
//! --try-state System,Staking \
//! live \
//! --uri ws://localhost:9999 \
//! --pallet System Staking
//! ```
//!
//! Will only run the `try-state` of the two given pallets. When running `try-state` against
//! some real chain data it can take a long time for the command to execute since it has to
//! query all the key-value pairs. In scenarios like above where we only want to run the
//! `try-state` for some specific pallets, we can use the `--pallet` option to specify from
//! which pallets we want to query the state. This will greatly decrease the execution time.
//!
//! See [`frame_try_runtime::TryStateSelect`] for more information.
//!
//! * Follow our live chain's blocks using `follow-chain`, whilst running the try-state of 3 pallets
//! in a round robin fashion
//!
//! ```bash
//! ./substrate-try-runtime \
//! try-runtime \
//! --runtime runtime-try-runtime.wasm \
//! -lruntime=debug \
//! follow-chain \
//! --uri ws://localhost:9999 \
//! --try-state rr-3
//! ```
//! It is no longer maintained here and will be removed in the future.

#![cfg(feature = "try-runtime")]

Expand Down Expand Up @@ -737,6 +409,8 @@ impl TryRuntimeCmd {
HostFns: HostFunctions,
BBIP: BlockBuildingInfoProvider<Block, Option<(InherentData, Digest)>>,
{
log::warn!("❗❗❗❗❗IMPORTANT: DEPRECATION NOTICE❗❗❗❗❗");
liamaharon marked this conversation as resolved.
Show resolved Hide resolved
log::warn!("Substrate's `try-runtime` subcommand has been migrated to a standalone CLI: https://github.com/paritytech/try-runtime-cli. It is no longer being maintained here and will be removed in the future. Please use the new standalone CLI.");
liamaharon marked this conversation as resolved.
Show resolved Hide resolved
match &self.command {
Command::OnRuntimeUpgrade(ref cmd) =>
commands::on_runtime_upgrade::on_runtime_upgrade::<Block, HostFns>(
Expand Down
Loading