Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

templates: make node compilation optional #5954

Merged
Merged
Show file tree
Hide file tree
Changes from 70 commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
29c9d09
templates: make node compilation optional
iulianbarbu Oct 7, 2024
0488023
don't add default-members for solochain
iulianbarbu Oct 9, 2024
a9fb4fe
remove extra spaces
iulianbarbu Oct 9, 2024
b20001c
added zombienet tests for omni node
iulianbarbu Oct 15, 2024
6a7946a
assert on the finality instead
iulianbarbu Oct 16, 2024
dae9b73
omni-node: remove in/out peers limitation
iulianbarbu Oct 17, 2024
5af79ea
Merge branch 'master' of github.com:paritytech/polkadot-sdk into ib-m…
iulianbarbu Oct 17, 2024
8e2450d
explained how to run minimal in multi-node setup
iulianbarbu Oct 21, 2024
b75bff2
Revert "omni-node: remove in/out peers limitation"
iulianbarbu Oct 21, 2024
e9ba647
polish minimal template docs
iulianbarbu Oct 21, 2024
67bc652
more polish to the docs
iulianbarbu Oct 21, 2024
f0232c4
add parachain id const
iulianbarbu Oct 21, 2024
73695bd
more docs polishing
iulianbarbu Oct 22, 2024
62c4a49
add prdoc
iulianbarbu Oct 22, 2024
868302d
Merge branch 'master' into ib-make-node-compilation-optional
iulianbarbu Oct 22, 2024
e1cbdfe
fix md checks
iulianbarbu Oct 22, 2024
3418a41
fix md to pass checker
iulianbarbu Oct 22, 2024
d02f136
remove line
iulianbarbu Oct 22, 2024
d0e7ee0
fix parachain id type
iulianbarbu Oct 22, 2024
94d6d0d
more polish to the READMEs
iulianbarbu Oct 22, 2024
16738e5
even more polish
iulianbarbu Oct 22, 2024
716b440
self review fixes
iulianbarbu Oct 22, 2024
adb3f92
fix clippy
iulianbarbu Oct 23, 2024
2ada6af
fix more clippy
iulianbarbu Oct 23, 2024
707e28c
more clippy fix
iulianbarbu Oct 23, 2024
a877784
finally fixed clippy
iulianbarbu Oct 23, 2024
245f7db
reorganizing sections and fixing parachain omni node run
iulianbarbu Oct 23, 2024
d51387a
some more fixes after self review
iulianbarbu Oct 23, 2024
aed2d66
small clarification
iulianbarbu Oct 23, 2024
6c5179e
fix markdown
iulianbarbu Oct 23, 2024
99bff3e
remove Cargo.toml comment
iulianbarbu Oct 23, 2024
ef9543d
Merge branch 'master' into ib-make-node-compilation-optional
iulianbarbu Oct 23, 2024
fe6b93f
remove todo for adding links
iulianbarbu Oct 23, 2024
bd8d547
resolve conflicts
iulianbarbu Oct 23, 2024
c091f84
small polish
iulianbarbu Oct 23, 2024
e8cbb68
added templates Cargo.toml comments
iulianbarbu Oct 24, 2024
3412c41
rlib is the default crate-type already
iulianbarbu Oct 24, 2024
6b17432
add high-level usage docs to omni-node crate
iulianbarbu Oct 24, 2024
c2cc3f6
polish omni-node README & add chain-spec-builder README
iulianbarbu Oct 24, 2024
0db0b59
fix capitalization
iulianbarbu Oct 24, 2024
ff56fc0
Update minimal README
iulianbarbu Oct 24, 2024
9dfce4f
deduplicate polkadot img tag
iulianbarbu Oct 24, 2024
69d213f
rever polkadot img tag removal
iulianbarbu Oct 25, 2024
3486884
refactored template zombienet tests
iulianbarbu Oct 25, 2024
58fd9bb
fix md and semvr
iulianbarbu Oct 25, 2024
cc4e036
fix fmt
iulianbarbu Oct 25, 2024
cc56e51
fix broken ToC links
iulianbarbu Oct 25, 2024
145e622
not quite
iulianbarbu Oct 25, 2024
d3b16ab
updated minimal & parachain README
iulianbarbu Oct 25, 2024
b74eacf
fix parachain template links
iulianbarbu Oct 25, 2024
9104370
fix typos
iulianbarbu Oct 25, 2024
ccad82f
chain spec builder README fixes
iulianbarbu Oct 29, 2024
8f4a0c1
make minimal single-node focused
iulianbarbu Oct 29, 2024
a01d114
include README as doc str
iulianbarbu Oct 29, 2024
24ec943
document chain-spec-builder and add doc tests too
iulianbarbu Oct 29, 2024
231802c
polish docs tests display in README.md
iulianbarbu Oct 30, 2024
37e43c0
made the links look better
iulianbarbu Oct 30, 2024
62f757c
improve polkadot-omni-node README
iulianbarbu Oct 31, 2024
e7dfb68
disable docs tests
iulianbarbu Oct 31, 2024
995af51
simplified tests after setting doctest to false
iulianbarbu Oct 31, 2024
5ff577b
more minimal README updates
iulianbarbu Oct 31, 2024
7326ee2
templates README installation sections updates
iulianbarbu Oct 31, 2024
922ae7f
adjusted README importing and compilation
iulianbarbu Oct 31, 2024
5b16b64
polish after michal review
iulianbarbu Oct 31, 2024
ef957f2
Merge branch 'master' into ib-make-node-compilation-optional
iulianbarbu Oct 31, 2024
f94402c
fixes after review
iulianbarbu Oct 31, 2024
e6c3ab4
fix generate readme
iulianbarbu Oct 31, 2024
ac3bba1
update wasm-builder docs
iulianbarbu Oct 31, 2024
6f9c200
compile markdown simplifications
iulianbarbu Nov 1, 2024
8b4884d
to remove: try clippy without cache
iulianbarbu Nov 1, 2024
d224db5
rever forklift but remove SKIP_WASM_BUILD
iulianbarbu Nov 1, 2024
0a79b74
remove forklift again
iulianbarbu Nov 1, 2024
51c1814
some more templates' READMEs fixes
iulianbarbu Nov 1, 2024
92dbdd8
update minimal template README
iulianbarbu Nov 1, 2024
fa71ef5
Merge branch 'master' into ib-make-node-compilation-optional
iulianbarbu Nov 1, 2024
46d0be7
fix clippy
iulianbarbu Nov 1, 2024
e6327ae
try one more time unsetting SKIP_WASM_BUILD
iulianbarbu Nov 1, 2024
28583e2
just disable forklift
iulianbarbu Nov 1, 2024
8ca2e65
temporary fix for doc tests disabling
iulianbarbu Nov 1, 2024
256d932
rever forklift
iulianbarbu Nov 1, 2024
8eed5a3
Merge branch 'master' into ib-make-node-compilation-optional
kianenigma Nov 4, 2024
4126edb
upgrade docify to 0.2.9
iulianbarbu Nov 4, 2024
9f4c95f
remove forklift based caching for clippy checks
iulianbarbu Nov 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ jobs:
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: script
run: |
forklift cargo clippy --all-targets --locked --workspace --quiet
forklift cargo clippy --all-targets --all-features --locked --workspace --quiet
cargo clippy --all-targets --locked --workspace --quiet
cargo clippy --all-targets --all-features --locked --workspace --quiet
check-try-runtime:
runs-on: ${{ needs.preflight.outputs.RUNNER }}
needs: [preflight]
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/misc-sync-templates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,12 @@ jobs:
homepage = "https://paritytech.github.io/polkadot-sdk/"

[workspace]
EOF
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EgorPopelyaev is this workflow called in the release process, or should we do it manually?

The person who made them has left, so a bit worried that it will go stale.

Copy link
Contributor Author

@iulianbarbu iulianbarbu Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It must be called manually: https://github.com/paritytech/polkadot-sdk/blob/master/.github/workflows/misc-sync-templates.yml#L18. I can do it after the release (or leave it to the release team if they handled it historically). We'll need to update this too: paritytech/polkadot-sdk-parachain-template#19.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created this #6329.


[ ${{ matrix.template }} != "solochain" ] && echo "# Leave out the node compilation from regular template usage." \
&& echo "\"default-members\" = [\"pallets/template\", \"runtime\"]" >> Cargo.toml
[ ${{ matrix.template }} == "solochain" ] && echo "# The node isn't yet replaceable by Omni Node."
cat << EOF >> Cargo.toml
members = [
"node",
"pallets/template",
Expand Down
139 changes: 113 additions & 26 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 11 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -556,7 +556,13 @@ default-members = [
[workspace.lints.rust]
suspicious_double_ref_op = { level = "allow", priority = 2 }
# `substrate_runtime` is a common `cfg` condition name used in the repo.
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(build_opt_level, values("3"))', 'cfg(build_profile, values("debug", "release"))', 'cfg(enable_alloc_error_handler)', 'cfg(fuzzing)', 'cfg(substrate_runtime)'] }
unexpected_cfgs = { level = "warn", check-cfg = [
'cfg(build_opt_level, values("3"))',
'cfg(build_profile, values("debug", "release"))',
'cfg(enable_alloc_error_handler)',
'cfg(fuzzing)',
'cfg(substrate_runtime)',
] }

[workspace.lints.clippy]
all = { level = "allow", priority = 0 }
Expand Down Expand Up @@ -677,6 +683,7 @@ cid = { version = "0.9.0" }
clap = { version = "4.5.13" }
clap-num = { version = "1.0.2" }
clap_complete = { version = "4.5.13" }
cmd_lib = { version = "1.9.5" }
coarsetime = { version = "0.1.22" }
codec = { version = "3.6.12", default-features = false, package = "parity-scale-codec" }
collectives-westend-emulated-chain = { path = "cumulus/parachains/integration-tests/emulated/chains/parachains/collectives/collectives-westend" }
Expand Down Expand Up @@ -1087,7 +1094,9 @@ polkavm-derive = "0.9.1"
polkavm-linker = "0.9.2"
portpicker = { version = "0.1.1" }
pretty_assertions = { version = "1.3.0" }
primitive-types = { version = "0.13.1", default-features = false, features = ["num-traits"] }
primitive-types = { version = "0.13.1", default-features = false, features = [
"num-traits",
] }
proc-macro-crate = { version = "3.0.0" }
proc-macro-warning = { version = "1.0.0", default-features = false }
proc-macro2 = { version = "1.0.86" }
Expand Down
65 changes: 65 additions & 0 deletions cumulus/polkadot-omni-node/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Polkadot Omni Node

This is a white labeled implementation based on [`polkadot-omni-node-lib`](https://crates.io/crates/polkadot-omni-node-lib).
It can be used to start a parachain node from a provided chain spec file. It is only compatible with runtimes that use block
number `u32` and `Aura` consensus.

## Installation

Download & expose it via `PATH`:

```bash
# Download and set it on PATH.
wget https://github.com/paritytech/polkadot-sdk/releases/download/<stable_release_tag>/polkadot-omni-node
chmod +x polkadot-omni-node
export PATH="$PATH:`pwd`"
```

Compile & install via `cargo`:

```bash
# Assuming ~/.cargo/bin is on the PATH
cargo install polkadot-omni-node
```

## Usage

A basic example for an Omni Node run starts from a runtime which implements the [`sp_genesis_builder::GenesisBuilder`](https://docs.rs/sp-genesis-builder/latest/sp_genesis_builder/trait.GenesisBuilder.html).
The interface mandates the runtime to expose a [`named-preset`](https://docs.rs/staging-chain-spec-builder/latest/staging_chain_spec_builder/#generate-chain-spec-using-runtime-provided-genesis-config-preset).

iulianbarbu marked this conversation as resolved.
Show resolved Hide resolved
### 1. Install chain-spec-builder

**Note**: `chain-spec-builder` binary is published on [`crates.io`](https://crates.io) under
[`staging-chain-spec-builder`](https://crates.io/crates/staging-chain-spec-builder) due to a name conflict.
Install it with `cargo` like bellow :

```bash
cargo install staging-chain-spec-builder
```

### 2. Generate a chain spec

Omni Node expects for the chain spec to contain parachains related fields like `relay_chain` and `para_id`.
These fields can be introduced by running [`staging-chain-spec-builder`](https://crates.io/crates/staging-chain-spec-builder)
with additional flags:

```bash
chain-spec-builder create --relay-chain <relay_chain_id> --para-id <id> -r <runtime.wasm> named-preset <preset_name>
```

### 3. Run Omni Node

And now with the generated chain spec we can start Omni Node like so:

```bash
polkadot-omni-node --chain <chain_spec.json>
```

## Useful links

* [`Omni Node Polkadot SDK Docs`](https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/reference_docs/omni_node/index.html)
* [`Chain Spec Genesis Reference Docs`](https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/reference_docs/chain_spec_genesis/index.html)
* [`polkadot-parachain-bin`](https://crates.io/crates/polkadot-parachain-bin)
* [`polkadot-sdk-parachain-template`](https://github.com/paritytech/polkadot-sdk-parachain-template)
* [`frame-omni-bencher`](https://crates.io/crates/frame-omni-bencher)
* [`staging-chain-spec-builder`](https://crates.io/crates/staging-chain-spec-builder)
Loading
Loading