lightclient: Add support for multi-chain usecase (#1238)

* lightclient: Make `smoldot::chainID` part of the RPC requests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Make `BackgroundTask` generic over `PlatformRef` and chain

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Construct from raw smoldot and target different chains

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Update cargo lock for wasm tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Reuse `new_from_client` method and removed unused imports

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Reexport smoldot client and RPC objects used in pub
interface

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Adjust `new_from_client` interface

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Extend background to poll over multiple RPC objects

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Build light client from raw and target different chains

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Add demo chain specs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Move artifacts to dedicated folder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Use SelectAll to drive all streams

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Fetch initial data from the target chain

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Reexport other smoldot objects

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Target chain with potentially different config

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt/rpc: Log chainID for debugging

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt/examples: Add smoldot client with parachain example

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Propagate chain ID together with rpc responses object

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Multiplex responses by request ID and chain ID

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Add raw light client builder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Add cargo feature flag for parachains example

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Derive default for internal structure

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Guard reexports by std feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update subxt/src/client/light_client/mod.rs

Co-authored-by: James Wilson <james@jsdw.me>

* lightclient: Update the builder pattern and chain targetting

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Fix documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Provide more insightful docs wrt native/wasm panics

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Adjust comment location

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Refactor UniqueChainId into the background task

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update lightclient/src/background.rs

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Update subxt/src/client/light_client/builder.rs

Co-authored-by: James Wilson <james@jsdw.me>

* lightclient: Update docs wrt panics

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Update docs wrt to smoldot instance -> client

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Use IntoIter instead of Iterator

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Adjsut docs wrt [`Self::new_from_client`]

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Remove RawRpc from LightClient in favor of chainID

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* lightclient: Reexport everything under smoldot module

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Use stateRootHash instead of genesis.raw

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

artifacts/demo_chain_specs/polkadot_asset_hub.json

@@ -0,0 +1,45 @@
+{
+    "name": "Polkadot Asset Hub",
+    "id": "asset-hub-polkadot",
+    "chainType": "Live",
+    "bootNodes": [
+      "/ip4/34.65.251.121/tcp/30334/p2p/12D3KooWG3GrM6XKMM4gp3cvemdwUvu96ziYoJmqmetLZBXE8bSa",
+      "/ip4/34.65.35.228/tcp/30334/p2p/12D3KooWMRyTLrCEPcAQD6c4EnudL3vVzg9zji3whvsMYPUYevpq",
+      "/dns/polkadot-asset-hub-connect-0.polkadot.io/tcp/30334/p2p/12D3KooWLHqbcQtoBygf7GJgVjVa3TaeLuf7VbicNdooaCmQM2JZ",
+      "/dns/polkadot-asset-hub-connect-0.polkadot.io/tcp/443/wss/p2p/12D3KooWLHqbcQtoBygf7GJgVjVa3TaeLuf7VbicNdooaCmQM2JZ",
+      "/dns/polkadot-asset-hub-connect-1.polkadot.io/tcp/30334/p2p/12D3KooWNDrKSayoZXGGE2dRSFW2g1iGPq3fTZE2U39ma9yZGKd3",
+      "/dns/polkadot-asset-hub-connect-1.polkadot.io/tcp/443/wss/p2p/12D3KooWNDrKSayoZXGGE2dRSFW2g1iGPq3fTZE2U39ma9yZGKd3",
+      "/dns/polkadot-asset-hub-connect-2.polkadot.io/tcp/30334/p2p/12D3KooWApa2JW4rbLtgzuK7fjLMupLS9HZheX9cdkQKyu6AnGrP",
+      "/dns/polkadot-asset-hub-connect-2.polkadot.io/tcp/443/wss/p2p/12D3KooWApa2JW4rbLtgzuK7fjLMupLS9HZheX9cdkQKyu6AnGrP",
+      "/dns/polkadot-asset-hub-connect-3.polkadot.io/tcp/30334/p2p/12D3KooWRsVeHqRs2iKmjLiguxp8myL4G2mDAWhtX2jHwyWujseV",
+      "/dns/polkadot-asset-hub-connect-3.polkadot.io/tcp/443/wss/p2p/12D3KooWRsVeHqRs2iKmjLiguxp8myL4G2mDAWhtX2jHwyWujseV",
+      "/dns/boot.stake.plus/tcp/35333/p2p/12D3KooWFrQjYaPZSSLLxEVmoaHFcrF6VoY4awG4KRSLaqy3JCdQ",
+      "/dns/boot.stake.plus/tcp/35334/wss/p2p/12D3KooWFrQjYaPZSSLLxEVmoaHFcrF6VoY4awG4KRSLaqy3JCdQ",
+      "/dns/boot.metaspan.io/tcp/16052/p2p/12D3KooWLwiJuvqQUB4kYaSjLenFKH9dWZhGZ4qi7pSb3sUYU651",
+      "/dns/boot.metaspan.io/tcp/16056/wss/p2p/12D3KooWLwiJuvqQUB4kYaSjLenFKH9dWZhGZ4qi7pSb3sUYU651",
+      "/dns/boot-cr.gatotech.network/tcp/33110/p2p/12D3KooWKgwQfAeDoJARdtxFNNWfbYmcu6s4yUuSifnNoDgzHZgm",
+      "/dns/boot-cr.gatotech.network/tcp/35110/wss/p2p/12D3KooWKgwQfAeDoJARdtxFNNWfbYmcu6s4yUuSifnNoDgzHZgm",
+      "/dns/statemint-bootnode.turboflakes.io/tcp/30315/p2p/12D3KooWL8CyLww3m3pRySQGGYGNJhWDMqko3j5xi67ckP7hDUvo",
+      "/dns/statemint-bootnode.turboflakes.io/tcp/30415/wss/p2p/12D3KooWL8CyLww3m3pRySQGGYGNJhWDMqko3j5xi67ckP7hDUvo",
+      "/dns/boot-node.helikon.io/tcp/10220/p2p/12D3KooW9uybhguhDjVJc3U3kgZC3i8rWmAnSpbnJkmuR7C6ZsRW",
+      "/dns/boot-node.helikon.io/tcp/10222/wss/p2p/12D3KooW9uybhguhDjVJc3U3kgZC3i8rWmAnSpbnJkmuR7C6ZsRW",
+      "/dns/statemint.bootnode.amforc.com/tcp/30341/p2p/12D3KooWByohP9FXn7ao8syS167qJsbFdpa7fY2Y24xbKtt3r7Ls",
+      "/dns/statemint.bootnode.amforc.com/tcp/30333/wss/p2p/12D3KooWByohP9FXn7ao8syS167qJsbFdpa7fY2Y24xbKtt3r7Ls",
+      "/dns/statemint-boot-ng.dwellir.com/tcp/30344/p2p/12D3KooWEFrNuNk8fPdQS2hf34Gmqi6dGSvrETshGJUrqrvfRDZr",
+      "/dns/statemint-boot-ng.dwellir.com/tcp/443/wss/p2p/12D3KooWEFrNuNk8fPdQS2hf34Gmqi6dGSvrETshGJUrqrvfRDZr"
+    ],
+    "telemetryEndpoints": null,
+    "protocolId": null,
+    "properties": {
+      "ss58Format": 0,
+      "tokenDecimals": 10,
+      "tokenSymbol": "DOT"
+    },
+    "relay_chain": "polkadot",
+    "para_id": 1000,
+    "consensusEngine": null,
+    "codeSubstitutes": {},
+    "genesis": {
+      "stateRootHash": "0xc1ef26b567de07159e4ecd415fbbb0340c56a09c4d72c82516d0f3bc2b782c80"
+    }
+  }

lightclient/src/client.rs

@@ -2,6 +2,8 @@
 // This file is dual-licensed as Apache-2.0 or GPL-3.0.
 // see LICENSE for license details.
 
+use std::iter;
+
 use super::{
     background::{BackgroundTask, FromSubxt, MethodResponse},
     LightClientRpcError,
@@ -11,14 +13,38 @@ use tokio::sync::{mpsc, mpsc::error::SendError, oneshot};
 
 use super::platform::build_platform;
 
-pub const LOG_TARGET: &str = "light-client";
+pub const LOG_TARGET: &str = "subxt-light-client";
+
+/// A raw light-client RPC implementation that can connect to multiple chains.
+#[derive(Clone)]
+pub struct RawLightClientRpc {
+    /// Communicate with the backend task that multiplexes the responses
+    /// back to the frontend.
+    to_backend: mpsc::UnboundedSender<FromSubxt>,
+}
+
+impl RawLightClientRpc {
+    /// Construct a [`LightClientRpc`] that can communicated with the provided chain.
+    ///
+    /// # Note
+    ///
+    /// This uses the same underlying instance created by [`LightClientRpc::new_from_client`].
+    pub fn for_chain(&self, chain_id: smoldot_light::ChainId) -> LightClientRpc {
+        LightClientRpc {
+            to_backend: self.to_backend.clone(),
+            chain_id,
+        }
+    }
+}
 
 /// The light-client RPC implementation that is used to connect with the chain.
 #[derive(Clone)]
 pub struct LightClientRpc {
     /// Communicate with the backend task that multiplexes the responses
     /// back to the frontend.
     to_backend: mpsc::UnboundedSender<FromSubxt>,
+    /// The chain ID to target for requests.
+    chain_id: smoldot_light::ChainId,
 }
 
 impl LightClientRpc {
@@ -31,37 +57,97 @@ impl LightClientRpc {
     ///
     /// ## Panics
     ///
-    /// Panics if being called outside of `tokio` runtime context.
+    /// The panic behaviour depends on the feature flag being used:
+    ///
+    /// ### Native
+    ///
+    /// Panics when called outside of a `tokio` runtime context.
+    ///
+    /// ### Web
+    ///
+    /// If smoldot panics, then the promise created will be leaked. For more details, see
+    /// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
     pub fn new(
-        config: smoldot_light::AddChainConfig<'_, (), impl Iterator<Item = smoldot_light::ChainId>>,
+        config: smoldot_light::AddChainConfig<
+            '_,
+            (),
+            impl IntoIterator<Item = smoldot_light::ChainId>,
+        >,
     ) -> Result<LightClientRpc, LightClientRpcError> {
         tracing::trace!(target: LOG_TARGET, "Create light client");
 
         let mut client = smoldot_light::Client::new(build_platform());
 
+        let config = smoldot_light::AddChainConfig {
+            specification: config.specification,
+            json_rpc: config.json_rpc,
+            database_content: config.database_content,
+            potential_relay_chains: config.potential_relay_chains.into_iter(),
+            user_data: config.user_data,
+        };
+
         let smoldot_light::AddChainSuccess {
             chain_id,
             json_rpc_responses,
         } = client
             .add_chain(config)
             .map_err(|err| LightClientRpcError::AddChainError(err.to_string()))?;
 
-        let (to_backend, backend) = mpsc::unbounded_channel();
-
-        // `json_rpc_responses` can only be `None` if we had passed `json_rpc: Disabled`.
         let rpc_responses = json_rpc_responses.expect("Light client RPC configured; qed");
 
+        let raw_client = Self::new_from_client(
+            client,
+            iter::once(AddedChain {
+                chain_id,
+                rpc_responses,
+            }),
+        );
+        Ok(raw_client.for_chain(chain_id))
+    }
+
+    /// Constructs a new [`RawLightClientRpc`] from the raw smoldot client.
+    ///
+    /// Receives a list of RPC objects as a result of calling `smoldot_light::Client::add_chain`.
+    /// This [`RawLightClientRpc`] can target different chains using [`RawLightClientRpc::for_chain`] method.
+    ///
+    /// ## Panics
+    ///
+    /// The panic behaviour depends on the feature flag being used:
+    ///
+    /// ### Native
+    ///
+    /// Panics when called outside of a `tokio` runtime context.
+    ///
+    /// ### Web
+    ///
+    /// If smoldot panics, then the promise created will be leaked. For more details, see
+    /// https://docs.rs/wasm-bindgen-futures/latest/wasm_bindgen_futures/fn.future_to_promise.html.
+    pub fn new_from_client<TPlat>(
+        client: smoldot_light::Client<TPlat>,
+        chains: impl IntoIterator<Item = AddedChain>,
+    ) -> RawLightClientRpc
+    where
+        TPlat: smoldot_light::platform::PlatformRef + Clone,
+    {
+        let (to_backend, backend) = mpsc::unbounded_channel();
+        let chains = chains.into_iter().collect();
+
         let future = async move {
-            let mut task = BackgroundTask::new(client, chain_id);
-            task.start_task(backend, rpc_responses).await;
+            let mut task = BackgroundTask::new(client);
+            task.start_task(backend, chains).await;
         };
 
         #[cfg(feature = "native")]
         tokio::spawn(future);
         #[cfg(feature = "web")]
         wasm_bindgen_futures::spawn_local(future);
 
-        Ok(LightClientRpc { to_backend })
+        RawLightClientRpc { to_backend }
+    }
+
+    /// Returns the chain ID of the current light-client.
+    pub fn chain_id(&self) -> smoldot_light::ChainId {
+        self.chain_id
     }
 
     /// Submits an RPC method request to the light-client.
@@ -79,6 +165,7 @@ impl LightClientRpc {
             method,
             params,
             sender,
+            chain_id: self.chain_id,
         })?;
 
         Ok(receiver)
@@ -107,8 +194,17 @@ impl LightClientRpc {
             params,
             sub_id,
             sender,
+            chain_id: self.chain_id,
         })?;
 
         Ok((sub_id_rx, receiver))
     }
 }
+
+/// The added chain of the light-client.
+pub struct AddedChain {
+    /// The id of the chain.
+    pub chain_id: smoldot_light::ChainId,
+    /// Producer of RPC responses for the chain.
+    pub rpc_responses: smoldot_light::JsonRpcResponses,
+}

lightclient/src/lib.rs

@@ -33,8 +33,18 @@ mod platform;
 #[allow(unused_imports)]
 pub use getrandom as _;
 
-pub use client::LightClientRpc;
-pub use smoldot_light::{AddChainConfig, AddChainConfigJsonRpc, ChainId};
+pub use client::{AddedChain, LightClientRpc, RawLightClientRpc};
+
+/// Re-exports of the smoldot related objects.
+pub mod smoldot {
+    pub use smoldot_light::{
+        platform::PlatformRef, AddChainConfig, AddChainConfigJsonRpc, ChainId, Client,
+        JsonRpcResponses,
+    };
+
+    #[cfg(feature = "native")]
+    pub use smoldot_light::platform::default::DefaultPlatform;
+}
 
 /// Light client error.
 #[derive(Debug, thiserror::Error)]

subxt/Cargo.toml

@@ -114,3 +114,8 @@ tracing-subscriber = { workspace = true }
 name = "unstable_light_client_tx_basic"
 path = "examples/unstable_light_client_tx_basic.rs"
 required-features = ["unstable-light-client", "jsonrpsee"]
+
+[[example]]
+name = "unstable_light_client_parachains"
+path = "examples/unstable_light_client_parachains.rs"
+required-features = ["unstable-light-client", "jsonrpsee", "native"]

subxt/examples/unstable_light_client_parachains.rs

@@ -0,0 +1,101 @@
+use futures::StreamExt;
+use std::{iter, num::NonZeroU32};
+use subxt::{
+    client::{LightClient, RawLightClient},
+    PolkadotConfig,
+};
+
+// Generate an interface that we can use from the node's metadata.
+#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
+pub mod polkadot {}
+
+const POLKADOT_SPEC: &str = include_str!("../../artifacts/demo_chain_specs/polkadot.json");
+const ASSET_HUB_SPEC: &str =
+    include_str!("../../artifacts/demo_chain_specs/polkadot_asset_hub.json");
+
+#[tokio::main]
+async fn main() -> Result<(), Box<dyn std::error::Error>> {
+    // The smoldot logs are informative:
+    tracing_subscriber::fmt::init();
+
+    // Connecting to a parachain is a multi step process.
+
+    // Step 1. Construct a new smoldot client.
+    let mut client =
+        subxt_lightclient::smoldot::Client::new(subxt_lightclient::smoldot::DefaultPlatform::new(
+            "subxt-example-light-client".into(),
+            "version-0".into(),
+        ));
+
+    // Step 2. Connect to the relay chain of the parachain. For this example, the Polkadot relay chain.
+    let polkadot_connection = client
+        .add_chain(subxt_lightclient::smoldot::AddChainConfig {
+            specification: POLKADOT_SPEC,
+            json_rpc: subxt_lightclient::smoldot::AddChainConfigJsonRpc::Enabled {
+                max_pending_requests: NonZeroU32::new(128).unwrap(),
+                max_subscriptions: 1024,
+            },
+            potential_relay_chains: iter::empty(),
+            database_content: "",
+            user_data: (),
+        })
+        .expect("Light client chain added with valid spec; qed");
+    let polkadot_json_rpc_responses = polkadot_connection
+        .json_rpc_responses
+        .expect("Light client configured with json rpc enabled; qed");
+    let polkadot_chain_id = polkadot_connection.chain_id;
+
+    // Step 3. Connect to the parachain. For this example, the Asset hub parachain.
+    let assethub_connection = client
+        .add_chain(subxt_lightclient::smoldot::AddChainConfig {
+            specification: ASSET_HUB_SPEC,
+            json_rpc: subxt_lightclient::smoldot::AddChainConfigJsonRpc::Enabled {
+                max_pending_requests: NonZeroU32::new(128).unwrap(),
+                max_subscriptions: 1024,
+            },
+            // The chain specification of the asset hub parachain mentions that the identifier
+            // of its relay chain is `polkadot`.
+            potential_relay_chains: [polkadot_chain_id].into_iter(),
+            database_content: "",
+            user_data: (),
+        })
+        .expect("Light client chain added with valid spec; qed");
+    let parachain_json_rpc_responses = assethub_connection
+        .json_rpc_responses
+        .expect("Light client configured with json rpc enabled; qed");
+    let parachain_chain_id = assethub_connection.chain_id;
+
+    // Step 4. Turn the smoldot client into a raw client.
+    let raw_light_client = RawLightClient::builder()
+        .add_chain(polkadot_chain_id, polkadot_json_rpc_responses)
+        .add_chain(parachain_chain_id, parachain_json_rpc_responses)
+        .build(client)
+        .await?;
+
+    // Step 5. Obtain a client to target the relay chain and the parachain.
+    let polkadot_api: LightClient<PolkadotConfig> =
+        raw_light_client.for_chain(polkadot_chain_id).await?;
+    let parachain_api: LightClient<PolkadotConfig> =
+        raw_light_client.for_chain(parachain_chain_id).await?;
+
+    // Step 6. Subscribe to the finalized blocks of the chains.
+    let polkadot_sub = polkadot_api
+        .blocks()
+        .subscribe_finalized()
+        .await?
+        .map(|block| ("Polkadot", block));
+    let parachain_sub = parachain_api
+        .blocks()
+        .subscribe_finalized()
+        .await?
+        .map(|block| ("AssetHub", block));
+    let mut stream_combinator = futures::stream::select(polkadot_sub, parachain_sub);
+
+    while let Some((chain, block)) = stream_combinator.next().await {
+        let block = block?;
+
+        println!("     Chain {:?} hash={:?}", chain, block.hash());
+    }
+
+    Ok(())
+}