substrate-api-client

The substrate-api-client is a Rust library for connecting to a substrate-based node via RPC. It's particularly useful for setups with no-std environment (which are typical for trusted execution environmnets or embedded devices). It provides similar functionalities as Polkadot-js, such as easy extrinsic submission and state queries. With an RPC client, developers can easily interact with any Polkadot or Kusama chain. There are several RPC clients available in different programming languages. For Rust, the most popular RPC client is subxt. The substrate-api-client provides a simpler, less extensive alternative to subxt, focused on providing as many features as possible for no-std environments.

The substrate-api-client connects to the substrate's RPC interface via WebSockets allowing to

• Compose extrinsics, send them (asynchronously and synchronously) and subscribe to updates (synchronously).
• Support no_std builds. Only the rpc-client is std only. For no_std builds, a custom rpc client needs to be implemented.
• Watch events and execute code upon events.
• Parse and print the node metadata.
• Support async and sync implementations.
• Support three different websocket crates (jsonrpsee, tungstenite and ws). See Cargo.toml for more information and limitations.

Prerequisites

In order to build the substrate-api-client and the examples, Rust and the wasm target are needed. For Linux:

curl https://sh.rustup.rs -sSf | sh
# Install the rust toolchain specified in rust-toolchain.toml
rustup show

Substrate node

To execute the examples, a running substrate node is needed. You can download a node artifact from substrate directly: https://github.com/paritytech/substrate or run the kitchensink-node with docker:

docker run -p 9944:9944 -p 9933:9933 -p 30333:30333 parity/substrate:latest --dev --rpc-external

For more information, please refer to the substrate repository.

Examples

The api-client provides several examples which show how to fetch node states or submit extrinsic. Examples are differentiated between sync and async implementations. Don't forget to check the feature import of the associated Cargo.toml. It shows how to import the api-client as an async or sync library. To run an example, clone the substrate-api-client repository and run the desired example directly with the cargo command:

git clone https://github.com/scs/substrate-api-client.git
cd substrate-api-client
# Run an async example:
cargo run -p ac-examples-async --example get_storage
# Run a sync example:
cargo run -p ac-examples-sync --example runtime_update_sync

or download the already built binaries from GitHub Actions and run them without any previous building:

# Enter the async or sync example directory and add execution rights to the chosen example.
cd examples-<sync/async>
chmod +x <example>
# And run it.
./<example>

Set the output verbosity by prepending RUST_LOG=info or RUST_LOG=debug.

The following async examples can be found in the async examples folder:

• benchmark_bulk_xt: Float the node with a series of transactions.
• check_extrinsic_events: Check and react according to events associated to an extrinsic.
• compose_extrinsic: Compose an extrinsic without interacting with the node or in no_std mode.
• custom_nonce: Compose an with a custom nonce.
• get_blocks: Read header, block and signed block from storage.
• get_storage: Read storage values.
• new_json_rpc_api_calls: Call the unstable rpc api with self defined functions.
• print_metadata: Print the metadata of the node in a readable way.
• query_runtime_api: How to query the runtime api.
• runtime_update_async: How to do an runtime upgrade asynchronously.
• staking_batch_payout: Batch reward payout for validator.
• subscribe_events: Subscribe and react on events.
• sudo: Create and send a sudo wrapped call.

The following sync examples can be found in the sync examples folder:

• runtime_update_sync: How to do an runtime upgrade synchronously.
• transfer_with_tungstenite_client: Transfer tokens by using a wrapper of compose_extrinsic with an account generated with a seed.
• transfer_with_ws_client: Transfer tokens by using a wrapper of compose_extrinsic with an account generated with a seed.

More, less well documented calls can be found in the testing crate.

no_std build

Almost everything in the api-client, except for the rpc-clients and a few additional features, is no_std compatible. Many helpful features, such as extrinsic and call creation (see the macros), metadata and event types (see the node-api and primitives) are available in no_std right away. However, to directly connect to a Substrate node a RPC client is necessary. Because websocket connection features are often hardware dependent, a generic no_std RPC client implementation is hardly possible. So for most use cases a self-implemented RPC client is required. To make this as simple as possible, the interface between the Api, which provides all the features, and the RPC client, providing the node connection, is kept very basic. Check out the following explanations for more info.

Import

To import the api-client in no_std make sure the default features are turned off and disable_target_static_assertions is enabled:

# In the Cargo.toml import the api-client as following:
substrate-api-client = { git = "https://github.com/scs/substrate-api-client.git", default-features = false, features = ["disable_target_static_assertions"] }

RPC Client

Depending on the usage, there are two traits that the RPC Client needs to implement. You can choose between the sync and async implementation. If you decide to use the async implementation, you need to use the library async-trait for now (until it is integrated into the rust toolchain).

Request

For simple requests (send one request and receive one answer) the trait Request is required:

/// Trait to be implemented by the ws-client for sending rpc requests and extrinsic.
pub trait Request {
	/// Sends a RPC request to the substrate node and returns the answer as string.
	(async) fn request<R: DeserializeOwned>(&self, method: &str, params: RpcParams) -> Result<R>;
}

By implementing this trait with a custom RPC client, most basic functionalities of the Api can already be used. Currently, there is no no_std example available. But the tungstenite_client provides a relatively simple std example. If a websocket library is available in your no_std environment, then your implementation may look similar.

Subscription

A little more complex is the second trait Subscribe, which does not only send a subscription request to the node, it also keeps listening and updating accordingly. Two traits need to be implemented for this feature. The Subscribe trait itself:

/// Trait to be implemented by the ws-client for subscribing to the substrate node.
pub trait Subscribe {
	type Subscription<Notification>: HandleSubscription<Notification>
	where
		Notification: DeserializeOwned;

	(async) fn subscribe<Notification: DeserializeOwned>(
		&self,
		sub: &str,
		params: RpcParams,
		unsub: &str,
	) -> Result<Self::Subscription<Notification>>;
}

and the HandleSubscription trait, which is returned by the subscribe function:

/// Trait to use the full functionality of jsonrpseee Subscription type
/// without actually enforcing it.
pub trait HandleSubscription<Notification: DeserializeOwned> {
	/// Returns the next notification from the stream.
	/// This may return `None` if the subscription has been terminated,
	/// which may happen if the channel becomes full or is dropped.
	(async) fn next(&mut self) -> Option<Result<Notification>>;

	/// Unsubscribe and consume the subscription.
	(async) fn unsubscribe(self) -> Result<()>;
}

Refering to the std example of the tungstenite, the HandleSubscription impl can be looked up here. It implements a simple channel receiver, waiting for the sender of the websocket client to send something. The Subscribe implementation can be found here.

A more complex RPC client, but also with more functionalities, is the jsonrpsee client.

Example Upgrades from older to newer versions

There have been some breaking API changes as of late to catch up with the newer Substrate versions and to fully support different Substrate nodes. An example project on how to upgrade from older tags can be found in the Integritee worker repository:

• tag v0.7.0 -> v0.9.0 (Upgrade to tag v0.8.0 is not recommended, directly upgrading to v0.9.0 saves you some extra work).
• tag v0.9.0 -> v0.10.0

If you still experience issues during upgrading, do not hesitate to create an issue for support.

Version numbers

Starting with release 1.16.0, we use a specific versioning schema in order to enable us to release the same version multiple times for different polkadot releases

• Major version numbers represent the version of the substrate-api-client
• Minor version numbers represent the polkadot release
  • Note that this implies that there can be breaking changes in minor releases.
• Patch/bugfix number is still used for bugfixes

Releases and backwards compatibility

• We usually create a release for each new polkadot release
  • This release contains all the features of our current master branch
  • We don't create releases for polkadot bugfix releases unless there are known issues
• We don't backport features and bugfixes per default. If requested we decide on a case by case basis.
• New releases can contain breaking changes. These are announced in the release notes.

Alternatives

Parity offers a Rust client with similar functionality: https://github.com/paritytech/subxt

Acknowledgements

The development of the substrate-api-client has been financed by:

• web3 foundation
• Integritee
• Kusama Treasury:
  • Maintenance Nov-22 to Jan-23
  • Maintenance Feb-23 to Apr-23
  • Maintenance May-23 to Jul-23
• Polkadot Treasury:
  • Maintenance Sep-23 to Dec-23

We also thank the teams at

• Parity Technologies for building substrate and supporting us during development.

Projects using substrate-api-client

If you intend to or are using substrate-api-client, please add your project here

In alphabetical order

• Ajuna Network
• Encointer
• Integritee Network
• Litentry
• Polkadex

FAQ

1. Q: Everything compiles but the Substrate node does not accept my extrinsic or returns an error even if the extrinsic should be correct.

   A: First, ensure the api-client and the Substrate node have a matching version. E.g. if the node is running on release-polkadot-v1.2.0, checkout and compile a matching branch of the api-client. We are using the same naming scheme as Parity does. Please note: Not all Polkadot releases are published for all api-client releases. Which Polkadot releases are supported by which api-client release are noted in the release notes. Don't find the release-match you're looking for? Feel free to request it via an issue.

2. Q: I get the error Bad input data provided to validate_transaction from the node when submitting an extrinsic. Even though I made sure the api-client and Polkadot releases are matching.

   A: Every extrinsic contains some node specific data. The tips for example may be provided by the Asset pallet or, by default, by the Balances pallet. The current api-client does not have access to this information. Therefore, these config data must be configured manually. Currently, there are two pre-defined Runtime Configs which should match most of the Substrate nodes:

   • Asset Runtime Config: This matches most nodes config that use the Asset pallet.
   • Default Runtime Config: This matches most of node runtimes that do not use the Asset pallet. The config, apart from the tip parameter, equals the asset runtime config.

   Ensure you're using a matching config. If you do not use default parameters as configured in one of the provided configs, you must provide your own config that implements the Config trait.

3. Q: I want to query a state from a substrate node via the api-client, but I do not get the expected value, respective the decoding fails. How come?

   A: When specifying your own state query, you must provide the return type of the state you're trying to retrieve. This is because the api-client only gets bytes from the node and must be able to deserialize these properly. That is not possible without knowing the type to decode to. This type may be for example a simple u64 for retrieving the Balance of an account. But careful: If you're looking at the pallet code and its return type, don't forget to take the Query type into consideration. The OptionQuery for example automatically wraps the return type into an Option (see the substrate docs "Handling query return values" for more information). Alternatively, you can always double check via polkadot.js. If you're importing a value directly from the runtime, as it's done in this example, remember to adapt it to the node you are querying from.