Iterating StorageDoubleMap problem.

[lx38]

I want to obtain ErasStakers for current_era but there is no way to set the first param of the map to be equal to current_era. So I have to download whole map first, then iterate over all items and filter only those which corresponds to current era. It would be good to set the current_era to append to StorateKey to limit amount of downloaded data. Any clues?

let mut addr = md::storage().staking().eras_stakers_root();
let mut iter = storage_client.iter(addr, 999).await.unwrap();

// while the map is keyed by current_era + account_id
md::storage().staking().eras_stakers(current_era, account_id)
``

[lx38]

Linking previous related ticket #526
It seems that the latest interface abolished ability to access StorageDoubleMap by prefix.

[jsdw]

Good question; this is an area where the storage APIs are weak at the moment.

The codegen doesn't support providing a "partial" key (ie not all of the required keys in the map to get to a single value) yet, but you can use the "raw" APIs to do whatever (it's just more cumbersome). I had a go at working up an example to do what you wanted, and the best approach I found was this:

use subxt::{OnlineClient, PolkadotConfig, utils::AccountId32 };
use codec::{ Decode, Encode };

#[subxt::subxt(
    runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale",
    derive_for_all_types = "Debug"
)]
pub mod polkadot {}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create a new API client, configured to talk to Polkadot nodes.
    let api = OnlineClient::<PolkadotConfig>::from_url("wss://rpc.polkadot.io:443").await?;

    // Get latest block to ensure all subsequent calls are for same block:
    let block_hash = api.blocks().at_latest().await?.hash();

    // Get current era:
    let current_era = api
        .storage()
        .at(block_hash)
        .fetch(&polkadot::storage().staking().current_era())
        .await?
        .unwrap();

    // get root key for this storage location:
    let mut addr_bytes = polkadot::storage()
        .staking()
        .eras_stakers_root()
        .to_root_bytes();

    // manually add key for current era (ie twox64_concat the key):
    {
        let current_era_bytes = current_era.encode();
        let hash = sp_core_hashing::twox_64(&current_era_bytes);
        addr_bytes.extend(&hash);
        addr_bytes.extend(&current_era_bytes);
    }

    // Get keys (first 100 here):
    let keys = api
        .storage()
        .at(block_hash)
        .fetch_keys(&addr_bytes, 100, None)
        .await?;

    // For each key, fetch the corresponding value bytes and decode them:
    for key in keys {
        let Some(val_bytes) = api
            .storage()
            .at(block_hash)
            .fetch_raw(key.as_ref())
            .await? else { continue };

        let exposure = <
            polkadot::runtime_types::pallet_staking::Exposure<
                AccountId32,
                u128,
            >
        >::decode(&mut &*val_bytes)?;

        // Print:
        println!("{key:?}: {exposure:?}");
    }

    Ok(())
}

I def think we can extend the codegen here to support providing partial keys and make this more ergonomic; in general revamping the storage APIs is something I've been meaning to look at for a while :)

[lx38]

Thank you for the full example. I guess you might want to include it in the examples for people who want this functionality. Warm thanks!

[jsdw]

Yup, I think that's a good idea!

I'll open an issue to improve storage iterating too, because I think it wouldn't be so hard to make this as ergonomic as iterating over everything :)