Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow for remapping type parameters in type substitutions #735

Merged
merged 7 commits into from
Jan 19, 2023
Merged

Allow for remapping type parameters in type substitutions #735

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
29f1ab5
feat!: Allow for remapping type parameters in type substitutions
Xanewok Nov 30, 2022
9276c63
chore: cargo fmt
Xanewok Jan 11, 2023
c9e7b64
chore: cargo clippy
Xanewok Jan 11, 2023
0866ca8
chore: Remove some old code
Xanewok Jan 11, 2023
cdab956
Merge branch 'master' into igor-type-substitutes
jsdw Jan 17, 2023
83f04ff
a little tidy
jsdw Jan 18, 2023
c66b77b
address comment nit
jsdw Jan 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion cli/src/commands/codegen.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ use std::{
io::Read,
path::PathBuf,
};
use subxt_codegen::DerivesRegistry;
use subxt_codegen::{
DerivesRegistry,
TypeSubstitutes,
};

/// Generate runtime API client code from metadata.
///
Expand Down Expand Up @@ -95,10 +98,13 @@ fn codegen(
derives.extend_for_type(ty, std::iter::once(derive), &crate_path)
}

let type_substitutes = TypeSubstitutes::new(&crate_path);

let runtime_api = subxt_codegen::generate_runtime_api_from_bytes(
item_mod,
metadata_bytes,
derives,
type_substitutes,
crate_path,
);
println!("{}", runtime_api);
Expand Down
75 changes: 22 additions & 53 deletions codegen/src/api/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ use crate::{
CompositeDef,
CompositeDefFields,
TypeGenerator,
TypeSubstitutes,
},
utils::{
fetch_metadata_bytes_blocking,
Expand All @@ -39,7 +40,6 @@ use quote::{
quote,
};
use std::{
collections::HashMap,
fs,
io::Read,
path,
Expand All @@ -54,13 +54,15 @@ use syn::parse_quote;
/// * `item_mod` - The module declaration for which the API is implemented.
/// * `path` - The path to the scale encoded metadata of the runtime node.
/// * `derives` - Provide custom derives for the generated types.
/// * `type_substitutes` - Provide custom type substitutes.
/// * `crate_path` - Path to the `subxt` crate.
///
/// **Note:** This is a wrapper over [RuntimeGenerator] for static metadata use-cases.
pub fn generate_runtime_api_from_path<P>(
item_mod: syn::ItemMod,
path: P,
derives: DerivesRegistry,
type_substitutes: TypeSubstitutes,
crate_path: CratePath,
) -> TokenStream2
where
Expand All @@ -74,7 +76,13 @@ where
file.read_to_end(&mut bytes)
.unwrap_or_else(|e| abort_call_site!("Failed to read metadata file: {}", e));

generate_runtime_api_from_bytes(item_mod, &bytes, derives, crate_path)
generate_runtime_api_from_bytes(
item_mod,
&bytes,
derives,
type_substitutes,
crate_path,
)
}

/// Generates the API for interacting with a substrate runtime, using metadata
Expand All @@ -86,19 +94,27 @@ where
/// * `item_mod` - The module declaration for which the API is implemented.
/// * `url` - HTTP/WS URL to the substrate node you'd like to pull metadata from.
/// * `derives` - Provide custom derives for the generated types.
/// * `type_substitutes` - Provide custom type substitutes.
/// * `crate_path` - Path to the `subxt` crate.
///
/// **Note:** This is a wrapper over [RuntimeGenerator] for static metadata use-cases.
pub fn generate_runtime_api_from_url(
item_mod: syn::ItemMod,
url: &Uri,
derives: DerivesRegistry,
type_substitutes: TypeSubstitutes,
crate_path: CratePath,
) -> TokenStream2 {
let bytes = fetch_metadata_bytes_blocking(url)
.unwrap_or_else(|e| abort_call_site!("Failed to obtain metadata: {}", e));

generate_runtime_api_from_bytes(item_mod, &bytes, derives, crate_path)
generate_runtime_api_from_bytes(
item_mod,
&bytes,
derives,
type_substitutes,
crate_path,
)
}

/// Generates the API for interacting with a substrate runtime, using metadata bytes.
Expand All @@ -115,13 +131,14 @@ pub fn generate_runtime_api_from_bytes(
item_mod: syn::ItemMod,
bytes: &[u8],
derives: DerivesRegistry,
type_substitutes: TypeSubstitutes,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super small nit: missing
/// * `type_substitutes` - Provide custom type substitutes.
on the doc.

Tho, I'm wondering if we should extend this documentation style to our repo, or just remove those # Arguments in the future. I guess the easiest would be later since we already have some comprehensive docs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot!

I added the extra param for now, though I'd note that these comments aren't exposed publically anywhere and so in general I'd be inclined to keep them simple as they are only for us devs who work on it anyway :)

crate_path: CratePath,
) -> TokenStream2 {
let metadata = frame_metadata::RuntimeMetadataPrefixed::decode(&mut &bytes[..])
.unwrap_or_else(|e| abort_call_site!("Failed to decode metadata: {}", e));

let generator = RuntimeGenerator::new(metadata);
generator.generate_runtime(item_mod, derives, crate_path)
generator.generate_runtime(item_mod, derives, type_substitutes, crate_path)
}

/// Create the API for interacting with a Substrate runtime.
Expand Down Expand Up @@ -152,61 +169,13 @@ impl RuntimeGenerator {
&self,
item_mod: syn::ItemMod,
derives: DerivesRegistry,
type_substitutes: TypeSubstitutes,
crate_path: CratePath,
) -> TokenStream2 {
let item_mod_attrs = item_mod.attrs.clone();
let item_mod_ir = ir::ItemMod::from(item_mod);
let default_derives = derives.default_derives();

// Some hardcoded default type substitutes, can be overridden by user
let mut type_substitutes = [
(
"bitvec::order::Lsb0",
parse_quote!(#crate_path::utils::bits::Lsb0),
),
(
"bitvec::order::Msb0",
parse_quote!(#crate_path::utils::bits::Msb0),
),
(
"sp_core::crypto::AccountId32",
parse_quote!(#crate_path::utils::AccountId32),
),
(
"sp_runtime::multiaddress::MultiAddress",
parse_quote!(#crate_path::utils::MultiAddress),
),
(
"primitive_types::H160",
parse_quote!(#crate_path::utils::H160),
),
(
"primitive_types::H256",
parse_quote!(#crate_path::utils::H256),
),
(
"primitive_types::H512",
parse_quote!(#crate_path::utils::H512),
),
(
"frame_support::traits::misc::WrapperKeepOpaque",
parse_quote!(#crate_path::utils::WrapperKeepOpaque),
),
// BTreeMap and BTreeSet impose an `Ord` constraint on their key types. This
// can cause an issue with generated code that doesn't impl `Ord` by default.
// Decoding them to Vec by default (KeyedVec is just an alias for Vec with
// suitable type params) avoids these issues.
("BTreeMap", parse_quote!(#crate_path::utils::KeyedVec)),
("BTreeSet", parse_quote!(::std::vec::Vec)),
]
.iter()
.map(|(path, substitute): &(&str, syn::TypePath)| {
(path.to_string(), substitute.clone())
})
.collect::<HashMap<_, _>>();

type_substitutes.extend(item_mod_ir.type_substitutes().into_iter());

let type_gen = TypeGenerator::new(
&self.metadata.types,
"runtime_types",
Expand Down
129 changes: 4 additions & 125 deletions codegen/src/ir.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,15 @@
// see LICENSE for license details.

use proc_macro_error::abort;
use std::collections::HashMap;
use syn::{
spanned::Spanned as _,
token,
};
use syn::token;

#[derive(Debug, PartialEq, Eq)]
pub struct ItemMod {
vis: syn::Visibility,
mod_token: token::Mod,
pub ident: syn::Ident,
brace: token::Brace,
items: Vec<Item>,
items: Vec<syn::Item>,
}

impl From<syn::ItemMod> for ItemMod {
Expand All @@ -32,130 +28,13 @@ impl From<syn::ItemMod> for ItemMod {
mod_token: module.mod_token,
ident: module.ident,
brace,
items: items.into_iter().map(From::from).collect(),
items,
}
}
}

impl ItemMod {
pub fn type_substitutes(&self) -> HashMap<String, syn::TypePath> {
self.items
.iter()
.filter_map(|item| {
if let Item::Subxt(SubxtItem::TypeSubstitute {
generated_type_path,
substitute_with: substitute_type,
}) = item
{
Some((generated_type_path.clone(), substitute_type.clone()))
} else {
None
}
})
.collect()
}

pub fn rust_items(&self) -> impl Iterator<Item = &syn::Item> {
self.items.iter().filter_map(Item::as_rust)
}
}

#[allow(clippy::large_enum_variant)]
#[derive(Debug, PartialEq, Eq)]
pub enum Item {
Rust(syn::Item),
Subxt(SubxtItem),
}

impl Item {
pub fn as_rust(&self) -> Option<&syn::Item> {
match self {
Item::Rust(item) => Some(item),
_ => None,
}
}
}

impl From<syn::Item> for Item {
fn from(item: syn::Item) -> Self {
if let syn::Item::Use(ref use_) = item {
let substitute_attrs = use_
.attrs
.iter()
.map(|attr| {
let meta = attr.parse_meta().unwrap_or_else(|e| {
abort!(attr.span(), "Error parsing attribute: {}", e)
});
<attrs::Subxt as darling::FromMeta>::from_meta(&meta).unwrap_or_else(
|e| abort!(attr.span(), "Error parsing attribute meta: {}", e),
)
})
.collect::<Vec<_>>();
if substitute_attrs.len() > 1 {
abort!(
use_.attrs[0].span(),
"Duplicate `substitute_type` attributes"
)
}
if let Some(attr) = substitute_attrs.get(0) {
let use_path = &use_.tree;
let substitute_with: syn::TypePath = syn::parse_quote!( #use_path );

let is_crate = substitute_with
.path
.segments
.first()
.map(|segment| segment.ident == "crate")
.unwrap_or(false);

// Check if the substitute path is a global absolute path, meaning it
// is prefixed with `::` or `crate`.
//
// Note: the leading colon is lost when parsing to `syn::TypePath` via
// `syn::parse_quote!`. Therefore, inspect `use_`'s leading colon.
if use_.leading_colon.is_none() && !is_crate {
abort!(
use_path.span(),
"The substitute path must be a global absolute path; try prefixing with `::` or `crate`"
)
}

let type_substitute = SubxtItem::TypeSubstitute {
generated_type_path: attr.substitute_type(),
substitute_with,
};
Self::Subxt(type_substitute)
} else {
Self::Rust(item)
}
} else {
Self::Rust(item)
}
}
}

#[derive(Debug, PartialEq, Eq)]
pub enum SubxtItem {
TypeSubstitute {
generated_type_path: String,
substitute_with: syn::TypePath,
},
}

mod attrs {
use darling::FromMeta;

#[derive(Debug, FromMeta)]
#[darling(rename_all = "snake_case")]
pub enum Subxt {
SubstituteType(String),
}

impl Subxt {
pub fn substitute_type(&self) -> String {
match self {
Self::SubstituteType(path) => path.clone(),
}
}
self.items.iter()
}
}
7 changes: 5 additions & 2 deletions codegen/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
//! use std::fs;
//! use codec::Decode;
//! use frame_metadata::RuntimeMetadataPrefixed;
//! use subxt_codegen::{CratePath, DerivesRegistry};
//! use subxt_codegen::{CratePath, DerivesRegistry, TypeSubstitutes};
//!
//! let encoded = fs::read("../artifacts/polkadot_metadata.scale").unwrap();
//!
Expand All @@ -33,9 +33,11 @@
//! );
//! // Default module derivatives.
//! let mut derives = DerivesRegistry::new(&CratePath::default());
//! // Default type substitutes.
//! let substs = TypeSubstitutes::new(&CratePath::default());
//! // Generate the Runtime API.
//! let generator = subxt_codegen::RuntimeGenerator::new(metadata);
//! let runtime_api = generator.generate_runtime(item_mod, derives, CratePath::default());
//! let runtime_api = generator.generate_runtime(item_mod, derives, substs, CratePath::default());
//! println!("{}", runtime_api);
//! ```

Expand All @@ -60,5 +62,6 @@ pub use self::{
DerivesRegistry,
Module,
TypeGenerator,
TypeSubstitutes,
},
};
Loading