Frequent dependency failures - substrate, polkadot, and subxt dependency mismatches

[drahnr]

We are building a CLI tool with subxt, but try to keep everything in a single workspace. This breaks frequently due to version mismatches of i.e. wasmtime and causes more friction than needed. We are not particularly keen on splitting the workspace into two.

Could it be viable to have a substrate/polkadot branch in subxt to be released in lockstep with the substrate versions that are used in polkadot?

Pretty sure this is the wrong place to file, but technically it'd need to be in all 3 repositories

[jsdw]

It's currently not possible to release a version of Subxt that aligns with latest Polkadot/Substrate crates since we have stopped publishing them to github. Once we have this publishing-to-github working again (@bkchr I think you were going to be helping a new hire on Mara's team with this), we'll either strive to stay in alignment with Polkadot releases (if substrate crates are published based on polkadot release branches) or just keep fairly uptodate (if Substrate crates are published on some regular cadence independently of Polkadot releases).

That said, Subxt can be used entirely independently of Substrate too if you disable the substrate-compat feature. You can use the provided subxt-signer to sign transactions, which is the main thing that substrate-compat provides. This would avoid any dependency conflicts with Substrate crates.

[jsdw]

(probably at some point substrate-compat will become "opt-in" instead of "opt-out" now, especially because it's been tricky to maintain any sort of sensible alignment with the versions of substrate crates that people are using elsewhere)

[jsdw]

See #1077; We'll aim to remove this flag because, while it can be useful, it's more a hinderence than a help nowadays for most cases I think.

@drahnr did removing this feature flag solve your case?

[drahnr]

The problem is much larger than substrate-compat unfortunately.

[jsdw]

Ah, sorry to hear that! Is there anything I can help with from the Subxt side at all?

[drahnr]

Lockstep releases and lockstep branching with polkadot releases would be a dream.

[jsdw]

For that, we can't really do anything until Substrate crates are being released again.

What issues did you encounter when disabling the "substrate-compat" feature flag? Doing this removes any ties to the Substrate crates and so Subxt should be able to coexist quite peacefully in a workspace with Substrate crates with this.

subxt-signer is a fairly new crate that we have released which can potentially replace sp_core et al for signing things (which was the only thing Substrate crates were needed for before).

[niklasad1]

subxt is now independent of substrate/polkadot-sdk after we removed the substrate-compat feature and the only polkadot-sdk dependency subxt has is sp-crypto-hashing which is just a hashing library.

So, subxt should really work with any version polkadot-sdk i.e, from crates.io/git or whatever you like..

I guess it's technically possible to have dependency mismatches within the "subxt macro" because it depends sp-executor if --feature runtime-metadata-path is enabled but otherwise I expect it work but maybe I'm missing something?

I think and hope that that this isn't any issue anymore.

Can you please double-check and let us know?