update sp-core, sp-runtime, and frame-metadata to point to their masters

[jakehemmerle]

These dependencies are reflecting the current docs.rs versions, which are quite out of date. Resultingly, its causing parity-util-mem versioning issues when using it in the same codebase as other substrate dependencies.

[jsdw]

We can't point at github dependencies on Subxt, because you can't publish something to crates.io that depends on anything other than other crates published to crates.io

There is an ongoing effort to start publishing the substrate crates we need at a proper regular interval so that we can keep uptodate, but currently the easiest thing for you to do would be to fork the Subxt repo and keep this tweak locally for now, I'm afraid, if you want to work with substrate master in your project. Sorry for the inconvenience; I know that it sucks!

[jsdw]

(One thing I can look at is publishing the subset of crates subxt uses to bring them more uptodate, if that might help. But they will not keep pace with substrate master of course; just be a more recent snapshot!)

[jakehemmerle]

Since subxt is dependant on lots of components of substrate, why doesn't master follow the same "always point to branches or tags on substrate master" pattern that the polkadot repo does? Do you know what justifies having this up-to-date as a crate at the expense of very out-of-date dependencies?

[niklasad1]

Since subxt is dependant on lots of components of substrate, why doesn't master follow the same "always point to branches or tags on substrate master" pattern that the polkadot repo does?

That's a good question but subxt has a daily compatibility job that runs the subxt tests against the latest substrate .

Do you know what justifies having this up-to-date as a crate at the expense of very out-of-date dependencies?

As long the daily compatibility check works then subxt is regarded as "up-to-date/compatible", see my answer above.

subxt only relies on sp-core and sp-runtime but if one pulls in dependencies that has substrate dependencies that has a different version it will end-up with a compile error where one have two options either patch the sp-*, sc-* crates or fork subxt which is far from ideal.

Not sure, if it's possible to provide something in between without having to maintain a bunch of branches in subxt. Any ideas?

[jsdw]

Just as a note; I had some chats and it sounds like the work to start publishing substrate crates to crates.io is underway, so I'm perhaps optimistically hoping that we'll start doing that by the end of the year. When that happens, we'll be able to keep subxt up to date with the latest sp-* crates in a much more sane and standard way.

afaik it's not a compile error anyway to have two versions of the same crate in a project, so I am curious as to what exactly the parity-util-mem error you encountered was @jakehemmerle? I'd love to understand this, and your use case in general, much better! (Perhaps open an issue with more detail so that we can discuss further? There might turn out to be other solutions we can look at.)

As a final note, Subxt tries its best to limit its usage of the sp-* crates, and in fact I would like to have a pass over and see whether I can further strip down what we actually use from them.

[jsdw]

Since subxt is dependant on lots of components of substrate, why doesn't master follow the same "always point to branches or tags on substrate master" pattern that the polkadot repo does?

And just on this point; simply because it can't point to things on github. If it did this, it couldn't be published to crates.io, which would in turn mean that anybody using Subxt in their projects also couldn't publish anything to crates.io.