feat: add embedded client back in

[sander2]

The integration test requires a slight modification on the substrate repo, which is why this PR is currently using a fork. I will make a PR on there soon.

One unfortunate thing about this PR is that while the client is feature-gated, it is enabled by default in order to make the integration test work. The number of transitive dependencies being pulled in is quite significant. The alternative is to not have it enabled by default, but then integration tests will fail to compile unless called with e.g. --all-features.

[sander2]

I had to disable this, because this feature is kind of broken. I kept getting license check failed from rustfmt even after doing cat FILE_TEMPLATE src/client.rs > /tmp/f && mv /tmp/f src/client.rs

[sander2]

@ascjones @dvdplm Could you guys have a look at this pr please?

[ascjones]

The reasoning for removing this were:

1. Big increase in CI build times because of the substrate dependencies, as you allude to above.
2. Tightly coupled to substrate and jsonrpsee APIs.
3. All the tests for it had been removed, replaced by spawning a test node (though I do see you have added one integration test back)

And generally to keep the library as lightweight as possible, in terms of amount of code to maintain, dependencies and build times etc. And also as much flexibility of supported substrate versions as possible.

I understand the benefit of being able to run an embedded light-client node for decentralization etc - so we do need a story for it, I'm just not convinced that it should necessarily exist inside this crate.

Can you explain to me how you this crate fits into your infrastructure?

[sander2]

I'm just not convinced that it should necessarily exist inside this crate.

Hmm maybe if we introduce some configuration trait we can actually create it as a separate crate outside of this repo.

Can you explain to me how you this crate fits into your infrastructure?

We use it for testing, which has the advantage of not having to host the binaries for the parachain, and it helps in debugging because you can set breakpoints inside of parachain code. It's also convenient that cargo test just work without having to manually download a binary.

Having said all that, the current setup we use also has its problems. Specifically, the versions of substrate crates used in the parachain need to align with the ones used in subxt, otherwise we get compilation errors because the correct trait versions are not implemented, which is a pain to maintain.

[ascjones]

Hmm maybe if we introduce some configuration trait we can actually create it as a separate crate outside of this repo.

By all means if we can have some abstraction here that would allow plugging in of an embedded client then I'm all for that.

Also @niklasad1 mentioned some changes that were coming to jsonrpsee that could make the glue code we had for the embedded client redundant. Hope he can provide more info.

[ascjones]

Linking #341

[jsdw]

We've decided that an embedded client shouldn't form part of the subxt library at the moment owing to the maintenance burden and complexity of keeping dependencies aligned. See #341 for a little more detail.

So I'm going to close this for now.