testing: Prepare light client testing with substrate binary and add subxt-test macro

[lexnv]

In this PR

• subxt-macro is added. The macro runs the tests with a default timeout (6 minutes if not specified)
• added cfg aliases for our testing with a new unstable-light-client-long-running feature flag to reuse subxt testing with the lightclient
• some tests are disabled in the context of the light-client
• chainHead initialized event supports the legacy format (to make it work with our outdated lightclient version)
• small testing refactoring to decouple the construction of transaction from the submit operation (might still be needed in the future if we want to retry failed light-client operations)

Builds on top of: #1459 (because the CI was not running anymore)

[jsdw]

Nit: also rename the folder to subxt-test-proc-macro (or subxt-test-macro if you prefer to shorten this crate name)?

[jsdw]

So, if using the light client, we try submitting a couple of additional times to handle light client specific issues?

I wonder if there's anything we can do to avoid needing to do this; does needing it imply that using the light client to submit transactions now is not that robust and will fail sometimes? Is there anything we can do before we try submitting that might help things (ie wait a few blocks), so that by the time we submit we don't need anything special? Is this just because of the legacy backend and everything works ok with the unstable backend (and in that case I think I'd only try testing the unstable backend with the light client)?

If we do need to keep this function then I think it'd be worth adding some docs to explain why it's used for future us :)

[lexnv]

Yep that makes sense. I've removed the tx_retries for now.
Instead maybe we should report those tx::Dropped events to smoldot if they still happen after an upgrade.

[jsdw]

LGTM!

Is the effect of removing that function that some tests will fail sometimes now when using the light client?

[lexnv]

When we run tests with the unstable-light-client-long-running we'll probably encounter more flaky failures to due TransactionEvent::Dropped. However, that is not enabled now in the CI and I would generally wait to enable that until we have: #1400.

I'll look into that issue shortly, hopefully we could handle that and create a PR in smoldot if the fix is not too involved

[niklasad1]

slightly weird comment; we could actually throw an error if both finalized_block_hash and finalized_block_hashes.

now, we checking finalized_block_hashes first and if that doesn't exist we fallback to finalized_block_hash which is probably fine but if one inserts both by accident it lead to confusing scenarios but fine :)

[niklasad1]

I know this only exists until we update smoldot :)

[niklasad1]

is it possible to merge the use stuff to avoid repeating #[cfg(fullclient)]?

[niklasad1]

would be nicer/more readable to use Duration here IMO

[lexnv]

We would still need to pass std::time::Duration to the quote! below, which does not impl toTokens , I've renamed this to illustrate better that we are counting seconds, thanks!

[niklasad1]

awesome, I really like this