metadata: Generate runtime outer enums if not present in V14

[lexnv]

This PR avoids panics while constructing the suxbt client from an unexpected metadata format.

In doing so, the subxt is more robust in connecting to custom chains that may define different metadata formats (such as wss://c1.hashed.live:443).

Panics originate from two paces while converting the metadata V14 to the latest format V15:
-expectation of extrinsic types in metadata
-expectation of outer enums RuntimeCall and RuntimeError being present

For the former case, errors are propagated back to the user avoiding panic.

In the latter case:

• RuntimeCall is searched in metadata
• an extra check is added to ensure it is an enum that was previously taken for granted
• when the RuntimeCall is not found, the Call enum is searched
• if neither RuntimeCall nor Call are found, an outer enum is generated

Closes: #1171

[jsdw]

One issue: RuntimeCall, RuntimeEvent and RuntimeError are fairly unique names which are unlikely to be used for random other types. but Call, Event and Error are not, and in fact all of these names are used for other enums in the type registry (each pallet could have one or more of these types declared).

Is the Call enum that you found actually one of those per-pallet ones? I have a feeling though that possibly an old version of Substrate used Call rather than RuntimeCall or something like that, but can't remember for sure.

Anyway, in the generated polkakdot metadata, the types we want (RuntimeCall etc) are all at paths like runtime_types::polkadot_runtime::RuntimeCall, runtime_types::polkadot_runtime::RuntimeError and runtime_types::polkadot_runtime::RuntimeEvent. (other types all live in runtime_types but in other places). I wonder whether the _runtime part of the polkadot_runtime path can be used to help find these things. Or else, maybe we just return an error and don't try falling back to anything else.

[lexnv]

Yep that makes sense, checking if a Call is indeed the enum we are looking for can become quite tedious :D

I've changed the code to not have fallbacks; propagate the error back to the user and only generate the RuntimeError if not present, which seems to be more straight forward

[jsdw]

It's a bit weird saving a path mutably like this I think. More code but IMO clearer would be something like this:

// here, return eg:
Some((ty.id, segments))

// and then:
let call_enumm = find_type("RuntimeCall");
let event_enumm = find_type("RuntimeEvent");
let error_enumm = find_type("RuntimeError");

// Pick a path to save any generated types to; aim to
// put them next to whatever existing type we find first,
// or fall back to dumping it in the "top level" runtime_types
// path.
let path = match (call_enum, event_enum, error_enum) {
    (Some((_,p)), None, None) => p,
    (None, Some((_,p)), None) => p,
    (None, None, Some((_,p))) => p,
    (None, None, None) => vec!["runtime_types".into()]
};

// ...

[lexnv]

I'll remove that bit; fun fact I had initially thought of something similar with call.or(event).or(error) :D

[jsdw]

Ooh yeah you're right; using .or would have been much more compact :D

[jsdw]

Is there any sensible way to test that we return gracefully (ie without a panic) in the cases where Runtime*'s aren't available; if so that would be really nice :)

[tadeohepperle]

Nice, good job!

[jsdw]

Looks good; nice one!

[kylezs]

We were on 0.31.0 and I was about to report a bug but thought I'd check later versions first, and saw it was fixed here 🙌 thanks.