Allow component::LinkerInstance::func_new functions to return complex types

[rvolosatovs]

Feature

At present, only simple types can be returned in component::LinkerInstance::func_new

The following variants of component::Val cannot be returned by dynamic functions defined via component::LinkerInstance::func_new:

• List(List)
• Record(Record)
• Tuple(Tuple)
• Variant(Variant)
• Enum(Enum)
• Option(OptionVal)
• Result(ResultVal)
• Flags(Flags)

The reason for this is the fact that constructors of these values require a type parameter, which cannot be constructed outside the crate. While such type parameter can be extracted from a function signature exported by a component, there is currently no means to access the type for functions imported by the component. At best, implementations may parse the type information using wit_parser crate, but there is currently no way to map from wit_parser type to wasmtime type.

For example:
Enum::new requires types::Enum value as a parameter, which does not export a constructor or any conversion mechanism. This type can only be returned by wasmtime crate, but functionality to extract this from a component import does not currently appear to exist.

Benefit

This would allow dynamically-typed function definitions in the linker returning more that the basic numeric/string types (and resources once #7688 lands)

Implementation

Perhaps func_new function parameter signatures could be updated to take in a Vec<Type> (https://docs.rs/wasmtime/latest/wasmtime/component/types/enum.Type.html) or, perhaps even better signatures could get Vec<(Val, Type)> as a parameter for the result parameter. Note, that params are free from this issue, because the type info is already embedded in Val passed to the function, which is not the case for results (and there is probably no way to change that)

Alternatives

The only alternative seems not supporting dynamically-typed function imports, or only supporting a subset of those, as currently done.

[alexcrichton]

I might recommend a route such as with #4593 where we'd flesh out the runtime reflection APIs on the types used in a component to be able to inspect imports/exports and then update func_new to taking a FuncType like its core wasm counterpart as well. The one major tricky part would be runtime construction of types if that's required which is not easy, but if you're using exact types from a component then that should still work ok.

[rvolosatovs]

I might recommend a route such as with #4593 where we'd flesh out the runtime reflection APIs on the types used in a component to be able to inspect imports/exports and then update func_new to taking a FuncType like its core wasm counterpart as well. The one major tricky part would be runtime construction of types if that's required which is not easy, but if you're using exact types from a component then that should still work ok.

Given existing functionality, I don't think this would work without significant changes in component::types - that is because at present construction of Type requires a component instance to lookup InstanceType from, e.g. consider OptionType, which relies on the Handle type, which, in turn, requires an InstanceType:

wasmtime/crates/wasmtime/src/component/types.rs

Lines 428 to 430 in f3b5478

	pub(crate) fn from(index: TypeOptionIndex, ty: &InstanceType<'_>) -> Self {
	OptionType(Handle::new(index, ty))
	}

. As far as I understand the implementation, at present the only way to produce an InstanceType is to instantiate the component.
Linker::func_new is called before instantiation takes place, therefore it's not possible to produce a FuncType parameter to pass to Linker::func_new - that would depend on instantiation.
Am I missing something here?

For my concrete use case, however, introspecting the Instance types from within the func_new closure implementation is most certainly "good enough" (a handle to it is simply kept in the store). What do you think about this approach? It feels like introspecting on Instance level would be a beneficial feature anyway

[alexcrichton]

I think that's reasonable yeah, good point. There's subtelty here around resources and while it's something I believe we can solve you're right it'd require some refactoring. In the long run we're going to want to query the type of and instance in addition to a component anyway, so I think it make sense to go ahead and implement this for instances at this time too.