[Wasm RyuJIT] Stub Generator Design/Development

[kg]

See also #121215 and #113692.

Wasm RyuJIT will need a stub generator component that's responsible for generating native-to-managed and managed-to-native transition stubs of various kinds with various signatures. Right now we're using a mix of hand written stubs in the runtime and stubs generated by a prototype stub generator, all in C++ that gets compiled and linked in.

This generator needs to be able to run during build, not just during publish, and needs to account for all the implementation details of both the native Wasm C ABI and our managed ABI.

Scenarios where we might need stubs:

• Indirect calls from AOT, since we don't know whether the target is interpreted or not. This includes things like virtuals, delegates and unmanaged calli
• Calling code we know wasn't AOT'd (this is a Wasm->interp transition)
• Calling from interp to AOT'd code
• Reverse pinvoke (unmanagedcallersonly)

Our approach here will be built on to enable pinvoke and general interop. We will be using Crossgen2 to generate stubs during build & publish because it knows all of the calls being performed by the application code and whether they are virtual, etc. Doing this requires at least having the JIT perform an import for all the methods that may need to perform a call using a stub, even if we don't generate AOT'd code for those methods.

• The presence of pinvokes in user code in Wasm can be sensed via a NativeFileReference in their csproj which indicates we need to generate stubs and relink.
• The presence of UnmanagedCallersOnly can be detected by scanning the CustomAttribute table in the user's binaries.
• The presence of unmanaged calli requires scanning method bodies or importing the code in the JIT.

Crossgen2's responsibility here will be to identify all the stubs that are needed, and then produce them as an object file that can be linked into the runtime - it can either do this by generating IL and then RyuJITing the IL, or by generating raw WASM. It will likely do the former. This is a separate responsibility from actually AOTing BCL or application code; if an end-user application contains new PInvokes crossgen2 will need to run to scan those pinvokes and generate stubs, but the application itself will not need to be AOTed so this invocation should be relatively fast.

This requires shipping a linker of some sort (likely wasm-ld) to link the stubs into the runtime but this is much cheaper than shipping a full clang toolchain. The runtime will need to be shipped in at least two pieces:

• Runtime without stubs object
• Stubs object for stock runtime

And then linked at build time, either using the stock runtime stubs object or using one that was generated by crossgen2 taking into account the additional stubs needed by user code. As an optimization, we can ship a pre-linked 'stock runtime with stock stubs' if necessary.

(incomplete - work in progress)

[dotnet-policy-service]

Tagging subscribers to 'arch-wasm': @lewing, @pavelsavara
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.