MacOS WasmVM is "broken": Link Error: "function types incompatible: expected func of type (i64) -> (), found func of type (i32) -> ()

[frol]

Describe the bug

nearcore 2.1.1+ built for macOS cannot deploy/execute any contract:

Error: Action #1: ExecutionError("Link Error: \"function types incompatible: expected func of type `(i64) -> ()`, found func of type `(i32) -> ()`\"")

https://github.com/near/near-workspaces-rs/actions/runs/10741891187/job/29793334139

This issue blocks all the tooling upgrades (near-sandbox-js, near-sandbox-rs, near-workspaces-*, ...).

It was reported before: #11782 (comment)

To Reproduce

Pull this commit: near/near-workspaces-rs@9186ff1, and run tests (cargo test)

Expected behavior

Tests should pass.

Version (please complete the following information):

• nearcore commit/branch: 2.1.1 (2.0.0 works fine)
• rust version (if local)
• docker (if using docker)
• local (MacOS)

[akorchyn]

The same issue happens on the 2.2.0 release:
https://github.com/near/near-workspaces-rs/actions/runs/10813176493/job/29996604197?pr=376

[akorchyn]

I have made some initial investigation and tried to determine the failure point.
Using some Github actions and binary search I have located the next commit:
872ac5c

I have adjusted to compile the nearcore sandbox (as some versions are not available on s3) and run tests:

This commit fails on the CI
https://github.com/near/near-workspaces-rs/actions/runs/10835306324/job/30066537415 - fails
The commit before passes on the CI
https://github.com/near/near-workspaces-rs/actions/runs/10835654098/job/30067647491

[akorchyn]

The issue is reproducible on any OS using:
cargo t -p near-vm-runner --features wasmtime_vm,near_vm

[akorchyn]

Weird, but it's only reproducible if you remove engine from WasmtimeVM and recreate it each time. Otherwise, test passes :(
As I did here for example: master...akorchyn:nearcore:test-fix

[akorchyn]

If the engine is new, it won't work. Potentially the issue is within: precompile and then prepare and run on another engine.

To reproduce the issue just use a new engine before the linker.instantiate()
It's a bit weird as the documentation says that if you have the same compatibility hash it should work.

Though, it doesn't

Returns a [std::hash::Hash] that can be used to check precompiled WebAssembly compatibility.

The outputs of [Engine::precompile_module] and [Engine::precompile_component]
are compatible with a different [Engine] instance only if the two engines use
compatible [Config]s. If this Hash matches between two [Engine]s then binaries
from one are guaranteed to deserialize in the other.