[dotnet] Support interpreter-only mode in the simulator #11421

lambdageek · 2021-05-03T15:32:37Z

For .NET 6 hot reload to work, and for other scenarios, we need to be able to run apps using the interpreter on the simulator.

Currently calling the following code from AppDelegate.FinishedLaunching returns "JIT"

using System.Runtime.CompilerServices;

public string GetExecutionMode()
{
  if (!RuntimeFeature.IsDynamicCodeSupported)
    return "AOT";
  if (RutnimeFeature.IsDynamicCodeCompiled)
    return "JIT";
  return "Interpreter";
}

The text was updated successfully, but these errors were encountered:

spouliot · 2021-05-03T15:38:11Z

Application.mtouch.cs is only used for mtouch, which is not used for dotnet (only legacy)

Are you seeing the MT0141 error ?

lambdageek · 2021-05-03T15:40:03Z

One way to test whether the interpreter is enabled is using

using System.Runtime.CompilerServices;

public string GetExecutionMode()
{
  if (!RuntimeFeature.IsDynamicCodeSupported)
    return "AOT";
  if (RutnimeFeature.IsDynamicCodeCompiled)
    return "JIT";
  return "Interpreter";
}

This test could be adjusted to (a) run, not just build; (b) try to use the above method to detect the execution engine.

https://github.com/xamarin/xamarin-macios/blob/5c12fdfac92277cd19cb4c73daa083ddc75d781b/tests/dotnet/MyInterpretedApp/MyInterpretedApp.csproj

spouliot · 2021-05-03T15:56:51Z

(a) run, not just build

No, those projects don't have a way to communicate success/failure back to the runner. They exists so the tooling, like msbuild can be verified, or to check for files expected presence (or absence), app bundle size...

Even if we added a success/failure execution report on run it would not tell us much. The interpreter could be enabled but could be crashing in various places that not part of the simple app.

To achieve more useful results we run existing unit tests under several configurations, including the interpreter (at least for legacy). That validates the build and that the feature works across a large set of tests.

If would be easy to add a dotnet specific test for RuntimeFeature so we know it returns the expected values under the different build configuration.

spouliot · 2021-05-03T15:59:45Z

Example from xharness: All configurations of monotouch-test for iOS Unified 64-bits - device

iOS Unified 64-bits - device (Ignored)Run all Build all
+AssemblyBuildTarget: dylib (debug) (Ignored) Run Build
+AssemblyBuildTarget: dylib (debug) [dotnet] (Ignored) Run Build
+AssemblyBuildTarget: dylib (debug, profiling) (Ignored) Run Build
+AssemblyBuildTarget: dylib (debug, profiling) [dotnet] (Ignored) Run Build
+AssemblyBuildTarget: SDK framework (debug) (Ignored) Run Build
+AssemblyBuildTarget: SDK framework (debug) [dotnet] (Ignored) Run Build
+AssemblyBuildTarget: SDK framework (debug, profiling) (Ignored) Run Build
+AssemblyBuildTarget: SDK framework (debug, profiling) [dotnet] (Ignored) Run Build
+AssemblyBuildTarget: SDK framework (release) (Ignored) Run Build
+AssemblyBuildTarget: SDK framework (release) [dotnet] (Ignored) Run Build
+Debug (Ignored) Run Build
+Debug (all optimizations) (Ignored) Run Build
+Debug (all optimizations) [dotnet] (Ignored) Run Build
+Debug (dynamic registrar) (Ignored) Run Build
+Debug (dynamic registrar) [dotnet] (Ignored) Run Build
+Debug (interpreter -mscorlib) (Ignored) Run Build
+Debug (interpreter -mscorlib) [dotnet] (Ignored) Run Build
+Debug (interpreter) (Ignored) Run Build
+Debug (interpreter) [dotnet] (Ignored) Run Build
+Debug [dotnet] (Ignored) Run Build
+Debug: SGenConc (Ignored) Run Build
+Debug: SGenConc [dotnet] (Ignored) Run Build
+Release (Ignored) Run Build
+Release (all optimizations) (Ignored) Run Build
+Release (all optimizations) [dotnet] (Ignored) Run Build
+Release (interpreter -mscorlib) (Ignored) Run Build
+Release (interpreter -mscorlib) [dotnet] (Ignored) Run Build
+Release [dotnet] (Ignored) Run Build

lambdageek · 2021-05-03T16:07:02Z

Application.mtouch.cs is only used for mtouch, which is not used for dotnet (only legacy)

Are you seeing the MT0141 error ?

No I'm not. I guess it must be something else that is affecting the execution engine choice. Sorry. let me update the PR description with just the observable behavior and not my guesses.

lambdageek · 2021-05-03T16:30:10Z

Update MONO_AOT_MODE_INTERP is the right thing to use for iOS device and iOS Simulator (particularly on Apple Silicon macs).

Hmm... I have a sneaking suspicion that this is due to MONO_AOT_MODE_INTERP

xamarin-macios/tools/common/Target.cs

Line 764 in b720b44

sw.WriteLine ("\tmono_jit_set_aot_mode (MONO_AOT_MODE_INTERP);");

I think that (as opposed to MONO_AOT_MODE_INTERP_ONLY) means "use interpreter as the fallback execution engine". which on simulator (since we're using the JIT) means "try to JIT; if it fails (it won't), use interpreter".

The mono tests in dotnet/runtime use MONO_AOT_MODE_INTERP_ONLY.

I'm going to try changing the runtime tests to use MONO_AOT_MODE_INTERP and check what behavior we get there.

lambdageek · 2021-05-04T03:59:20Z

I think that [MONO_AOT_MODE_INTERP] (as opposed to MONO_AOT_MODE_INTERP_ONLY) means "use interpreter as the fallback execution engine". which on simulator (since we're using the JIT) means "try to JIT; if it fails (it won't), use interpreter".

Hmm... actually I don't know that this is correct. I think it's possible the IsDynamicCodeSupported/Compiled intrinsics might not be telling me the right things. Need to check some actual dynamic code and see if it's JITed or interpreted

lambdageek · 2021-06-08T18:13:11Z

So here's what needs to happen to get the interpreter working on iOS simulator:

We need to compile at least System.Private.CoreLib.dll with mono --aot=interp and make the resulting aot image available to the runtime. It's ok to AOT other assemblies with --aot=interp, but it's not necessary. the S.P.C AOT image contains all the necessary interpreter trampolines that we care about.
Runtime should start with MONO_AOT_MODE_INTERP.

On iOSSimulator-arm64 we will run into W^X issues until a runtime pack that includes dotnet/runtime#53197 is used (I think .NET 6 Preview 6), but iOSSimulator-x64 should be ok now.

@spouliot I think the steps above are similar to what is used for the interpreter on ios devices (although maybe it uses mono --aot=interp,full, not mono --aot=interp) do you know what is the best way to fix enable the interp for the simulators too?

and #11724. (#12211) * Add support for the interpreter everywhere. * Add support for the AOT compiler everywhere we didn't support it before, because the interpreter needs it (at least System.Private.CoreLib.dll must be AOT-compiled when using the interpreter). * Do FullAOT compilation on Mac Catalyst/ARM64 if we're not using the interpreter, since we can't use the JIT. * Fix monotouch-test to be green on Mac Catalyst/ARM64. Fixes #11724. Fixes #11421.

… AOT when needed. Fixes #11421 and #11724. (#12237) * [dotnet] Refactor and extend the logic to determine whether we should run the AOT compiler or not. We also can't link dynamically with libmonosgen-2.0.dylib if we AOT compile anything, so make sure we don't do that. * [runtime] AOT is no longer limited to device builds. * [runtime/tools] Implement finding native mono lib for Mac Catalyst. This also meant propagating how libmono is linked from the MSBuild code to the Application class so that our existing logic is able to correctly determine which native mono lib to use. * [tools] Implement interpreter + AOT compilation for simulator + Mac Catalyst. The interpreter requires running the AOT compiler for System.Private.CoreLib.dll, so the first step in implementing the interpreter is to implement support for the AOT compiler as well. For .NET the logic is now as follows: * If the interpreter is enabled, AOT compile System.Private.CoreLib.dll. * If the interpreter is disabled, AOT everything on device + Mac Catalyst on ARM64. * [tests] Add a MyInterpretedApp test solution * [dotnet] Parse --dlsym and pass it to the dotnet-linker tasks. * [dotnet] Only list linker flags for assemblies after we've computed them. This means not listing per-assembly linker flags for only binding projects, but delay it until we've computed the linker flags for all assemblies (and reflect this in variable names as well). * [dotnet-linker] Add native references to libSystem.Security.Cryptography.Native.Apple for Mac Catalyst. Fixes these test failures: Ctor_Trust: System.DllNotFoundException : libSystem.Security.Cryptography.Native.Apple MailX1: System.DllNotFoundException : libSystem.Security.Cryptography.Native.Apple Encrypt_Empty: System.DllNotFoundException : libSystem.Security.Cryptography.Native.Apple KeyRecordTest: System.DllNotFoundException : libSystem.Security.Cryptography.Native.Apple Basic_Leaf_Only: System.DllNotFoundException : libSystem.Security.Cryptography.Native.Apple which are all because of Xamarin.MacCatalyst: Unable to resolve P/Invoke 'AppleCryptoNative_X509GetContentType' in the library 'libSystem.Security.Cryptography.Native.Apple' which happens because without this change we're not linking with the static version of libSystem.Security.Cryptography.Native.Apple. * [monotouch-test] Fix some formatting and add assert messages * [tools] Disable direct-pinvoke for AOT-compiled code on Mac Catalyst. Using 'direct-pinvoke' will tell the AOT compiler to emit a direct call to the native function declared in the DllImport, which doesn't work when we want to redirect to a different native funcion (in our xamarin_pinvoke_override/PINVOKE_OVERRIDE implementation). This fixes our exception marshalling tests in monotouch-test. * [monotouch-test] Improve the MTLDeviceTest to work on Mac Catalyst. * [tools] Always use '__Internal' as the native library for .NET. This is not the final fix, but this code will be reworked extensively, and this fix works for now to make the interpreter work. * [monotouch-test] Ignore the MTLDeviceTest.ReturnReleaseTest on Mac Catalyst. It requires a backport of other test fixes (which we're not really interested in), so just ignore it. Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

spouliot added dotnet An issue or pull request related to .NET (6) dotnet-pri0 .NET 6: required for stable release labels May 3, 2021

spouliot added this to the .NET 6 milestone May 3, 2021

lambdageek changed the title ~~[dotnet] Cannot use interpreter in the simulator~~ [dotnet] Support interpreter-only mode in the simulator May 3, 2021

lambdageek changed the title ~~[dotnet] Support interpreter-only mode in the simulator~~ [dotnet] Support interpreter-first mode in the simulator May 3, 2021

lambdageek changed the title ~~[dotnet] Support interpreter-first mode in the simulator~~ [dotnet] Support interpreter-only mode in the simulator May 3, 2021

lambdageek mentioned this issue Jun 15, 2021

Mono Method Body Replacement dotnet/runtime#44806

Closed

51 tasks

rolfbjarne self-assigned this Jun 24, 2021

rolfbjarne mentioned this issue Jul 23, 2021

[dotnet] Add support for the interpreter + AOT when needed. Fixes #11421 and #11724. #12211

Merged

rolfbjarne closed this as completed in #12211 Jul 27, 2021

vs-mobiletools-engineering-service2 mentioned this issue Jul 27, 2021

[release/6.0.1xx-preview7] [dotnet] Add support for the interpreter + AOT when needed. Fixes #11421 and #11724. #12237

Merged

ghost locked as resolved and limited conversation to collaborators Apr 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dotnet] Support interpreter-only mode in the simulator #11421

[dotnet] Support interpreter-only mode in the simulator #11421

lambdageek commented May 3, 2021 •

edited

Loading

spouliot commented May 3, 2021

lambdageek commented May 3, 2021 •

edited

Loading

spouliot commented May 3, 2021

spouliot commented May 3, 2021

lambdageek commented May 3, 2021

lambdageek commented May 3, 2021 •

edited

Loading

lambdageek commented May 4, 2021

lambdageek commented Jun 8, 2021 •

edited

Loading

[dotnet] Support interpreter-only mode in the simulator #11421

[dotnet] Support interpreter-only mode in the simulator #11421

Comments

lambdageek commented May 3, 2021 • edited Loading

spouliot commented May 3, 2021

lambdageek commented May 3, 2021 • edited Loading

spouliot commented May 3, 2021

spouliot commented May 3, 2021

lambdageek commented May 3, 2021

lambdageek commented May 3, 2021 • edited Loading

lambdageek commented May 4, 2021

lambdageek commented Jun 8, 2021 • edited Loading

lambdageek commented May 3, 2021 •

edited

Loading

lambdageek commented May 3, 2021 •

edited

Loading

lambdageek commented May 3, 2021 •

edited

Loading

lambdageek commented Jun 8, 2021 •

edited

Loading