Expand existing tests matrix to MonoVM

[marek-safar]

This is a tracking issue for expanding existing test coverage to include workloads that will be introduced as part of the .NET6 workloads

MonoVM testing matrix for managed libraries

Platform (RID)	AOT-LLVM	Interpreter	JIT-mini
osx-x64	🚫	🚫	✅
osx-x64-ios14 (sim)	🚫	✅	🚫
osx-x64-tvos14 (sim)	🚫	✅	🚫
osx-arm64-ios14 (sim)	🚫	✅	🚫
osx-arm64-tvos14 (sim)	🚫	✅	🚫
osx-x64-maccatalyst	⬜	✅	🚫
osx-arm64-maccatalyst	⬜	✅	🚫
linux-x64	🚫	✅	✅
linux-arm64	🚫	✅	✅
browser-wasm (linux/v8)	✅	✅	🚫
linux-x64-android5 (sim)	🚫	⬜	✅
ios14-arm64	⬜	🚫	🚫
ios10-arm32	⬜	🚫	🚫
tvos-arm64	✅	⬜	🚫
android-arm64 (api21)	🚫	🚫	✅
android-arm32 (api21)	🚫	🚫	✅

MonoVM testing matrix for runtime tests

Platform (RID)	AOT-LLVM	Interpreter	JIT-mini
osx-x64	⬜	✅	✅
osx-x64-ios (sim)	⬜	⬜	✅
osx-arm64-ios (sim)	⬜	⬜	🚫
linux-x64-android5 (sim)	🚫	✅	✅
linux-x64	✅	✅	✅
linux-arm64	✅	✅	✅
browser-wasm (linux/v8)	⬜	✅	🚫
ios-arm64	⬜	🚫	🚫
ios10-arm32	⬜	🚫	🚫
tvos-arm64	⬜	🚫	🚫
android-arm64	🚫	🚫	✅
android-arm32	🚫	🚫	⬜

Updated for .NET6 priorities

---

Legend
✅ - enabled
⬜ - to be enabled
🚫 - out of scope

[Dotnet-GitSync-Bot]

I couldn't add an area label to this Issue.

Checkout this page to find out which area owner to ping, or please add exactly one area label to help train me in the future.

[ghost]

Tagging subscribers to this area: @directhex
Notify danmosemsft if you want to be subscribed.

[marek-safar]

Priority for the first milestone for both managed libraries & runtime tests

• linux-wasm-js - INT
• linux-arm64 - AOT, INT, JIT
• osx-x64-ios13 (sim) - AOT, INT
• osx-x64-tvos13 (sim) - AOT, INT
• osx-x64-android5 (sim) - INT, JIT

[naricc]

I spoke with @jashook and he was concerned about the resource usage from running the AOT-LLVM builds on PR, and suggested running them as an outer loop job, especially on arm.

I polled Mono folks, and it seems like the consensus is we would like to have at least some AOT stuff on PR because changes causing issues only on AOT are fairly common, but some platforms an be done on an outer loop.

Therefore, I propose doing iOS on PR probably makes the most sense, since it is the most important target for AOT (and we cannot JIT on iOS any way). The CI handles iOS test runs with a simulator, so it is not using physical arm hardware.

AOT builds for other platforms can be done in an outer loop.

[marek-safar]

AOT is our primary target and we could consider moving arm32 to outer loop but don't see a good reason for anything else to be in outer loop only

[naricc]

AOT is our primary target and we could consider moving arm32 to outer loop but don't see a good reason for anything else to be in outer loop only

Then maybe we should start the process of getting additional arm resources, if we don't want PR builds to be queued all day? @jashook Any idea how we do that?

[lambdageek]

@marek-safar We should consider doing some x86 CI:

1. The 32-bit x86 Android emulator is pretty popular, so we should test it some place. We've previously had bugs that only showed there.
2. Now that we have the x86 interpreter port - [interp] Add x86 support #35966 - we should also test that.

I think we can probably use a linux desktop build with a 32-bit userspace. And have the JIT and interp scenarios.

[jashook]

/cc @jaredpar

[steveisok]

@marek-safar We should be able to bring up interp lanes for mobile now pretty easily. Is that still important for net6?

[marek-safar]

Is that still important for net6?

@steveisok yes

[steveisok]

Changing the milestone to 8.0 in case we want to add to the matrix.