From 39b87ee17b1d31a845289e1a534de0691da53680 Mon Sep 17 00:00:00 2001 From: Katelyn Gadd Date: Wed, 31 Aug 2022 19:42:45 -0700 Subject: [PATCH 1/3] Initial checkpoint of jiterpreter prototype integration Jiterpreter traces work (yay alignment) Use function pointers to invoke jiterpreter traces Implement binops, basic arithmetic, and some unary ops cknull Strlen ldloca_s + print statistics at exit getchr, track bailout count Use a JS dispatcher for traces because emscripten addFunction is bad. Use unchecked memory setters in traces. Enable tracing for more code. Checkpoint ldfld Checkpoint stfld Checkpoint: F32 arithmetic Add enable flag Move back to addFunction because while the overhead of adding the trace functions is higher, the invoke performance is better Checkpoint: Traces continue past brfalse/brtrue if the branch is not taken Checkpoint wasm generation Generates dummy traces Checkpoint: Works again Optimized trace generator Traces work again Fixed codegen for memmove and ldloca Use wasm memmove instruction Implement cknull Implement brfalse and brtrue Refactor null checks stloc refactoring Basic ldfld/stfld support Checkpoint ldind (broken) ldind works ldfld_o stfld_o ldfld_vt Unary math intrinsics and ldlen/ldelem_ref Fix math intrinsics, add some i8 operations Working (u|i)32 -> i64 promotion stind Add fp remainder operation, adjust thresholds to avoid making tests way slower Fix remainder codegen, implement mono_ldptr ldelema1 ldelema1 error handling ldelem for ints and floats Checkpoint trace enter optimization (broken) Generate moves instead of a memmove for small constant-sized memmoves More accurate trace length measurement, higher trace length requirement for better perf Check in simple raytracer sample code De-inline intersection test because the branches cause lots of trace bailouts add_iN_imm and mul_iN_imm ldc_i8 for small constants 64-bit relops and a few 64-bit superinsns Checkpoint relop branches Fixed relop branches Specify alignment of ldloc/stloc where possible Fix imm add/mul opcodes and improve debugging Pass locals to traces instead of frame. Hard-code data items into the trace instead of loading them. Pass method name to prepare_jiterpreter, include method name in trace name, add flag to trace method name when aborts happen. Add Min and Max interpreter opcodes Try to keep interpreter tiering working Add abs interp opcode, add missing float32 interp opcodes. Fix codegen for f32 pow/min/etc. Don't generate additional jiterpreter enter opcodes when inlining. Fix a warning Fix more warnings, don't pass ip to traces Checkpoint work on static fields; fix unbalanced stack on stfld Static fields work Support for generating trace entry points at the target of all backward branches. When bailing out for a branch, add the displacement to the bailout target. Restructure the simple raytracer Categorize bailouts and count each type separately when counting is active Integer divide and remainder Implement MUL_OVF_I4 and MUL_OVF_UN_I4. Fix the getelema failure check being inverted. Add support for more precise tracking of what method calls cause traces to abort Implement INITOBJ Implement STIND_REF i4 to u1/u2 conversions Add i4->i2 and i4->i1 conversions Fix min and max interp opcodes Detect when a trace is approaching the browser wasm size limit (4kb) and abort it so that it doesn't fail to compile Debugging and logging improvements Optimize generated module boilerplate for size Rework branch patch point system to be less sloppy Handle different branch depths Checkpoint new approach to branches Fix ldc failures unbalancing the stack. Fix some branch problems Implement GETCHR and STRLEN Fully qualified trace names Move some stuff into a new file Optimize some null checks to not perform a double memory load Cache i4 mul/div/rem operands in locals instead of re-loading them from memory after range checks Less aggressive 'a branch can never skip past this' abort for calls Fix math operand caching Fix ldlen Improved error handling and instrumentation u4 -> i4 with underflow check Shifts with immediates ldind with constant offset Re-enable stelem Add ldlen_span, fix incorrect base in trace error messages newobj_vt_inlined and newobj_inlined ldc_i8 for full value range ldobj_vt and cpobj_vt stobj_vt br(true|false)_i8_s and ldftn_addr Fix local index assignment. implement unsigned i8 div and rem intrins_span_ctor Improved instrumentation for estimated opcode importance, implement more intrinsics If a method has the Intrinsic attribute, treat it as if AggressiveInlining were set in the interpreter. This makes System.Numerics.Vectors types much faster. Fix remainder opcodes. Implement castclass and isinst Implement INTRINS_MEMORYMARSHAL_GETARRAYDATAREF. Add partial support for safepoint branches and with-immediate relop branches ldelem_vt Replace JS remainder implementation with a wrapper around C fmod. Implement atan2. Fix casing so aot-cross can be located during build of samples Fix i32/i64 type mismatch for an opcode. Rebase damage fixes. When rejecting a trace, set its abort reason to trace-too-small. Implement r8 conditional branches. Implement ldind_offset stfld_vt Checkpoint: Easier enable/disable for jiterpreter, mess up the raytracer sample so that it causes AOT to bail into the interp Implement MINT_UNBOX, fix counters Optimize some jiterpreter entry points by moving them into interp.c so they can be inlined Don't call stackval_from_data for pointers Checkpoint backwards branch support Backwards branch entry points seem to work now Rename various jiterpreter entry points and make sure they don't use external APIs or volatile Add some missing indirect opcodes and rework the indirect code Fix negative constant indirect offsets Implement MINT_BOX Adjust thresholds and add some missing offset opcodes Code cleanup Move various jiterpreter code to its own .c file Checkpoint Checkpoint Add heuristic filter to avoid inserting trace enter opcodes in places where they are unlikely to produce a trace Fix crash at startup with jiterpreter turned off Branch block heuristic fixes When estimating trace lengths, don't count opcodes like nop and dummy_use Lower trace length threshold since it ignores dummy opcodes now, adjust test benchmark to be slower, prevent leb being linked out when jit wrappers are enabled Repair rebase damage eslint and wasi build fixes --amend C4206 workaround Diff cleanup Diff cleanup Fix jiterp opdefs Checkpoint: Move most jiterpreter options into c so they can be accessed from everywhere Move the opcode abort counts into C and update them in the heuristic abort logic as well. More options improvements Revert inlining limit to see if it fixes the crash; fix osx build Manually enable jiterpreter stats in browser-bench lint fix; implement ld_delegate_method_ptr since it shows up in browser bench wrappers Disable jiterpreter and wrapper jit by default if threading is enabled. Update comments. Checkpoint code cleanup Code cleanup Code and project file cleanup Implement castclass_interface and isinst_interface Implement localloc and newarr. Pass frame pointer to traces. CI build fix Rework call target tracking so it's more useful. Move jiterpreter APIs to INTERNAL. Implement add_ovf for i4 and u4, implement leave_s, improve usefulness of hottest failed traces output Implement LDTSFLDA, improve trace list filtering Implement BOX_VT Unify overflow check and castclass/isinst helpers to reduce number of imports. Implement more conversions and cast/isinst variants Unify overflow conversions and implement a few more. Implement i8 immediate comparison branches. Fix overflow in bailout counting Checkpoint: add jit for specialized do_jit_call trampolines Checkpoint Optimized out wasmtable.get overhead when invoking jit call wrappers Revert test changes Checkpoint repairing merge damage Temporarily disable getitem_span since it changed Implement getitem_span and getitem_localspan CI build fixes Address PR feedback Address PR feedback Address PR feedback Address PR feedback Refactor import section generation; checkpoint non-indirect jit call wrappers Use direct call instead of indirect call for jit call wrappers. Optimize some of the __linker_exports generated wrappers to use .call instead of .apply so there is less runtime overhead on C-to-JS calls Improved linker export wrappers for lower c-to-js overhead Replace linker export wrappers with the actual export functions at startup to remove overhead Remove debug print and fix infinitely growing import list Revert lib.js changes since the optimized wrapper complexity is not needed anymore lint fix for CI build Fix computed goto non-wasm interpreter build due to jiterpreter opcodes Implement LEAVE and BR alongside the short versions Implement support for wasm exception handling in jit_call wrappers Aligned memops in a few key places Remove MINT_NEWARR support since it appears to break NDPin test and it's not terribly important to have. Fix instrumented method support Remove instrumented method JIT do_jit_call wrappers in groups if possible instead of one at a time. If multiple do_jit_call cinfos share the same target function, share a single wrapper Improve error message for missing lazy cwraps Lower initial heap size for browser-bench because the default of 512mb causes OOM during the appstart tests Also generate groups of interp_entry wrappers in one go instead of generating one module each Repair merge damage Fix name collision when generating multiple interp entry wrappers at once Inline stackval_from_data logic for common types into interp_entry wrappers Optimize jitcall hit counter Fix jitcall counter continuing to increment even if jit fails Detect whether the runtime supports WASM EH and if not, disable it automatically Improve formatting of log messages Code cleanup Fix crash due to not handling the unbox flag for the this-reference in interp_entry wrappers Rename some of the jiterpreter configuration bools Add a missing jiterpreter config switch Don't rely on atob for the wasm EH support probe, just convert a hex literal by hand instead Clean up some options, adjust some thresholds, and add a couple new comments Code cleanups, rearrangement, and add comments Add support for optimized JS and WASM EH based implementations of mono_llvm_cpp_catch_exception Formatting improvements --- src/mono/mono/mini/CMakeLists.txt | 3 +- src/mono/mono/mini/driver.c | 7 + src/mono/mono/mini/interp/interp-internals.h | 37 + src/mono/mono/mini/interp/interp.c | 327 +- src/mono/mono/mini/interp/jiterpreter.c | 1083 ++++++ src/mono/mono/mini/interp/jiterpreter.h | 96 + src/mono/mono/mini/interp/mintops.def | 5 + src/mono/mono/mini/interp/transform.c | 21 +- src/mono/mono/mini/interp/transform.h | 7 + src/mono/sample/wasm/browser-bench/main.js | 9 +- .../sample/wasm/simple-raytracer/Makefile | 11 + .../sample/wasm/simple-raytracer/Program.cs | 304 ++ .../wasm/simple-raytracer/RayTracer.csproj | 23 + .../sample/wasm/simple-raytracer/index.html | 26 + src/mono/sample/wasm/simple-raytracer/main.js | 26 + src/mono/wasm/runtime/cwraps.ts | 38 + src/mono/wasm/runtime/do-jit-call.wasm | Bin 0 -> 103 bytes src/mono/wasm/runtime/do-jit-call.wat | 14 + src/mono/wasm/runtime/dotnet.d.ts | 2 + src/mono/wasm/runtime/driver.c | 12 +- src/mono/wasm/runtime/es6/dotnet.es6.lib.js | 8 + src/mono/wasm/runtime/exports-internal.ts | 7 + src/mono/wasm/runtime/exports-linker.ts | 12 +- .../wasm/runtime/jiterpreter-interp-entry.ts | 542 +++ src/mono/wasm/runtime/jiterpreter-jit-call.ts | 504 +++ src/mono/wasm/runtime/jiterpreter-opcodes.ts | 1833 ++++++++++ src/mono/wasm/runtime/jiterpreter-support.ts | 788 +++++ src/mono/wasm/runtime/jiterpreter.ts | 3066 +++++++++++++++++ .../wasm/runtime/net6-legacy/export-types.ts | 2 +- src/mono/wasm/runtime/run.ts | 5 +- src/mono/wasm/runtime/types/emscripten.ts | 2 + src/mono/wasm/wasm.proj | 4 + 32 files changed, 8800 insertions(+), 24 deletions(-) create mode 100644 src/mono/mono/mini/interp/jiterpreter.c create mode 100644 src/mono/mono/mini/interp/jiterpreter.h create mode 100644 src/mono/sample/wasm/simple-raytracer/Makefile create mode 100644 src/mono/sample/wasm/simple-raytracer/Program.cs create mode 100644 src/mono/sample/wasm/simple-raytracer/RayTracer.csproj create mode 100644 src/mono/sample/wasm/simple-raytracer/index.html create mode 100644 src/mono/sample/wasm/simple-raytracer/main.js create mode 100644 src/mono/wasm/runtime/do-jit-call.wasm create mode 100755 src/mono/wasm/runtime/do-jit-call.wat create mode 100644 src/mono/wasm/runtime/jiterpreter-interp-entry.ts create mode 100644 src/mono/wasm/runtime/jiterpreter-jit-call.ts create mode 100644 src/mono/wasm/runtime/jiterpreter-opcodes.ts create mode 100644 src/mono/wasm/runtime/jiterpreter-support.ts create mode 100644 src/mono/wasm/runtime/jiterpreter.ts diff --git a/src/mono/mono/mini/CMakeLists.txt b/src/mono/mono/mini/CMakeLists.txt index c68af8beb34ca9..d9ceca747682ea 100644 --- a/src/mono/mono/mini/CMakeLists.txt +++ b/src/mono/mono/mini/CMakeLists.txt @@ -284,7 +284,8 @@ set(interp_sources interp/mintops.c interp/transform.c interp/tiering.h - interp/tiering.c) + interp/tiering.c + interp/jiterpreter.c) set(interp_stub_sources interp-stubs.c) diff --git a/src/mono/mono/mini/driver.c b/src/mono/mono/mini/driver.c index 2b8fb62b44b8a1..96dbce859c0caa 100644 --- a/src/mono/mono/mini/driver.c +++ b/src/mono/mono/mini/driver.c @@ -62,6 +62,10 @@ #include "mini-runtime.h" #include "interp/interp.h" +#if HOST_BROWSER +#include "interp/jiterpreter.h" +#endif + #include #include #include @@ -1843,6 +1847,9 @@ mono_jit_parse_options (int argc, char * argv[]) } else if (strncmp (argv [i], "--profile=", 10) == 0) { mini_add_profiler_argument (argv [i] + 10); } else if (argv [i][0] == '-' && argv [i][1] == '-' && mini_parse_debug_option (argv [i] + 2)) { +#if HOST_BROWSER + } else if (argv [i][0] == '-' && argv [i][1] == '-' && mono_jiterp_parse_option (argv [i] + 2)) { +#endif } else { fprintf (stderr, "Unsupported command line option: '%s'\n", argv [i]); exit (1); diff --git a/src/mono/mono/mini/interp/interp-internals.h b/src/mono/mono/mini/interp/interp-internals.h index e2ec36b82bdba0..4a61ef9591bacf 100644 --- a/src/mono/mono/mini/interp/interp-internals.h +++ b/src/mono/mono/mini/interp/interp-internals.h @@ -298,6 +298,43 @@ mono_interp_jit_call_supported (MonoMethod *method, MonoMethodSignature *sig); void mono_interp_error_cleanup (MonoError *error); +gboolean +mono_interp_is_method_multicastdelegate_invoke (MonoMethod *method); + +MONO_NEVER_INLINE void +mono_interp_exec_method (InterpFrame *frame, ThreadContext *context, FrameClauseArgs *clause_args); + +#if HOST_BROWSER + +gboolean +mono_jiterp_isinst (MonoObject* object, MonoClass* klass); + +void +mono_jiterp_check_pending_unwind (ThreadContext *context); + +void * +mono_jiterp_get_context (void); + +int +mono_jiterp_overflow_check_i4 (gint32 lhs, gint32 rhs, int opcode); + +int +mono_jiterp_overflow_check_u4 (guint32 lhs, guint32 rhs, int opcode); + +void +mono_jiterp_ld_delegate_method_ptr (gpointer *destination, MonoDelegate **source); + +int +mono_jiterp_stackval_to_data (MonoType *type, stackval *val, void *data); + +int +mono_jiterp_stackval_from_data (MonoType *type, stackval *result, const void *data); + +gpointer +mono_jiterp_frame_data_allocator_alloc (FrameDataAllocator *stack, InterpFrame *frame, int size); + +#endif + static inline int mint_type(MonoType *type) { diff --git a/src/mono/mono/mini/interp/interp.c b/src/mono/mono/mini/interp/interp.c index 4f1fc52b5caf38..062b9f290c577c 100644 --- a/src/mono/mono/mini/interp/interp.c +++ b/src/mono/mono/mini/interp/interp.c @@ -82,6 +82,11 @@ #endif #include +#ifdef HOST_BROWSER +#include "jiterpreter.h" +#include +#endif + /* Arguments that are passed when invoking only a finally/filter clause from the frame */ struct FrameClauseArgs { /* Where we start the frame execution from */ @@ -248,9 +253,6 @@ static gboolean interp_init_done = FALSE; static gboolean debugger_enabled = FALSE; #endif -static void -interp_exec_method (InterpFrame *frame, ThreadContext *context, FrameClauseArgs *clause_args); - static MonoException* do_transform_method (InterpMethod *imethod, InterpFrame *method, ThreadContext *context); static InterpMethod* lookup_method_pointer (gpointer addr); @@ -1168,7 +1170,7 @@ INTERP_GET_EXCEPTION_CHAR_ARG(argument_out_of_range) } \ } while (0) -// Reduce duplicate code in interp_exec_method +// Reduce duplicate code in mono_interp_exec_method static MONO_NEVER_INLINE void do_safepoint (InterpFrame *frame, ThreadContext *context, const guint16 *ip) { @@ -2113,13 +2115,13 @@ interp_runtime_invoke (MonoMethod *method, void *obj, void **params, MonoObject // The method to execute might not be transformed yet, so we don't know how much stack // it uses. We bump the stack_pointer here so any code triggered by method compilation // will not attempt to use the space that we used to push the args for this method. - // The real top of stack for this method will be set in interp_exec_method once the + // The real top of stack for this method will be set in mono_interp_exec_method once the // method is transformed. context->stack_pointer = (guchar*)(sp + 4); g_assert (context->stack_pointer < context->stack_end); MONO_ENTER_GC_UNSAFE; - interp_exec_method (&frame, context, NULL); + mono_interp_exec_method (&frame, context, NULL); MONO_EXIT_GC_UNSAFE; context->stack_pointer = (guchar*)sp; @@ -2224,7 +2226,7 @@ interp_entry (InterpEntryData *data) g_assert (context->stack_pointer < context->stack_end); MONO_ENTER_GC_UNSAFE; - interp_exec_method (&frame, context, NULL); + mono_interp_exec_method (&frame, context, NULL); MONO_EXIT_GC_UNSAFE; context->stack_pointer = (guchar*)sp; @@ -2521,6 +2523,10 @@ struct _JitCallInfo { gint32 res_size; int ret_mt; gboolean no_wrapper; +#if HOST_BROWSER + int hit_count; + WasmJitCallThunk jiterp_thunk; +#endif }; static MONO_NEVER_INLINE void @@ -2614,11 +2620,19 @@ init_jit_call_info (InterpMethod *rmethod, MonoError *error) rmethod->jit_call_info = cinfo; } +#if HOST_BROWSER +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_register_jit_call_thunk (void *cinfo, WasmJitCallThunk thunk) { + ((JitCallInfo*)cinfo)->jiterp_thunk = thunk; +} +#endif + static MONO_NEVER_INLINE void do_jit_call (ThreadContext *context, stackval *ret_sp, stackval *sp, InterpFrame *frame, InterpMethod *rmethod, MonoError *error) { MonoLMFExt ext; JitCallInfo *cinfo; + gboolean thrown = FALSE; //printf ("jit_call: %s\n", mono_method_full_name (rmethod->method, 1)); @@ -2630,8 +2644,50 @@ do_jit_call (ThreadContext *context, stackval *ret_sp, stackval *sp, InterpFrame init_jit_call_info (rmethod, error); mono_error_assert_ok (error); } + cinfo = (JitCallInfo*)rmethod->jit_call_info; +#if JITERPRETER_ENABLE_JIT_CALL_TRAMPOLINES + // FIXME: thread safety + if (jiterpreter_jit_call_enabled) { + WasmJitCallThunk thunk = cinfo->jiterp_thunk; + if (thunk) { + MonoFtnDesc ftndesc = {0}; + void *extra_arg; + ftndesc.addr = cinfo->addr; + ftndesc.arg = cinfo->extra_arg; + extra_arg = cinfo->no_wrapper ? cinfo->extra_arg : &ftndesc; + interp_push_lmf (&ext, frame); + if ( + jiterpreter_wasm_eh_enabled || + (mono_aot_mode != MONO_AOT_MODE_LLVMONLY_INTERP) + ) { + thunk (extra_arg, ret_sp, sp, &thrown); + } else { + mono_interp_invoke_wasm_jit_call_trampoline ( + thunk, extra_arg, ret_sp, sp, &thrown + ); + } + interp_pop_lmf (&ext); + goto epilogue; + } else { + int count = cinfo->hit_count; + if (count == JITERPRETER_JIT_CALL_TRAMPOLINE_HIT_COUNT) { + void *fn = cinfo->no_wrapper ? cinfo->addr : cinfo->wrapper; + mono_interp_jit_wasm_jit_call_trampoline ( + rmethod, cinfo, fn, rmethod->hasthis, rmethod->param_count, + rmethod->arg_offsets, mono_aot_mode == MONO_AOT_MODE_LLVMONLY_INTERP + ); + } else { + if (count <= JITERPRETER_JIT_CALL_QUEUE_FLUSH_THRESHOLD) + cinfo->hit_count++; + if (count == JITERPRETER_JIT_CALL_QUEUE_FLUSH_THRESHOLD) + mono_interp_flush_jitcall_queue (); + } + } + } +#endif + /* * Convert the arguments on the interpreter stack to the format expected by the gsharedvt_out wrapper. */ @@ -2669,14 +2725,32 @@ do_jit_call (ThreadContext *context, stackval *ret_sp, stackval *sp, InterpFrame } interp_push_lmf (&ext, frame); - gboolean thrown = FALSE; + if (mono_aot_mode == MONO_AOT_MODE_LLVMONLY_INTERP) { +#if JITERPRETER_ENABLE_SPECIALIZED_JIT_CALL + /* + * invoke jit_call_cb via a single indirect function call that dispatches to + * either a specialized JS implementation or a specialized WASM EH version + * see jiterpreter-jit-call.ts and do-jit-call.wat + * NOTE: the first argument must ALWAYS be jit_call_cb for the specialization. + * the actual implementation cannot verify this at runtime, so get it right + * this is faster than mono_llvm_cpp_catch_exception by avoiding the use of + * emscripten invoke_vi to find and invoke jit_call_cb indirectly + */ + jiterpreter_do_jit_call(jit_call_cb, &cb_data, &thrown); +#else /* Catch the exception thrown by the native code using a try-catch */ mono_llvm_cpp_catch_exception (jit_call_cb, &cb_data, &thrown); +#endif } else { jit_call_cb (&cb_data); } + interp_pop_lmf (&ext); + +#if JITERPRETER_ENABLE_JIT_CALL_TRAMPOLINES +epilogue: +#endif if (thrown) { if (context->has_resume_state) /* @@ -3002,7 +3076,7 @@ interp_entry_from_trampoline (gpointer ccontext_untyped, gpointer rmethod_untype g_assert (context->stack_pointer < context->stack_end); MONO_ENTER_GC_UNSAFE; - interp_exec_method (&frame, context, NULL); + mono_interp_exec_method (&frame, context, NULL); MONO_EXIT_GC_UNSAFE; context->stack_pointer = (guchar*)sp; @@ -3094,7 +3168,7 @@ no_llvmonly_interp_method_pointer (void) static MonoFtnDesc* interp_create_method_pointer_llvmonly (MonoMethod *method, gboolean unbox, MonoError *error) { - gpointer addr, entry_func, entry_wrapper; + gpointer addr, entry_func = NULL, entry_wrapper; MonoMethodSignature *sig; MonoMethod *wrapper; InterpMethod *imethod; @@ -3145,10 +3219,33 @@ interp_create_method_pointer_llvmonly (MonoMethod *method, gboolean unbox, MonoE } g_assert (entry_func); +#if HOST_BROWSER + // FIXME: We don't support generating wasm trampolines for high arg counts yet + if ( + (sig->param_count <= MAX_INTERP_ENTRY_ARGS) && + jiterpreter_interp_entry_enabled + ) { + jiterp_preserve_module(); + + const char *name = mono_method_full_name (method, FALSE); + gpointer wasm_entry_func = mono_interp_jit_wasm_entry_trampoline ( + imethod, method, sig->param_count, (MonoType *)sig->params, + unbox, sig->hasthis, sig->ret->type != MONO_TYPE_VOID, + name, entry_func + ); + g_free((void *)name); + + // Compiling a trampoline can fail for various reasons, so in that case we will fall back to the pre-existing ones below + if (wasm_entry_func) + entry_func = wasm_entry_func; + } +#endif + /* Encode unbox in the lower bit of imethod */ gpointer entry_arg = imethod; if (unbox) entry_arg = (gpointer)(((gsize)entry_arg) | 1); + MonoFtnDesc *entry_ftndesc = mini_llvmonly_create_ftndesc (method, entry_func, entry_arg); addr = mini_llvmonly_create_ftndesc (method, entry_wrapper, entry_ftndesc); @@ -3600,8 +3697,8 @@ max_d (double lhs, double rhs) * to return error information. * FRAME is only valid until the next call to alloc_frame (). */ -static MONO_NEVER_INLINE void -interp_exec_method (InterpFrame *frame, ThreadContext *context, FrameClauseArgs *clause_args) +MONO_NEVER_INLINE void +mono_interp_exec_method (InterpFrame *frame, ThreadContext *context, FrameClauseArgs *clause_args) { InterpMethod *cmethod; ERROR_DECL(error); @@ -7408,6 +7505,113 @@ MINT_IN_CASE(MINT_BRTRUE_I8_SP) ZEROP_SP(gint64, !=); MINT_IN_BREAK; MINT_IN_BREAK; } +#ifdef HOST_BROWSER + MINT_IN_CASE(MINT_TIER_NOP_JITERPRETER) { + ip += 3; + MINT_IN_BREAK; + } + + MINT_IN_CASE(MINT_TIER_PREPARE_JITERPRETER) { + if (jiterpreter_traces_enabled) { + // We may lose a race with another thread here so we need to use volatile and be careful + volatile guint16 *mutable_ip = (volatile guint16*)ip; + /* + * prepare_jiterpreter will update the trace's hit count and potentially either JIT it or + * disable this entry point based on whether it fails to JIT. the hit counting is necessary + * because a given method may contain many jiterpreter entry points, but some of them will + * not be actually hit often enough to justify the cost of jitting them. (for example, a + * trace that only runs inside an unlikely branch for throwing exceptions.) + * thanks to the heuristic that runs during transform.c's codegen, most (95%+) of these + * entry points will JIT successfully, which will keep the number of NOT_JITTED nops low. + * note: threading doesn't work yet, we will need to broadcast jitted traces to all of our + * JS workers in order to register them at the appropriate slots in the function pointer + * table. when growing the function pointer table we will also need to synchronize that. + */ + JiterpreterThunk prepare_result = mono_interp_tier_prepare_jiterpreter(frame, frame->imethod->method, ip, frame->imethod->jinfo->code_start, frame->imethod->jinfo->code_size); + switch ((guint32)(void*)prepare_result) { + case JITERPRETER_TRAINING: + // jiterpreter still updating hit count before deciding to generate a trace, + // so skip this opcode. + ip += 3; + break; + case JITERPRETER_NOT_JITTED: + // Patch opcode to disable it because this trace failed to JIT. + mono_memory_barrier(); + *mutable_ip = MINT_TIER_NOP_JITERPRETER; + mono_memory_barrier(); + ip += 3; + break; + default: + /* + * trace generated. patch opcode to disable it, then write the function + * pointer, then patch opcode again to turn this trace on. + * we do this to ensure that other threads won't see an ENTER_JITERPRETER + * opcode that has no function pointer stored inside of it. + * (note that right now threading doesn't work, but it's worth being correct + * here so that implementing thread support will be easier later.) + */ + *mutable_ip = MINT_TIER_NOP_JITERPRETER; + mono_memory_barrier(); + *(volatile JiterpreterThunk*)(ip + 1) = prepare_result; + mono_memory_barrier(); + *mutable_ip = MINT_TIER_ENTER_JITERPRETER; + ip += 3; + break; + } + } else { + ip += 3; + } + + MINT_IN_BREAK; + } + + MINT_IN_CASE(MINT_TIER_ENTER_JITERPRETER) { + JiterpreterThunk thunk = (void*)READ32(ip + 1); + gboolean trace_requires_safepoint = FALSE; + g_assert(thunk); + ptrdiff_t offset = thunk(frame, locals); + /* + * The trace signals that we need to perform a safepoint by adding a very + * large amount to the relative displacement. This is because setting a bit + * in JS via the | operator doesn't work for negative numbers + */ + if (offset >= 0xE000000) { + offset -= 0xF000000; + trace_requires_safepoint = TRUE; + } + /* + * Verify that the offset returned by the thunk is not total garbage + * FIXME: These constants might actually be too small since a method + * could have massive amounts of IL - maybe we should disable the jiterpreter + * for methods that big + */ + g_assertf((offset >= -0xFFFFF) && (offset <= 0xFFFFF), "thunk returned an obviously invalid offset: %i", offset); + if (offset <= 0) { + BACK_BRANCH_PROFILE (offset); + } + if (trace_requires_safepoint) { + SAFEPOINT; + } + ip = (guint16*) (((guint8*)ip) + offset); + MINT_IN_BREAK; + } +#else + MINT_IN_CASE(MINT_TIER_NOP_JITERPRETER) { + g_assert_not_reached (); + MINT_IN_BREAK; + } + + MINT_IN_CASE(MINT_TIER_PREPARE_JITERPRETER) { + g_assert_not_reached (); + MINT_IN_BREAK; + } + + MINT_IN_CASE(MINT_TIER_ENTER_JITERPRETER) { + g_assert_not_reached (); + MINT_IN_BREAK; + } +#endif + #if !USE_COMPUTED_GOTO default: interp_error_xsx ("Unimplemented opcode: %04x %s at 0x%x\n", *ip, mono_interp_opname (*ip), GPTRDIFF_TO_INT (ip - frame->imethod->code)); @@ -7605,7 +7809,7 @@ interp_run_finally (StackFrameInfo *frame, int clause_index) // this informs MINT_ENDFINALLY to return to EH *(guint16**)(frame_locals (iframe) + iframe->imethod->clause_data_offsets [clause_index]) = NULL; - interp_exec_method (iframe, context, &clause_args); + mono_interp_exec_method (iframe, context, &clause_args); iframe->next_free = next_free; iframe->state.ip = state_ip; @@ -7656,7 +7860,7 @@ interp_run_filter (StackFrameInfo *frame, MonoException *ex, int clause_index, g clause_args.end_at_ip = (const guint16*)handler_ip_end; clause_args.exec_frame = &child_frame; - interp_exec_method (&child_frame, context, &clause_args); + mono_interp_exec_method (&child_frame, context, &clause_args); /* Copy back the updated frame */ memcpy (iframe->stack, child_frame.stack, iframe->imethod->locals_size); @@ -7771,7 +7975,7 @@ interp_run_clause_with_il_state (gpointer il_state_ptr, int clause_index, MonoOb /* Set in mono_handle_exception () */ context->has_resume_state = FALSE; - interp_exec_method (&frame, context, &clause_args); + mono_interp_exec_method (&frame, context, &clause_args); /* Write back args */ sp_args = sp; @@ -8302,3 +8506,96 @@ mono_ee_interp_init (const char *opts) debugger_enabled = mini_get_debug_options ()->mdb_optimizations; #endif } + +#ifdef HOST_BROWSER +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_stackval_to_data (MonoType *type, stackval *val, void *data) +{ + return stackval_to_data (type, val, data, FALSE); +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_stackval_from_data (MonoType *type, stackval *result, const void *data) +{ + return stackval_from_data (type, result, data, FALSE); +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_overflow_check_i4 (gint32 lhs, gint32 rhs, int opcode) +{ + switch (opcode) { + case MINT_MUL_OVF_I4: + if (CHECK_MUL_OVERFLOW (lhs, rhs)) + return 1; + break; + case MINT_ADD_OVF_I4: + if (CHECK_ADD_OVERFLOW (lhs, rhs)) + return 1; + break; + } + + return 0; +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_overflow_check_u4 (guint32 lhs, guint32 rhs, int opcode) +{ + switch (opcode) { + case MINT_MUL_OVF_UN_I4: + if (CHECK_MUL_OVERFLOW_UN (lhs, rhs)) + return 1; + break; + case MINT_ADD_OVF_UN_I4: + if (CHECK_ADD_OVERFLOW_UN (lhs, rhs)) + return 1; + break; + } + + return 0; +} + +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_ld_delegate_method_ptr (gpointer *destination, MonoDelegate **source) +{ + MonoDelegate *del = *source; + if (!del->interp_method) { + /* Not created from interpreted code */ + g_assert (del->method); + del->interp_method = mono_interp_get_imethod (del->method); + } else if (((InterpMethod*)del->interp_method)->optimized_imethod) { + del->interp_method = ((InterpMethod*)del->interp_method)->optimized_imethod; + } + g_assert (del->interp_method); + *destination = imethod_to_ftnptr (del->interp_method, FALSE); +} + +void +mono_jiterp_check_pending_unwind (ThreadContext *context) +{ + return check_pending_unwind (context); +} + +void * +mono_jiterp_get_context (void) +{ + return get_context (); +} + +gpointer +mono_jiterp_frame_data_allocator_alloc (FrameDataAllocator *stack, InterpFrame *frame, int size) +{ + return frame_data_allocator_alloc(stack, frame, size); +} + +gboolean +mono_jiterp_isinst (MonoObject* object, MonoClass* klass) +{ + return mono_interp_isinst (object, klass); +} + +gboolean +mono_interp_is_method_multicastdelegate_invoke (MonoMethod *method) +{ + return is_method_multicastdelegate_invoke (method); +} +#endif diff --git a/src/mono/mono/mini/interp/jiterpreter.c b/src/mono/mono/mini/interp/jiterpreter.c new file mode 100644 index 00000000000000..7075302d54cbea --- /dev/null +++ b/src/mono/mono/mini/interp/jiterpreter.c @@ -0,0 +1,1083 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// This file contains icalls used in jitted interpreter traces and wrappers, +// along with infrastructure to support code generration + +#ifndef __USE_ISOC99 +#define __USE_ISOC99 +#endif +#include "config.h" + +#if 0 +#define jiterp_assert(b) g_assert(b) +#else +#define jiterp_assert(b) +#endif + +void jiterp_preserve_module (void); + +#if HOST_BROWSER + +#include + +#include +#include +#include + +#include +#include +#include + +#include "interp.h" +#include "interp-internals.h" +#include "mintops.h" +#include "transform.h" +#include "interp-intrins.h" +#include "tiering.h" + +#include +#include +#include +#include +#include + +#define JITERPRETER_IMPLEMENTATION +#include "jiterpreter.h" + +static gint32 jiterpreter_abort_counts[MINT_LASTOP + 1] = { 0 }; +static int64_t jiterp_trace_bailout_counts[256] = { 0 }; + +#if FEATURE_WASM_THREADS +// the jiterpreter is not yet thread safe due to the need to synchronize function pointers +// and wasm modules between threads. before these can be enabled we need to implement all that +gboolean jiterpreter_traces_enabled = FALSE, + jiterpreter_interp_entry_enabled = FALSE, + jiterpreter_jit_call_enabled = FALSE, +#else +// traces_enabled controls whether the jiterpreter will JIT individual interpreter opcode traces +gboolean jiterpreter_traces_enabled = TRUE, +// interp_entry_enabled controls whether specialized interp_entry wrappers will be jitted + jiterpreter_interp_entry_enabled = TRUE, +// jit_call_enabled controls whether do_jit_call will use specialized trampolines for hot call sites + jiterpreter_jit_call_enabled = TRUE, +#endif +// if enabled, we will insert trace entry points at backwards branch targets, so that we can +// JIT loop bodies + jiterpreter_backward_branch_entries_enabled = TRUE, +// if enabled, after a call instruction terminates a trace, we will attempt to start a new +// one at the next basic block. this allows jitting loop bodies that start with 'if (x) continue' etc + jiterpreter_call_resume_enabled = TRUE, +// enables using WASM try/catch_all instructions where appropriate (currently only do_jit_call), +// will be automatically turned off if the instructions are not available. + jiterpreter_wasm_eh_enabled = TRUE, +// For locations where the jiterpreter heuristic says we will be unable to generate +// a trace, insert an entry point opcode anyway. This enables collecting accurate +// stats for options like estimateHeat, but raises overhead. + jiterpreter_always_generate = FALSE, +// Automatically prints stats at app exit or when jiterpreter_dump_stats is called + jiterpreter_stats_enabled = FALSE, +// Continue counting hits for traces that fail to compile and use it to estimate +// the relative importance of the opcode that caused them to abort + jiterpreter_estimate_heat = FALSE, +// Count the number of times a trace bails out (branch taken, etc) and for what reason + jiterpreter_count_bailouts = FALSE; + +gint32 jiterpreter_options_version = 0, +// any trace that doesn't have at least this many meaningful (non-nop) opcodes in it will be rejected + jiterpreter_minimum_trace_length = 8; + +// This function pointer is used by interp.c to invoke jit_call_cb for exception handling purposes +// See jiterpreter-jit-call.ts mono_jiterp_do_jit_call_indirect +WasmDoJitCall jiterpreter_do_jit_call = mono_jiterp_do_jit_call_indirect; + +// We disable this diagnostic because EMSCRIPTEN_KEEPALIVE makes it a false alarm, the keepalive +// functions are being used externally. Having a bunch of prototypes is pointless since these +// functions are not consumed by C anywhere else +#pragma clang diagnostic ignored "-Wmissing-prototypes" + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_encode_leb64_ref (unsigned char * destination, void * source, int valueIsSigned) { + if (!destination || !source) + return 0; + + unsigned char b; + unsigned char * originalDestination = destination; + if (valueIsSigned) { + int64_t value = *((int64_t*)source); + int more = 1, signBit; + + while (more) { + b = (unsigned char)(value & 0x7FL); + value >>= 7; + + signBit = (b & 0x40u) != 0; + if ( + ((value == 0) && !signBit) || + ((value == -1) && signBit) + ) + more = 0; + else + b |= 0x80; + + *destination++ = b; + } + } else { + uint64_t value = *((uint64_t*)source); + + do { + b = (unsigned char)(value & 0x7Ful); + value >>= 7; + + if (value != 0) + b |= 0x80; + + *destination++ = b; + } while (value != 0); + } + + return (int)(destination - originalDestination); +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_encode_leb52 (unsigned char * destination, double doubleValue, int valueIsSigned) { + if (!destination) + return 0; + + if (valueIsSigned) { + int64_t value = (int64_t)doubleValue; + if (((double)value) != doubleValue) + return 0; + + return mono_jiterp_encode_leb64_ref(destination, &value, valueIsSigned); + } else { + uint64_t value = (uint64_t)doubleValue; + if (((double)value) != doubleValue) + return 0; + + return mono_jiterp_encode_leb64_ref(destination, &value, valueIsSigned); + } +} + +// Many of the following functions implement various opcodes or provide support for opcodes +// so that jiterpreter traces don't have to inline dozens of wasm instructions worth of +// complex logic - these are designed to match interp.c + +EMSCRIPTEN_KEEPALIVE double +mono_jiterp_fmod (double lhs, double rhs) { + return fmod(lhs, rhs); +} + +EMSCRIPTEN_KEEPALIVE double +mono_jiterp_atan2 (double lhs, double rhs) { + return atan2(lhs, rhs); +} + +// If a trace is jitted for a method that hasn't been tiered yet, we need to +// update the interpreter entry count for the method. +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_increase_entry_count (void *_imethod) { + InterpMethod *imethod = (InterpMethod*)_imethod; + imethod->entry_count++; + // Return whether the trace should bail out because the method needs to be tiered + return imethod->entry_count >= INTERP_TIER_ENTRY_LIMIT; +} + +EMSCRIPTEN_KEEPALIVE void* +mono_jiterp_object_unbox (MonoObject *obj) { + return mono_object_unbox_internal(obj); +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_try_unbox_ref ( + MonoClass *klass, void **dest, MonoObject **src +) { + if (!klass) + return 0; + + MonoObject *o = *src; + if (!o) + return 0; + + if ( + !( + (m_class_get_rank (o->vtable->klass) == 0) && + (m_class_get_element_class (o->vtable->klass) == m_class_get_element_class (klass)) + ) + ) + return 0; + + *dest = mono_object_unbox_internal(o); + return 1; +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_type_is_byref (MonoType *type) { + if (!type) + return 0; + return m_type_is_byref(type); +} + +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_value_copy (void *dest, void *src, MonoClass *klass) { + mono_value_copy_internal(dest, src, klass); +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_strlen_ref (MonoString **ppString, int *result) { + MonoString *pString = *ppString; + if (!pString) + return 0; + + *result = mono_string_length_internal(pString); + return 1; +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_getchr_ref (MonoString **ppString, int *pIndex, int *result) { + int index = *pIndex; + MonoString *pString = *ppString; + if (!pString) + return 0; + if ((index < 0) || (index >= mono_string_length_internal(pString))) + return 0; + + *result = mono_string_chars_internal(pString)[index]; + return 1; +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_try_newobj_inlined (MonoObject **destination, MonoVTable *vtable) { + *destination = 0; + if (!vtable->initialized) + return 0; + + *destination = mono_gc_alloc_obj(vtable, m_class_get_instance_size(vtable->klass)); + if (!destination) + return 0; + + return 1; +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_getitem_span ( + void **destination, MonoSpanOfVoid *span, int index, size_t element_size +) { + if (!span) + return 0; + + const gint32 length = span->_length; + if ((index < 0) || (index >= length)) + return 0; + + unsigned char * pointer = (unsigned char *)span->_reference; + *destination = pointer + (index * element_size); + return 1; +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_gettype_ref ( + MonoObject **destination, MonoObject **source +) { + MonoObject *obj = *source; + if (obj) { + *destination = (obj)->vtable->type; + return 1; + } else + return 0; +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_cast_ref ( + MonoObject **destination, MonoObject **source, + MonoClass *klass, MintOpcode opcode +) { + if (!klass) + return 0; + + MonoObject *obj = *source; + if (!obj) { + *destination = 0; + return 1; + } + + switch (opcode) { + case MINT_CASTCLASS: + case MINT_ISINST: { + if (obj) { + // FIXME push/pop LMF + if (!mono_jiterp_isinst (obj, klass)) { // FIXME: do not swallow the error + if (opcode == MINT_ISINST) + *destination = NULL; + else + return 0; // bailout + } else { + *destination = obj; + } + } else { + *destination = NULL; + } + return 1; + } + case MINT_CASTCLASS_INTERFACE: + case MINT_ISINST_INTERFACE: { + gboolean isinst; + // FIXME: Perform some of this work at JIT time + if (MONO_VTABLE_IMPLEMENTS_INTERFACE (obj->vtable, m_class_get_interface_id (klass))) { + isinst = TRUE; + } else if (m_class_is_array_special_interface (klass)) { + /* slow path */ + // FIXME push/pop LMF + isinst = mono_jiterp_isinst (obj, klass); // FIXME: do not swallow the error + } else { + isinst = FALSE; + } + + if (!isinst) { + if (opcode == MINT_ISINST_INTERFACE) + *destination = NULL; + else + return 0; // bailout + } else { + *destination = obj; + } + return 1; + } + case MINT_CASTCLASS_COMMON: + case MINT_ISINST_COMMON: { + if (obj) { + gboolean isinst = mono_class_has_parent_fast (obj->vtable->klass, klass); + + if (!isinst) { + if (opcode == MINT_ISINST_COMMON) + *destination = NULL; + else + return 0; // bailout + } else { + *destination = obj; + } + } else { + *destination = NULL; + } + return 1; + } + } + + return 0; +} + +EMSCRIPTEN_KEEPALIVE void* +mono_jiterp_array_get_element_address_with_size_ref (MonoArray **array, int size, int index) +{ + // HACK: This does not need to be volatile because we know array is visible to + // the GC and this is called from interp traces in gc unsafe mode + MonoArray* _array = *array; + if (!_array) + return NULL; + if (index >= mono_array_length_internal(_array)) + return NULL; + return mono_array_addr_with_size_fast (_array, size, index); +} + +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_localloc (gpointer *destination, gint32 len, InterpFrame *frame) +{ + ThreadContext *context = mono_jiterp_get_context(); + gpointer mem; + if (len > 0) { + mem = mono_jiterp_frame_data_allocator_alloc (&context->data_stack, frame, ALIGN_TO (len, MINT_VT_ALIGNMENT)); + + if (frame->imethod->init_locals) + memset (mem, 0, len); + } else { + mem = NULL; + } + *destination = mem; +} + +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_ldtsflda (gpointer *destination, guint32 offset) { + MonoInternalThread *thread = mono_thread_internal_current (); + *destination = ((char*)thread->static_data [offset & 0x3f]) + (offset >> 6); +} + +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_box_ref (MonoVTable *vtable, MonoObject **dest, void *src, gboolean vt) { + HANDLE_FUNCTION_ENTER (); + + MonoObjectHandle tmp_handle = MONO_HANDLE_NEW (MonoObject, NULL); + + // FIXME push/pop LMF + MonoObject *o = mono_gc_alloc_obj (vtable, m_class_get_instance_size (vtable->klass)); + MONO_HANDLE_ASSIGN_RAW (tmp_handle, o); + if (vt) + mono_value_copy_internal (mono_object_get_data (o), src, vtable->klass); + else + mono_jiterp_stackval_to_data (m_class_get_byval_arg (vtable->klass), (stackval*)(src), mono_object_get_data (o)); + MONO_HANDLE_ASSIGN_RAW (tmp_handle, NULL); + + *dest = o; + + HANDLE_FUNCTION_RETURN (); +} + +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_conv_ovf (void *dest, void *src, int opcode) { + switch (opcode) { + case MINT_CONV_OVF_I4_I8: { + gint64 val = *(gint64*)src; + if (val < G_MININT32 || val > G_MAXINT32) + return 0; + *(gint32*)dest = (gint32) val; + return 1; + } + + case MINT_CONV_OVF_U4_I8: { + gint64 val = *(gint64*)src; + if (val < 0 || val > G_MAXUINT32) + return 0; + *(guint32*)dest = (guint32) val; + return 1; + } + + case MINT_CONV_OVF_I4_U8: { + guint64 val = *(guint64*)src; + if (val > G_MAXINT32) + return 0; + *(gint32*)dest = (gint32) val; + return 1; + } + + case MINT_CONV_OVF_U4_I4: { + gint32 val = *(gint32*)src; + if (val < 0) + return 0; + *(guint32*)dest = (guint32) val; + return 1; + } + + case MINT_CONV_OVF_I4_R8: + case MINT_CONV_OVF_I4_R4: { + double val; + if (opcode == MINT_CONV_OVF_I4_R4) + val = *(float*)src; + else + val = *(double*)src; + + if (val > ((double)G_MININT32 - 1) && val < ((double)G_MAXINT32 + 1)) { + *(gint32*)dest = (gint32) val; + return 1; + } + return 0; + } + } + + // TODO: return 0 on success and a unique bailout code on failure? + // Probably not necessary right now and would bloat traces slightly + return 0; +} + +// we use these helpers at JIT time to figure out where to do memory loads and stores +EMSCRIPTEN_KEEPALIVE size_t +mono_jiterp_get_offset_of_vtable_initialized_flag () { + return offsetof(MonoVTable, initialized); +} + +EMSCRIPTEN_KEEPALIVE size_t +mono_jiterp_get_offset_of_array_data () { + return MONO_STRUCT_OFFSET (MonoArray, vector); +} + +EMSCRIPTEN_KEEPALIVE size_t +mono_jiterp_get_size_of_stackval () { + return sizeof(stackval); +} + +// jiterpreter-interp-entry.ts uses this information to decide whether to call +// stackval_from_data for a given type or just do a raw value copy of N bytes +EMSCRIPTEN_KEEPALIVE int +mono_jiterp_type_get_raw_value_size (MonoType *type) { + // We use a NULL type to indicate that we want a raw ptr copy + if ((type == NULL) || m_type_is_byref (type)) + return 256; + + switch (type->type) { + // for unsigned types we return a negative size to communicate to + // the jiterpreter implementation that it should use the _u version + // of the wasm load opcodes instead of the _s version + case MONO_TYPE_U1: + return -1; + case MONO_TYPE_U2: + case MONO_TYPE_CHAR: + return -2; + + case MONO_TYPE_I1: + return 1; + case MONO_TYPE_I2: + return 2; + case MONO_TYPE_I: + case MONO_TYPE_I4: + case MONO_TYPE_U: + case MONO_TYPE_U4: + case MONO_TYPE_PTR: + case MONO_TYPE_FNPTR: + return 4; + + default: + return 0; + } +} + +// we use these helpers to record when a trace bails out (in countBailouts mode) +EMSCRIPTEN_KEEPALIVE void* +mono_jiterp_trace_bailout (void* rip, int reason) +{ + if (reason < 256) + jiterp_trace_bailout_counts[reason]++; + return rip; +} + +EMSCRIPTEN_KEEPALIVE double +mono_jiterp_get_trace_bailout_count (int reason) +{ + if (reason > 255) + return -1; + + int64_t result = jiterp_trace_bailout_counts[reason]; + return (double)result; +} + +// we use this to record how many times a trace has aborted due to a given opcode. +// this is done in C because the heuristic updates it along with typescript updating it +EMSCRIPTEN_KEEPALIVE gint32 +mono_jiterp_adjust_abort_count (MintOpcode opcode, gint32 delta) { + if ((opcode < 0) || (opcode >= MINT_LASTOP)) + return 0; + if (delta != 0) + jiterpreter_abort_counts[opcode] += delta; + return jiterpreter_abort_counts[opcode]; +} + +typedef struct { + InterpMethod *rmethod; + ThreadContext *context; + gpointer orig_domain; + gpointer attach_cookie; +} JiterpEntryDataHeader; + +// we optimize delegate calls by attempting to cache the delegate invoke +// target - this will improve performance when the same delegate is invoked +// repeatedly inside a loop +typedef struct { + MonoDelegate *delegate_invoke_is_for; + MonoMethod *delegate_invoke; + InterpMethod *delegate_invoke_rmethod; +} JiterpEntryDataCache; + +// jitted interp_entry wrappers use custom tracking data structures +// that are allocated in the heap, one per wrapper +// FIXME: For thread safety we need to make these thread-local or stack-allocated +// Note that if we stack allocate these the cache will need to move somewhere else +typedef struct { + // We split the cache out from the important data so that when + // jiterp_interp_entry copies the important data it doesn't have + // to also copy the cache. This reduces overhead slightly + JiterpEntryDataHeader header; + JiterpEntryDataCache cache; +} JiterpEntryData; + +// at the start of a jitted interp_entry wrapper, this is called to perform initial setup +// like resolving the target for delegates and setting up the thread context +// inlining this into the wrappers would make them unnecessarily big and complex +EMSCRIPTEN_KEEPALIVE stackval * +mono_jiterp_interp_entry_prologue (JiterpEntryData *data, void *this_arg) +{ + stackval *sp_args; + MonoMethod *method; + InterpMethod *rmethod; + ThreadContext *context; + + // unbox implemented by jit + + jiterp_assert(data); + rmethod = data->header.rmethod; + jiterp_assert(rmethod); + method = rmethod->method; + jiterp_assert(method); + + if (mono_interp_is_method_multicastdelegate_invoke(method)) { + // Copy the current state of the cache before using it + JiterpEntryDataCache cache = data->cache; + if (this_arg && (cache.delegate_invoke_is_for == (MonoDelegate*)this_arg)) { + // We previously cached the invoke for this delegate + method = cache.delegate_invoke; + data->header.rmethod = rmethod = cache.delegate_invoke_rmethod; + } else { + /* + * This happens when AOT code for the invoke wrapper is not found. + * Have to replace the method with the wrapper here, since the wrapper depends on the delegate. + */ + MonoDelegate *del = (MonoDelegate*)this_arg; + method = mono_marshal_get_delegate_invoke (method, del); + data->header.rmethod = rmethod = mono_interp_get_imethod (method); + + // Cache the delegate invoke. This works because data was allocated statically + // when the jitted trampoline was created, so it will stick around. + // FIXME: Thread safety + data->cache.delegate_invoke_is_for = NULL; + data->cache.delegate_invoke = method; + data->cache.delegate_invoke_rmethod = rmethod; + data->cache.delegate_invoke_is_for = del; + } + } + + // FIXME: Thread safety + + if (rmethod->needs_thread_attach) + data->header.orig_domain = mono_threads_attach_coop (mono_domain_get (), &data->header.attach_cookie); + else + data->header.orig_domain = data->header.attach_cookie = NULL; + + data->header.context = context = mono_jiterp_get_context (); + sp_args = (stackval*)context->stack_pointer; + + return sp_args; +} + +// after interp_entry_prologue the wrapper will set up all the argument values +// in the correct place and compute the stack offset, then it passes that in to this +// function in order to actually enter the interpreter and process the return value +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_interp_entry (JiterpEntryData *_data, stackval *sp_args, void *res) +{ + JiterpEntryDataHeader header; + MonoType *type; + + // Copy the scratch buffer into a local variable. This is necessary for us to be + // reentrant-safe because mono_interp_exec_method could end up hitting the trampoline + // again + jiterp_assert(_data); + header = _data->header; + + jiterp_assert(header.rmethod); + jiterp_assert(header.rmethod->method); + jiterp_assert(sp_args); + + stackval *sp = (stackval*)header.context->stack_pointer; + + InterpFrame frame = {0}; + frame.imethod = header.rmethod; + frame.stack = sp; + frame.retval = sp; + + header.context->stack_pointer = (guchar*)sp_args; + g_assert ((guchar*)sp_args < header.context->stack_end); + + MONO_ENTER_GC_UNSAFE; + mono_interp_exec_method (&frame, header.context, NULL); + MONO_EXIT_GC_UNSAFE; + + header.context->stack_pointer = (guchar*)sp; + + if (header.rmethod->needs_thread_attach) + mono_threads_detach_coop (header.orig_domain, &header.attach_cookie); + + mono_jiterp_check_pending_unwind (header.context); + + if (mono_llvm_only) { + if (header.context->has_resume_state) + /* The exception will be handled in a frame above us */ + mono_llvm_cpp_throw_exception (); + } else { + g_assert (!header.context->has_resume_state); + } + + // The return value is at the bottom of the stack, after the locals space + type = header.rmethod->rtype; + if (type->type != MONO_TYPE_VOID) + mono_jiterp_stackval_to_data (type, frame.stack, res); +} + +// should_abort_trace returns one of these codes depending on the opcode and current state +#define TRACE_IGNORE -1 +#define TRACE_CONTINUE 0 +#define TRACE_ABORT 1 + +/* + * This function provides an approximate answer for "will this instruction cause the jiterpreter + * to abort trace compilation here?" so that we can decide whether it's worthwhile to have + * a trace entry instruction at various points in a method. It doesn't need to be exact, it just + * needs to provide correct answers often enough so that we avoid generating lots of expensive + * trace nops while still ensuring we put entry points where we need them. + * At present this is around 94-97% accurate, which is more than good enough + */ +static int +jiterp_should_abort_trace (InterpInst *ins, gboolean *inside_branch_block) +{ + guint16 opcode = ins->opcode; + switch (opcode) { + // Individual instructions that never abort traces. + // Please keep this in sync with jiterpreter.ts:generate_wasm_body + case MINT_TIER_ENTER_METHOD: + case MINT_TIER_PATCHPOINT: + case MINT_TIER_PREPARE_JITERPRETER: + case MINT_TIER_NOP_JITERPRETER: + case MINT_TIER_ENTER_JITERPRETER: + case MINT_NOP: + case MINT_DEF: + case MINT_DUMMY_USE: + case MINT_IL_SEQ_POINT: + case MINT_TIER_PATCHPOINT_DATA: + case MINT_MONO_MEMORY_BARRIER: + case MINT_SDB_BREAKPOINT: + case MINT_SDB_INTR_LOC: + case MINT_SDB_SEQ_POINT: + return TRACE_IGNORE; + + case MINT_INITLOCAL: + case MINT_INITLOCALS: + case MINT_LOCALLOC: + case MINT_INITOBJ: + case MINT_CKNULL: + case MINT_LDLOCA_S: + case MINT_LDTOKEN: + case MINT_LDSTR: + case MINT_LDFTN_ADDR: + case MINT_MONO_LDPTR: + case MINT_CPOBJ_VT: + case MINT_LDOBJ_VT: + case MINT_STOBJ_VT: + case MINT_STRLEN: + case MINT_GETCHR: + case MINT_GETITEM_SPAN: + case MINT_INTRINS_SPAN_CTOR: + case MINT_INTRINS_UNSAFE_BYTE_OFFSET: + case MINT_INTRINS_GET_TYPE: + case MINT_INTRINS_MEMORYMARSHAL_GETARRAYDATAREF: + case MINT_CASTCLASS: + case MINT_CASTCLASS_COMMON: + case MINT_CASTCLASS_INTERFACE: + case MINT_ISINST: + case MINT_ISINST_COMMON: + case MINT_ISINST_INTERFACE: + case MINT_BOX: + case MINT_BOX_VT: + case MINT_UNBOX: + case MINT_NEWOBJ_INLINED: + case MINT_NEWOBJ_VT_INLINED: + case MINT_LD_DELEGATE_METHOD_PTR: + case MINT_LDTSFLDA: + return TRACE_CONTINUE; + + case MINT_BR: + case MINT_BR_S: + if (*inside_branch_block) + return TRACE_CONTINUE; + + return TRACE_ABORT; + + case MINT_THROW: + case MINT_LEAVE: + case MINT_LEAVE_S: + if (*inside_branch_block) + return TRACE_CONTINUE; + + return TRACE_ABORT; + + case MINT_LEAVE_CHECK: + case MINT_LEAVE_S_CHECK: + return TRACE_ABORT; + + case MINT_CALL_HANDLER: + case MINT_CALL_HANDLER_S: + case MINT_ENDFINALLY: + case MINT_RETHROW: + case MINT_MONO_RETHROW: + case MINT_PROF_EXIT: + case MINT_PROF_EXIT_VOID: + case MINT_SAFEPOINT: + return TRACE_ABORT; + + default: + if ( + // branches + // FIXME: some of these abort traces because the trace compiler doesn't + // implement them, but they are rare + (opcode >= MINT_BRFALSE_I4) && + (opcode <= MINT_BLT_UN_I8_IMM_SP) + ) { + *inside_branch_block = TRUE; + return TRACE_CONTINUE; + } + else if ( + // calls + // FIXME: many of these abort traces unconditionally because the trace + // compiler doesn't implement them, but that's fixable + (opcode >= MINT_CALL) && + (opcode <= MINT_CALLI_NAT_FAST) + // (opcode <= MINT_JIT_CALL2) + ) + return *inside_branch_block ? TRACE_CONTINUE : TRACE_ABORT; + else if ( + // returns + (opcode >= MINT_RET) && + (opcode <= MINT_RET_U2) + ) + return *inside_branch_block ? TRACE_CONTINUE : TRACE_ABORT; + else if ( + (opcode >= MINT_LDC_I4_M1) && + (opcode <= MINT_LDC_R8) + ) + return TRACE_CONTINUE; + else if ( + (opcode >= MINT_MOV_SRC_OFF) && + (opcode <= MINT_MOV_8_4) + ) + return TRACE_CONTINUE; + else if ( + // binops + (opcode >= MINT_ADD_I4) && + (opcode <= MINT_CLT_UN_R8) + ) + return TRACE_CONTINUE; + else if ( + // unops and some superinsns + // fixme: a lot of these aren't actually implemented. but they're also uncommon + (opcode >= MINT_ADD1_I4) && + (opcode <= MINT_SHR_I8_IMM) + ) + return TRACE_CONTINUE; + else if ( + // math intrinsics + (opcode >= MINT_ASIN) && + (opcode <= MINT_MAXF) + ) + return TRACE_CONTINUE; + else if ( + // field operations + // the trace compiler currently implements most, but not all of these + (opcode >= MINT_LDFLD_I1) && + (opcode <= MINT_LDTSFLDA) + ) + return TRACE_CONTINUE; + else if ( + // indirect operations + // there are also a few of these not implemented by the trace compiler yet + (opcode >= MINT_LDLOCA_S) && + (opcode <= MINT_STIND_OFFSET_IMM_I8) + ) + return TRACE_CONTINUE; + else if ( + // array operations + // some of these like the _I ones aren't implemented yet but are rare + (opcode >= MINT_LDELEM_I) && + (opcode <= MINT_GETITEM_LOCALSPAN) + ) + return TRACE_CONTINUE; + else + return TRACE_ABORT; + } +} + +static gboolean +should_generate_trace_here (InterpBasicBlock *bb, InterpInst *last_ins) { + int current_trace_length = 0; + // A preceding trace may have been in a branch block, but we only care whether the current + // trace will have a branch block opened, because that determines whether calls and branches + // will unconditionally abort the trace or not. + gboolean inside_branch_block = FALSE; + + // We scan forward through the entire method body starting from the current block, not just + // the current block (since the actual trace compiler doesn't know about block boundaries). + for (InterpInst *ins = bb->first_ins; (ins != NULL) && (ins != last_ins); ins = ins->next) { + int category = jiterp_should_abort_trace(ins, &inside_branch_block); + switch (category) { + case TRACE_ABORT: { + jiterpreter_abort_counts[ins->opcode]++; + return current_trace_length >= jiterpreter_minimum_trace_length; + } + case TRACE_IGNORE: + break; + default: + current_trace_length++; + break; + } + + // Once we know the trace is long enough we can stop scanning. + if (current_trace_length >= jiterpreter_minimum_trace_length) + return TRUE; + } + + return FALSE; +} + +/* + * Insert jiterpreter entry points at the correct candidate locations: + * The first basic block of the function, + * Backward branch targets (if enabled), + * The next basic block after a call instruction (if enabled) + * To determine whether it is appropriate to insert an entry point at a given candidate location + * we have to scan through all the instructions to estimate whether it is possible to generate + * a suitably large trace. If it's not, we should avoid the overhead of the jiterpreter nop + * instruction that would end up there instead and not waste any resources trying to compile it. + */ +void +jiterp_insert_entry_points (void *_td) +{ + if (!jiterpreter_traces_enabled) + return; + TransformData *td = (TransformData *)_td; + + // Insert an entry opcode for the next basic block (call resume and first bb) + // FIXME: Should we do this based on relationships between BBs instead of insn sequence? + gboolean enter_at_next = TRUE; + + for (InterpBasicBlock *bb = td->entry_bb; bb != NULL; bb = bb->next_bb) { + // Enter trace at top of functions + gboolean is_backwards_branch = FALSE, + is_resume_or_first = enter_at_next; + + // If backwards branches target a block, enter a trace there so that + // after the backward branch we can re-enter jitted code + if (jiterpreter_backward_branch_entries_enabled && bb->backwards_branch_target) + is_backwards_branch = TRUE; + + gboolean enabled = (is_backwards_branch || is_resume_or_first); + // FIXME: This scan will likely proceed forward all the way out of the current block, + // which means that for large methods we will sometimes scan the same instruction + // multiple times and waste some work. At present this is unavoidable because + // control flow means we can end up with two traces covering different subsets + // of the same method in order to handle loops and resuming + gboolean should_generate = enabled && should_generate_trace_here(bb, td->last_ins); + + if (jiterpreter_call_resume_enabled && bb->contains_call_instruction) + enter_at_next = TRUE; + + if (jiterpreter_always_generate) + should_generate = TRUE; + + if (enabled && should_generate) { + td->cbb = bb; + interp_insert_ins (td, NULL, MINT_TIER_PREPARE_JITERPRETER); + // Note that we only clear enter_at_next here, after generating a trace. + // This means that the flag will stay set intentionally if we keep failing + // to generate traces, perhaps due to a string of small basic blocks + // or multiple call instructions. + enter_at_next = bb->contains_call_instruction; + } + } +} + +// Used to parse runtime options that control the jiterpreter. This is *also* used at runtime +// by the jiterpreter typescript to reconfigure the jiterpreter, for example if WASM EH is not +// actually available even though it was enabled (to turn it off). +EMSCRIPTEN_KEEPALIVE gboolean +mono_jiterp_parse_option (const char *option) +{ + if (!option || (*option == 0)) + return FALSE; + + const char *mtl = "jiterpreter-minimum-trace-length="; + const int mtl_l = strlen(mtl); + + if (!strcmp (option, "jiterpreter-enable-traces")) + jiterpreter_traces_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-traces")) + jiterpreter_traces_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-enable-interp-entry")) + jiterpreter_interp_entry_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-interp-entry")) + jiterpreter_interp_entry_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-enable-jit-call")) + jiterpreter_jit_call_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-jit-call")) + jiterpreter_jit_call_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-enable-backward-branches")) + jiterpreter_backward_branch_entries_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-backward-branches")) + jiterpreter_backward_branch_entries_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-enable-call-resume")) + jiterpreter_call_resume_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-call-resume")) + jiterpreter_call_resume_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-enable-wasm-eh")) + jiterpreter_wasm_eh_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-wasm-eh")) + jiterpreter_wasm_eh_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-always-generate")) + jiterpreter_always_generate = TRUE; + else if (!strcmp (option, "jiterpreter-enable-all")) + jiterpreter_traces_enabled = jiterpreter_interp_entry_enabled = jiterpreter_jit_call_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-all")) + jiterpreter_traces_enabled = jiterpreter_interp_entry_enabled = jiterpreter_jit_call_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-enable-stats")) + jiterpreter_stats_enabled = TRUE; + else if (!strcmp (option, "jiterpreter-disable-stats")) + jiterpreter_stats_enabled = FALSE; + else if (!strcmp (option, "jiterpreter-estimate-heat")) + jiterpreter_always_generate = jiterpreter_estimate_heat = TRUE; + else if (!strcmp (option, "jiterpreter-count-bailouts")) + jiterpreter_always_generate = jiterpreter_count_bailouts = TRUE; + else if (!strncmp (option, mtl, mtl_l)) + jiterpreter_minimum_trace_length = atoi(option + mtl_l); + else + return FALSE; + + jiterpreter_options_version++; + return TRUE; +} + +// When jiterpreter options change we increment this version so that the typescript knows +// it will have to re-query all the option values +EMSCRIPTEN_KEEPALIVE gint32 +mono_jiterp_get_options_version () { + return jiterpreter_options_version; +} + +// The typescript uses this to query the full jiterpreter configuration so it can behave +// appropriately (minimum trace length, EH enabled state, etc) +EMSCRIPTEN_KEEPALIVE gint32 +mono_jiterp_get_option (const char * option) { + if (!strcmp (option, "jiterpreter-enable-traces")) + return jiterpreter_traces_enabled; + else if (!strcmp (option, "jiterpreter-enable-interp-entry")) + return jiterpreter_interp_entry_enabled; + else if (!strcmp (option, "jiterpreter-enable-jit-call")) + return jiterpreter_jit_call_enabled; + else if (!strcmp (option, "jiterpreter-enable-all")) + return jiterpreter_traces_enabled && jiterpreter_interp_entry_enabled && jiterpreter_jit_call_enabled; + else if (!strcmp (option, "jiterpreter-enable-backward-branches")) + return jiterpreter_backward_branch_entries_enabled; + else if (!strcmp (option, "jiterpreter-enable-call-resume")) + return jiterpreter_call_resume_enabled; + else if (!strcmp (option, "jiterpreter-enable-wasm-eh")) + return jiterpreter_wasm_eh_enabled; + else if (!strcmp (option, "jiterpreter-always-generate")) + return jiterpreter_always_generate; + else if (!strcmp (option, "jiterpreter-enable-stats")) + return jiterpreter_stats_enabled; + else if (!strcmp (option, "jiterpreter-minimum-trace-length")) + return jiterpreter_minimum_trace_length; + else if (!strcmp (option, "jiterpreter-estimate-heat")) + return jiterpreter_estimate_heat; + else if (!strcmp (option, "jiterpreter-count-bailouts")) + return jiterpreter_count_bailouts; + else + return -1; +} + +EMSCRIPTEN_KEEPALIVE void +mono_jiterp_update_jit_call_dispatcher (WasmDoJitCall dispatcher) { + // If we received a 0 dispatcher that means the TS side failed to compile + // any kind of dispatcher - this likely indicates that content security policy + // blocked the use of Module.addFunction + if (!dispatcher) + dispatcher = (WasmDoJitCall)mono_llvm_cpp_catch_exception; + jiterpreter_do_jit_call = dispatcher; +} + +// HACK: fix C4206 +EMSCRIPTEN_KEEPALIVE +#endif + +void jiterp_preserve_module () { +} diff --git a/src/mono/mono/mini/interp/jiterpreter.h b/src/mono/mono/mini/interp/jiterpreter.h new file mode 100644 index 00000000000000..55b3be429d63d7 --- /dev/null +++ b/src/mono/mono/mini/interp/jiterpreter.h @@ -0,0 +1,96 @@ +#ifdef HOST_BROWSER + +// enables specialized mono_llvm_cpp_catch_exception replacement (see jiterpreter-jit-call.ts) +// works even if the jiterpreter is otherwise disabled. +#define JITERPRETER_ENABLE_SPECIALIZED_JIT_CALL 1 + +// mono_interp_tier_prepare_jiterpreter will return these special values if it doesn't +// have a function pointer for a specific entry point. +// TRAINING indicates that the hit count is not high enough yet +#define JITERPRETER_TRAINING 0 +// NOT_JITTED indicates that the trace was not jitted and it should be turned into a NOP +#define JITERPRETER_NOT_JITTED 1 + +#define JITERPRETER_ENABLE_JIT_CALL_TRAMPOLINES 1 +// After a do_jit_call call site is hit this many times, we will queue it to be jitted +#define JITERPRETER_JIT_CALL_TRAMPOLINE_HIT_COUNT 2999 +// If a do_jit_call site is hit this many times without being jitted (due to waiting in +// the queue), we will flush the queue immediately +#define JITERPRETER_JIT_CALL_QUEUE_FLUSH_THRESHOLD 10000 + +typedef const ptrdiff_t (*JiterpreterThunk) (void *frame, void *pLocals); +typedef void (*WasmJitCallThunk) (void *extra_arg, void *ret_sp, void *sp, gboolean *thrown); +typedef void (*WasmDoJitCall) (gpointer cb, gpointer arg, gboolean *out_thrown); + +// Parses a single jiterpreter runtime option. This is used both by driver.c and our typescript +gboolean +mono_jiterp_parse_option (const char *option); + +// HACK: Prevent jiterpreter.c from being entirely linked out, because the typescript relies on it +void +jiterp_preserve_module (); + +// HACK: Pass void* so that this header can include safely in files without definition for TransformData +void +jiterp_insert_entry_points (void *td); + +// used by the typescript JIT implementation to notify the runtime that it has finished jitting a thunk +// for a specific callsite, since it can take a while before it happens +void +mono_jiterp_register_jit_call_thunk (void *cinfo, WasmJitCallThunk thunk); + +// jiterpreter-interp-entry.ts +// HACK: Pass void* so that this header can include safely in files without definition for InterpMethod +extern gpointer +mono_interp_jit_wasm_entry_trampoline ( + void *imethod, MonoMethod *method, int argument_count, MonoType *param_types, + int unbox, int has_this, int has_return, const char *name, void *default_implementation +); + +// HACK: Pass void* so that this header can include safely in files without definition for InterpFrame +extern JiterpreterThunk +mono_interp_tier_prepare_jiterpreter ( + void *frame, MonoMethod *method, const guint16 *ip, + const guint16 *start_of_body, int size_of_body +); + +// HACK: Pass void* so that this header can include safely in files without definition for InterpMethod, +// or JitCallInfo +extern void +mono_interp_jit_wasm_jit_call_trampoline ( + void *rmethod, void *cinfo, void *func, + gboolean has_this, int param_count, + guint32 *arg_offsets, gboolean catch_exceptions +); + +// synchronously jits everything waiting in the do_jit_call jit queue +extern void +mono_interp_flush_jitcall_queue (); + +// invokes a specific jit call trampoline with JS exception handling. this is only used if +// we were unable to use WASM EH to perform exception handling, either because it was +// disabled or because the current runtime environment does not support it +extern void +mono_interp_invoke_wasm_jit_call_trampoline ( + WasmJitCallThunk thunk, void *extra_arg, + void *ret_sp, void *sp, gboolean *thrown +); + +extern void +mono_jiterp_do_jit_call_indirect ( + gpointer cb, gpointer arg, gboolean *out_thrown +); + + +#ifndef JITERPRETER_IMPLEMENTATION + +extern gboolean jiterpreter_traces_enabled; +extern gboolean jiterpreter_interp_entry_enabled; +extern gboolean jiterpreter_jit_call_enabled; +extern gboolean jiterpreter_wasm_eh_enabled; +extern WasmDoJitCall jiterpreter_do_jit_call; + +#endif + + +#endif diff --git a/src/mono/mono/mini/interp/mintops.def b/src/mono/mono/mini/interp/mintops.def index 89439b541b0f39..8fa0c024a41d4e 100644 --- a/src/mono/mono/mini/interp/mintops.def +++ b/src/mono/mono/mini/interp/mintops.def @@ -799,3 +799,8 @@ OPDEF(MINT_INTRINS_ORDINAL_IGNORE_CASE_ASCII, "intrins_ordinal_ignore_case_ascii OPDEF(MINT_INTRINS_64ORDINAL_IGNORE_CASE_ASCII, "intrins_64ordinal_ignore_case_ascii", 4, 1, 2, MintOpNoArgs) OPDEF(MINT_INTRINS_U32_TO_DECSTR, "intrins_u32_to_decstr", 5, 1, 1, MintOpTwoShorts) OPDEF(MINT_INTRINS_WIDEN_ASCII_TO_UTF16, "intrins_widen_ascii_to_utf16", 5, 1, 3, MintOpNoArgs) + +// TODO: Make this wasm only +OPDEF(MINT_TIER_PREPARE_JITERPRETER, "tier_prepare_jiterpreter", 3, 0, 0, MintOpInt) +OPDEF(MINT_TIER_NOP_JITERPRETER, "tier_nop_jiterpreter", 3, 0, 0, MintOpInt) +OPDEF(MINT_TIER_ENTER_JITERPRETER, "tier_enter_jiterpreter", 3, 0, 0, MintOpInt) diff --git a/src/mono/mono/mini/interp/transform.c b/src/mono/mono/mini/interp/transform.c index 938e1300387781..5167810ecfdfe5 100644 --- a/src/mono/mono/mini/interp/transform.c +++ b/src/mono/mono/mini/interp/transform.c @@ -35,6 +35,10 @@ #include "transform.h" #include "tiering.h" +#if HOST_BROWSER +#include "jiterpreter.h" +#endif + MonoInterpStats mono_interp_stats; #define DEBUG 0 @@ -256,7 +260,7 @@ interp_insert_ins_bb (TransformData *td, InterpBasicBlock *bb, InterpInst *prev_ } /* Inserts a new instruction after prev_ins. prev_ins must be in cbb */ -static InterpInst* +InterpInst* interp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode) { return interp_insert_ins_bb (td, td->cbb, prev_ins, opcode); @@ -836,6 +840,9 @@ handle_branch (TransformData *td, int long_op, int offset) InterpBasicBlock *target_bb = td->offset_to_bb [target]; g_assert (target_bb); + if (offset < 0) + target_bb->backwards_branch_target = TRUE; + if (offset < 0 && td->sp == td->stack && !td->inlined_method) { // Backwards branch inside unoptimized method where the IL stack is empty // This is candidate for a patchpoint @@ -4909,6 +4916,8 @@ generate_code (TransformData *td, MonoMethod *method, MonoMethodHeader *header, case CEE_CALL: { gboolean need_seq_point = FALSE; + td->cbb->contains_call_instruction = TRUE; + if (sym_seq_points && !mono_bitset_test_fast (seq_point_locs, td->ip + 5 - header->code)) need_seq_point = TRUE; @@ -5749,6 +5758,7 @@ generate_code (TransformData *td, MonoMethod *method, MonoMethodHeader *header, td->sp -= csignature->param_count; guint32 params_stack_size = tos_offset - get_tos_offset (td); + td->cbb->contains_call_instruction = TRUE; interp_add_ins (td, MINT_NEWOBJ_STRING_UNOPT); td->last_ins->data [0] = get_data_item_index (td, mono_interp_get_imethod (m)); td->last_ins->data [1] = params_stack_size; @@ -5765,6 +5775,7 @@ generate_code (TransformData *td, MonoMethod *method, MonoMethodHeader *header, } call_args [csignature->param_count + 1] = -1; + td->cbb->contains_call_instruction = TRUE; interp_add_ins (td, MINT_NEWOBJ_STRING); td->last_ins->data [0] = get_data_item_index_imethod (td, mono_interp_get_imethod (m)); push_type (td, stack_type [ret_mt], klass); @@ -5792,6 +5803,7 @@ generate_code (TransformData *td, MonoMethod *method, MonoMethodHeader *header, td->sp -= csignature->param_count; int param_size = tos - get_tos_offset (td); + td->cbb->contains_call_instruction = TRUE; interp_add_ins (td, MINT_NEWOBJ_SLOW_UNOPT); td->last_ins->data [0] = get_data_item_index_imethod (td, mono_interp_get_imethod (m)); td->last_ins->data [1] = param_size; @@ -5848,12 +5860,14 @@ generate_code (TransformData *td, MonoMethod *method, MonoMethodHeader *header, InterpInst *newobj_fast; if (is_vt) { + td->cbb->contains_call_instruction = TRUE; newobj_fast = interp_add_ins (td, MINT_NEWOBJ_VT); interp_ins_set_dreg (newobj_fast, dreg); newobj_fast->data [1] = GUINTPTR_TO_UINT16 (ALIGN_TO (vtsize, MINT_STACK_SLOT_SIZE)); } else { MonoVTable *vtable = mono_class_vtable_checked (klass, error); goto_if_nok (error, exit); + td->cbb->contains_call_instruction = TRUE; newobj_fast = interp_add_ins (td, MINT_NEWOBJ); interp_ins_set_dreg (newobj_fast, dreg); newobj_fast->data [1] = get_data_item_index (td, vtable); @@ -5865,6 +5879,7 @@ generate_code (TransformData *td, MonoMethod *method, MonoMethodHeader *header, if (!td->aggressive_inlining) INLINE_FAILURE; } else { + td->cbb->contains_call_instruction = TRUE; interp_add_ins (td, MINT_NEWOBJ_SLOW); g_assert (!m_class_is_valuetype (klass)); interp_ins_set_dreg (td->last_ins, dreg); @@ -8123,6 +8138,7 @@ generate_compacted_code (TransformData *td) *ip++ = MINT_TIER_PATCHPOINT; *ip++ = (guint16)bb->index; } + while (ins) { if (ins->opcode == MINT_TIER_PATCHPOINT_DATA) { int native_offset = (int)(ip - td->new_code); @@ -10004,6 +10020,9 @@ generate (MonoMethod *method, MonoMethodHeader *header, InterpMethod *rtm, MonoG if (td->optimized) { interp_optimize_code (td); interp_alloc_offsets (td); +#if HOST_BROWSER + jiterp_insert_entry_points (td); +#endif } generate_compacted_code (td); diff --git a/src/mono/mono/mini/interp/transform.h b/src/mono/mono/mini/interp/transform.h index 7b5526738f055e..88387b2dc1170e 100644 --- a/src/mono/mono/mini/interp/transform.h +++ b/src/mono/mono/mini/interp/transform.h @@ -133,6 +133,9 @@ struct _InterpBasicBlock { // optimized method we will map the bb_index to the corresponding native offset. int patchpoint_data: 1; int emit_patchpoint: 1; + // used by jiterpreter + int backwards_branch_target: 1; + int contains_call_instruction: 1; }; typedef enum { @@ -266,6 +269,10 @@ mono_test_interp_generate_code (TransformData *td, MonoMethod *method, MonoMetho void mono_test_interp_method_compute_offsets (TransformData *td, InterpMethod *imethod, MonoMethodSignature *signature, MonoMethodHeader *header); +/* used by jiterpreter */ +InterpInst* +interp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode); + /* debugging aid */ void mono_interp_print_td_code (TransformData *td); diff --git a/src/mono/sample/wasm/browser-bench/main.js b/src/mono/sample/wasm/browser-bench/main.js index e287f0eb8b6ef3..67d71d0187e9bd 100644 --- a/src/mono/sample/wasm/browser-bench/main.js +++ b/src/mono/sample/wasm/browser-bench/main.js @@ -12,6 +12,7 @@ let legacyExportTargetInt; let jsExportTargetInt; let legacyExportTargetString; let jsExportTargetString; +let _jiterpreter_dump_stats; function runLegacyExportInt(count) { for (let i = 0; i < count; i++) { @@ -59,8 +60,12 @@ function importTargetThrows(value) { } class MainApp { - async init({ getAssemblyExports, setModuleImports, BINDING }) { + async init({ getAssemblyExports, setModuleImports, BINDING, INTERNAL }) { const exports = await getAssemblyExports("Wasm.Browser.Bench.Sample.dll"); + INTERNAL.jiterpreter_apply_options({ + enableStats: true + }); + _jiterpreter_dump_stats = INTERNAL.jiterpreter_dump_stats.bind(INTERNAL); runBenchmark = exports.Sample.Test.RunBenchmark; setTasks = exports.Sample.Test.SetTasks; getFullJsonResults = exports.Sample.Test.GetFullJsonResults; @@ -104,6 +109,7 @@ class MainApp { if (ret.length > 0) { setTimeout(() => { this.yieldBench(); }, 0); } else { + _jiterpreter_dump_stats(); document.getElementById("out").innerHTML += "Finished"; fetch("/results.json", { method: 'POST', @@ -166,6 +172,7 @@ try { globalThis.mainApp.PageShow = globalThis.mainApp.pageShow.bind(globalThis.mainApp); const runtime = await dotnet + .withRuntimeOptions(["--jiterpreter-enable-stats"]) .withElementOnExit() .withExitCodeLogging() .create(); diff --git a/src/mono/sample/wasm/simple-raytracer/Makefile b/src/mono/sample/wasm/simple-raytracer/Makefile new file mode 100644 index 00000000000000..2baa239a988437 --- /dev/null +++ b/src/mono/sample/wasm/simple-raytracer/Makefile @@ -0,0 +1,11 @@ +TOP=../../../../.. + +include ../wasm.mk + +ifneq ($(AOT),) +override MSBUILD_ARGS+=/p:RunAOTCompilation=true +endif + +PROJECT_NAME=Wasm.Browser.SimpleRaytracer.csproj + +run: run-browser diff --git a/src/mono/sample/wasm/simple-raytracer/Program.cs b/src/mono/sample/wasm/simple-raytracer/Program.cs new file mode 100644 index 00000000000000..d5e99d3e50b443 --- /dev/null +++ b/src/mono/sample/wasm/simple-raytracer/Program.cs @@ -0,0 +1,304 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +using System; +using System.Runtime.InteropServices.JavaScript; +using System.Runtime.InteropServices; +using System.Runtime.CompilerServices; + +[StructLayout(LayoutKind.Sequential, Pack=1)] +public struct Vec3f { + public float x, y, z; + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public Vec3f (float x, float y, float z = 0f) { + this.x = x; + this.y = y; + this.z = z; + } +} + +public delegate void SceneObjectReader (ref T obj, out float radius, out Vec3f center); + +public interface ISceneObject { +} + +public struct Sphere : ISceneObject { + public Vec3f Center; + public float Radius; + public Vec3f Color; + + public static void Read (ref Sphere obj, out float radius, out Vec3f center) { + center = obj.Center; + radius = obj.Radius; + } +} + +public static unsafe class Raytrace { + public const int BytesPerPixel = 4, + width = 640, height = 480; + + private static byte[] FrameBuffer; + private static Sphere[] Scene; + + // Convert a linear color value to a gamma-space int in [0, 255] + // Square root approximates gamma-correct rendering. + public static int l2gi (float v) { + // sqrt, clamp to [0, 1], then scale to [0, 255] and truncate to int + return (int)((MathF.Min(MathF.Max(MathF.Sqrt(v), 0.0f), 1.0f)) * 255.0f); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void vecStore (float x, float y, float z, ref Vec3f ptr) { + ptr.x = x; + ptr.y = y; + ptr.z = z; + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void vecAdd (ref Vec3f a, ref Vec3f b, ref Vec3f ptr) { + ptr.x = a.x + b.x; + ptr.y = a.y + b.y; + ptr.z = a.z + b.z; + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void vecScale (ref Vec3f a, float scale, ref Vec3f ptr) { + ptr.x = a.x * scale; + ptr.y = a.y * scale; + ptr.z = a.z * scale; + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void vecNormalize (ref Vec3f ptr) { + var x = ptr.x; + var y = ptr.y; + var z = ptr.z; + + float invLen = (1.0f / MathF.Sqrt((x * x) + (y * y) + (z * z))); + ptr.x *= invLen; + ptr.y *= invLen; + ptr.z *= invLen; + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static float vecDot (ref Vec3f a, ref Vec3f b) { + return (a.x * b.x) + (a.y * b.y) + (a.z * b.z); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static float vecNLDot (ref Vec3f a, ref Vec3f b) { + var value = vecDot(ref a, ref b); + if (value < 0) + return 0; + else + return value; + } + + public static void sampleEnv (ref Vec3f dir, ref Vec3f ptr) { + var y = dir.y; + var amt = y * 0.5f + 0.5f; + var keep = 1.0f - amt; + vecStore( + keep * 0.1f + amt * 0.1f, + keep * 1.0f + amt * 0.1f, + keep * 0.1f + amt * 1.0f, + ref ptr + ); + } + + public abstract class Intersector { + public abstract void SetReader (Delegate d); + public abstract unsafe bool Intersect (ref Vec3f pos, ref Vec3f dir, void * obj, ref Vec3f intersection_normal); + } + + public class Intersector : Intersector + where T : unmanaged, ISceneObject + { + public SceneObjectReader Reader; + + public override void SetReader (Delegate d) { + Reader = (SceneObjectReader)d; + } + + public override bool Intersect (ref Vec3f pos, ref Vec3f dir, void * obj, ref Vec3f intersection_normal) { + var so = Unsafe.AsRef(obj); + Reader(ref so, out var radius, out var center); + return Intersect(ref pos, ref dir, radius, ref center, ref intersection_normal); + } + + public bool Intersect (ref Vec3f pos, ref Vec3f dir, float radius, ref Vec3f center, ref Vec3f intersection_normal) { + var vx = dir.x; + var vy = dir.y; + var vz = dir.z; + + // The sphere. + var cx = center.x; + var cy = center.y; // (float)Math.Sin(phase); + var cz = center.z; + + // Calculate the position relative to the center of the sphere. + var ox = pos.x - cx; + var oy = pos.y - cy; + var oz = pos.z - cz; + + var dot = vx * ox + vy * oy + vz * oz; + + var partial = dot * dot + radius * radius - (ox * ox + oy * oy + oz * oz); + if (partial >= 0.0f) { + var d = -dot - MathF.Sqrt(partial); + + if (d >= 0.0f) { + intersection_normal.x = pos.x + vx * d - cx; + intersection_normal.y = pos.y + vy * d - cy; + intersection_normal.z = pos.z + vz * d - cz; + vecNormalize(ref intersection_normal); + return true; + } + } + + return false; + } + } + + private static void renderPixel (int i, int j, ref Vec3f light, Intersector intersector) { + var fb = FrameBuffer; + var scene = Scene; + + var x = (float)(i) / (float)(width) - 0.5f; + var y = 0.5f - (float)(j) / (float)(height); + Vec3f pos = new Vec3f(x, y), + dir = new Vec3f(x, y, -0.5f), + half = default, intersection_normal = default, + color = default; + vecNormalize(ref dir); + + // Compute the half vector; + vecScale(ref dir, -1.0f, ref half); + vecAdd(ref half, ref light, ref half); + vecNormalize(ref half); + + // Light accumulation + var r = 0.0f; + var g = 0.0f; + var b = 0.0f; + + // Surface diffuse. + var dr = 0.7f; + var dg = 0.7f; + var db = 0.7f; + + float hitZ = -999; + bool didHitZ = false; + for (int s = 0; s < scene.Length; s++) { + ref var sphere = ref scene[s]; + + if (didHitZ && (hitZ > sphere.Center.z)) + continue; + + if (intersector.Intersect(ref pos, ref dir, Unsafe.AsPointer(ref sphere), ref intersection_normal)) { + sampleEnv(ref intersection_normal, ref color); + + const float ambientScale = 0.2f; + r = dr * color.x * ambientScale; + g = dg * color.y * ambientScale; + b = db * color.z * ambientScale; + + var diffuse = vecNLDot(ref intersection_normal, ref light); + var specular = vecNLDot(ref intersection_normal, ref half); + + // Take it to the 64th power, manually. + specular *= specular; + specular *= specular; + specular *= specular; + specular *= specular; + specular *= specular; + specular *= specular; + + specular = specular * 0.6f; + + r += dr * (diffuse * sphere.Color.x) + specular; + g += dg * (diffuse * sphere.Color.y) + specular; + b += db * (diffuse * sphere.Color.z) + specular; + // FIXME: Compute z of intersection point and check that instead + hitZ = sphere.Center.z; + didHitZ = true; + } + } + + if (!didHitZ) { + sampleEnv(ref dir, ref color); + r = color.x; + g = color.y; + b = color.z; + } + + var index = (i + (j * width)) * BytesPerPixel; + + fb[index + 0] = (byte)l2gi(r); + fb[index + 1] = (byte)l2gi(g); + fb[index + 2] = (byte)l2gi(b); + fb[index + 3] = 255; + } + + public static byte[] renderFrame () { + Vec3f light = default; + vecStore(20.0f, 20.0f, 15.0f, ref light); + vecNormalize(ref light); + + var reader = (SceneObjectReader)Sphere.Read; + var tIntersector = typeof(Intersector<>).MakeGenericType(new[] { typeof(Sphere) }); + var intersector = (Intersector)Activator.CreateInstance(tIntersector); + intersector.SetReader(reader); + for (int j = 0; j < height; j++) { + for (int i = 0; i < width; i++) + renderPixel(i, j, ref light, intersector); + } + + return FrameBuffer; + } + + public static void init () { + FrameBuffer = new byte[width * height * BytesPerPixel]; + var rng = new Random(1); + const int count = 128; + Scene = new Sphere[count]; + for (int i = 0; i < count; i++) { + Scene[i] = new Sphere { + Center = new Vec3f( + (rng.NextSingle() * 8f) - 5.5f, + (rng.NextSingle() * 8f) - 5.5f, + (rng.NextSingle() * -8f) - 2f + ), + Color = new Vec3f( + rng.NextSingle(), + rng.NextSingle(), + rng.NextSingle() + ), + Radius = (rng.NextSingle() * 0.85f) + 0.075f + }; + } + } +} + +public static partial class Program { + public static void Main() + { + Raytrace.init(); + Console.WriteLine ("Hello, World!"); + } + + [JSImport("renderCanvas", "main.js")] + static partial void RenderCanvas([JSMarshalAs] ArraySegment rgba); + + [JSExport] + internal static void OnClick(){ + var now = DateTime.UtcNow; + Console.WriteLine ("Rendering started"); + + var bytes = Raytrace.renderFrame(); + + Console.WriteLine ("Rendering finished in "+ (DateTime.UtcNow - now).TotalMilliseconds+ " ms"); + RenderCanvas(bytes); + } +} diff --git a/src/mono/sample/wasm/simple-raytracer/RayTracer.csproj b/src/mono/sample/wasm/simple-raytracer/RayTracer.csproj new file mode 100644 index 00000000000000..4318c27f3ed886 --- /dev/null +++ b/src/mono/sample/wasm/simple-raytracer/RayTracer.csproj @@ -0,0 +1,23 @@ + + + main.js + Exe + true + preview + true + $(EnableAOTAndTrimming) + $(EnableAOTAndTrimming) + $(EnableAOTAndTrimming) + false + $(NetCoreAppCurrent) + + false + true + true + + + + + + + diff --git a/src/mono/sample/wasm/simple-raytracer/index.html b/src/mono/sample/wasm/simple-raytracer/index.html new file mode 100644 index 00000000000000..945cffd15621a7 --- /dev/null +++ b/src/mono/sample/wasm/simple-raytracer/index.html @@ -0,0 +1,26 @@ + + + + + + + Simple wasm raytracer + + + + + + + +

Simple wasm raytracer

+ +

+ + +

+

+ Original implementation by Nicholas C. Bray +

+ + + diff --git a/src/mono/sample/wasm/simple-raytracer/main.js b/src/mono/sample/wasm/simple-raytracer/main.js new file mode 100644 index 00000000000000..f2c3f74bc9e7b9 --- /dev/null +++ b/src/mono/sample/wasm/simple-raytracer/main.js @@ -0,0 +1,26 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +import { dotnet } from './dotnet.js' + +function renderCanvas(rgbaView) { + const canvas = document.getElementById("out"); + const ctx = canvas.getContext('2d'); + const clamped = new Uint8ClampedArray(rgbaView.slice()); + const image = new ImageData(clamped, 640, 480); + ctx.putImageData(image, 0, 0); + rgbaView.dispose(); + canvas.style = ""; +} + +const { setModuleImports, getAssemblyExports, getConfig } = await dotnet.create(); +setModuleImports("main.js", { renderCanvas }); +const config = getConfig(); +const exports = await getAssemblyExports(config.mainAssemblyName); +globalThis.onClick = exports.Program.OnClick; + +await dotnet + .withRuntimeOptions(["--jiterpreter-enable-stats"]) + .run(); +const btnRender = document.getElementById("btnRender"); +btnRender.disabled = false; diff --git a/src/mono/wasm/runtime/cwraps.ts b/src/mono/wasm/runtime/cwraps.ts index 6a8f0dbfa0e8b2..57fa125f5a8404 100644 --- a/src/mono/wasm/runtime/cwraps.ts +++ b/src/mono/wasm/runtime/cwraps.ts @@ -97,6 +97,24 @@ const fn_signatures: SigLine[] = [ [true, "mono_wasm_f64_to_i52", "number", ["number", "number"]], [true, "mono_wasm_f64_to_u52", "number", ["number", "number"]], [true, "mono_wasm_method_get_name", "number", ["number"]], + [true, "mono_wasm_method_get_full_name", "number", ["number"]], + + // jiterpreter + [true, "mono_jiterp_get_trace_bailout_count", "number", ["number"]], + [true, "mono_jiterp_value_copy", "void", ["number", "number", "number"]], + [true, "mono_jiterp_get_offset_of_vtable_initialized_flag", "number", []], + [true, "mono_jiterp_get_offset_of_array_data", "number", []], + [false, "mono_jiterp_encode_leb52", "number", ["number", "number", "number"]], + [false, "mono_jiterp_encode_leb64_ref", "number", ["number", "number", "number"]], + [true, "mono_jiterp_type_is_byref", "number", ["number"]], + [true, "mono_jiterp_get_size_of_stackval", "number", []], + [true, "mono_jiterp_parse_option", "number", ["string"]], + [true, "mono_jiterp_get_option", "number", ["string"]], + [true, "mono_jiterp_get_options_version", "number", []], + [true, "mono_jiterp_adjust_abort_count", "number", ["number", "number"]], + [true, "mono_jiterp_register_jit_call_thunk", "void", ["number", "number"]], + [true, "mono_jiterp_type_get_raw_value_size", "number", ["number"]], + [true, "mono_jiterp_update_jit_call_dispatcher", "void", ["number"]], ]; export interface t_Cwraps { @@ -210,6 +228,26 @@ export interface t_Cwraps { mono_wasm_f64_to_u52(destination: VoidPtr, value: number): I52Error; mono_wasm_runtime_run_module_cctor(assembly: MonoAssembly): void; mono_wasm_method_get_name(method: MonoMethod): CharPtr; + mono_wasm_method_get_full_name(method: MonoMethod): CharPtr; + + mono_jiterp_get_trace_bailout_count(reason: number): number; + mono_jiterp_value_copy(destination: VoidPtr, source: VoidPtr, klass: MonoClass): void; + mono_jiterp_get_offset_of_vtable_initialized_flag(): number; + mono_jiterp_get_offset_of_array_data(): number; + // Returns bytes written (or 0 if writing failed) + mono_jiterp_encode_leb52 (destination: VoidPtr, value: number, valueIsSigned: number): number; + // Returns bytes written (or 0 if writing failed) + // Source is the address of a 64-bit int or uint + mono_jiterp_encode_leb64_ref (destination: VoidPtr, source: VoidPtr, valueIsSigned: number): number; + mono_jiterp_type_is_byref (type: MonoType): number; + mono_jiterp_get_size_of_stackval (): number; + mono_jiterp_type_get_raw_value_size (type: MonoType): number; + mono_jiterp_parse_option (name: string): number; + mono_jiterp_get_option (name: string): number; + mono_jiterp_get_options_version (): number; + mono_jiterp_adjust_abort_count (opcode: number, delta: number): number; + mono_jiterp_register_jit_call_thunk (cinfo: number, func: number): void; + mono_jiterp_update_jit_call_dispatcher (fn: number): void; } const wrapped_c_functions: t_Cwraps = {}; diff --git a/src/mono/wasm/runtime/do-jit-call.wasm b/src/mono/wasm/runtime/do-jit-call.wasm new file mode 100644 index 0000000000000000000000000000000000000000..a9ff8f6c0b9ee7bc2d8f4d6f121f86cc2ba830b5 GIT binary patch literal 103 zcmZQbEY4+QU|?Y6W=deJXGmbKudiodl4WAdWXny>%`d8C0?K7_XJwYeCnx6Q#3v^) xFfcPQGO|lBilpSn!^Jc6QZkEDlS>#Fxr7-78Q2^Y7zG$46_^|u&6pUtxdE*H73u&0 literal 0 HcmV?d00001 diff --git a/src/mono/wasm/runtime/do-jit-call.wat b/src/mono/wasm/runtime/do-jit-call.wat new file mode 100755 index 00000000000000..b08ca58bf0d065 --- /dev/null +++ b/src/mono/wasm/runtime/do-jit-call.wat @@ -0,0 +1,14 @@ +(module + (import "i" "memory" (memory 0)) + (func $jit_call_cb (import "i" "jit_call_cb") (param i32)) + (func $do_jit_call_indirect (export "do_jit_call_indirect") (param $unused i32) (param $cb_data i32) (param $thrown i32) + try + local.get $cb_data + call $jit_call_cb + catch_all + local.get $thrown + i32.const 1 + i32.store + end + ) +) diff --git a/src/mono/wasm/runtime/dotnet.d.ts b/src/mono/wasm/runtime/dotnet.d.ts index b166ec4261975a..b0a2679c1c8f35 100644 --- a/src/mono/wasm/runtime/dotnet.d.ts +++ b/src/mono/wasm/runtime/dotnet.d.ts @@ -60,6 +60,8 @@ declare interface EmscriptenModule { FS_readFile(filename: string, opts: any): any; removeRunDependency(id: string): void; addRunDependency(id: string): void; + addFunction(fn: Function, signature: string): number; + getWasmTableEntry(index: number): any; stackSave(): VoidPtr; stackRestore(stack: VoidPtr): void; stackAlloc(size: number): VoidPtr; diff --git a/src/mono/wasm/runtime/driver.c b/src/mono/wasm/runtime/driver.c index 76f0f29f544119..9b145963b0d4c5 100644 --- a/src/mono/wasm/runtime/driver.c +++ b/src/mono/wasm/runtime/driver.c @@ -497,13 +497,14 @@ mono_wasm_load_runtime (const char *unused, int debug_level) monoeg_g_setenv ("MONO_SLEEP_ABORT_LIMIT", "5000", 0); #endif + // monoeg_g_setenv ("COMPlus_DebugWriteToStdErr", "1", 0); + #ifdef DEBUG // monoeg_g_setenv ("MONO_LOG_LEVEL", "debug", 0); // monoeg_g_setenv ("MONO_LOG_MASK", "gc", 0); // Setting this env var allows Diagnostic.Debug to write to stderr. In a browser environment this // output will be sent to the console. Right now this is the only way to emit debug logging from // corlib assemblies. - // monoeg_g_setenv ("COMPlus_DebugWriteToStdErr", "1", 0); #endif // When the list of app context properties changes, please update RuntimeConfigReservedProperties for // target _WasmGenerateRuntimeConfig in WasmApp.targets file @@ -1488,6 +1489,11 @@ EMSCRIPTEN_KEEPALIVE int mono_wasm_f64_to_i52 (int64_t *destination, double valu return I52_ERROR_NONE; } -EMSCRIPTEN_KEEPALIVE const char* mono_wasm_method_get_name (MonoMethod *method) { - return mono_method_full_name(method, 0); +// JS is responsible for freeing this +EMSCRIPTEN_KEEPALIVE const char * mono_wasm_method_get_full_name (MonoMethod *method) { + return mono_method_get_full_name(method); +} + +EMSCRIPTEN_KEEPALIVE const char * mono_wasm_method_get_name (MonoMethod *method) { + return mono_method_get_name(method); } diff --git a/src/mono/wasm/runtime/es6/dotnet.es6.lib.js b/src/mono/wasm/runtime/es6/dotnet.es6.lib.js index 612330c9df5977..46e0c07a4193a8 100644 --- a/src/mono/wasm/runtime/es6/dotnet.es6.lib.js +++ b/src/mono/wasm/runtime/es6/dotnet.es6.lib.js @@ -78,6 +78,14 @@ const linked_functions = [ "mono_wasm_trace_logger", "mono_wasm_event_pipe_early_startup_callback", + // jiterpreter.c / interp.c / transform.c + "mono_interp_tier_prepare_jiterpreter", + "mono_interp_jit_wasm_entry_trampoline", + "mono_interp_jit_wasm_jit_call_trampoline", + "mono_interp_invoke_wasm_jit_call_trampoline", + "mono_interp_flush_jitcall_queue", + "mono_jiterp_do_jit_call_indirect", + // corebindings.c "mono_wasm_invoke_js_with_args_ref", "mono_wasm_get_object_property_ref", diff --git a/src/mono/wasm/runtime/exports-internal.ts b/src/mono/wasm/runtime/exports-internal.ts index 0db08cb407f496..c3ff6a75d3bbe3 100644 --- a/src/mono/wasm/runtime/exports-internal.ts +++ b/src/mono/wasm/runtime/exports-internal.ts @@ -12,6 +12,8 @@ import { mono_intern_string } from "./strings"; import { mono_wasm_stringify_as_error_with_stack } from "./logging"; import { ws_wasm_create, ws_wasm_open, ws_wasm_send, ws_wasm_receive, ws_wasm_close, ws_wasm_abort } from "./web-socket"; import { mono_wasm_get_loaded_files } from "./assets"; +import { jiterpreter_dump_stats } from "./jiterpreter"; +import { getOptions, applyOptions } from "./jiterpreter-support"; export function export_internal(): any { return { @@ -74,6 +76,11 @@ export function export_internal(): any { http_wasm_get_response_bytes, http_wasm_get_response_length, http_wasm_get_streamed_response_bytes, + + // jiterpreter + jiterpreter_dump_stats, + jiterpreter_apply_options: applyOptions, + jiterpreter_get_options: getOptions }; } diff --git a/src/mono/wasm/runtime/exports-linker.ts b/src/mono/wasm/runtime/exports-linker.ts index 2ff03d0c39bafb..4f3b2ce45f6e18 100644 --- a/src/mono/wasm/runtime/exports-linker.ts +++ b/src/mono/wasm/runtime/exports-linker.ts @@ -7,6 +7,9 @@ import { mono_wasm_release_cs_owned_object } from "./gc-handles"; import { mono_wasm_load_icu_data, mono_wasm_get_icudt_name } from "./icu"; import { mono_wasm_bind_cs_function } from "./invoke-cs"; import { mono_wasm_bind_js_function, mono_wasm_invoke_bound_function, mono_wasm_invoke_import } from "./invoke-js"; +import { mono_interp_tier_prepare_jiterpreter } from "./jiterpreter"; +import { mono_interp_jit_wasm_entry_trampoline } from "./jiterpreter-interp-entry"; +import { mono_interp_jit_wasm_jit_call_trampoline, mono_interp_invoke_wasm_jit_call_trampoline, mono_interp_flush_jitcall_queue, mono_jiterp_do_jit_call_indirect } from "./jiterpreter-jit-call"; import { mono_wasm_typed_array_from_ref } from "./net6-legacy/buffers"; import { mono_wasm_invoke_js_blazor, mono_wasm_invoke_js_with_args_ref, mono_wasm_get_object_property_ref, mono_wasm_set_object_property_ref, @@ -52,7 +55,14 @@ export function export_linker(): any { // mono-threads-wasm.c schedule_background_exec, - // interp.c + // interp.c and jiterpreter.c + mono_interp_tier_prepare_jiterpreter, + mono_interp_jit_wasm_entry_trampoline, + mono_interp_jit_wasm_jit_call_trampoline, + mono_interp_invoke_wasm_jit_call_trampoline, + mono_interp_flush_jitcall_queue, + mono_jiterp_do_jit_call_indirect, + mono_wasm_profiler_enter, mono_wasm_profiler_leave, diff --git a/src/mono/wasm/runtime/jiterpreter-interp-entry.ts b/src/mono/wasm/runtime/jiterpreter-interp-entry.ts new file mode 100644 index 00000000000000..cbea9b90e7bc5f --- /dev/null +++ b/src/mono/wasm/runtime/jiterpreter-interp-entry.ts @@ -0,0 +1,542 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +import { mono_assert, MonoMethod, MonoType } from "./types"; +import { NativePointer } from "./types/emscripten"; +import { Module } from "./imports"; +import { + getU32, _zero_region +} from "./memory"; +import { WasmOpcode } from "./jiterpreter-opcodes"; +import cwraps from "./cwraps"; +import { + WasmValtype, WasmBuilder, addWasmFunctionPointer, + _now, elapsedTimes, counters, getRawCwrap, importDef, + getWasmFunctionTable +} from "./jiterpreter-support"; + +// Controls miscellaneous diagnostic output. +const trace = 0; +const + // Dumps all compiled wrappers + dumpWrappers = false; + +/* +typedef struct { + InterpMethod *rmethod; + gpointer this_arg; + gpointer res; + gpointer args [16]; + gpointer *many_args; +} InterpEntryData; +*/ + +const // offsetOfStack = 12, + maxInlineArgs = 16, + // just allocate a bunch of extra space + sizeOfJiterpEntryData = 64, + offsetOfRMethod = 0; + +const maxJitQueueLength = 4, + queueFlushDelayMs = 10; + +let trampBuilder : WasmBuilder; +let trampImports : Array<[string, string, Function]> | undefined; +let fnTable : WebAssembly.Table; +let jitQueueTimeout = 0; +const jitQueue : TrampolineInfo[] = []; + +/* +const enum WasmReftype { + funcref = 0x70, + externref = 0x6F, +} +*/ + +function getTrampImports () { + if (trampImports) + return trampImports; + + trampImports = [ + importDef("interp_entry_prologue", getRawCwrap("mono_jiterp_interp_entry_prologue")), + importDef("interp_entry", getRawCwrap("mono_jiterp_interp_entry")), + importDef("unbox", getRawCwrap("mono_jiterp_object_unbox")), + importDef("stackval_from_data", getRawCwrap("mono_jiterp_stackval_from_data")), + ]; + + return trampImports; +} + +class TrampolineInfo { + imethod: number; + method: MonoMethod; + paramTypes: Array; + + argumentCount: number; + hasThisReference: boolean; + unbox: boolean; + hasReturnValue: boolean; + name: string; + traceName: string; + + defaultImplementation: number; + result: number; + + constructor ( + imethod: number, method: MonoMethod, argumentCount: number, pParamTypes: NativePointer, + unbox: boolean, hasThisReference: boolean, hasReturnValue: boolean, name: string, + defaultImplementation: number + ) { + this.imethod = imethod; + this.method = method; + this.argumentCount = argumentCount; + this.unbox = unbox; + this.hasThisReference = hasThisReference; + this.hasReturnValue = hasReturnValue; + this.name = name; + this.paramTypes = new Array(argumentCount); + for (let i = 0; i < argumentCount; i++) + this.paramTypes[i] = getU32(pParamTypes + (i * 4)); + this.defaultImplementation = defaultImplementation; + this.result = 0; + let subName = name; + if (!subName) { + subName = `${this.imethod.toString(16)}_${this.hasThisReference ? "i" : "s"}${this.hasReturnValue ? "_r" : ""}_${this.argumentCount}`; + } else { + // truncate the real method name so that it doesn't make the module too big. this isn't a big deal for module-per-function, + // but since we jit in groups now we need to keep the sizes reasonable. we keep the tail end of the name + // since it is likely to contain the method name and/or signature instead of type and noise + const maxLength = 24; + if (subName.length > maxLength) + subName = subName.substring(subName.length - maxLength, subName.length); + subName = `${this.imethod.toString(16)}_${subName}`; + } + this.traceName = subName; + } +} + +// returns function pointer +export function mono_interp_jit_wasm_entry_trampoline ( + imethod: number, method: MonoMethod, argumentCount: number, pParamTypes: NativePointer, + unbox: boolean, hasThisReference: boolean, hasReturnValue: boolean, name: NativePointer, + defaultImplementation: number +) : number { + // HACK + if (argumentCount > maxInlineArgs) + return 0; + + const info = new TrampolineInfo( + imethod, method, argumentCount, pParamTypes, + unbox, hasThisReference, hasReturnValue, Module.UTF8ToString(name), + defaultImplementation + ); + if (!fnTable) + fnTable = getWasmFunctionTable(); + + jitQueue.push(info); + + // We start by creating a function pointer for this interp_entry trampoline, but instead of + // compiling it right away, we make it point to the default implementation for that signature + // This gives us time to wait before jitting it so we can jit multiple trampolines at once. + const defaultImplementationFn = fnTable.get(defaultImplementation); + info.result = addWasmFunctionPointer(defaultImplementationFn); + + if (jitQueue.length >= maxJitQueueLength) + flush_wasm_entry_trampoline_jit_queue(); + else + ensure_jit_is_scheduled(); + + return info.result; +} + +function ensure_jit_is_scheduled () { + if (jitQueueTimeout > 0) + return; + + if (typeof (globalThis.setTimeout) !== "function") + return; + + // We only want to wait a short period of time before jitting the trampolines. + // In practice the queue should fill up pretty fast during startup, and we just + // want to make sure we catch the last few stragglers with this timeout handler. + // Note that in console JS runtimes this means we will never automatically flush + // the queue unless it fills up, which is unfortunate but not fixable since + // there is no realistic way to efficiently maintain a hit counter for these trampolines + jitQueueTimeout = globalThis.setTimeout(() => { + jitQueueTimeout = 0; + flush_wasm_entry_trampoline_jit_queue(); + }, queueFlushDelayMs); +} + +function flush_wasm_entry_trampoline_jit_queue () { + if (jitQueue.length <= 0) + return; + + let builder = trampBuilder; + if (!builder) + trampBuilder = builder = new WasmBuilder(); + else + builder.clear(); + const started = _now(); + let compileStarted = 0; + let rejected = true, threw = false; + + try { + // Magic number and version + builder.appendU32(0x6d736100); + builder.appendU32(1); + + builder.defineType( + "unbox", { + "pMonoObject": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "interp_entry_prologue", { + "pData": WasmValtype.i32, + "this_arg": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "interp_entry", { + "pData": WasmValtype.i32, + "sp_args": WasmValtype.i32, + "res": WasmValtype.i32, + }, WasmValtype.void + ); + builder.defineType( + "stackval_from_data", { + "type": WasmValtype.i32, + "result": WasmValtype.i32, + "value": WasmValtype.i32 + }, WasmValtype.i32 + ); + + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + + const sig : any = {}; + if (info.hasThisReference) + sig["this_arg"] = WasmValtype.i32; + if (info.hasReturnValue) + sig["res"] = WasmValtype.i32; + for (let i = 0; i < info.argumentCount; i++) + sig[`arg${i}`] = WasmValtype.i32; + sig["rmethod"] = WasmValtype.i32; + + // Function type for compiled traces + builder.defineType( + info.traceName, sig, WasmValtype.void + ); + } + + builder.generateTypeSection(); + + // Import section + const trampImports = getTrampImports(); + const compress = true; + + // Emit function imports + for (let i = 0; i < trampImports.length; i++) { + mono_assert(trampImports[i], () => `trace #${i} missing`); + const wasmName = compress ? i.toString(16) : undefined; + builder.defineImportedFunction("i", trampImports[i][0], trampImports[i][1], wasmName); + } + + builder.generateImportSection(); + + // Function section + builder.beginSection(3); + builder.appendULeb(jitQueue.length); + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + // Function type for our compiled trace + mono_assert(builder.functionTypes[info.traceName], "func type missing"); + builder.appendULeb(builder.functionTypes[info.traceName][0]); + } + + // Export section + builder.beginSection(7); + builder.appendULeb(jitQueue.length); + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + builder.appendName(info.traceName); + builder.appendU8(0); + // Imports get added to the function index space, so we need to add + // the count of imported functions to get the index of our compiled trace + builder.appendULeb(builder.importedFunctionCount + i); + } + + // Code section + builder.beginSection(10); + builder.appendULeb(jitQueue.length); + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + builder.beginFunction(info.traceName, { + "sp_args": WasmValtype.i32, + "need_unbox": WasmValtype.i32, + }); + + const ok = generate_wasm_body(builder, info); + if (!ok) + throw new Error(`Failed to generate ${info.traceName}`); + + builder.appendU8(WasmOpcode.end); + } + + builder.endSection(); + + compileStarted = _now(); + const buffer = builder.getArrayView(); + if (trace > 0) + console.log(`jit queue generated ${buffer.length} byte(s) of wasm`); + const traceModule = new WebAssembly.Module(buffer); + + const imports : any = { + h: (Module).asm.memory + }; + // Place our function imports into the import dictionary + for (let i = 0; i < trampImports.length; i++) { + const wasmName = compress ? i.toString(16) : trampImports[i][0]; + imports[wasmName] = trampImports[i][2]; + } + + const traceInstance = new WebAssembly.Instance(traceModule, { + i: imports + }); + + // Now that we've jitted the trampolines, go through and fix up the function pointers + // to point to the new jitted trampolines instead of the default implementations + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + + // Get the exported trampoline + const fn = traceInstance.exports[info.traceName]; + // Patch the function pointer for this function to use the trampoline now + fnTable.set(info.result, fn); + + rejected = false; + counters.entryWrappersCompiled++; + } + } catch (exc: any) { + threw = true; + rejected = false; + // console.error(`${traceName} failed: ${exc} ${exc.stack}`); + // HACK: exc.stack is enormous garbage in v8 console + console.error(`MONO_WASM: interp_entry trampoline jit failed: ${exc}`); + } finally { + const finished = _now(); + if (compileStarted) { + elapsedTimes.generation += compileStarted - started; + elapsedTimes.compilation += finished - compileStarted; + } else { + elapsedTimes.generation += finished - started; + } + + if (threw || (!rejected && ((trace >= 2) || dumpWrappers))) { + console.log(`// MONO_WASM: ${jitQueue.length} trampolines generated, blob follows //`); + let s = "", j = 0; + try { + if (builder.inSection) + builder.endSection(); + } catch { + // eslint-disable-next-line @typescript-eslint/no-extra-semi + ; + } + + const buf = builder.getArrayView(); + for (let i = 0; i < buf.length; i++) { + const b = buf[i]; + if (b < 0x10) + s += "0"; + s += b.toString(16); + s += " "; + if ((s.length % 10) === 0) { + console.log(`${j}\t${s}`); + s = ""; + j = i + 1; + } + } + console.log(`${j}\t${s}`); + console.log("// end blob //"); + } else if (rejected && !threw) { + console.error("MONO_WASM: failed to generate trampoline for unknown reason"); + } + + jitQueue.length = 0; + } +} + +function append_stackval_from_data ( + builder: WasmBuilder, type: MonoType, valueName: string +) { + const stackvalSize = cwraps.mono_jiterp_get_size_of_stackval(); + const rawSize = cwraps.mono_jiterp_type_get_raw_value_size(type); + + switch (rawSize) { + case 256: { + // Copy pointers directly + builder.local("sp_args"); + builder.local(valueName); + + builder.appendU8(WasmOpcode.i32_store); + builder.appendMemarg(0, 2); + + // Fixed stackval size + builder.i32_const(stackvalSize); + break; + } + + case -1: + case -2: + case 1: + case 2: + case 4: { + // De-reference small primitives and then store them directly + builder.local("sp_args"); + builder.local(valueName); + + switch (rawSize) { + case -1: + builder.appendU8(WasmOpcode.i32_load8_u); + builder.appendMemarg(0, 0); + break; + case 1: + builder.appendU8(WasmOpcode.i32_load8_s); + builder.appendMemarg(0, 0); + break; + case -2: + builder.appendU8(WasmOpcode.i32_load16_u); + builder.appendMemarg(0, 0); + break; + case 2: + builder.appendU8(WasmOpcode.i32_load16_s); + builder.appendMemarg(0, 0); + break; + case 4: + builder.appendU8(WasmOpcode.i32_load); + builder.appendMemarg(0, 2); + break; + // FIXME: 8-byte ints (unaligned) + // FIXME: 4 and 8-byte floats (unaligned) + } + + builder.appendU8(WasmOpcode.i32_store); + builder.appendMemarg(0, 2); + + // Fixed stackval size + builder.i32_const(stackvalSize); + break; + } + + default: { + // Call stackval_from_data to copy the value and get its size + builder.i32_const(type); + // result + builder.local("sp_args"); + // value + builder.local(valueName); + + builder.callImport("stackval_from_data"); + break; + } + } + + // Value size is on the stack, add it to sp_args and update it + builder.local("sp_args"); + builder.appendU8(WasmOpcode.i32_add); + builder.local("sp_args", WasmOpcode.set_local); +} + +function generate_wasm_body ( + builder: WasmBuilder, info: TrampolineInfo +) : boolean { + // FIXME: This is not thread-safe, but the alternative of alloca makes the trampoline + // more expensive + const scratchBuffer = Module._malloc(sizeOfJiterpEntryData); + _zero_region(scratchBuffer, sizeOfJiterpEntryData); + + // the this-reference may be a boxed struct that needs to be unboxed, for example calling + // methods like object.ToString on structs will end up with the unbox flag set + // instead of passing an extra 'unbox' argument to every wrapper, though, the flag is hidden + // inside the rmethod/imethod parameter in the lowest bit (1), so we need to check it + if (info.hasThisReference) { + builder.block(); + // Find the unbox-this-reference flag in rmethod + builder.local("rmethod"); + builder.i32_const(0x1); + builder.appendU8(WasmOpcode.i32_and); + // If the flag is not set (rmethod & 0x1) == 0 then skip the unbox operation + builder.appendU8(WasmOpcode.i32_eqz); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + // otherwise, the flag was set, so unbox the this reference and update the local + builder.local("this_arg"); + builder.callImport("unbox"); + builder.local("this_arg", WasmOpcode.set_local); + builder.endBlock(); + } + + // Populate the scratch buffer containing call data + builder.i32_const(scratchBuffer); + + builder.local("rmethod"); + // Clear the unbox-this-reference flag if present (see above) so that rmethod is a valid ptr + builder.i32_const(~0x1); + builder.appendU8(WasmOpcode.i32_and); + + // Store the cleaned up rmethod value into the data.rmethod field of the scratch buffer + builder.appendU8(WasmOpcode.i32_store); + builder.appendMemarg(offsetOfRMethod, 0); // data.rmethod + + // prologue takes data->rmethod and initializes data->context, then returns a value for sp_args + // prologue also performs thread attach + builder.i32_const(scratchBuffer); + // prologue takes this_arg so it can handle delegates + if (info.hasThisReference) + builder.local("this_arg"); + else + builder.i32_const(0); + builder.callImport("interp_entry_prologue"); + builder.local("sp_args", WasmOpcode.set_local); + + /* + if (sig->hasthis) { + sp_args->data.p = data->this_arg; + sp_args++; + } + */ + + if (info.hasThisReference) { + // null type for raw ptr copy + append_stackval_from_data(builder, 0, "this_arg"); + } + + /* + for (i = 0; i < sig->param_count; ++i) { + if (m_type_is_byref (sig->params [i])) { + sp_args->data.p = params [i]; + sp_args++; + } else { + int size = stackval_from_data (sig->params [i], sp_args, params [i], FALSE); + sp_args = STACK_ADD_BYTES (sp_args, size); + } + } + */ + + for (let i = 0; i < info.paramTypes.length; i++) { + const type = info.paramTypes[i]; + append_stackval_from_data(builder, type, `arg${i}`); + } + + builder.i32_const(scratchBuffer); + builder.local("sp_args"); + if (info.hasReturnValue) + builder.local("res"); + else + builder.i32_const(0); + builder.callImport("interp_entry"); + builder.appendU8(WasmOpcode.return_); + + return true; +} diff --git a/src/mono/wasm/runtime/jiterpreter-jit-call.ts b/src/mono/wasm/runtime/jiterpreter-jit-call.ts new file mode 100644 index 00000000000000..7994ba44489306 --- /dev/null +++ b/src/mono/wasm/runtime/jiterpreter-jit-call.ts @@ -0,0 +1,504 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +import { mono_assert } from "./types"; +import { NativePointer, Int32Ptr } from "./types/emscripten"; +import { Module } from "./imports"; +import { + getU8, getI32, getU32, setU32_unchecked +} from "./memory"; +import { WasmOpcode } from "./jiterpreter-opcodes"; +import { + WasmValtype, WasmBuilder, addWasmFunctionPointer as addWasmFunctionPointer, + _now, elapsedTimes, counters, getWasmFunctionTable, applyOptions +} from "./jiterpreter-support"; +import cwraps from "./cwraps"; + +// Controls miscellaneous diagnostic output. +const trace = 0; +const + // Dumps all compiled wrappers + dumpWrappers = false; + +/* +struct _JitCallInfo { + gpointer addr; // 0 + gpointer extra_arg; // 4 + gpointer wrapper; // 8 + MonoMethodSignature *sig; // 12 + guint8 *arginfo; // 16 + gint32 res_size; // 20 + int ret_mt; // 24 + gboolean no_wrapper; // 28 +#if HOST_BROWSER + int hit_count; + WasmJitCallThunk jiterp_thunk; +#endif +}; +*/ + +const offsetOfArgInfo = 16, + offsetOfRetMt = 24; + +const maxJitQueueLength = 4, + maxSharedQueueLength = 12, + flushParamThreshold = 7; + +let trampBuilder : WasmBuilder; +let fnTable : WebAssembly.Table; +let wasmEhSupported : boolean | undefined = undefined; +const fnCache : Array = []; +const targetCache : { [target: number] : TrampolineInfo } = {}; +const jitQueue : TrampolineInfo[] = []; + +class TrampolineInfo { + rmethod: NativePointer; + cinfo: NativePointer; + hasThisReference: boolean; + hasReturnValue: boolean; + paramCount: number; + argOffsets: number[]; + catchExceptions: boolean; + target: number; + name: string; + result: number; + queue: NativePointer[] = []; + + constructor ( + rmethod: NativePointer, cinfo: NativePointer, has_this: boolean, param_count: number, + arg_offsets: NativePointer, catch_exceptions: boolean, func: number + ) { + this.rmethod = rmethod; + this.cinfo = cinfo; + this.hasThisReference = has_this; + this.paramCount = param_count; + this.catchExceptions = catch_exceptions; + this.argOffsets = new Array(param_count); + this.hasReturnValue = getI32(cinfo + offsetOfRetMt) !== -1; + for (let i = 0, c = param_count + (has_this ? 1 : 0); i < c; i++) + this.argOffsets[i] = getU32(arg_offsets + (i * 4)); + this.target = func; + this.name = `jitcall_${func.toString(16)}`; + this.result = 0; + } +} + +function getWasmTableEntry (index: number) { + let result = fnCache[index]; + if (!result) { + if (index >= fnCache.length) + fnCache.length = index + 1; + fnCache[index] = result = fnTable.get(index); + } + return result; +} + +export function mono_interp_invoke_wasm_jit_call_trampoline ( + thunkIndex: number, extra_arg: number, + ret_sp: number, sp: number, thrown: NativePointer +) { + // FIXME: It's impossible to get emscripten to export this for some reason + // const thunk = Module.getWasmTableEntry(thunkIndex); + const thunk = getWasmTableEntry(thunkIndex); + try { + thunk(extra_arg, ret_sp, sp, thrown); + } catch (exc) { + setU32_unchecked(thrown, 1); + } +} + +export function mono_interp_jit_wasm_jit_call_trampoline ( + rmethod: NativePointer, cinfo: NativePointer, func: number, + has_this: number, param_count: number, + arg_offsets: NativePointer, catch_exceptions: number +) : void { + // multiple cinfos can share the same target function, so for that scenario we want to + // use the same TrampolineInfo for all of them. if that info has already been jitted + // we want to immediately store its pointer into the cinfo, otherwise we add it to + // a queue inside the info object so that all the cinfos will get updated once a + // jit operation happens + const existing = targetCache[func]; + if (existing) { + if (existing.result > 0) + cwraps.mono_jiterp_register_jit_call_thunk(cinfo, existing.result); + else { + existing.queue.push(cinfo); + // the jitQueue might never fill up if we have a bunch of cinfos that share + // the same target function, and they might never hit the call count threshold + // to flush the jit queue from the C side. since entering the queue at all + // requires hitting a minimum hit count on the C side, flush if we have too many + // shared cinfos all waiting for a JIT to happen. + if (existing.queue.length > maxSharedQueueLength) + mono_interp_flush_jitcall_queue(); + } + return; + } + + const info = new TrampolineInfo( + rmethod, cinfo, has_this !== 0, param_count, + arg_offsets, catch_exceptions !== 0, func + ); + targetCache[func] = info; + jitQueue.push(info); + + // we don't want the queue to get too long, both because jitting too many trampolines + // at once can hit the 4kb limit and because it makes it more likely that we will + // fail to jit them early enough + // HACK: we also want to flush the queue when we get a function with many parameters, + // since it's going to generate a lot more code and push us closer to 4kb + if ((info.paramCount >= flushParamThreshold) || (jitQueue.length >= maxJitQueueLength)) + mono_interp_flush_jitcall_queue(); +} + +// pure wasm implementation of do_jit_call_indirect (using wasm EH). see do-jit-call.wat / do-jit-call.wasm +const doJitCall16 = + "0061736d01000000010b0260017f0060037f7f7f00021d020169066d656d6f727902000001690b6a69745f63616c6c5f636200000302010107180114646f5f6a69745f63616c6c5f696e64697265637400010a1301110006402001100019200241013602000b0b"; +let doJitCallModule : WebAssembly.Module | undefined = undefined; + +function getIsWasmEhSupported () : boolean { + if (wasmEhSupported !== undefined) + return wasmEhSupported; + + // Probe whether the current environment can handle wasm exceptions + try { + // Load and compile the wasm version of do_jit_call_indirect. This serves as a way to probe for wasm EH + const bytes = new Uint8Array(doJitCall16.length / 2); + for (let i = 0; i < doJitCall16.length; i += 2) + bytes[i / 2] = parseInt(doJitCall16.substring(i, i + 2), 16); + + doJitCallModule = new WebAssembly.Module(bytes); + wasmEhSupported = true; + } catch (exc) { + console.log("MONO_WASM: Disabling WASM EH support due to JIT failure", exc); + wasmEhSupported = false; + } + + return wasmEhSupported; +} + +// this is the generic entry point for do_jit_call that is registered by default at runtime startup. +// its job is to do initialization for the optimized do_jit_call path, which will either use a jitted +// wasm trampoline or will use a specialized JS function. +export function mono_jiterp_do_jit_call_indirect ( + jit_call_cb: number, cb_data: NativePointer, thrown: Int32Ptr +) : void { + const table = getWasmFunctionTable(); + const jitCallCb = table.get(jit_call_cb); + + // This should perform better than the regular mono_llvm_cpp_catch_exception because the call target + // is statically known, not being pulled out of a table. + const do_jit_call_indirect_js = function (unused: number, _cb_data: NativePointer, _thrown: Int32Ptr) { + try { + jitCallCb(_cb_data); + } catch { + setU32_unchecked(_thrown, 1); + } + }; + + let failed = !getIsWasmEhSupported(); + if (!failed) { + // Wasm EH is supported which means doJitCallModule was loaded and compiled. + // Now that we have jit_call_cb, we can instantiate it. + try { + const instance = new WebAssembly.Instance(doJitCallModule!, { + i: { + jit_call_cb: jitCallCb, + memory: (Module).asm.memory + } + }); + const impl = instance.exports.do_jit_call_indirect; + if (typeof (impl) !== "function") + throw new Error("Did not find exported do_jit_call handler"); + + // console.log("registering wasm jit call dispatcher"); + // We successfully instantiated it so we can register it as the new do_jit_call handler + const result = addWasmFunctionPointer(impl); + cwraps.mono_jiterp_update_jit_call_dispatcher(result); + failed = false; + } catch (exc) { + console.error("MONO_WASM: failed to compile do_jit_call handler", exc); + failed = true; + } + // If wasm EH support was detected, a native wasm implementation of the dispatcher was already registered. + } + + if (failed) { + // console.log("registering JS jit call dispatcher"); + try { + const result = Module.addFunction(do_jit_call_indirect_js, "viii"); + cwraps.mono_jiterp_update_jit_call_dispatcher(result); + } catch { + // CSP policy or some other problem could break Module.addFunction, so in that case, pass 0 + // This will cause the runtime to use mono_llvm_cpp_catch_exception + cwraps.mono_jiterp_update_jit_call_dispatcher(0); + } + } + + do_jit_call_indirect_js(jit_call_cb, cb_data, thrown); +} + +export function mono_interp_flush_jitcall_queue () : void { + if (jitQueue.length === 0) + return; + + let builder = trampBuilder; + if (!builder) + trampBuilder = builder = new WasmBuilder(); + else + builder.clear(); + + if (builder.options.enableWasmEh) { + if (!getIsWasmEhSupported()) { + // The user requested to enable wasm EH but it's not supported, so turn the option back off + applyOptions({enableWasmEh: false}); + builder.options.enableWasmEh = false; + } + } + + const started = _now(); + let compileStarted = 0; + let rejected = true, threw = false; + + const trampImports : Array<[string, string, Function]> = []; + + try { + if (!fnTable) + fnTable = getWasmFunctionTable(); + + // Magic number and version + builder.appendU32(0x6d736100); + builder.appendU32(1); + + // Function type for compiled trampolines + builder.defineType( + "trampoline", { + "extra_arg": WasmValtype.i32, + "ret_sp": WasmValtype.i32, + "sp": WasmValtype.i32, + "thrown": WasmValtype.i32, + }, WasmValtype.void + ); + + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + const ctn = `fn${info.target.toString(16)}`; + + const actualParamCount = (info.hasThisReference ? 1 : 0) + (info.hasReturnValue ? 1 : 0) + info.paramCount; + const sig : any = {}; + for (let j = 0; j < actualParamCount; j++) + sig[`arg${j}`] = WasmValtype.i32; + sig["extra_arg"] = WasmValtype.i32; + builder.defineType( + ctn, sig, WasmValtype.void + ); + + const callTarget = getWasmTableEntry(info.target); + mono_assert(typeof (callTarget) === "function", () => `expected call target to be function but was ${callTarget}`); + trampImports.push([ctn, ctn, callTarget]); + } + + builder.generateTypeSection(); + + const compress = true; + // Emit function imports + for (let i = 0; i < trampImports.length; i++) { + const wasmName = compress ? i.toString(16) : undefined; + builder.defineImportedFunction("i", trampImports[i][0], trampImports[i][1], wasmName); + } + builder.generateImportSection(); + + // Function section + builder.beginSection(3); + builder.appendULeb(jitQueue.length); + // Function type for our compiled trampoline + mono_assert(builder.functionTypes["trampoline"], "func type missing"); + + for (let i = 0; i < jitQueue.length; i++) + builder.appendULeb(builder.functionTypes["trampoline"][0]); + + // Export section + builder.beginSection(7); + builder.appendULeb(jitQueue.length); + + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + builder.appendName(info.name); + builder.appendU8(0); + // Imports get added to the function index space, so we need to add + // the count of imported functions to get the index of our compiled trace + builder.appendULeb(builder.importedFunctionCount + i); + } + + // Code section + builder.beginSection(10); + builder.appendULeb(jitQueue.length); + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + builder.beginFunction("trampoline", {}); + + const ok = generate_wasm_body(builder, info); + // FIXME + if (!ok) + throw new Error(`Failed to generate ${info.name}`); + builder.appendU8(WasmOpcode.end); + } + + builder.endSection(); + + compileStarted = _now(); + const buffer = builder.getArrayView(); + if (trace > 0) + console.log(`do_jit_call queue flush generated ${buffer.length} byte(s) of wasm`); + const traceModule = new WebAssembly.Module(buffer); + + const imports : any = { + h: (Module).asm.memory + }; + // Place our function imports into the import dictionary + for (let i = 0; i < trampImports.length; i++) { + const wasmName = compress ? i.toString(16) : trampImports[i][0]; + imports[wasmName] = trampImports[i][2]; + } + + const traceInstance = new WebAssembly.Instance(traceModule, { + i: imports + }); + + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + + // Get the exported trace function + const jitted = traceInstance.exports[info.name]; + const idx = addWasmFunctionPointer(jitted); + if (!idx) + throw new Error("add_function_pointer returned a 0 index"); + else if (trace >= 2) + console.log(`${info.name} -> fn index ${idx}`); + + info.result = idx; + cwraps.mono_jiterp_register_jit_call_thunk(info.cinfo, idx); + for (let j = 0; j < info.queue.length; j++) + cwraps.mono_jiterp_register_jit_call_thunk(info.queue[j], idx); + + counters.jitCallsCompiled++; + info.queue.length = 0; + rejected = false; + } + } catch (exc: any) { + threw = true; + rejected = false; + // console.error(`${traceName} failed: ${exc} ${exc.stack}`); + // HACK: exc.stack is enormous garbage in v8 console + console.error(`jit failed: ${exc}`); + } finally { + const finished = _now(); + if (compileStarted) { + elapsedTimes.generation += compileStarted - started; + elapsedTimes.compilation += finished - compileStarted; + } else { + elapsedTimes.generation += finished - started; + } + + if (threw || rejected) { + for (let i = 0; i < jitQueue.length; i++) { + const info = jitQueue[i]; + info.result = -1; + } + } + + // FIXME + if (threw || (!rejected && ((trace >= 2) || dumpWrappers))) { + console.log(`// MONO_WASM: ${jitQueue.length} jit call wrappers generated, blob follows //`); + let s = "", j = 0; + try { + if (builder.inSection) + builder.endSection(); + } catch { + // eslint-disable-next-line @typescript-eslint/no-extra-semi + ; + } + + const buf = builder.getArrayView(); + for (let i = 0; i < buf.length; i++) { + const b = buf[i]; + if (b < 0x10) + s += "0"; + s += b.toString(16); + s += " "; + if ((s.length % 10) === 0) { + console.log(`${j}\t${s}`); + s = ""; + j = i + 1; + } + } + console.log(`${j}\t${s}`); + console.log("// end blob //"); + } else if (rejected && !threw) { + console.error("MONO_WASM: failed to generate trampoline for unknown reason"); + } + + jitQueue.length = 0; + } +} + +function append_ldloc (builder: WasmBuilder, offset: number, opcode: WasmOpcode) { + builder.local("sp"); + builder.appendU8(opcode); + builder.appendMemarg(offset, 2); +} + +const JIT_ARG_BYVAL = 0; + +function generate_wasm_body ( + builder: WasmBuilder, info: TrampolineInfo +) : boolean { + let stack_index = 0; + + if (builder.options.enableWasmEh) + builder.block(WasmValtype.void, WasmOpcode.try_); + + if (info.hasThisReference) { + append_ldloc(builder, 0, WasmOpcode.i32_load); + stack_index++; + } + + /* return address */ + if (info.hasReturnValue) + builder.local("ret_sp"); + + for (let i = 0; i < info.paramCount; i++) { + // FIXME: STACK_ADD_BYTES does alignment, but we probably don't need to? + const svalOffset = info.argOffsets[stack_index + i]; + const argInfoOffset = getU32(info.cinfo + offsetOfArgInfo) + i; + const argInfo = getU8(argInfoOffset); + if (argInfo == JIT_ARG_BYVAL) { + // pass the first four bytes of the stackval data union, + // which is 'p' where pointers live + builder.local("sp"); + builder.appendU8(WasmOpcode.i32_load); + builder.appendMemarg(svalOffset, 2); + } else { + // pass the address of the stackval data union + builder.local("sp"); + builder.i32_const(svalOffset); + builder.appendU8(WasmOpcode.i32_add); + } + } + + builder.local("extra_arg"); + builder.callImport(`fn${info.target.toString(16)}`); + + if (builder.options.enableWasmEh) { + builder.appendU8(WasmOpcode.catch_all); + builder.local("thrown"); + builder.i32_const(1); + builder.appendU8(WasmOpcode.i32_store); + builder.appendMemarg(0, 2); + + builder.endBlock(); + } + + builder.appendU8(WasmOpcode.return_); + + return true; +} diff --git a/src/mono/wasm/runtime/jiterpreter-opcodes.ts b/src/mono/wasm/runtime/jiterpreter-opcodes.ts new file mode 100644 index 00000000000000..9b714eacdd75e5 --- /dev/null +++ b/src/mono/wasm/runtime/jiterpreter-opcodes.ts @@ -0,0 +1,1833 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +// Keep this file in sync with mintops.def. The order and values need to match exactly. + +export const enum MintOpcode { + MINT_NOP = 0, + MINT_NIY, + MINT_DEF, + MINT_IL_SEQ_POINT, + MINT_DUMMY_USE, + MINT_TIER_PATCHPOINT_DATA, + MINT_BREAK, + MINT_BREAKPOINT, + + MINT_RET, + MINT_RET_VOID, + MINT_RET_VT, + MINT_RET_LOCALLOC, + MINT_RET_VOID_LOCALLOC, + MINT_RET_VT_LOCALLOC, + + MINT_RET_I1, + MINT_RET_U1, + MINT_RET_I2, + MINT_RET_U2, + + MINT_LDC_I4_M1, + MINT_LDC_I4_0, + MINT_LDC_I4_1, + MINT_LDC_I4_2, + MINT_LDC_I4_3, + MINT_LDC_I4_4, + MINT_LDC_I4_5, + MINT_LDC_I4_6, + MINT_LDC_I4_7, + MINT_LDC_I4_8, + MINT_LDC_I4_S, + MINT_LDC_I4, + + MINT_LDC_I8_0, + MINT_LDC_I8_S, + MINT_LDC_I8, + + MINT_LDC_R4, + MINT_LDC_R8, + + MINT_INIT_ARGLIST, + + MINT_LDFLD_I1, + MINT_LDFLD_U1, + MINT_LDFLD_I2, + MINT_LDFLD_U2, + MINT_LDFLD_I4, + MINT_LDFLD_I8, + MINT_LDFLD_R4, + MINT_LDFLD_R8, + MINT_LDFLD_O, + MINT_LDFLD_VT, + MINT_LDFLD_I8_UNALIGNED, + MINT_LDFLD_R8_UNALIGNED, + + MINT_LDFLDA, + MINT_LDFLDA_UNSAFE, + + MINT_STFLD_I1, + MINT_STFLD_U1, + MINT_STFLD_I2, + MINT_STFLD_U2, + MINT_STFLD_I4, + MINT_STFLD_I8, + MINT_STFLD_R4, + MINT_STFLD_R8, + MINT_STFLD_O, + MINT_STFLD_VT, + MINT_STFLD_VT_NOREF, + MINT_STFLD_I8_UNALIGNED, + MINT_STFLD_R8_UNALIGNED, + + MINT_LDSFLD_I1, + MINT_LDSFLD_U1, + MINT_LDSFLD_I2, + MINT_LDSFLD_U2, + MINT_LDSFLD_I4, + MINT_LDSFLD_I8, + MINT_LDSFLD_R4, + MINT_LDSFLD_R8, + MINT_LDSFLD_O, + MINT_LDSFLD_VT, + MINT_LDSFLD_W, + + MINT_STSFLD_I1, + MINT_STSFLD_U1, + MINT_STSFLD_I2, + MINT_STSFLD_U2, + MINT_STSFLD_I4, + MINT_STSFLD_I8, + MINT_STSFLD_R4, + MINT_STSFLD_R8, + MINT_STSFLD_O, + MINT_STSFLD_VT, + MINT_STSFLD_W, + MINT_LDSFLDA, + MINT_LDTSFLDA, + + MINT_MOV_SRC_OFF, + MINT_MOV_DST_OFF, + + MINT_MOV_I4_I1, + MINT_MOV_I4_U1, + MINT_MOV_I4_I2, + MINT_MOV_I4_U2, + MINT_MOV_1, + MINT_MOV_2, + MINT_MOV_4, + MINT_MOV_8, + MINT_MOV_VT, + + // These opcodes represent multiple moves stacked together. They have multiple src and dst + // but they are not represented here. They are generated by the var offset allocator. + MINT_MOV_8_2, + MINT_MOV_8_3, + MINT_MOV_8_4, + + MINT_LDLOCA_S, + + MINT_LDIND_I1, + MINT_LDIND_U1, + MINT_LDIND_I2, + MINT_LDIND_U2, + MINT_LDIND_I4, + MINT_LDIND_I8, + MINT_LDIND_R4, + MINT_LDIND_R8, + + MINT_LDIND_OFFSET_I1, + MINT_LDIND_OFFSET_U1, + MINT_LDIND_OFFSET_I2, + MINT_LDIND_OFFSET_U2, + MINT_LDIND_OFFSET_I4, + MINT_LDIND_OFFSET_I8, + + MINT_LDIND_OFFSET_IMM_I1, + MINT_LDIND_OFFSET_IMM_U1, + MINT_LDIND_OFFSET_IMM_I2, + MINT_LDIND_OFFSET_IMM_U2, + MINT_LDIND_OFFSET_IMM_I4, + MINT_LDIND_OFFSET_IMM_I8, + + MINT_STIND_I1, + MINT_STIND_I2, + MINT_STIND_I4, + MINT_STIND_I8, + MINT_STIND_R4, + MINT_STIND_R8, + MINT_STIND_REF, + + MINT_STIND_OFFSET_I1, + MINT_STIND_OFFSET_I2, + MINT_STIND_OFFSET_I4, + MINT_STIND_OFFSET_I8, + + MINT_STIND_OFFSET_IMM_I1, + MINT_STIND_OFFSET_IMM_I2, + MINT_STIND_OFFSET_IMM_I4, + MINT_STIND_OFFSET_IMM_I8, + + MINT_BR, + MINT_LEAVE, + MINT_LEAVE_CHECK, + MINT_BR_S, + MINT_LEAVE_S, + MINT_LEAVE_S_CHECK, + MINT_CALL_HANDLER, + MINT_CALL_HANDLER_S, + + MINT_THROW, + MINT_RETHROW, + MINT_ENDFINALLY, + MINT_MONO_RETHROW, + + MINT_SAFEPOINT, + + MINT_BRFALSE_I4, + MINT_BRFALSE_I8, + MINT_BRFALSE_R4, + MINT_BRFALSE_R8, + MINT_BRTRUE_I4, + MINT_BRTRUE_I8, + MINT_BRTRUE_R4, + MINT_BRTRUE_R8, + + MINT_BRFALSE_I4_S, + MINT_BRFALSE_I8_S, + MINT_BRFALSE_R4_S, + MINT_BRFALSE_R8_S, + MINT_BRTRUE_I4_S, + MINT_BRTRUE_I8_S, + MINT_BRTRUE_R4_S, + MINT_BRTRUE_R8_S, + + MINT_BEQ_I4, + MINT_BEQ_I8, + MINT_BEQ_R4, + MINT_BEQ_R8, + MINT_BGE_I4, + MINT_BGE_I8, + MINT_BGE_R4, + MINT_BGE_R8, + MINT_BGT_I4, + MINT_BGT_I8, + MINT_BGT_R4, + MINT_BGT_R8, + MINT_BLT_I4, + MINT_BLT_I8, + MINT_BLT_R4, + MINT_BLT_R8, + MINT_BLE_I4, + MINT_BLE_I8, + MINT_BLE_R4, + MINT_BLE_R8, + + MINT_BNE_UN_I4, + MINT_BNE_UN_I8, + MINT_BNE_UN_R4, + MINT_BNE_UN_R8, + MINT_BGE_UN_I4, + MINT_BGE_UN_I8, + MINT_BGE_UN_R4, + MINT_BGE_UN_R8, + MINT_BGT_UN_I4, + MINT_BGT_UN_I8, + MINT_BGT_UN_R4, + MINT_BGT_UN_R8, + MINT_BLE_UN_I4, + MINT_BLE_UN_I8, + MINT_BLE_UN_R4, + MINT_BLE_UN_R8, + MINT_BLT_UN_I4, + MINT_BLT_UN_I8, + MINT_BLT_UN_R4, + MINT_BLT_UN_R8, + + MINT_BEQ_I4_S, + MINT_BEQ_I8_S, + MINT_BEQ_R4_S, + MINT_BEQ_R8_S, + MINT_BGE_I4_S, + MINT_BGE_I8_S, + MINT_BGE_R4_S, + MINT_BGE_R8_S, + MINT_BGT_I4_S, + MINT_BGT_I8_S, + MINT_BGT_R4_S, + MINT_BGT_R8_S, + MINT_BLT_I4_S, + MINT_BLT_I8_S, + MINT_BLT_R4_S, + MINT_BLT_R8_S, + MINT_BLE_I4_S, + MINT_BLE_I8_S, + MINT_BLE_R4_S, + MINT_BLE_R8_S, + + MINT_BNE_UN_I4_S, + MINT_BNE_UN_I8_S, + MINT_BNE_UN_R4_S, + MINT_BNE_UN_R8_S, + MINT_BGE_UN_I4_S, + MINT_BGE_UN_I8_S, + MINT_BGE_UN_R4_S, + MINT_BGE_UN_R8_S, + MINT_BGT_UN_I4_S, + MINT_BGT_UN_I8_S, + MINT_BGT_UN_R4_S, + MINT_BGT_UN_R8_S, + MINT_BLE_UN_I4_S, + MINT_BLE_UN_I8_S, + MINT_BLE_UN_R4_S, + MINT_BLE_UN_R8_S, + MINT_BLT_UN_I4_S, + MINT_BLT_UN_I8_S, + MINT_BLT_UN_R4_S, + MINT_BLT_UN_R8_S, + + MINT_BRFALSE_I4_SP, + MINT_BRFALSE_I8_SP, + MINT_BRTRUE_I4_SP, + MINT_BRTRUE_I8_SP, + + MINT_BEQ_I4_SP, + MINT_BEQ_I8_SP, + MINT_BGE_I4_SP, + MINT_BGE_I8_SP, + MINT_BGT_I4_SP, + MINT_BGT_I8_SP, + MINT_BLT_I4_SP, + MINT_BLT_I8_SP, + MINT_BLE_I4_SP, + MINT_BLE_I8_SP, + + MINT_BNE_UN_I4_SP, + MINT_BNE_UN_I8_SP, + MINT_BGE_UN_I4_SP, + MINT_BGE_UN_I8_SP, + MINT_BGT_UN_I4_SP, + MINT_BGT_UN_I8_SP, + MINT_BLE_UN_I4_SP, + MINT_BLE_UN_I8_SP, + MINT_BLT_UN_I4_SP, + MINT_BLT_UN_I8_SP, + + MINT_BEQ_I4_IMM_SP, + MINT_BEQ_I8_IMM_SP, + MINT_BGE_I4_IMM_SP, + MINT_BGE_I8_IMM_SP, + MINT_BGT_I4_IMM_SP, + MINT_BGT_I8_IMM_SP, + MINT_BLT_I4_IMM_SP, + MINT_BLT_I8_IMM_SP, + MINT_BLE_I4_IMM_SP, + MINT_BLE_I8_IMM_SP, + + MINT_BNE_UN_I4_IMM_SP, + MINT_BNE_UN_I8_IMM_SP, + MINT_BGE_UN_I4_IMM_SP, + MINT_BGE_UN_I8_IMM_SP, + MINT_BGT_UN_I4_IMM_SP, + MINT_BGT_UN_I8_IMM_SP, + MINT_BLE_UN_I4_IMM_SP, + MINT_BLE_UN_I8_IMM_SP, + MINT_BLT_UN_I4_IMM_SP, + MINT_BLT_UN_I8_IMM_SP, + + + MINT_SWITCH, + + MINT_LDSTR, + MINT_LDSTR_TOKEN, + + MINT_JMP, + + MINT_ENDFILTER, + + MINT_NEWOBJ_SLOW_UNOPT, + MINT_NEWOBJ_STRING_UNOPT, + MINT_NEWOBJ_SLOW, + MINT_NEWOBJ_ARRAY, + MINT_NEWOBJ_STRING, + MINT_NEWOBJ, + MINT_NEWOBJ_INLINED, + MINT_NEWOBJ_VT, + MINT_NEWOBJ_VT_INLINED, + MINT_INITOBJ, + MINT_CASTCLASS, + MINT_ISINST, + MINT_CASTCLASS_INTERFACE, + MINT_ISINST_INTERFACE, + MINT_CASTCLASS_COMMON, + MINT_ISINST_COMMON, + MINT_NEWARR, + MINT_BOX, + MINT_BOX_VT, + MINT_BOX_PTR, + MINT_BOX_NULLABLE_PTR, + MINT_UNBOX, + MINT_LDTOKEN, + MINT_LDFTN, + MINT_LDFTN_ADDR, + MINT_LDFTN_DYNAMIC, + MINT_LDVIRTFTN, + MINT_CPOBJ, + MINT_CPOBJ_VT, + MINT_LDOBJ_VT, + MINT_STOBJ_VT, + MINT_CPBLK, + MINT_INITBLK, + MINT_LOCALLOC, + MINT_INITLOCAL, + MINT_INITLOCALS, + + MINT_LDELEM_I, + MINT_LDELEM_I1, + MINT_LDELEM_U1, + MINT_LDELEM_I2, + MINT_LDELEM_U2, + MINT_LDELEM_I4, + MINT_LDELEM_U4, + MINT_LDELEM_I8, + MINT_LDELEM_R4, + MINT_LDELEM_R8, + MINT_LDELEM_REF, + MINT_LDELEM_VT, + + MINT_LDELEMA1, + MINT_LDELEMA, + MINT_LDELEMA_TC, + + MINT_STELEM_I, + MINT_STELEM_I1, + MINT_STELEM_U1, + MINT_STELEM_I2, + MINT_STELEM_U2, + MINT_STELEM_I4, + MINT_STELEM_I8, + MINT_STELEM_R4, + MINT_STELEM_R8, + MINT_STELEM_REF, + MINT_STELEM_VT, + + MINT_LDLEN, + + MINT_GETITEM_SPAN, + MINT_GETITEM_LOCALSPAN, + + /* binops */ + MINT_ADD_I4, + MINT_ADD_I8, + MINT_ADD_R4, + MINT_ADD_R8, + + MINT_SUB_I4, + MINT_SUB_I8, + MINT_SUB_R4, + MINT_SUB_R8, + + MINT_MUL_I4, + MINT_MUL_I8, + MINT_MUL_R4, + MINT_MUL_R8, + + MINT_DIV_I4, + MINT_DIV_I8, + MINT_DIV_R4, + MINT_DIV_R8, + + MINT_DIV_UN_I4, + MINT_DIV_UN_I8, + + MINT_ADD_OVF_I4, + MINT_ADD_OVF_I8, + + MINT_ADD_OVF_UN_I4, + MINT_ADD_OVF_UN_I8, + + MINT_MUL_OVF_I4, + MINT_MUL_OVF_I8, + + MINT_MUL_OVF_UN_I4, + MINT_MUL_OVF_UN_I8, + + MINT_SUB_OVF_I4, + MINT_SUB_OVF_I8, + + MINT_SUB_OVF_UN_I4, + MINT_SUB_OVF_UN_I8, + + MINT_AND_I4, + MINT_AND_I8, + + MINT_OR_I4, + MINT_OR_I8, + + MINT_XOR_I4, + MINT_XOR_I8, + + MINT_REM_I4, + MINT_REM_I8, + MINT_REM_R4, + MINT_REM_R8, + + MINT_REM_UN_I4, + MINT_REM_UN_I8, + + // Shifts, keep in order with imm versions + MINT_SHR_UN_I4, + MINT_SHR_UN_I8, + MINT_SHL_I4, + MINT_SHL_I8, + MINT_SHR_I4, + MINT_SHR_I8, + + MINT_CEQ_I4, + MINT_CEQ_I8, + MINT_CEQ_R4, + MINT_CEQ_R8, + + MINT_CNE_I4, + MINT_CNE_I8, + MINT_CNE_R4, + MINT_CNE_R8, + + MINT_CGT_I4, + MINT_CGT_I8, + MINT_CGT_R4, + MINT_CGT_R8, + + MINT_CGE_I4, + MINT_CGE_I8, + MINT_CGE_R4, + MINT_CGE_R8, + + MINT_CGE_UN_I4, + MINT_CGE_UN_I8, + + MINT_CGT_UN_I4, + MINT_CGT_UN_I8, + MINT_CGT_UN_R4, + MINT_CGT_UN_R8, + + MINT_CLT_I4, + MINT_CLT_I8, + MINT_CLT_R4, + MINT_CLT_R8, + + MINT_CLE_I4, + MINT_CLE_I8, + MINT_CLE_R4, + MINT_CLE_R8, + + MINT_CLE_UN_I4, + MINT_CLE_UN_I8, + + MINT_CLT_UN_I4, + MINT_CLT_UN_I8, + MINT_CLT_UN_R4, + MINT_CLT_UN_R8, + /* binops end */ + + /* unops */ + MINT_ADD1_I4, + MINT_ADD1_I8, + MINT_SUB1_I4, + MINT_SUB1_I8, + + MINT_NEG_I4, + MINT_NEG_I8, + MINT_NEG_R4, + MINT_NEG_R8, + + MINT_NOT_I4, + MINT_NOT_I8, + + MINT_CONV_R_UN_I4, + MINT_CONV_R_UN_I8, + + MINT_CONV_I1_I4, + MINT_CONV_I1_I8, + MINT_CONV_I1_R4, + MINT_CONV_I1_R8, + + MINT_CONV_U1_I4, + MINT_CONV_U1_I8, + MINT_CONV_U1_R4, + MINT_CONV_U1_R8, + + MINT_CONV_I2_I4, + MINT_CONV_I2_I8, + MINT_CONV_I2_R4, + MINT_CONV_I2_R8, + + MINT_CONV_U2_I4, + MINT_CONV_U2_I8, + MINT_CONV_U2_R4, + MINT_CONV_U2_R8, + + MINT_CONV_I4_R4, + MINT_CONV_I4_R8, + + MINT_CONV_U4_R4, + MINT_CONV_U4_R8, + + MINT_CONV_I8_I4, + MINT_CONV_I8_U4, + MINT_CONV_I8_R4, + MINT_CONV_I8_R8, + + MINT_CONV_R4_I4, + MINT_CONV_R4_I8, + MINT_CONV_R4_R8, + + MINT_CONV_R8_I4, + MINT_CONV_R8_I8, + MINT_CONV_R8_R4, + + MINT_CONV_U8_R4, + MINT_CONV_U8_R8, + + MINT_CONV_OVF_I1_I4, + MINT_CONV_OVF_I1_I8, + MINT_CONV_OVF_I1_R4, + MINT_CONV_OVF_I1_R8, + + MINT_CONV_OVF_I1_U4, + MINT_CONV_OVF_I1_U8, + + MINT_CONV_OVF_U1_I4, + MINT_CONV_OVF_U1_I8, + MINT_CONV_OVF_U1_R4, + MINT_CONV_OVF_U1_R8, + + MINT_CONV_OVF_I2_I4, + MINT_CONV_OVF_I2_I8, + MINT_CONV_OVF_I2_R4, + MINT_CONV_OVF_I2_R8, + + MINT_CONV_OVF_I2_U4, + MINT_CONV_OVF_I2_U8, + + MINT_CONV_OVF_U2_I4, + MINT_CONV_OVF_U2_I8, + MINT_CONV_OVF_U2_R4, + MINT_CONV_OVF_U2_R8, + + MINT_CONV_OVF_I4_U4, + MINT_CONV_OVF_I4_I8, + MINT_CONV_OVF_I4_U8, + MINT_CONV_OVF_I4_R4, + MINT_CONV_OVF_I4_R8, + + MINT_CONV_OVF_U4_I4, + MINT_CONV_OVF_U4_I8, + MINT_CONV_OVF_U4_R4, + MINT_CONV_OVF_U4_R8, + + MINT_CONV_OVF_I8_U8, + MINT_CONV_OVF_I8_R4, + MINT_CONV_OVF_I8_R8, + + MINT_CONV_OVF_U8_I4, + MINT_CONV_OVF_U8_I8, + MINT_CONV_OVF_U8_R4, + MINT_CONV_OVF_U8_R8, + + MINT_CEQ0_I4, + /* unops end */ + + /* super instructions */ + MINT_RET_I4_IMM, + MINT_RET_I8_IMM, + + MINT_ADD_I4_IMM, + MINT_ADD_I8_IMM, + + MINT_MUL_I4_IMM, + MINT_MUL_I8_IMM, + + MINT_SHR_UN_I4_IMM, + MINT_SHR_UN_I8_IMM, + MINT_SHL_I4_IMM, + MINT_SHL_I8_IMM, + MINT_SHR_I4_IMM, + MINT_SHR_I8_IMM, + + + MINT_CKFINITE_R4, + MINT_CKFINITE_R8, + MINT_MKREFANY, + MINT_REFANYTYPE, + MINT_REFANYVAL, + + MINT_CKNULL, + + MINT_GETCHR, + MINT_STRLEN, + MINT_ARRAY_RANK, + MINT_ARRAY_ELEMENT_SIZE, + MINT_ARRAY_IS_PRIMITIVE, + + /* Calls */ + MINT_CALL, + MINT_CALLVIRT, + MINT_CALLVIRT_FAST, + MINT_CALL_DELEGATE, + MINT_CALLI, + MINT_CALLI_NAT, + MINT_CALLI_NAT_DYNAMIC, + MINT_CALLI_NAT_FAST, + MINT_CALL_VARARG, + MINT_CALLRUN, + MINT_TAILCALL, + MINT_TAILCALL_VIRT, + + MINT_ICALL_V_V, + MINT_ICALL_V_P, + MINT_ICALL_P_V, + MINT_ICALL_P_P, + MINT_ICALL_PP_V, + MINT_ICALL_PP_P, + MINT_ICALL_PPP_V, + MINT_ICALL_PPP_P, + MINT_ICALL_PPPP_V, + MINT_ICALL_PPPP_P, + MINT_ICALL_PPPPP_V, + MINT_ICALL_PPPPP_P, + MINT_ICALL_PPPPPP_V, + MINT_ICALL_PPPPPP_P, + // FIXME: MintOp + MINT_JIT_CALL, + MINT_JIT_CALL2, + + MINT_MONO_LDPTR, + MINT_MONO_SGEN_THREAD_INFO, + MINT_MONO_NEWOBJ, + MINT_MONO_RETOBJ, + MINT_MONO_ATOMIC_STORE_I4, + MINT_MONO_MEMORY_BARRIER, + MINT_MONO_EXCHANGE_I8, + MINT_MONO_LDDOMAIN, + MINT_MONO_ENABLE_GCTRANS, + + MINT_SDB_INTR_LOC, + MINT_SDB_SEQ_POINT, + MINT_SDB_BREAKPOINT, + MINT_LD_DELEGATE_METHOD_PTR, + + // Math intrinsics + // double + MINT_ASIN, + MINT_ASINH, + MINT_ACOS, + MINT_ACOSH, + MINT_ATAN, + MINT_ATANH, + MINT_ATAN2, + MINT_CEILING, + MINT_COS, + MINT_CBRT, + MINT_COSH, + MINT_EXP, + MINT_FMA, + MINT_FLOOR, + MINT_LOG, + MINT_LOG2, + MINT_LOG10, + MINT_POW, + MINT_SCALEB, + MINT_SIN, + MINT_SQRT, + MINT_SINH, + MINT_TAN, + MINT_TANH, + MINT_ABS, + MINT_MIN, + MINT_MAX, + + // float. These must be kept in the same order as their double counterpart + MINT_ASINF, + MINT_ASINHF, + MINT_ACOSF, + MINT_ACOSHF, + MINT_ATANF, + MINT_ATANHF, + MINT_ATAN2F, + MINT_CEILINGF, + MINT_COSF, + MINT_CBRTF, + MINT_COSHF, + MINT_EXPF, + MINT_FMAF, + MINT_FLOORF, + MINT_LOGF, + MINT_LOG2F, + MINT_LOG10F, + MINT_POWF, + MINT_SCALEBF, + MINT_SINF, + MINT_SQRTF, + MINT_SINHF, + MINT_TANF, + MINT_TANHF, + MINT_ABSF, + MINT_MINF, + MINT_MAXF, + + MINT_PROF_ENTER, + MINT_PROF_EXIT, + MINT_PROF_EXIT_VOID, + MINT_PROF_COVERAGE_STORE, + + MINT_TIER_ENTER_METHOD, + MINT_TIER_PATCHPOINT, + + MINT_INTRINS_ENUM_HASFLAG, + MINT_INTRINS_GET_HASHCODE, + MINT_INTRINS_GET_TYPE, + MINT_INTRINS_SPAN_CTOR, + MINT_INTRINS_UNSAFE_BYTE_OFFSET, + MINT_INTRINS_RUNTIMEHELPERS_OBJECT_HAS_COMPONENT_SIZE, + MINT_INTRINS_CLEAR_WITH_REFERENCES, + MINT_INTRINS_MARVIN_BLOCK, + MINT_INTRINS_ASCII_CHARS_TO_UPPERCASE, + MINT_INTRINS_MEMORYMARSHAL_GETARRAYDATAREF, + MINT_INTRINS_ORDINAL_IGNORE_CASE_ASCII, + MINT_INTRINS_64ORDINAL_IGNORE_CASE_ASCII, + MINT_INTRINS_U32_TO_DECSTR, + MINT_INTRINS_WIDEN_ASCII_TO_UTF16, + + // TODO: Make this wasm only + MINT_TIER_PREPARE_JITERPRETER, + MINT_TIER_NOP_JITERPRETER, + MINT_TIER_ENTER_JITERPRETER, + + MINT_LASTOP +} + +export const enum MintOpArgType { + MintOpNoArgs = 0, + MintOpShortInt, + MintOpUShortInt, + MintOpInt, + MintOpLongInt, + MintOpFloat, + MintOpDouble, + MintOpBranch, + MintOpShortBranch, + MintOpSwitch, + MintOpMethodToken, + MintOpFieldToken, + MintOpClassToken, + MintOpTwoShorts, + MintOpTwoInts, + MintOpShortAndInt, + MintOpShortAndShortBranch, + MintOpPair2, + MintOpPair3, + MintOpPair4 +} + +type OpcodeInfoTable = { + [key: number]: [name: string, length_u16: number, dregs: number, sregs: number, optype: MintOpArgType]; +} + +export const OpcodeInfo : OpcodeInfoTable = { + [MintOpcode.MINT_NOP]: [ "nop", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_NIY]: [ "niy", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_DEF]: [ "def", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_IL_SEQ_POINT]: [ "il_seq_point", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_DUMMY_USE]: [ "dummy_use", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_TIER_PATCHPOINT_DATA]: [ "tier_patchpoint_data", 2, 0, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_BREAK]: [ "break", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_BREAKPOINT]: [ "breakpoint", 1, 0, 0, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_RET]: [ "ret", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RET_VOID]: [ "ret.void", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RET_VT]: [ "ret.vt", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_RET_LOCALLOC]: [ "ret.localloc", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RET_VOID_LOCALLOC]: [ "ret.void.localloc", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RET_VT_LOCALLOC]: [ "ret.vt.localloc", 3, 0, 1, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_RET_I1]: [ "ret.i1", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RET_U1]: [ "ret.u1", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RET_I2]: [ "ret.i2", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RET_U2]: [ "ret.u2", 2, 0, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_LDC_I4_M1]: [ "ldc.i4.m1", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_0]: [ "ldc.i4.0", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_1]: [ "ldc.i4.1", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_2]: [ "ldc.i4.2", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_3]: [ "ldc.i4.3", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_4]: [ "ldc.i4.4", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_5]: [ "ldc.i4.5", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_6]: [ "ldc.i4.6", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_7]: [ "ldc.i4.7", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_8]: [ "ldc.i4.8", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I4_S]: [ "ldc.i4.s", 3, 1, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDC_I4]: [ "ldc.i4", 4, 1, 0, MintOpArgType.MintOpInt], + + [MintOpcode.MINT_LDC_I8_0]: [ "ldc.i8.0", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDC_I8_S]: [ "ldc.i8.s", 3, 1, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDC_I8]: [ "ldc.i8", 6, 1, 0, MintOpArgType.MintOpLongInt], + + [MintOpcode.MINT_LDC_R4]: [ "ldc.r4", 4, 1, 0, MintOpArgType.MintOpFloat], + [MintOpcode.MINT_LDC_R8]: [ "ldc.r8", 6, 1, 0, MintOpArgType.MintOpDouble], + + [MintOpcode.MINT_INIT_ARGLIST]: [ "init_arglist", 3, 1, 0, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_LDFLD_I1]: [ "ldfld.i1", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_U1]: [ "ldfld.u1", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_I2]: [ "ldfld.i2", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_U2]: [ "ldfld.u2", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_I4]: [ "ldfld.i4", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_I8]: [ "ldfld.i8", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_R4]: [ "ldfld.r4", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_R8]: [ "ldfld.r8", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_O]: [ "ldfld.o", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_VT]: [ "ldfld.vt", 5, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDFLD_I8_UNALIGNED]: [ "ldfld.i8.unaligned", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLD_R8_UNALIGNED]: [ "ldfld.r8.unaligned", 4, 1, 1, MintOpArgType.MintOpUShortInt], + + [MintOpcode.MINT_LDFLDA]: [ "ldflda", 4, 1, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDFLDA_UNSAFE]: [ "ldflda.unsafe", 4, 1, 1, MintOpArgType.MintOpUShortInt], + + [MintOpcode.MINT_STFLD_I1]: [ "stfld.i1", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_U1]: [ "stfld.u1", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_I2]: [ "stfld.i2", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_U2]: [ "stfld.u2", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_I4]: [ "stfld.i4", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_I8]: [ "stfld.i8", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_R4]: [ "stfld.r4", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_R8]: [ "stfld.r8", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_O]: [ "stfld.o", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_VT]: [ "stfld.vt", 5, 0, 2, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_STFLD_VT_NOREF]: [ "stfld.vt.noref", 5, 0, 2, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_STFLD_I8_UNALIGNED]: [ "stfld.i8.unaligned", 4, 0, 2, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STFLD_R8_UNALIGNED]: [ "stfld.r8.unaligned", 4, 0, 2, MintOpArgType.MintOpUShortInt], + + [MintOpcode.MINT_LDSFLD_I1]: [ "ldsfld.i1", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_U1]: [ "ldsfld.u1", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_I2]: [ "ldsfld.i2", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_U2]: [ "ldsfld.u2", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_I4]: [ "ldsfld.i4", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_I8]: [ "ldsfld.i8", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_R4]: [ "ldsfld.r4", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_R8]: [ "ldsfld.r8", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_O]: [ "ldsfld.o", 4, 1, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_LDSFLD_VT]: [ "ldsfld.vt", 5, 1, 0, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_LDSFLD_W]: [ "ldsfld.w", 8, 1, 0, MintOpArgType.MintOpTwoInts], + + [MintOpcode.MINT_STSFLD_I1]: [ "stsfld.i1", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_U1]: [ "stsfld.u1", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_I2]: [ "stsfld.i2", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_U2]: [ "stsfld.u2", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_I4]: [ "stsfld.i4", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_I8]: [ "stsfld.i8", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_R4]: [ "stsfld.r4", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_R8]: [ "stsfld.r8", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_O]: [ "stsfld.o", 4, 0, 1, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_STSFLD_VT]: [ "stsfld.vt", 5, 0, 1, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_STSFLD_W]: [ "stsfld.w", 8, 0, 1, MintOpArgType.MintOpTwoInts], + [MintOpcode.MINT_LDSFLDA]: [ "ldsflda", 4, 1, 0, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_LDTSFLDA]: [ "ldtsflda", 4, 1, 0, MintOpArgType.MintOpInt], + + [MintOpcode.MINT_MOV_SRC_OFF]: [ "mov.src.off", 6, 1, 1, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_MOV_DST_OFF]: [ "mov.dst.off", 6, 1, 1, MintOpArgType.MintOpTwoShorts], + + [MintOpcode.MINT_MOV_I4_I1]: [ "mov.i4.i1", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_I4_U1]: [ "mov.i4.u1", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_I4_I2]: [ "mov.i4.i2", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_I4_U2]: [ "mov.i4.u2", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_1]: [ "mov.1", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_2]: [ "mov.2", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_4]: [ "mov.4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_8]: [ "mov.8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MOV_VT]: [ "mov.vt", 4, 1, 1, MintOpArgType.MintOpShortInt], + + // These opcodes represent multiple moves stacked together. They have multiple src and dst + // but they are not represented here. They are generated by the var offset allocator. + [MintOpcode.MINT_MOV_8_2]: [ "mov.8.2", 5, 0, 0, MintOpArgType.MintOpPair2], + [MintOpcode.MINT_MOV_8_3]: [ "mov.8.3", 7, 0, 0, MintOpArgType.MintOpPair3], + [MintOpcode.MINT_MOV_8_4]: [ "mov.8.4", 9, 0, 0, MintOpArgType.MintOpPair4], + + [MintOpcode.MINT_LDLOCA_S]: [ "ldloca.s", 3, 1, 0, MintOpArgType.MintOpUShortInt], + + [MintOpcode.MINT_LDIND_I1]: [ "ldind.i1", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_U1]: [ "ldind.u1", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_I2]: [ "ldind.i2", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_U2]: [ "ldind.u2", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_I4]: [ "ldind.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_I8]: [ "ldind.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_R4]: [ "ldind.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_R8]: [ "ldind.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_LDIND_OFFSET_I1]: [ "ldind_off.i1", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_OFFSET_U1]: [ "ldind_off.u1", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_OFFSET_I2]: [ "ldind_off.i2", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_OFFSET_U2]: [ "ldind_off.u2", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_OFFSET_I4]: [ "ldind_off.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDIND_OFFSET_I8]: [ "ldind_off.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_LDIND_OFFSET_IMM_I1]: [ "ldind_off_imm.i1", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDIND_OFFSET_IMM_U1]: [ "ldind_off_imm.u1", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDIND_OFFSET_IMM_I2]: [ "ldind_off_imm.i2", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDIND_OFFSET_IMM_U2]: [ "ldind_off_imm.u2", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDIND_OFFSET_IMM_I4]: [ "ldind_off_imm.i4", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDIND_OFFSET_IMM_I8]: [ "ldind_off_imm.i8", 4, 1, 1, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_STIND_I1]: [ "stind.i1", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_I2]: [ "stind.i2", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_I4]: [ "stind.i4", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_I8]: [ "stind.i8", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_R4]: [ "stind.r4", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_R8]: [ "stind.r8", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_REF]: [ "stind.ref", 3, 0, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_STIND_OFFSET_I1]: [ "stind_off.i1", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_OFFSET_I2]: [ "stind_off.i2", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_OFFSET_I4]: [ "stind_off.i4", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STIND_OFFSET_I8]: [ "stind_off.i8", 4, 0, 3, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_STIND_OFFSET_IMM_I1]: [ "stind_off_imm.i1", 4, 0, 2, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_STIND_OFFSET_IMM_I2]: [ "stind_off_imm.i2", 4, 0, 2, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_STIND_OFFSET_IMM_I4]: [ "stind_off_imm.i4", 4, 0, 2, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_STIND_OFFSET_IMM_I8]: [ "stind_off_imm.i8", 4, 0, 2, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_BR]: [ "br", 3, 0, 0, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_LEAVE]: [ "leave", 3, 0, 0, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_LEAVE_CHECK]: [ "leave.check", 3, 0, 0, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BR_S]: [ "br.s", 2, 0, 0, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_LEAVE_S]: [ "leave.s", 2, 0, 0, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_LEAVE_S_CHECK]: [ "leave.s.check", 2, 0, 0, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_CALL_HANDLER]: [ "call_handler", 4, 0, 0, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_CALL_HANDLER_S]: [ "call_handler.s", 3, 0, 0, MintOpArgType.MintOpShortBranch], + + [MintOpcode.MINT_THROW]: [ "throw", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_RETHROW]: [ "rethrow", 2, 0, 0, MintOpArgType.MintOpUShortInt], + [MintOpcode.MINT_ENDFINALLY]: [ "endfinally", 2, 0, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_MONO_RETHROW]: [ "mono_rethrow", 2, 0, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_SAFEPOINT]: [ "safepoint", 1, 0, 0, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_BRFALSE_I4]: [ "brfalse.i4", 4, 0, 1, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BRFALSE_I8]: [ "brfalse.i8", 4, 0, 1, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BRFALSE_R4]: [ "brfalse.r4", 4, 0, 1, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BRFALSE_R8]: [ "brfalse.r8", 4, 0, 1, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BRTRUE_I4]: [ "brtrue.i4", 4, 0, 1, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BRTRUE_I8]: [ "brtrue.i8", 4, 0, 1, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BRTRUE_R4]: [ "brtrue.r4", 4, 0, 1, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BRTRUE_R8]: [ "brtrue.r8", 4, 0, 1, MintOpArgType.MintOpBranch], + + [MintOpcode.MINT_BRFALSE_I4_S]: [ "brfalse.i4.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRFALSE_I8_S]: [ "brfalse.i8.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRFALSE_R4_S]: [ "brfalse.r4.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRFALSE_R8_S]: [ "brfalse.r8.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRTRUE_I4_S]: [ "brtrue.i4.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRTRUE_I8_S]: [ "brtrue.i8.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRTRUE_R4_S]: [ "brtrue.r4.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRTRUE_R8_S]: [ "brtrue.r8.s", 3, 0, 1, MintOpArgType.MintOpShortBranch], + + [MintOpcode.MINT_BEQ_I4]: [ "beq.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BEQ_I8]: [ "beq.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BEQ_R4]: [ "beq.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BEQ_R8]: [ "beq.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_I4]: [ "bge.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_I8]: [ "bge.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_R4]: [ "bge.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_R8]: [ "bge.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_I4]: [ "bgt.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_I8]: [ "bgt.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_R4]: [ "bgt.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_R8]: [ "bgt.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_I4]: [ "blt.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_I8]: [ "blt.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_R4]: [ "blt.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_R8]: [ "blt.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_I4]: [ "ble.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_I8]: [ "ble.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_R4]: [ "ble.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_R8]: [ "ble.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + + [MintOpcode.MINT_BNE_UN_I4]: [ "bne.un.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BNE_UN_I8]: [ "bne.un.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BNE_UN_R4]: [ "bne.un.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BNE_UN_R8]: [ "bne.un.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_UN_I4]: [ "bge.un.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_UN_I8]: [ "bge.un.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_UN_R4]: [ "bge.un.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGE_UN_R8]: [ "bge.un.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_UN_I4]: [ "bgt.un.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_UN_I8]: [ "bgt.un.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_UN_R4]: [ "bgt.un.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BGT_UN_R8]: [ "bgt.un.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_UN_I4]: [ "ble.un.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_UN_I8]: [ "ble.un.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_UN_R4]: [ "ble.un.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLE_UN_R8]: [ "ble.un.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_UN_I4]: [ "blt.un.i4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_UN_I8]: [ "blt.un.i8", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_UN_R4]: [ "blt.un.r4", 5, 0, 2, MintOpArgType.MintOpBranch], + [MintOpcode.MINT_BLT_UN_R8]: [ "blt.un.r8", 5, 0, 2, MintOpArgType.MintOpBranch], + + [MintOpcode.MINT_BEQ_I4_S]: [ "beq.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BEQ_I8_S]: [ "beq.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BEQ_R4_S]: [ "beq.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BEQ_R8_S]: [ "beq.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_I4_S]: [ "bge.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_I8_S]: [ "bge.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_R4_S]: [ "bge.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_R8_S]: [ "bge.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_I4_S]: [ "bgt.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_I8_S]: [ "bgt.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_R4_S]: [ "bgt.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_R8_S]: [ "bgt.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_I4_S]: [ "blt.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_I8_S]: [ "blt.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_R4_S]: [ "blt.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_R8_S]: [ "blt.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_I4_S]: [ "ble.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_I8_S]: [ "ble.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_R4_S]: [ "ble.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_R8_S]: [ "ble.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + + [MintOpcode.MINT_BNE_UN_I4_S]: [ "bne.un.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BNE_UN_I8_S]: [ "bne.un.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BNE_UN_R4_S]: [ "bne.un.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BNE_UN_R8_S]: [ "bne.un.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_UN_I4_S]: [ "bge.un.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_UN_I8_S]: [ "bge.un.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_UN_R4_S]: [ "bge.un.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_UN_R8_S]: [ "bge.un.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_UN_I4_S]: [ "bgt.un.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_UN_I8_S]: [ "bgt.un.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_UN_R4_S]: [ "bgt.un.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_UN_R8_S]: [ "bgt.un.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_UN_I4_S]: [ "ble.un.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_UN_I8_S]: [ "ble.un.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_UN_R4_S]: [ "ble.un.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_UN_R8_S]: [ "ble.un.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_UN_I4_S]: [ "blt.un.i4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_UN_I8_S]: [ "blt.un.i8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_UN_R4_S]: [ "blt.un.r4.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_UN_R8_S]: [ "blt.un.r8.s", 4, 0, 2, MintOpArgType.MintOpShortBranch], + + [MintOpcode.MINT_BRFALSE_I4_SP]: [ "brfalse.i4.sp", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRFALSE_I8_SP]: [ "brfalse.i8.sp", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRTRUE_I4_SP]: [ "brtrue.i4.sp", 3, 0, 1, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BRTRUE_I8_SP]: [ "brtrue.i8.sp", 3, 0, 1, MintOpArgType.MintOpShortBranch], + + [MintOpcode.MINT_BEQ_I4_SP]: [ "beq.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BEQ_I8_SP]: [ "beq.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_I4_SP]: [ "bge.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_I8_SP]: [ "bge.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_I4_SP]: [ "bgt.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_I8_SP]: [ "bgt.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_I4_SP]: [ "blt.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_I8_SP]: [ "blt.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_I4_SP]: [ "ble.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_I8_SP]: [ "ble.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + + [MintOpcode.MINT_BNE_UN_I4_SP]: [ "bne.un.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BNE_UN_I8_SP]: [ "bne.un.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_UN_I4_SP]: [ "bge.un.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGE_UN_I8_SP]: [ "bge.un.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_UN_I4_SP]: [ "bgt.un.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BGT_UN_I8_SP]: [ "bgt.un.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_UN_I4_SP]: [ "ble.un.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLE_UN_I8_SP]: [ "ble.un.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_UN_I4_SP]: [ "blt.un.i4.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + [MintOpcode.MINT_BLT_UN_I8_SP]: [ "blt.un.i8.sp", 4, 0, 2, MintOpArgType.MintOpShortBranch], + + [MintOpcode.MINT_BEQ_I4_IMM_SP]: [ "beq.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BEQ_I8_IMM_SP]: [ "beq.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGE_I4_IMM_SP]: [ "bge.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGE_I8_IMM_SP]: [ "bge.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGT_I4_IMM_SP]: [ "bgt.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGT_I8_IMM_SP]: [ "bgt.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLT_I4_IMM_SP]: [ "blt.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLT_I8_IMM_SP]: [ "blt.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLE_I4_IMM_SP]: [ "ble.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLE_I8_IMM_SP]: [ "ble.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + + [MintOpcode.MINT_BNE_UN_I4_IMM_SP]: [ "bne.un.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BNE_UN_I8_IMM_SP]: [ "bne.un.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGE_UN_I4_IMM_SP]: [ "bge.un.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGE_UN_I8_IMM_SP]: [ "bge.un.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGT_UN_I4_IMM_SP]: [ "bgt.un.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BGT_UN_I8_IMM_SP]: [ "bgt.un.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLE_UN_I4_IMM_SP]: [ "ble.un.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLE_UN_I8_IMM_SP]: [ "ble.un.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLT_UN_I4_IMM_SP]: [ "blt.un.i4.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + [MintOpcode.MINT_BLT_UN_I8_IMM_SP]: [ "blt.un.i8.imm.sp", 4, 0, 1, MintOpArgType.MintOpShortAndShortBranch], + + + [MintOpcode.MINT_SWITCH]: [ "switch", 0, 0, 1, MintOpArgType.MintOpSwitch], + + [MintOpcode.MINT_LDSTR]: [ "ldstr", 3, 1, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDSTR_TOKEN]: [ "ldstr.token", 3, 1, 0, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_JMP]: [ "jmp", 2, 0, 0, MintOpArgType.MintOpMethodToken], + + [MintOpcode.MINT_ENDFILTER]: [ "endfilter", 2, 0, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_NEWOBJ_SLOW_UNOPT]: [ "newobj_slow_unopt", 5, 1, 0, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ_STRING_UNOPT]: [ "newobj_string_unopt", 4, 1, 0, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ_SLOW]: [ "newobj_slow", 4, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ_ARRAY]: [ "newobj_array", 5, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ_STRING]: [ "newobj_string", 4, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ]: [ "newobj", 5, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ_INLINED]: [ "newobj_inlined", 3, 1, 0, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ_VT]: [ "newobj_vt", 5, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_NEWOBJ_VT_INLINED]: [ "newobj_vt_inlined", 4, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_INITOBJ]: [ "initobj", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_CASTCLASS]: [ "castclass", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_ISINST]: [ "isinst", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_CASTCLASS_INTERFACE]: [ "castclass.interface", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_ISINST_INTERFACE]: [ "isinst.interface", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_CASTCLASS_COMMON]: [ "castclass.common", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_ISINST_COMMON]: [ "isinst.common", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_NEWARR]: [ "newarr", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_BOX]: [ "box", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_BOX_VT]: [ "box.vt", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_BOX_PTR]: [ "box.ptr", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_BOX_NULLABLE_PTR]: [ "box.nullable.ptr", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_UNBOX]: [ "unbox", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_LDTOKEN]: [ "ldtoken", 3, 1, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDFTN]: [ "ldftn", 3, 1, 0, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_LDFTN_ADDR]: [ "ldftn_addr", 3, 1, 0, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_LDFTN_DYNAMIC]: [ "ldftn.dynamic", 3, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_LDVIRTFTN]: [ "ldvirtftn", 4, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CPOBJ]: [ "cpobj", 4, 0, 2, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_CPOBJ_VT]: [ "cpobj.vt", 4, 0, 2, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_LDOBJ_VT]: [ "ldobj.vt", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_STOBJ_VT]: [ "stobj.vt", 4, 0, 2, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_CPBLK]: [ "cpblk", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INITBLK]: [ "initblk", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LOCALLOC]: [ "localloc", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INITLOCAL]: [ "initlocal", 3, 1, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_INITLOCALS]: [ "initlocals", 3, 0, 0, MintOpArgType.MintOpTwoShorts], + + [MintOpcode.MINT_LDELEM_I]: [ "ldelem.i", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_I1]: [ "ldelem.i1", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_U1]: [ "ldelem.u1", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_I2]: [ "ldelem.i2", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_U2]: [ "ldelem.u2", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_I4]: [ "ldelem.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_U4]: [ "ldelem.u4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_I8]: [ "ldelem.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_R4]: [ "ldelem.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_R8]: [ "ldelem.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_REF]: [ "ldelem.ref", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LDELEM_VT]: [ "ldelem.vt", 5, 1, 2, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_LDELEMA1]: [ "ldelema1", 5, 1, 2, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_LDELEMA]: [ "ldelema", 5, 1, 1, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_LDELEMA_TC]: [ "ldelema.tc", 4, 1, 1, MintOpArgType.MintOpTwoShorts], + + [MintOpcode.MINT_STELEM_I]: [ "stelem.i", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_I1]: [ "stelem.i1", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_U1]: [ "stelem.u1", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_I2]: [ "stelem.i2", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_U2]: [ "stelem.u2", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_I4]: [ "stelem.i4", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_I8]: [ "stelem.i8", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_R4]: [ "stelem.r4", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_R8]: [ "stelem.r8", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_REF]: [ "stelem.ref", 4, 0, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STELEM_VT]: [ "stelem.vt", 6, 0, 3, MintOpArgType.MintOpTwoShorts], + + [MintOpcode.MINT_LDLEN]: [ "ldlen", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_GETITEM_SPAN]: [ "getitem.span", 5, 1, 2, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_GETITEM_LOCALSPAN]: [ "getitem.localspan", 5, 1, 2, MintOpArgType.MintOpTwoShorts], + + /* binops */ + [MintOpcode.MINT_ADD_I4]: [ "add.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ADD_I8]: [ "add.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ADD_R4]: [ "add.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ADD_R8]: [ "add.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_SUB_I4]: [ "sub.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SUB_I8]: [ "sub.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SUB_R4]: [ "sub.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SUB_R8]: [ "sub.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_MUL_I4]: [ "mul.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MUL_I8]: [ "mul.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MUL_R4]: [ "mul.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MUL_R8]: [ "mul.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_DIV_I4]: [ "div.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_DIV_I8]: [ "div.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_DIV_R4]: [ "div.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_DIV_R8]: [ "div.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_DIV_UN_I4]: [ "div.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_DIV_UN_I8]: [ "div.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_ADD_OVF_I4]: [ "add.ovf.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ADD_OVF_I8]: [ "add.ovf.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_ADD_OVF_UN_I4]: [ "add.ovf.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ADD_OVF_UN_I8]: [ "add.ovf.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_MUL_OVF_I4]: [ "mul.ovf.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MUL_OVF_I8]: [ "mul.ovf.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_MUL_OVF_UN_I4]: [ "mul.ovf.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MUL_OVF_UN_I8]: [ "mul.ovf.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_SUB_OVF_I4]: [ "sub.ovf.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SUB_OVF_I8]: [ "sub.ovf.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_SUB_OVF_UN_I4]: [ "sub.ovf.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SUB_OVF_UN_I8]: [ "sub.ovf.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_AND_I4]: [ "and.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_AND_I8]: [ "and.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_OR_I4]: [ "or.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_OR_I8]: [ "or.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_XOR_I4]: [ "xor.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_XOR_I8]: [ "xor.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_REM_I4]: [ "rem.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_REM_I8]: [ "rem.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_REM_R4]: [ "rem.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_REM_R8]: [ "rem.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_REM_UN_I4]: [ "rem.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_REM_UN_I8]: [ "rem.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + // Shifts, keep in order with imm versions + [MintOpcode.MINT_SHR_UN_I4]: [ "shr.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SHR_UN_I8]: [ "shr.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SHL_I4]: [ "shl.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SHL_I8]: [ "shl.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SHR_I4]: [ "shr.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SHR_I8]: [ "shr.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CEQ_I4]: [ "ceq.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CEQ_I8]: [ "ceq.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CEQ_R4]: [ "ceq.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CEQ_R8]: [ "ceq.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CNE_I4]: [ "cne.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CNE_I8]: [ "cne.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CNE_R4]: [ "cne.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CNE_R8]: [ "cne.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CGT_I4]: [ "cgt.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGT_I8]: [ "cgt.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGT_R4]: [ "cgt.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGT_R8]: [ "cgt.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CGE_I4]: [ "cge.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGE_I8]: [ "cge.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGE_R4]: [ "cge.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGE_R8]: [ "cge.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CGE_UN_I4]: [ "cge.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGE_UN_I8]: [ "cge.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CGT_UN_I4]: [ "cgt.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGT_UN_I8]: [ "cgt.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGT_UN_R4]: [ "cgt.un.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CGT_UN_R8]: [ "cgt.un.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CLT_I4]: [ "clt.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLT_I8]: [ "clt.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLT_R4]: [ "clt.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLT_R8]: [ "clt.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CLE_I4]: [ "cle.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLE_I8]: [ "cle.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLE_R4]: [ "cle.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLE_R8]: [ "cle.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CLE_UN_I4]: [ "cle.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLE_UN_I8]: [ "cle.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CLT_UN_I4]: [ "clt.un.i4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLT_UN_I8]: [ "clt.un.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLT_UN_R4]: [ "clt.un.r4", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CLT_UN_R8]: [ "clt.un.r8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + /* binops end */ + + /* unops */ + [MintOpcode.MINT_ADD1_I4]: [ "add1.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ADD1_I8]: [ "add1.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SUB1_I4]: [ "sub1.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SUB1_I8]: [ "sub1.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_NEG_I4]: [ "neg.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_NEG_I8]: [ "neg.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_NEG_R4]: [ "neg.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_NEG_R8]: [ "neg.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_NOT_I4]: [ "not.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_NOT_I8]: [ "not.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_R_UN_I4]: [ "conv.r.un.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_R_UN_I8]: [ "conv.r.un.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_I1_I4]: [ "conv.i1.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I1_I8]: [ "conv.i1.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I1_R4]: [ "conv.i1.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I1_R8]: [ "conv.i1.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_U1_I4]: [ "conv.u1.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U1_I8]: [ "conv.u1.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U1_R4]: [ "conv.u1.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U1_R8]: [ "conv.u1.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_I2_I4]: [ "conv.i2.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I2_I8]: [ "conv.i2.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I2_R4]: [ "conv.i2.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I2_R8]: [ "conv.i2.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_U2_I4]: [ "conv.u2.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U2_I8]: [ "conv.u2.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U2_R4]: [ "conv.u2.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U2_R8]: [ "conv.u2.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_I4_R4]: [ "conv.i4.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I4_R8]: [ "conv.i4.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_U4_R4]: [ "conv.u4.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U4_R8]: [ "conv.u4.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_I8_I4]: [ "conv.i8.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I8_U4]: [ "conv.i8.u4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I8_R4]: [ "conv.i8.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_I8_R8]: [ "conv.i8.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_R4_I4]: [ "conv.r4.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_R4_I8]: [ "conv.r4.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_R4_R8]: [ "conv.r4.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_R8_I4]: [ "conv.r8.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_R8_I8]: [ "conv.r8.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_R8_R4]: [ "conv.r8.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_U8_R4]: [ "conv.u8.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_U8_R8]: [ "conv.u8.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_I1_I4]: [ "conv.ovf.i1.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I1_I8]: [ "conv.ovf.i1.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I1_R4]: [ "conv.ovf.i1.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I1_R8]: [ "conv.ovf.i1.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_I1_U4]: [ "conv.ovf.i1.u4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I1_U8]: [ "conv.ovf.i1.u8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_U1_I4]: [ "conv.ovf.u1.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U1_I8]: [ "conv.ovf.u1.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U1_R4]: [ "conv.ovf.u1.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U1_R8]: [ "conv.ovf.u1.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_I2_I4]: [ "conv.ovf.i2.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I2_I8]: [ "conv.ovf.i2.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I2_R4]: [ "conv.ovf.i2.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I2_R8]: [ "conv.ovf.i2.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_I2_U4]: [ "conv.ovf.i2.u4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I2_U8]: [ "conv.ovf.i2.u8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_U2_I4]: [ "conv.ovf.u2.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U2_I8]: [ "conv.ovf.u2.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U2_R4]: [ "conv.ovf.u2.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U2_R8]: [ "conv.ovf.u2.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_I4_U4]: [ "conv.ovf.i4.u4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I4_I8]: [ "conv.ovf.i4.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I4_U8]: [ "conv.ovf.i4.u8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I4_R4]: [ "conv.ovf.i4.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I4_R8]: [ "conv.ovf.i4.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_U4_I4]: [ "conv.ovf.u4.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U4_I8]: [ "conv.ovf.u4.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U4_R4]: [ "conv.ovf.u4.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U4_R8]: [ "conv.ovf.u4.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_I8_U8]: [ "conv.ovf.i8.u8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I8_R4]: [ "conv.ovf.i8.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_I8_R8]: [ "conv.ovf.i8.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CONV_OVF_U8_I4]: [ "conv.ovf.u8.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U8_I8]: [ "conv.ovf.u8.i8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U8_R4]: [ "conv.ovf.u8.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CONV_OVF_U8_R8]: [ "conv.ovf.u8.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CEQ0_I4]: [ "ceq0.i4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + /* unops end */ + + /* super instructions */ + [MintOpcode.MINT_RET_I4_IMM]: [ "ret.i4.imm", 2, 0, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_RET_I8_IMM]: [ "ret.i8.imm", 2, 0, 0, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_ADD_I4_IMM]: [ "add.i4.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ADD_I8_IMM]: [ "add.i8.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_MUL_I4_IMM]: [ "mul.i4.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_MUL_I8_IMM]: [ "mul.i8.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_SHR_UN_I4_IMM]: [ "shr.un.i4.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_SHR_UN_I8_IMM]: [ "shr.un.i8.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_SHL_I4_IMM]: [ "shl.i4.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_SHL_I8_IMM]: [ "shl.i8.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_SHR_I4_IMM]: [ "shr.i4.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_SHR_I8_IMM]: [ "shr.i8.imm", 4, 1, 1, MintOpArgType.MintOpShortInt], + + + [MintOpcode.MINT_CKFINITE_R4]: [ "ckfinite.r4", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CKFINITE_R8]: [ "ckfinite.r8", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MKREFANY]: [ "mkrefany", 4, 1, 1, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_REFANYTYPE]: [ "refanytype", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_REFANYVAL]: [ "refanyval", 4, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_CKNULL]: [ "cknull", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_GETCHR]: [ "getchr", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_STRLEN]: [ "strlen", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ARRAY_RANK]: [ "array_rank", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ARRAY_ELEMENT_SIZE]: [ "array_element_size", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ARRAY_IS_PRIMITIVE]: [ "array_is_primitive", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + /* Calls */ + [MintOpcode.MINT_CALL]: [ "call", 4, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CALLVIRT]: [ "callvirt", 4, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CALLVIRT_FAST]: [ "callvirt.fast", 5, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CALL_DELEGATE]: [ "call.delegate", 5, 1, 1, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_CALLI]: [ "calli", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CALLI_NAT]: [ "calli.nat", 8, 1, 2, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CALLI_NAT_DYNAMIC]: [ "calli.nat.dynamic", 5, 1, 2, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CALLI_NAT_FAST]: [ "calli.nat.fast", 7, 1, 2, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CALL_VARARG]: [ "call.vararg", 6, 1, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_CALLRUN]: [ "callrun", 5, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_TAILCALL]: [ "tailcall", 4, 0, 1, MintOpArgType.MintOpMethodToken], + [MintOpcode.MINT_TAILCALL_VIRT]: [ "tailcall.virt", 5, 0, 1, MintOpArgType.MintOpMethodToken], + + [MintOpcode.MINT_ICALL_V_V]: [ "mono_icall_v_v", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_V_P]: [ "mono_icall_v_p", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_P_V]: [ "mono_icall_p_v", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_P_P]: [ "mono_icall_p_p", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PP_V]: [ "mono_icall_pp_v", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PP_P]: [ "mono_icall_pp_p", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPP_V]: [ "mono_icall_ppp_v", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPP_P]: [ "mono_icall_ppp_p", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPPP_V]: [ "mono_icall_pppp_v", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPPP_P]: [ "mono_icall_pppp_p", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPPPP_V]: [ "mono_icall_ppppp_v", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPPPP_P]: [ "mono_icall_ppppp_p", 4, 1, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPPPPP_V]: [ "mono_icall_pppppp_v", 3, 0, 1, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_ICALL_PPPPPP_P]: [ "mono_icall_pppppp_p", 4, 1, 1, MintOpArgType.MintOpShortInt], + // FIXME: MintOp + [MintOpcode.MINT_JIT_CALL]: [ "mono_jit_call", 4, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_JIT_CALL2]: [ "mono_jit_call2", 7, 1, 1, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_MONO_LDPTR]: [ "mono_ldptr", 3, 1, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_MONO_SGEN_THREAD_INFO]: [ "mono_sgen_thread_info", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MONO_NEWOBJ]: [ "mono_newobj", 3, 1, 0, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_MONO_RETOBJ]: [ "mono_retobj", 2, 0, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MONO_ATOMIC_STORE_I4]: [ "mono_atomic.store.i4", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MONO_MEMORY_BARRIER]: [ "mono_memory_barrier", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MONO_EXCHANGE_I8]: [ "mono_interlocked.xchg.i8", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MONO_LDDOMAIN]: [ "mono_lddomain", 2, 1, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MONO_ENABLE_GCTRANS]: [ "mono_enable_gctrans", 1, 0, 0, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_SDB_INTR_LOC]: [ "sdb_intr_loc", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SDB_SEQ_POINT]: [ "sdb_seq_point", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SDB_BREAKPOINT]: [ "sdb_breakpoint", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LD_DELEGATE_METHOD_PTR]: [ "ld_delegate_method_ptr", 3, 1, 1, MintOpArgType.MintOpNoArgs], + + // Math intrinsics + // double + [MintOpcode.MINT_ASIN]: [ "asin", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ASINH]: [ "asinh", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ACOS]: [ "acos", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ACOSH]: [ "acosh", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ATAN]: [ "atan", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ATANH]: [ "atanh", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ATAN2]: [ "atan2", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CEILING]: [ "ceiling", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_COS]: [ "cos", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CBRT]: [ "cbrt", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_COSH]: [ "cosh", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_EXP]: [ "exp", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_FMA]: [ "fma", 5, 1, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_FLOOR]: [ "floor", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LOG]: [ "log", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LOG2]: [ "log2", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LOG10]: [ "log10", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_POW]: [ "pow", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SCALEB]: [ "scaleb", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SIN]: [ "sin", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SQRT]: [ "sqrt", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SINH]: [ "sinh", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_TAN]: [ "tan", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_TANH]: [ "tanh", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ABS]: [ "abs_d", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MIN]: [ "min_d", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MAX]: [ "max_d", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + // float. These must be kept in the same order as their double counterpart + [MintOpcode.MINT_ASINF]: [ "asinf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ASINHF]: [ "asinhf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ACOSF]: [ "acosf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ACOSHF]: [ "acoshf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ATANF]: [ "atanf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ATANHF]: [ "atanhf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ATAN2F]: [ "atan2f", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CEILINGF]: [ "ceilingf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_COSF]: [ "cosf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_CBRTF]: [ "cbrtf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_COSHF]: [ "coshf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_EXPF]: [ "expf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_FMAF]: [ "fmaf", 5, 1, 3, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_FLOORF]: [ "floorf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LOGF]: [ "logf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LOG2F]: [ "log2f", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_LOG10F]: [ "log10f", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_POWF]: [ "powf", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SCALEBF]: [ "scalebf", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SINF]: [ "sinf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SQRTF]: [ "sqrtf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_SINHF]: [ "sinhf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_TANF]: [ "tanf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_TANHF]: [ "tanhf", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_ABSF]: [ "abs_f", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MINF]: [ "min_f", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_MAXF]: [ "max_f", 4, 1, 2, MintOpArgType.MintOpNoArgs], + + [MintOpcode.MINT_PROF_ENTER]: [ "prof_enter", 2, 0, 0, MintOpArgType.MintOpShortInt], + [MintOpcode.MINT_PROF_EXIT]: [ "prof_exit", 5, 0, 1, MintOpArgType.MintOpShortAndInt], + [MintOpcode.MINT_PROF_EXIT_VOID]: [ "prof_exit_void", 2, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_PROF_COVERAGE_STORE]: [ "prof_coverage_store", 5, 0, 0, MintOpArgType.MintOpLongInt], + + [MintOpcode.MINT_TIER_ENTER_METHOD]: [ "tier_enter_method", 1, 0, 0, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_TIER_PATCHPOINT]: [ "tier_patchpoint", 2, 0, 0, MintOpArgType.MintOpShortInt], + + [MintOpcode.MINT_INTRINS_ENUM_HASFLAG]: [ "intrins_enum_hasflag", 5, 1, 2, MintOpArgType.MintOpClassToken], + [MintOpcode.MINT_INTRINS_GET_HASHCODE]: [ "intrins_get_hashcode", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_GET_TYPE]: [ "intrins_get_type", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_SPAN_CTOR]: [ "intrins_span_ctor", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_UNSAFE_BYTE_OFFSET]: [ "intrins_unsafe_byte_offset", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_RUNTIMEHELPERS_OBJECT_HAS_COMPONENT_SIZE]: [ "intrins_runtimehelpers_object_has_component_size", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_CLEAR_WITH_REFERENCES]: [ "intrin_clear_with_references", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_MARVIN_BLOCK]: [ "intrins_marvin_block", 3, 0, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_ASCII_CHARS_TO_UPPERCASE]: [ "intrins_ascii_chars_to_uppercase", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_MEMORYMARSHAL_GETARRAYDATAREF]: [ "intrins_memorymarshal_getarraydataref", 3, 1, 1, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_ORDINAL_IGNORE_CASE_ASCII]: [ "intrins_ordinal_ignore_case_ascii", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_64ORDINAL_IGNORE_CASE_ASCII]: [ "intrins_64ordinal_ignore_case_ascii", 4, 1, 2, MintOpArgType.MintOpNoArgs], + [MintOpcode.MINT_INTRINS_U32_TO_DECSTR]: [ "intrins_u32_to_decstr", 5, 1, 1, MintOpArgType.MintOpTwoShorts], + [MintOpcode.MINT_INTRINS_WIDEN_ASCII_TO_UTF16]: [ "intrins_widen_ascii_to_utf16", 5, 1, 3, MintOpArgType.MintOpNoArgs], + + // TODO: Make this wasm only + [MintOpcode.MINT_TIER_PREPARE_JITERPRETER]: [ "tier_prepare_jiterpreter", 3, 0, 0, MintOpArgType.MintOpInt], + [MintOpcode.MINT_TIER_NOP_JITERPRETER]: [ "tier_nop_jiterpreter", 3, 0, 0, MintOpArgType.MintOpInt], + [MintOpcode.MINT_TIER_ENTER_JITERPRETER]: [ "tier_enter_jiterpreter", 3, 0, 0, MintOpArgType.MintOpInt], +}; + +// Keep this in sync with the wasm spec (but I don't think any changes will impact it), +// Note that prefix opcodes aren't in this enum, since making them write properly is awkward. + +export const enum WasmOpcode { + unreachable = 0x00, + nop, + block, + loop, + if_, + else_, + + try_ = 0x06, + catch_, + catch_all = 0x19, + throw_ = 0x08, + rethrow_ = 0x09, + + end = 0x0b, + br, + br_if, + br_table, + return_, + call, + call_indirect, + + drop = 0x1a, + select, + + get_local = 0x20, + set_local, + tee_local, + get_global, + set_global, + + i32_load = 0x28, + i64_load, + f32_load, + f64_load, + i32_load8_s, + i32_load8_u, + i32_load16_s, + i32_load16_u, + i64_load8_s, + i64_load8_u, + i64_load16_s, + i64_load16_u, + i64_load32_s, + i64_load32_u, + i32_store, + i64_store, + f32_store, + f64_store, + i32_store8, + i32_store16, + i64_store8, + i64_store16, + i64_store32, + current_memory, + grow_memory, + + i32_const = 0x41, + i64_const, + f32_const, + f64_const, + + i32_eqz = 0x45, + i32_eq, + i32_ne, + i32_lt_s, + i32_lt_u, + i32_gt_s, + i32_gt_u, + i32_le_s, + i32_le_u, + i32_ge_s, + i32_ge_u, + i64_eqz, + i64_eq, + i64_ne, + i64_lt_s, + i64_lt_u, + i64_gt_s, + i64_gt_u, + i64_le_s, + i64_le_u, + i64_ge_s, + i64_ge_u, + f32_eq, + f32_ne, + f32_lt, + f32_gt, + f32_le, + f32_ge, + f64_eq, + f64_ne, + f64_lt, + f64_gt, + f64_le, + f64_ge, + + i32_clz = 0x67, + i32_ctz, + i32_popcnt, + i32_add, + i32_sub, + i32_mul, + i32_div_s, + i32_div_u, + i32_rem_s, + i32_rem_u, + i32_and, + i32_or, + i32_xor, + i32_shl, + i32_shr_s, + i32_shr_u, + i32_rotl, + i32_rotr, + i64_clz, + i64_ctz, + i64_popcnt, + i64_add, + i64_sub, + i64_mul, + i64_div_s, + i64_div_u, + i64_rem_s, + i64_rem_u, + i64_and, + i64_or, + i64_xor, + i64_shl, + i64_shr_s, + i64_shr_u, + i64_rotl, + i64_rotr, + f32_abs, + f32_neg, + f32_ceil, + f32_floor, + f32_trunc, + f32_nearest, + f32_sqrt, + f32_add, + f32_sub, + f32_mul, + f32_div, + f32_min, + f32_max, + f32_copysign, + f64_abs, + f64_neg, + f64_ceil, + f64_floor, + f64_trunc, + f64_nearest, + f64_sqrt, + f64_add, + f64_sub, + f64_mul, + f64_div, + f64_min, + f64_max, + f64_copysign, + + i32_wrap_i64 = 0xa7, + i32_trunc_s_f32, + i32_trunc_u_f32, + i32_trunc_s_f64, + i32_trunc_u_f64, + i64_extend_s_i32, + i64_extend_u_i32, + i64_trunc_s_f32, + i64_trunc_u_f32, + i64_trunc_s_f64, + i64_trunc_u_f64, + f32_convert_s_i32, + f32_convert_u_i32, + f32_convert_s_i64, + f32_convert_u_i64, + f32_demote_f64, + f64_convert_s_i32, + f64_convert_u_i32, + f64_convert_s_i64, + f64_convert_u_i64, + f64_promote_f32, + + i32_reinterpret_f32 = 0xbc, + i64_reinterpret_f64, + f32_reinterpret_i32, + f64_reinterpret_i64, + + i32_extend_8_s = 0xc0, + i32_extend_16_s, + i64_extend_8_s, + i64_extend_16_s, + i64_extend_32_s, + + PREFIX_sat = 0xfc, + PREFIX_atomic = 0xfe +} diff --git a/src/mono/wasm/runtime/jiterpreter-support.ts b/src/mono/wasm/runtime/jiterpreter-support.ts new file mode 100644 index 00000000000000..06512c2ad08535 --- /dev/null +++ b/src/mono/wasm/runtime/jiterpreter-support.ts @@ -0,0 +1,788 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +import { NativePointer, VoidPtr } from "./types/emscripten"; +import { Module } from "./imports"; +import { WasmOpcode } from "./jiterpreter-opcodes"; +import cwraps from "./cwraps"; + +// uint16 +export declare interface MintOpcodePtr extends NativePointer { + __brand: "MintOpcodePtr" +} + +export class WasmBuilder { + stack: Array; + stackSize!: number; + inSection!: boolean; + inFunction!: boolean; + locals = new Map(); + functionTypeCount!: number; + functionTypes!: { [name: string] : [number, { [name: string]: WasmValtype }, WasmValtype, string] }; + functionTypesByShape!: { [shape: string] : number }; + functionTypesByIndex: Array = []; + importedFunctionCount!: number; + importedFunctions!: { [name: string] : [number, number, string] }; + importsToEmit!: Array<[string, string, number, number]>; + argumentCount!: number; + activeBlocks!: number; + base!: MintOpcodePtr; + traceBuf: Array = []; + branchTargets = new Set(); + options!: JiterpreterOptions; + + constructor () { + this.stack = [new BlobBuilder()]; + this.clear(); + } + + clear () { + this.options = getOptions(); + this.stackSize = 1; + this.inSection = false; + this.inFunction = false; + this.locals.clear(); + this.functionTypeCount = 0; + this.functionTypes = {}; + this.functionTypesByShape = {}; + this.functionTypesByIndex.length = 0; + this.importedFunctionCount = 0; + this.importedFunctions = {}; + this.importsToEmit = []; + this.argumentCount = 0; + this.current.clear(); + this.traceBuf.length = 0; + this.branchTargets.clear(); + this.activeBlocks = 0; + } + + push () { + this.stackSize++; + if (this.stackSize >= this.stack.length) + this.stack.push(new BlobBuilder()); + this.current.clear(); + } + + pop () { + if (this.stackSize <= 1) + throw new Error("Stack empty"); + + const current = this.current; + this.stackSize--; + + this.appendULeb(current.size); + const av = current.getArrayView(); + this.appendBytes(av); + } + + get bytesGeneratedSoFar () { + return this.stack[0].size; + } + + get current() { + return this.stack[this.stackSize - 1]; + } + + get size() { + return this.current.size; + } + + appendU8 (value: number | WasmOpcode) { + if ((value != value >>> 0) || (value > 255)) + throw new Error(`Byte out of range: ${value}`); + return this.current.appendU8(value); + } + + appendU32 (value: number) { + return this.current.appendU32(value); + } + + appendF32 (value: number) { + return this.current.appendF32(value); + } + + appendF64 (value: number) { + return this.current.appendF64(value); + } + + appendULeb (value: number | MintOpcodePtr) { + return this.current.appendULeb(value); + } + + appendLeb (value: number) { + return this.current.appendLeb(value); + } + + appendLebRef (sourceAddress: VoidPtr, signed: boolean) { + return this.current.appendLebRef(sourceAddress, signed); + } + + appendBytes (bytes: Uint8Array) { + return this.current.appendBytes(bytes); + } + + appendName (text: string) { + return this.current.appendName(text); + } + + ret (ip: MintOpcodePtr) { + this.ip_const(ip); + this.appendU8(WasmOpcode.return_); + } + + i32_const (value: number) { + this.appendU8(WasmOpcode.i32_const); + this.appendLeb(value); + } + + ip_const (value: MintOpcodePtr, highBit?: boolean) { + this.appendU8(WasmOpcode.i32_const); + let relativeValue = value - this.base; + if (highBit) { + // it is impossible to do this in JS as far as i can tell + // relativeValue |= 0x80000000; + relativeValue += 0xF000000; + } + this.appendLeb(relativeValue); + } + + i52_const (value: number) { + this.appendU8(WasmOpcode.i64_const); + this.appendLeb(value); + } + + defineType (name: string, parameters: { [name: string]: WasmValtype }, returnType: WasmValtype) { + if (this.functionTypes[name]) + throw new Error(`Function type ${name} already defined`); + + let index: number; + let shape = ""; + for (const k in parameters) + shape += parameters[k] + ","; + shape += returnType; + index = this.functionTypesByShape[shape]; + + if (!index) { + index = this.functionTypeCount++; + this.functionTypesByShape[shape] = index; + this.functionTypesByIndex[index] = [parameters, returnType]; + } + + this.functionTypes[name] = [ + index, parameters, returnType, `(${JSON.stringify(parameters)}) -> ${returnType}` + ]; + return index; + } + + generateTypeSection () { + this.beginSection(1); + this.appendULeb(this.functionTypeCount); + /* + if (trace > 1) + console.log(`Generated ${this.functionTypeCount} wasm type(s) from ${Object.keys(this.functionTypes).length} named function types`); + */ + for (let i = 0; i < this.functionTypesByIndex.length; i++) { + const parameters = this.functionTypesByIndex[i][0]; + const returnType = this.functionTypesByIndex[i][1]; + this.appendU8(0x60); + // Parameters + this.appendULeb(Object.keys(parameters).length); + for (const k in parameters) + this.appendU8(parameters[k]); + // Return type(s) + if (returnType !== WasmValtype.void) { + this.appendULeb(1); + this.appendU8(returnType); + } else + this.appendULeb(0); + } + this.endSection(); + } + + generateImportSection () { + // Import section + this.beginSection(2); + this.appendULeb(1 + this.importsToEmit.length); + + for (let i = 0; i < this.importsToEmit.length; i++) { + const tup = this.importsToEmit[i]; + this.appendName(tup[0]); + this.appendName(tup[1]); + this.appendU8(tup[2]); + this.appendULeb(tup[3]); + } + + this.appendName("i"); + this.appendName("h"); + // memtype (limits = { min=0x01, max=infinity }) + this.appendU8(0x02); + this.appendU8(0x00); + // Minimum size is in 64k pages, not bytes + this.appendULeb(0x01); + } + + defineImportedFunction ( + module: string, name: string, functionTypeName: string, + wasmName?: string + ) { + const index = this.importedFunctionCount++; + const type = this.functionTypes[functionTypeName]; + if (!type) + throw new Error("No function type named " + functionTypeName); + const typeIndex = type[0]; + this.importedFunctions[name] = [ + index, typeIndex, type[3] + ]; + this.importsToEmit.push([module, wasmName || name, 0, typeIndex]); + return index; + } + + callImport (name: string) { + const func = this.importedFunctions[name]; + if (!func) + throw new Error("No imported function named " + name); + this.appendU8(WasmOpcode.call); + this.appendULeb(func[0]); + } + + beginSection (type: number) { + if (this.inSection) + this.pop(); + this.appendU8(type); + this.push(); + this.inSection = true; + } + + endSection () { + if (!this.inSection) + throw new Error("Not in section"); + if (this.inFunction) + this.endFunction(); + this.pop(); + this.inSection = false; + } + + beginFunction ( + type: string, + locals?: {[name: string]: WasmValtype} + ) { + if (this.inFunction) + this.endFunction(); + this.push(); + + const signature = this.functionTypes[type]; + this.locals.clear(); + this.branchTargets.clear(); + let counts: any = {}; + const tk = [WasmValtype.i32, WasmValtype.i64, WasmValtype.f32, WasmValtype.f64]; + + const assignParameterIndices = (parms: {[name: string] : WasmValtype}) => { + let result = 0; + for (const k in parms) { + const parm = parms[k]; + this.locals.set(k, [parm, result]); + // console.log(`parm ${k} -> ${result}`); + result++; + } + return result; + }; + + let localGroupCount = 0; + + // We first assign the parameters local indices and then + // we assign the named locals indices, because parameters + // come first in the local space. Imagine if parameters + // had their own opcode and weren't mutable?????? + const assignLocalIndices = (locals: {[name: string] : WasmValtype}, base: number) => { + Object.assign(counts, { + [WasmValtype.i32]: 0, + [WasmValtype.i64]: 0, + [WasmValtype.f32]: 0, + [WasmValtype.f64]: 0, + }); + for (const k in locals) { + const ty = locals[k]; + if (counts[ty] <= 0) + localGroupCount++; + counts[ty]++; + } + + const offi32 = 0, + offi64 = counts[WasmValtype.i32], + offf32 = offi64 + counts[WasmValtype.i64], + offf64 = offf32 + counts[WasmValtype.f32]; + Object.assign(counts,{ + [WasmValtype.i32]: 0, + [WasmValtype.i64]: 0, + [WasmValtype.f32]: 0, + [WasmValtype.f64]: 0, + }); + for (const k in locals) { + const ty = locals[k]; + let idx = 0; + switch (ty) { + case WasmValtype.i32: + idx = (counts[ty]++) + offi32 + base; + this.locals.set(k, [ty, idx]); + break; + case WasmValtype.i64: + idx = (counts[ty]++) + offi64 + base; + this.locals.set(k, [ty, idx]); + break; + case WasmValtype.f32: + idx = (counts[ty]++) + offf32 + base; + this.locals.set(k, [ty, idx]); + break; + case WasmValtype.f64: + idx = (counts[ty]++) + offf64 + base; + this.locals.set(k, [ty, idx]); + break; + } + // console.log(`local ${k} ${locals[k]} -> ${idx}`); + } + }; + + // Assign indices for the parameter list from the function signature + const localBaseIndex = assignParameterIndices(signature[1]); + if (locals) + // Now if we have any locals, assign indices for those + assignLocalIndices(locals, localBaseIndex); + else + // Otherwise erase the counts table from the parameter assignment + counts = {}; + + // Write the number of types and then write a count for each type + this.appendULeb(localGroupCount); + for (let i = 0; i < tk.length; i++) { + const k = tk[i]; + const c = counts[k]; + if (!c) + continue; + // console.log(`${k} x${c}`); + this.appendULeb(c); + this.appendU8(k); + } + + this.inFunction = true; + } + + endFunction () { + if (!this.inFunction) + throw new Error("Not in function"); + if (this.activeBlocks > 0) + throw new Error(`${this.activeBlocks} unclosed block(s) at end of function`); + this.pop(); + this.inFunction = false; + } + + block (type?: WasmValtype, opcode?: WasmOpcode) { + const result = this.appendU8(opcode || WasmOpcode.block); + if (type) + this.appendU8(type); + else + this.appendU8(WasmValtype.void); + this.activeBlocks++; + return result; + } + + endBlock () { + if (this.activeBlocks <= 0) + throw new Error("No blocks active"); + this.activeBlocks--; + this.appendU8(WasmOpcode.end); + } + + arg (name: string | number, opcode?: WasmOpcode) { + const index = typeof(name) === "string" + ? (this.locals.has(name) ? this.locals.get(name)![1] : undefined) + : name; + if (typeof (index) !== "number") + throw new Error("No local named " + name); + if (opcode) + this.appendU8(opcode); + this.appendULeb(index); + } + + local (name: string | number, opcode?: WasmOpcode) { + const index = typeof(name) === "string" + ? (this.locals.has(name) ? this.locals.get(name)![1] : undefined) + : name + this.argumentCount; + if (typeof (index) !== "number") + throw new Error("No local named " + name); + if (opcode) + this.appendU8(opcode); + else + this.appendU8(WasmOpcode.get_local); + this.appendULeb(index); + } + + appendMemarg (offset: number, alignPower: number) { + this.appendULeb(alignPower); + this.appendULeb(offset); + } + + /* + generates either (u32)get_local(ptr) + offset or (u32)ptr1 + offset + */ + lea (ptr1: string | number, offset: number) { + if (typeof (ptr1) === "string") + this.local(ptr1); + else + this.i32_const(ptr1); + + this.i32_const(offset); + // FIXME: How do we make sure this has correct semantics for pointers over 2gb? + this.appendU8(WasmOpcode.i32_add); + } + + getArrayView (fullCapacity?: boolean) { + if (this.stackSize > 1) + throw new Error("Stack not empty"); + return this.stack[0].getArrayView(fullCapacity); + } +} + +export class BlobBuilder { + buffer: number; + view!: DataView; + size: number; + capacity: number; + encoder?: TextEncoder; + + constructor () { + this.capacity = 32000; + this.buffer = Module._malloc(this.capacity); + this.size = 0; + this.clear(); + } + + // It is necessary for you to call this before using the builder so that the DataView + // can be reconstructed in case the heap grew since last use + clear () { + // FIXME: This should not be necessary + Module.HEAPU8.fill(0, this.buffer, this.buffer + this.size); + this.size = 0; + this.view = new DataView(Module.HEAPU8.buffer, this.buffer, this.capacity); + } + + appendU8 (value: number | WasmOpcode) { + if (this.size >= this.capacity) + throw new Error("Buffer full"); + + const result = this.size; + Module.HEAPU8[this.buffer + (this.size++)] = value; + return result; + } + + appendU16 (value: number) { + const result = this.size; + this.view.setUint16(this.size, value, true); + this.size += 2; + return result; + } + + appendI16 (value: number) { + const result = this.size; + this.view.setInt16(this.size, value, true); + this.size += 2; + return result; + } + + appendU32 (value: number) { + const result = this.size; + this.view.setUint32(this.size, value, true); + this.size += 4; + return result; + } + + appendI32 (value: number) { + const result = this.size; + this.view.setInt32(this.size, value, true); + this.size += 4; + return result; + } + + appendF32 (value: number) { + const result = this.size; + this.view.setFloat32(this.size, value, true); + this.size += 4; + return result; + } + + appendF64 (value: number) { + const result = this.size; + this.view.setFloat64(this.size, value, true); + this.size += 8; + return result; + } + + appendULeb (value: number) { + if (this.size + 8 >= this.capacity) + throw new Error("Buffer full"); + + const bytesWritten = cwraps.mono_jiterp_encode_leb52((this.buffer + this.size), value, 0); + if (bytesWritten < 1) + throw new Error(`Failed to encode value '${value}' as unsigned leb`); + this.size += bytesWritten; + return bytesWritten; + } + + appendLeb (value: number) { + if (this.size + 8 >= this.capacity) + throw new Error("Buffer full"); + + const bytesWritten = cwraps.mono_jiterp_encode_leb52((this.buffer + this.size), value, 1); + if (bytesWritten < 1) + throw new Error(`Failed to encode value '${value}' as signed leb`); + this.size += bytesWritten; + return bytesWritten; + } + + appendLebRef (sourceAddress: VoidPtr, signed: boolean) { + if (this.size + 8 >= this.capacity) + throw new Error("Buffer full"); + + const bytesWritten = cwraps.mono_jiterp_encode_leb64_ref((this.buffer + this.size), sourceAddress, signed ? 1 : 0); + if (bytesWritten < 1) + throw new Error("Failed to encode value as leb"); + this.size += bytesWritten; + return bytesWritten; + } + + appendBytes (bytes: Uint8Array) { + const result = this.size; + const av = this.getArrayView(true); + av.set(bytes, this.size); + this.size += bytes.length; + return result; + } + + appendName (text: string) { + let bytes: any = null; + + if (typeof (TextEncoder) === "function") { + if (!this.encoder) + this.encoder = new TextEncoder(); + bytes = this.encoder.encode(text); + } else { + bytes = new Uint8Array(text.length); + for (let i = 0; i < text.length; i++) { + const ch = text.charCodeAt(i); + if (ch > 0x7F) + throw new Error("Out of range character and no TextEncoder available"); + else + bytes[i] = ch; + } + } + this.appendULeb(bytes.length); + this.appendBytes(bytes); + } + + getArrayView (fullCapacity?: boolean) { + return new Uint8Array(Module.HEAPU8.buffer, this.buffer, fullCapacity ? this.capacity : this.size); + } +} + +export const enum WasmValtype { + void = 0x40, + i32 = 0x7F, + i64 = 0x7E, + f32 = 0x7D, + f64 = 0x7C, +} + +let wasmTable : WebAssembly.Table | undefined; +let wasmNextFunctionIndex = -1, wasmFunctionIndicesFree = 0; + +// eslint-disable-next-line prefer-const +export const elapsedTimes = { + generation: 0, + compilation: 0 +}; + +export const counters = { + traceCandidates: 0, + tracesCompiled: 0, + entryWrappersCompiled: 0, + jitCallsCompiled: 0, +}; + +export const _now = (globalThis.performance && globalThis.performance.now) + ? globalThis.performance.now.bind(globalThis.performance) + : Date.now; + +let scratchBuffer : NativePointer = 0; + +export function copyIntoScratchBuffer (src: NativePointer, size: number) : NativePointer { + if (!scratchBuffer) + scratchBuffer = Module._malloc(64); + if (size > 64) + throw new Error("Scratch buffer size is 64"); + + Module.HEAPU8.copyWithin(scratchBuffer, src, src + size); + return scratchBuffer; +} + +export function getWasmFunctionTable () { + if (!wasmTable) + wasmTable = (Module)["asm"]["__indirect_function_table"]; + if (!wasmTable) + throw new Error("Module did not export the indirect function table"); + return wasmTable; +} + +export function addWasmFunctionPointer (f: Function) { + if (!f) + throw new Error("Attempting to set null function into table"); + const table = getWasmFunctionTable(); + if (wasmFunctionIndicesFree <= 0) { + wasmNextFunctionIndex = table.length; + wasmFunctionIndicesFree = 512; + table.grow(wasmFunctionIndicesFree); + } + const index = wasmNextFunctionIndex; + wasmNextFunctionIndex++; + wasmFunctionIndicesFree--; + table.set(index, f); + return index; +} + +export function append_memset_dest (builder: WasmBuilder, value: number, count: number) { + // spec: pop n, pop val, pop d, fill from d[0] to d[n] with value val + builder.i32_const(value); + builder.i32_const(count); + builder.appendU8(WasmOpcode.PREFIX_sat); + builder.appendU8(11); + builder.appendU8(0); +} + +// expects dest then source to have been pushed onto wasm stack +export function append_memmove_dest_src (builder: WasmBuilder, count: number) { + switch (count) { + case 1: + builder.appendU8(WasmOpcode.i32_load8_u); + builder.appendMemarg(0, 0); + builder.appendU8(WasmOpcode.i32_store8); + builder.appendMemarg(0, 0); + return true; + case 2: + builder.appendU8(WasmOpcode.i32_load16_u); + builder.appendMemarg(0, 0); + builder.appendU8(WasmOpcode.i32_store16); + builder.appendMemarg(0, 0); + return true; + case 4: + builder.appendU8(WasmOpcode.i32_load); + builder.appendMemarg(0, 0); + builder.appendU8(WasmOpcode.i32_store); + builder.appendMemarg(0, 0); + return true; + case 8: + builder.appendU8(WasmOpcode.i64_load); + builder.appendMemarg(0, 0); + builder.appendU8(WasmOpcode.i64_store); + builder.appendMemarg(0, 0); + return true; + default: + // spec: pop n, pop s, pop d, copy n bytes from s to d + builder.i32_const(count); + // great encoding isn't it + builder.appendU8(WasmOpcode.PREFIX_sat); + builder.appendU8(10); + builder.appendU8(0); + builder.appendU8(0); + return true; + } +} + +export function getRawCwrap (name: string): Function { + const result = (Module)["asm"][name]; + if (typeof (result) !== "function") + throw new Error(`raw cwrap ${name} not found`); + return result; +} + +export function importDef (name: string, fn: Function): [string, string, Function] { + return [name, name, fn]; +} + +export type JiterpreterOptions = { + enableAll?: boolean; + enableTraces: boolean; + enableInterpEntry: boolean; + enableJitCall: boolean; + enableBackwardBranches: boolean; + enableCallResume: boolean; + enableWasmEh: boolean; + // For locations where the jiterpreter heuristic says we will be unable to generate + // a trace, insert an entry point opcode anyway. This enables collecting accurate + // stats for options like estimateHeat, but raises overhead. + alwaysGenerate: boolean; + enableStats: boolean; + // Continue counting hits for traces that fail to compile and use it to estimate + // the relative importance of the opcode that caused them to abort + estimateHeat: boolean; + // Count the number of times a trace bails out (branch taken, etc) and for what reason + countBailouts: boolean; + minimumTraceLength: number; +} + +const optionNames : { [jsName: string] : [string, string] | string } = { + "enableAll": ["jiterpreter-enable-all", "jiterpreter-disable-all"], + "enableTraces": ["jiterpreter-enable-traces", "jiterpreter-disable-traces"], + "enableInterpEntry": ["jiterpreter-enable-interp-entry", "jiterpreter-disable-interp-entry"], + "enableJitCall": ["jiterpreter-enable-jit-call", "jiterpreter-disable-jit-call"], + "enableBackwardBranches": ["jiterpreter-enable-backward-branches", "jiterpreter-disable-backward-branches"], + "enableCallResume": ["jiterpreter-enable-call-resume", "jiterpreter-disable-call-resume"], + "enableWasmEh": ["jiterpreter-enable-wasm-eh", "jiterpreter-disable-wasm-eh"], + "enableStats": ["jiterpreter-enable-stats", "jiterpreter-disable-stats"], + "alwaysGenerate": ["jiterpreter-always-generate", ""], + "estimateHeat": ["jiterpreter-estimate-heat", ""], + "countBailouts": ["jiterpreter-count-bailouts", ""], + "minimumTraceLength": "jiterpreter-minimum-trace-length", +}; + +let optionsVersion = -1; +let optionTable : JiterpreterOptions = {}; + +// applies one or more jiterpreter options to change the current jiterpreter configuration. +export function applyOptions (options: JiterpreterOptions) { + for (const k in options) { + const info = optionNames[k]; + if (!info) { + console.error(`Unrecognized jiterpreter option: ${k}`); + continue; + } + + const v = (options)[k]; + if (typeof (v) === "boolean") + cwraps.mono_jiterp_parse_option(v ? info[0] : info[1]); + else if (typeof (v) === "number") + cwraps.mono_jiterp_parse_option(`${info}=${v}`); + else + console.error(`Jiterpreter option must be a boolean or a number but was ${typeof(v)} '${v}'`); + } +} + +// returns the current jiterpreter configuration. do not mutate the return value! +export function getOptions () { + const currentVersion = cwraps.mono_jiterp_get_options_version(); + if (currentVersion !== optionsVersion) { + updateOptions(); + optionsVersion = currentVersion; + } + return optionTable; +} + +function updateOptions () { + const table = {}; + optionTable = table; + for (const k in optionNames) { + const info = optionNames[k]; + if (Array.isArray(info)) + table[k] = cwraps.mono_jiterp_get_option(info[0]) > 0; + else + table[k] = cwraps.mono_jiterp_get_option(info); + } + // console.log(`options=${JSON.stringify(table)}`); +} diff --git a/src/mono/wasm/runtime/jiterpreter.ts b/src/mono/wasm/runtime/jiterpreter.ts new file mode 100644 index 00000000000000..404bf7155cba01 --- /dev/null +++ b/src/mono/wasm/runtime/jiterpreter.ts @@ -0,0 +1,3066 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +import { mono_assert, MonoMethod } from "./types"; +import { NativePointer } from "./types/emscripten"; +import { Module } from "./imports"; +import { + getU16, getI16, + getU32, getI32, getF32, getF64, +} from "./memory"; +import { MintOpcode, OpcodeInfo, WasmOpcode } from "./jiterpreter-opcodes"; +import cwraps from "./cwraps"; +import { + MintOpcodePtr, WasmValtype, WasmBuilder, addWasmFunctionPointer, + copyIntoScratchBuffer, _now, elapsedTimes, append_memset_dest, + append_memmove_dest_src, counters, getRawCwrap, importDef, + JiterpreterOptions, getOptions +} from "./jiterpreter-support"; + +// Controls miscellaneous diagnostic output. +const trace = 0; + +const + // Record a trace of all managed interpreter opcodes then dump it to console + // if an error occurs while compiling the output wasm + traceOnError = false, + // Record trace but dump it when the trace has a runtime error instead + // requires trapTraceErrors to work and will slow trace compilation + + // increase memory usage + traceOnRuntimeError = false, + // Trace the method name, location and reason for each abort + traceAbortLocations = false, + // Count the number of times a given method is seen as a call target, then + // dump a list of the most common ones when dumping stats + countCallTargets = false, + // Trace when encountering branches + traceBranchDisplacements = false, + // Trace when we reject something for being too small + traceTooSmall = false, + // Wraps traces in a JS function that will trap errors and log the trace responsible. + // Very expensive!!!! + trapTraceErrors = false, + // Dumps all compiled traces + dumpTraces = false, + // Emit a wasm nop between each managed interpreter opcode + emitPadding = false, + // Generate compressed names for imports so that modules have more space for code + compressImportNames = true; + +const callTargetCounts : { [method: number] : number } = {}; + +const instrumentedMethodNames : Array = [ + // "int NDPin:RunTest ()" +]; + +class InstrumentedTraceState { + name: string; + eip: MintOpcodePtr; + operand1: number | undefined; + operand2: number | undefined; + + constructor (name: string) { + this.name = name; + this.eip = 0; + } +} + +class TraceInfo { + ip: MintOpcodePtr; + hitCount: number; + name: string | undefined; + abortReason: string | undefined; + fnPtr: Number | undefined; + + constructor (ip: MintOpcodePtr) { + this.ip = ip; + this.hitCount = 1; + } +} + +const instrumentedTraces : { [key: number]: InstrumentedTraceState } = {}; +let nextInstrumentedTraceId = 1; +const abortCounts : { [key: string] : number } = {}; +const traceInfo : { [key: string] : TraceInfo } = {}; + +// It is critical to only jit traces that contain a significant +// number of opcodes, because the indirect call into a trace +// is pretty expensive. We have both MINT opcode and WASM byte +// thresholds, and as long as a trace is above one of these two +// thresholds, we will keep it. +const minimumHitCount = 10000, + minTraceLengthMintOpcodes = 8, + minTraceLengthWasmBytes = 360; + +const // offsetOfStack = 12, + offsetOfImethod = 4, + offsetOfDataItems = 20, + sizeOfJiterpreterOpcode = 6, // opcode + 4 bytes for thunk id/fn ptr + sizeOfDataItem = 4, + // HACK: Typically we generate ~12 bytes of extra gunk after the function body so we are + // subtracting 20 from the maximum size to make sure we don't produce too much + // Also subtract some more size since the wasm we generate for one opcode could be big + // WASM implementations only allow compiling 4KB of code at once :-) + maxModuleSize = 4000 - 20 - 100; + +/* +struct MonoVTable { + MonoClass *klass; // 0 + MonoGCDescriptor gc_descr; // 4 + MonoDomain *domain; // 8 + gpointer type; // 12 + guint8 *interface_bitmap; // 16 + guint32 max_interface_id; // 20 + guint8 rank; // 21 + guint8 initialized; // 22 + guint8 flags; +*/ + +/* +struct InterpFrame { + InterpFrame *parent; // 0 + InterpMethod *imethod; // 4 + stackval *retval; // 8 + stackval *stack; // 12 + InterpFrame *next_free; // 16 + InterpState state; // 20 +}; + +struct InterpMethod { + MonoMethod *method; + InterpMethod *next_jit_code_hash; + + // Sort pointers ahead of integers to minimize padding for alignment. + + unsigned short *code; + MonoPIFunc func; + MonoExceptionClause *clauses; // num_clauses + void **data_items; +*/ + +const enum BailoutReason { + Unknown, + InterpreterTiering, + NullCheck, + VtableNotInitialized, + Branch, + BackwardBranch, + ConditionalBranch, + ConditionalBackwardBranch, + ComplexBranch, + ArrayLoadFailed, + StringOperationFailed, + DivideByZero, + Overflow, + Return, + Call, + Throw, + AllocFailed, + SpanOperationFailed, + CastFailed, + SafepointBranchTaken, + UnboxFailed, + CallDelegate +} + +const BailoutReasonNames = [ + "Unknown", + "InterpreterTiering", + "NullCheck", + "VtableNotInitialized", + "Branch", + "BackwardBranch", + "ConditionalBranch", + "ConditionalBackwardBranch", + "ComplexBranch", + "ArrayLoadFailed", + "StringOperationFailed", + "DivideByZero", + "Overflow", + "Return", + "Call", + "Throw", + "AllocFailed", + "SpanOperationFailed", + "CastFailed", + "SafepointBranchTaken", + "UnboxFailed", + "CallDelegate" +]; + +let traceBuilder : WasmBuilder; +let traceImports : Array<[string, string, Function]> | undefined; + +let _wrap_trace_function: Function; + +// indexPlusOne so that ip[1] in the interpreter becomes getArgU16(ip, 1) +function getArgU16 (ip: MintOpcodePtr, indexPlusOne: number) { + return getU16(ip + (2 * indexPlusOne)); +} + +function getArgI16 (ip: MintOpcodePtr, indexPlusOne: number) { + return getI16(ip + (2 * indexPlusOne)); +} + +function getArgI32 (ip: MintOpcodePtr, indexPlusOne: number) { + const src = copyIntoScratchBuffer(ip + (2 * indexPlusOne), 4); + return getI32(src); +} + +function getArgF32 (ip: MintOpcodePtr, indexPlusOne: number) { + const src = copyIntoScratchBuffer(ip + (2 * indexPlusOne), 4); + return getF32(src); +} + +function getArgF64 (ip: MintOpcodePtr, indexPlusOne: number) { + const src = copyIntoScratchBuffer(ip + (2 * indexPlusOne), 8); + return getF64(src); +} + +/* +const enum WasmReftype { + funcref = 0x70, + externref = 0x6F, +} +*/ + +const mathOps1 = [ + "acos", + "cos", + "sin", + "asin", + "tan", + "atan" +]; + +function getTraceImports () { + if (traceImports) + return traceImports; + + traceImports = [ + importDef("bailout", getRawCwrap("mono_jiterp_trace_bailout")), + importDef("copy_pointer", getRawCwrap("mono_wasm_copy_managed_pointer")), + importDef("array_length", getRawCwrap("mono_wasm_array_length_ref")), + importDef("array_address", getRawCwrap("mono_jiterp_array_get_element_address_with_size_ref")), + importDef("entry", getRawCwrap("mono_jiterp_increase_entry_count")), + importDef("value_copy", getRawCwrap("mono_jiterp_value_copy")), + importDef("strlen", getRawCwrap("mono_jiterp_strlen_ref")), + importDef("getchr", getRawCwrap("mono_jiterp_getchr_ref")), + importDef("getspan", getRawCwrap("mono_jiterp_getitem_span")), + importDef("gettype", getRawCwrap("mono_jiterp_gettype_ref")), + importDef("cast", getRawCwrap("mono_jiterp_cast_ref")), + importDef("try_unbox", getRawCwrap("mono_jiterp_try_unbox_ref")), + importDef("box", getRawCwrap("mono_jiterp_box_ref")), + importDef("localloc", getRawCwrap("mono_jiterp_localloc")), + ["ckovr_i4", "overflow_check_i4", getRawCwrap("mono_jiterp_overflow_check_i4")], + ["ckovr_u4", "overflow_check_i4", getRawCwrap("mono_jiterp_overflow_check_u4")], + ["rem", "mathop_dd_d", getRawCwrap("mono_jiterp_fmod")], + ["atan2", "mathop_dd_d", getRawCwrap("mono_jiterp_atan2")], + ["newobj_i", "newobj_i", getRawCwrap("mono_jiterp_try_newobj_inlined")], + ["ld_del_ptr", "ld_del_ptr", getRawCwrap("mono_jiterp_ld_delegate_method_ptr")], + ["ldtsflda", "ldtsflda", getRawCwrap("mono_jiterp_ldtsflda")], + ["conv_ovf", "conv_ovf", getRawCwrap("mono_jiterp_conv_ovf")], + ]; + + if (instrumentedMethodNames.length > 0) { + traceImports.push(["trace_eip", "trace_eip", trace_current_ip]); + traceImports.push(["trace_args", "trace_eip", trace_operands]); + } + + for (let i = 0; i < mathOps1.length; i++) { + const mop = mathOps1[i]; + traceImports.push([mop, "mathop_d_d", (Math)[mop]]); + } + + return traceImports; +} + +function wrap_trace_function ( + f: Function, name: string, traceBuf: any, + base: MintOpcodePtr, instrumentedTraceId: number +) { + const tup = instrumentedTraces[instrumentedTraceId]; + if (instrumentedTraceId) + console.log(`instrumented ${tup.name}`); + + if (!_wrap_trace_function) { + // If we used a regular closure, the js console would print the entirety of + // dotnet.js when printing an error stack trace, which is... not helpful + const js = `return function trace_enter (locals) { + let threw = true; + try { + let result = trace(locals); + threw = false; + return result; + } finally { + if (threw) { + let msg = "Unhandled error in trace '" + name + "'"; + if (tup) { + msg += " at offset " + (tup.eip + base).toString(16); + msg += " with most recent operands " + tup.operand1.toString(16) + ", " + tup.operand2.toString(16); + } + console.error(msg); + if (traceBuf) { + for (let i = 0, l = traceBuf.length; i < l; i++) + console.log(traceBuf[i]); + } + } + } + };`; + _wrap_trace_function = new Function("trace", "name", "traceBuf", "tup", "base", js); + } + return _wrap_trace_function( + f, name, traceBuf, instrumentedTraces[instrumentedTraceId], base + ); +} + +// returns function id +function generate_wasm ( + frame: NativePointer, methodName: string, ip: MintOpcodePtr, + startOfBody: MintOpcodePtr, sizeOfBody: MintOpcodePtr, + methodFullName: string | undefined +) : number { + let builder = traceBuilder; + if (!builder) + traceBuilder = builder = new WasmBuilder(); + else + builder.clear(); + + mostRecentOptions = builder.options; + + // skip jiterpreter_enter + // const _ip = ip; + const traceOffset = ip - startOfBody; + const endOfBody = startOfBody + sizeOfBody; + const traceName = `${methodName}:${(traceOffset).toString(16)}`; + + const started = _now(); + let compileStarted = 0; + let rejected = true, threw = false; + + const instrument = methodFullName && (instrumentedMethodNames.indexOf(methodFullName) >= 0); + const instrumentedTraceId = instrument ? nextInstrumentedTraceId++ : 0; + if (instrument) { + console.log(`instrumenting: ${methodFullName}`); + instrumentedTraces[instrumentedTraceId] = new InstrumentedTraceState(methodFullName); + } + const compress = compressImportNames && !instrument; + + try { + // Magic number and version + builder.appendU32(0x6d736100); + builder.appendU32(1); + + // Function type for compiled traces + builder.defineType( + "trace", { + "frame": WasmValtype.i32, + "pLocals": WasmValtype.i32 + }, WasmValtype.i32 + ); + builder.defineType( + "bailout", { + "ip": WasmValtype.i32, + "reason": WasmValtype.i32 + }, WasmValtype.i32 + ); + builder.defineType( + "copy_pointer", { + "dest": WasmValtype.i32, + "src": WasmValtype.i32 + }, WasmValtype.void + ); + builder.defineType( + "value_copy", { + "dest": WasmValtype.i32, + "src": WasmValtype.i32, + "klass": WasmValtype.i32, + }, WasmValtype.void + ); + builder.defineType( + "array_length", { + "ppArray": WasmValtype.i32 + }, WasmValtype.i32 + ); + builder.defineType( + "array_address", { + "ppArray": WasmValtype.i32, + "elementSize": WasmValtype.i32, + "index": WasmValtype.i32 + }, WasmValtype.i32 + ); + builder.defineType( + "entry", { + "imethod": WasmValtype.i32 + }, WasmValtype.i32 + ); + builder.defineType( + "strlen", { + "ppString": WasmValtype.i32, + "pResult": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "getchr", { + "ppString": WasmValtype.i32, + "pIndex": WasmValtype.i32, + "pResult": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "getspan", { + "destination": WasmValtype.i32, + "span": WasmValtype.i32, + "index": WasmValtype.i32, + "element_size": WasmValtype.i32 + }, WasmValtype.i32 + ); + builder.defineType( + "overflow_check_i4", { + "lhs": WasmValtype.i32, + "rhs": WasmValtype.i32, + "opcode": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "mathop_d_d", { + "value": WasmValtype.f64, + }, WasmValtype.f64 + ); + builder.defineType( + "mathop_dd_d", { + "lhs": WasmValtype.f64, + "rhs": WasmValtype.f64, + }, WasmValtype.f64 + ); + builder.defineType( + "trace_eip", { + "traceId": WasmValtype.i32, + "eip": WasmValtype.i32, + }, WasmValtype.void + ); + builder.defineType( + "newobj_i", { + "ppDestination": WasmValtype.i32, + "vtable": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "localloc", { + "destination": WasmValtype.i32, + "len": WasmValtype.i32, + "frame": WasmValtype.i32, + }, WasmValtype.void + ); + builder.defineType( + "ld_del_ptr", { + "ppDestination": WasmValtype.i32, + "ppSource": WasmValtype.i32, + }, WasmValtype.void + ); + builder.defineType( + "ldtsflda", { + "ppDestination": WasmValtype.i32, + "offset": WasmValtype.i32, + }, WasmValtype.void + ); + builder.defineType( + "gettype", { + "destination": WasmValtype.i32, + "source": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "cast", { + "destination": WasmValtype.i32, + "source": WasmValtype.i32, + "klass": WasmValtype.i32, + "opcode": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "try_unbox", { + "klass": WasmValtype.i32, + "destination": WasmValtype.i32, + "source": WasmValtype.i32, + }, WasmValtype.i32 + ); + builder.defineType( + "box", { + "vtable": WasmValtype.i32, + "destination": WasmValtype.i32, + "source": WasmValtype.i32, + "vt": WasmValtype.i32, + }, WasmValtype.void + ); + builder.defineType( + "conv_ovf", { + "destination": WasmValtype.i32, + "source": WasmValtype.i32, + "opcode": WasmValtype.i32, + }, WasmValtype.i32 + ); + + builder.generateTypeSection(); + + // Import section + const traceImports = getTraceImports(); + + // Emit function imports + for (let i = 0; i < traceImports.length; i++) { + mono_assert(traceImports[i], () => `trace #${i} missing`); + const wasmName = compress ? i.toString(16) : undefined; + builder.defineImportedFunction("i", traceImports[i][0], traceImports[i][1], wasmName); + } + + builder.generateImportSection(); + + // Function section + builder.beginSection(3); + builder.appendULeb(1); + // Function type for our compiled trace + mono_assert(builder.functionTypes["trace"], "func type missing"); + builder.appendULeb(builder.functionTypes["trace"][0]); + + // Export section + builder.beginSection(7); + builder.appendULeb(1); + builder.appendName(traceName); + builder.appendU8(0); + // Imports get added to the function index space, so we need to add + // the count of imported functions to get the index of our compiled trace + builder.appendULeb(builder.importedFunctionCount + 0); + + // Code section + builder.beginSection(10); + builder.appendULeb(1); + builder.beginFunction("trace", { + "eip": WasmValtype.i32, + "temp_ptr": WasmValtype.i32, + "cknull_ptr": WasmValtype.i32, + "math_lhs32": WasmValtype.i32, + "math_rhs32": WasmValtype.i32, + "math_lhs64": WasmValtype.i64, + "math_rhs64": WasmValtype.i64 + // "tempi64": WasmValtype.i64 + }); + + if (emitPadding) { + builder.appendU8(WasmOpcode.nop); + builder.appendU8(WasmOpcode.nop); + } + + builder.base = ip; + if (getU16(ip) !== MintOpcode.MINT_TIER_PREPARE_JITERPRETER) + throw new Error(`Expected *ip to be MINT_TIER_PREPARE_JITERPRETER but was ${getU16(ip)}`); + + const opcodes_processed = generate_wasm_body( + frame, traceName, ip, endOfBody, builder, + instrumentedTraceId + ); + const keep = (opcodes_processed >= minTraceLengthMintOpcodes) || + (builder.current.size >= minTraceLengthWasmBytes); + + if (!keep) { + const ti = traceInfo[ip]; + if (ti && (ti.abortReason === "end-of-body")) + ti.abortReason = "trace-too-small"; + + if (traceTooSmall && (opcodes_processed > 1)) + console.log(`${traceName} too small: ${opcodes_processed} opcodes, ${builder.current.size} wasm bytes`); + return 0; + } + + builder.appendU8(WasmOpcode.end); + builder.endSection(); + + compileStarted = _now(); + const buffer = builder.getArrayView(); + if (trace > 0) + console.log(`${traceName} generated ${buffer.length} byte(s) of wasm`); + const traceModule = new WebAssembly.Module(buffer); + + const imports : any = { + h: (Module).asm.memory + }; + // Place our function imports into the import dictionary + for (let i = 0; i < traceImports.length; i++) { + const ifn = traceImports[i][2]; + const iname = traceImports[i][0]; + if (!ifn || (typeof (ifn) !== "function")) + throw new Error(`Import '${iname}' not found or not a function`); + const wasmName = compress ? i.toString(16) : iname; + imports[wasmName] = ifn; + } + + const traceInstance = new WebAssembly.Instance(traceModule, { + i: imports + }); + + // Get the exported trace function + const fn = traceInstance.exports[traceName]; + + // FIXME: Before threading can be supported, we will need to ensure that + // once we assign a function pointer index to a given trace, the trace is + // broadcast to all the JS workers and compiled + installed at the appropriate + // index in every worker's function pointer table. This also means that we + // would need to fill empty slots with a dummy function when growing the table + // so that any erroneous ENTERs will skip the opcode instead of crashing due + // to calling a null function pointer. + // Table grow operations will need to be synchronized between workers somehow, + // probably by storing the table size in a volatile global or something so that + // we know the range of indexes available to us and can ensure that threads + // independently jitting traces will not stomp on each other and all threads + // have a globally consistent view of which function pointer maps to each trace. + rejected = false; + const idx = + trapTraceErrors + ? Module.addFunction( + wrap_trace_function( + fn, methodFullName || methodName, traceOnRuntimeError ? builder.traceBuf : undefined, + builder.base, instrumentedTraceId + ), "iii" + ) + : addWasmFunctionPointer(fn); + if (!idx) + throw new Error("add_function_pointer returned a 0 index"); + else if (trace >= 2) + console.log(`${traceName} -> fn index ${idx}`); + + return idx; + } catch (exc: any) { + threw = true; + rejected = false; + console.error(`MONO_WASM: ${traceName} failed: ${exc} ${exc.stack}`); + return 0; + } finally { + const finished = _now(); + if (compileStarted) { + elapsedTimes.generation += compileStarted - started; + elapsedTimes.compilation += finished - compileStarted; + } else { + elapsedTimes.generation += finished - started; + } + + if (threw || (!rejected && ((trace >= 2) || dumpTraces)) || instrument) { + if (threw || (trace >= 3) || dumpTraces || instrument) { + for (let i = 0; i < builder.traceBuf.length; i++) + console.log(builder.traceBuf[i]); + } + + console.log(`// MONO_WASM: ${traceName} generated, blob follows //`); + let s = "", j = 0; + try { + if (builder.inSection) + builder.endSection(); + } catch { + // eslint-disable-next-line @typescript-eslint/no-extra-semi + ; + } + + const buf = builder.getArrayView(); + for (let i = 0; i < buf.length; i++) { + const b = buf[i]; + if (b < 0x10) + s += "0"; + s += b.toString(16); + s += " "; + if ((s.length % 10) === 0) { + console.log(`${j}\t${s}`); + s = ""; + j = i + 1; + } + } + console.log(`${j}\t${s}`); + console.log("// end blob //"); + } + } +} + +let mostRecentTrace : InstrumentedTraceState | undefined; + +function trace_current_ip (traceId: number, eip: MintOpcodePtr) { + const tup = instrumentedTraces[traceId]; + if (!tup) + throw new Error(`Unrecognized instrumented trace id ${traceId}`); + tup.eip = eip; + mostRecentTrace = tup; +} + +function trace_operands (a: number, b: number) { + if (!mostRecentTrace) + throw new Error("No trace active"); + mostRecentTrace.operand1 = a >>> 0; + mostRecentTrace.operand2 = b >>> 0; +} + +function record_abort (traceIp: MintOpcodePtr, ip: MintOpcodePtr, traceName: string, reason: string | MintOpcode) { + if (typeof (reason) === "number") { + cwraps.mono_jiterp_adjust_abort_count(reason, 1); + reason = OpcodeInfo[reason][0]; + } else { + let abortCount = abortCounts[reason]; + if (typeof (abortCount) !== "number") + abortCount = 1; + else + abortCount++; + + abortCounts[reason] = abortCount; + } + + if ((traceAbortLocations && (reason !== "end-of-body")) || (trace >= 2)) + console.log(`abort ${traceIp} ${traceName}@${ip} ${reason}`); + + traceInfo[traceIp].abortReason = reason; +} + +function get_imethod_data (frame: NativePointer, index: number) { + // FIXME: Encoding this data directly into the trace will prevent trace reuse + const iMethod = getU32(frame + offsetOfImethod); + const pData = getU32(iMethod + offsetOfDataItems); + const dataOffset = pData + (index * sizeOfDataItem); + return getU32(dataOffset); +} + +function append_branch_target_block (builder: WasmBuilder, ip: MintOpcodePtr) { + // End the current branch block, then create a new one that conditionally executes + // if eip matches the offset of its first instruction. + builder.endBlock(); + builder.local("eip"); + builder.ip_const(ip); + builder.appendU8(WasmOpcode.i32_eq); + builder.block(WasmValtype.void, WasmOpcode.if_); +} + +function generate_wasm_body ( + frame: NativePointer, traceName: string, ip: MintOpcodePtr, + endOfBody: MintOpcodePtr, builder: WasmBuilder, instrumentedTraceId: number +) : number { + const abort = 0; + let isFirstInstruction = true; + let result = 0; + const traceIp = ip; + + ip += sizeOfJiterpreterOpcode; + let rip = ip; + + // Initialize eip, so that we will never return a 0 displacement + // Otherwise we could return 0 in the scenario where none of our blocks executed + // (This shouldn't happen though!) + builder.ip_const(ip); + builder.local("eip", WasmOpcode.set_local); + + while (ip) { + if (ip >= endOfBody) { + record_abort(traceIp, ip, traceName, "end-of-body"); + break; + } + if (builder.size >= maxModuleSize - builder.bytesGeneratedSoFar) { + record_abort(traceIp, ip, traceName, "trace-too-big"); + break; + } + + if (instrumentedTraceId) { + builder.i32_const(instrumentedTraceId); + builder.ip_const(ip); + builder.callImport("trace_eip"); + } + + const _ip = ip, + opcode = getU16(ip), + info = OpcodeInfo[opcode]; + mono_assert(info, () => `invalid opcode ${opcode}`); + const opname = info[0]; + let is_dead_opcode = false; + /* This doesn't work for some reason + const endOfOpcode = ip + (info[1] * 2); + if (endOfOpcode > endOfBody) { + record_abort(ip, traceName, "end-of-body"); + break; + } + */ + + // We wrap all instructions in a 'branch block' that is used + // when performing a branch and will be skipped over if the + // current instruction pointer does not match. This means + // that if ip points to a branch target we don't handle, + // the trace will automatically bail out at the end after + // skipping past all the branch targets + if (isFirstInstruction) { + isFirstInstruction = false; + // FIXME: If we allow entering into the middle of a trace, this needs + // to become an if that checks the ip + builder.block(); + } else if (builder.branchTargets.has(ip)) { + // If execution runs past the end of the current branch block, ensure + // that the instruction pointer is updated appropriately. This will + // also guarantee that the branch target block's comparison will + // succeed so that execution continues. + builder.ip_const(rip); + builder.local("eip", WasmOpcode.set_local); + append_branch_target_block(builder, ip); + } + + switch (opcode) { + case MintOpcode.MINT_INITLOCAL: + case MintOpcode.MINT_INITLOCALS: { + // FIXME: We should move the first entry point after initlocals if it exists + const startOffsetInBytes = getArgU16(ip, 1), + sizeInBytes = getArgU16(ip, 2); + append_memset_local(builder, startOffsetInBytes, 0, sizeInBytes); + break; + } + case MintOpcode.MINT_LOCALLOC: { + // dest + append_ldloca(builder, getArgU16(ip, 1)); + // len + append_ldloc(builder, getArgU16(ip, 2), WasmOpcode.i32_load); + // frame + builder.local("frame"); + builder.callImport("localloc"); + break; + } + case MintOpcode.MINT_INITOBJ: { + append_ldloc(builder, getArgU16(ip, 1), WasmOpcode.i32_load); + append_memset_dest(builder, 0, getArgU16(ip, 2)); + break; + } + + // Other conditional branch types are handled by the relop table. + case MintOpcode.MINT_BRFALSE_I4_S: + case MintOpcode.MINT_BRTRUE_I4_S: + case MintOpcode.MINT_BRFALSE_I4_SP: + case MintOpcode.MINT_BRTRUE_I4_SP: + case MintOpcode.MINT_BRFALSE_I8_S: + case MintOpcode.MINT_BRTRUE_I8_S: + case MintOpcode.MINT_LEAVE_S: + case MintOpcode.MINT_BR_S: + if (!emit_branch(builder, ip, opcode)) + ip = abort; + break; + + case MintOpcode.MINT_CKNULL: + // if (locals[ip[2]]) locals[ip[1]] = locals[ip[2]] + builder.local("pLocals"); + append_ldloc_cknull(builder, getArgU16(ip, 2), ip, true); + append_stloc_tail(builder, getArgU16(ip, 1), WasmOpcode.i32_store); + break; + + case MintOpcode.MINT_TIER_ENTER_METHOD: + case MintOpcode.MINT_TIER_PATCHPOINT: { + // We need to make sure to notify the interpreter about tiering opcodes + // so that tiering up will still happen + const iMethod = getU32(frame + offsetOfImethod); + builder.i32_const(iMethod); + // increase_entry_count will return 1 if we can continue, otherwise + // we need to bail out into the interpreter so it can perform tiering + builder.callImport("entry"); + builder.block(WasmValtype.void, WasmOpcode.if_); + append_bailout(builder, ip, BailoutReason.InterpreterTiering); + builder.endBlock(); + break; + } + + case MintOpcode.MINT_TIER_PREPARE_JITERPRETER: + case MintOpcode.MINT_TIER_NOP_JITERPRETER: + case MintOpcode.MINT_TIER_ENTER_JITERPRETER: + case MintOpcode.MINT_NOP: + case MintOpcode.MINT_DEF: + case MintOpcode.MINT_DUMMY_USE: + case MintOpcode.MINT_IL_SEQ_POINT: + case MintOpcode.MINT_TIER_PATCHPOINT_DATA: + case MintOpcode.MINT_MONO_MEMORY_BARRIER: + case MintOpcode.MINT_SDB_BREAKPOINT: + case MintOpcode.MINT_SDB_INTR_LOC: + case MintOpcode.MINT_SDB_SEQ_POINT: + is_dead_opcode = true; + break; + + case MintOpcode.MINT_LDLOCA_S: + // Pre-load locals for the store op + builder.local("pLocals"); + // locals[ip[1]] = &locals[ip[2]] + append_ldloca(builder, getArgU16(ip, 2)); + append_stloc_tail(builder, getArgU16(ip, 1), WasmOpcode.i32_store); + break; + case MintOpcode.MINT_LDTOKEN: + case MintOpcode.MINT_LDSTR: + case MintOpcode.MINT_LDFTN_ADDR: + case MintOpcode.MINT_MONO_LDPTR: { + // Pre-load locals for the store op + builder.local("pLocals"); + + // frame->imethod->data_items [ip [2]] + const data = get_imethod_data(frame, getArgU16(ip, 2)); + builder.i32_const(data); + + append_stloc_tail(builder, getArgU16(ip, 1), WasmOpcode.i32_store); + break; + } + + case MintOpcode.MINT_CPOBJ_VT: { + const klass = get_imethod_data(frame, getArgU16(ip, 3)); + append_ldloc(builder, getArgU16(ip, 1), WasmOpcode.i32_load); + append_ldloc(builder, getArgU16(ip, 2), WasmOpcode.i32_load); + builder.i32_const(klass); + builder.callImport("value_copy"); + break; + } + case MintOpcode.MINT_LDOBJ_VT: { + const size = getArgU16(ip, 3); + append_ldloca(builder, getArgU16(ip, 1)); + append_ldloc_cknull(builder, getArgU16(ip, 2), ip, true); + append_memmove_dest_src(builder, size); + break; + } + case MintOpcode.MINT_STOBJ_VT: { + const klass = get_imethod_data(frame, getArgU16(ip, 3)); + append_ldloc(builder, getArgU16(ip, 1), WasmOpcode.i32_load); + append_ldloca(builder, getArgU16(ip, 2)); + builder.i32_const(klass); + builder.callImport("value_copy"); + break; + } + + case MintOpcode.MINT_STRLEN: + builder.block(); + append_ldloca(builder, getArgU16(ip, 2)); + append_ldloca(builder, getArgU16(ip, 1)); + builder.callImport("strlen"); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.StringOperationFailed); + builder.endBlock(); + break; + case MintOpcode.MINT_GETCHR: + builder.block(); + append_ldloca(builder, getArgU16(ip, 2)); + append_ldloca(builder, getArgU16(ip, 3)); + append_ldloca(builder, getArgU16(ip, 1)); + builder.callImport("getchr"); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.StringOperationFailed); + builder.endBlock(); + break; + + /* + EMSCRIPTEN_KEEPALIVE int mono_jiterp_getitem_span ( + void **destination, MonoSpanOfVoid *span, int index, size_t element_size + ) { + */ + case MintOpcode.MINT_GETITEM_SPAN: + case MintOpcode.MINT_GETITEM_LOCALSPAN: + // FIXME + builder.block(); + // destination = &locals[1] + append_ldloca(builder, getArgU16(ip, 1)); + if (opcode === MintOpcode.MINT_GETITEM_SPAN) { + // span = (MonoSpanOfVoid *)locals[2] + append_ldloc(builder, getArgU16(ip, 2), WasmOpcode.i32_load); + } else { + // span = (MonoSpanOfVoid)locals[2] + append_ldloca(builder, getArgU16(ip, 2)); + } + // index = locals[3] + append_ldloc(builder, getArgU16(ip, 3), WasmOpcode.i32_load); + // element_size = ip[4] + builder.i32_const(getArgI16(ip, 4)); + builder.callImport("getspan"); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.SpanOperationFailed); + builder.endBlock(); + break; + + case MintOpcode.MINT_INTRINS_SPAN_CTOR: { + // if (len < 0) bailout + builder.block(); + // int len = LOCAL_VAR (ip [3], gint32); + append_ldloc(builder, getArgU16(ip, 3), WasmOpcode.i32_load); + builder.local("math_rhs32", WasmOpcode.tee_local); + builder.i32_const(0); + builder.appendU8(WasmOpcode.i32_ge_s); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.SpanOperationFailed); + builder.endBlock(); + // gpointer span = locals + ip [1]; + append_ldloca(builder, getArgU16(ip, 1)); + builder.local("math_lhs32", WasmOpcode.tee_local); + // *(gpointer*)span = ptr; + append_ldloc(builder, getArgU16(ip, 2), WasmOpcode.i32_load); + builder.appendU8(WasmOpcode.i32_store); + builder.appendMemarg(0, 0); + // *(gint32*)((gpointer*)span + 1) = len; + builder.local("math_lhs32"); + builder.local("math_rhs32"); + builder.appendU8(WasmOpcode.i32_store); + builder.appendMemarg(4, 0); + break; + } + case MintOpcode.MINT_LD_DELEGATE_METHOD_PTR: { + append_ldloca(builder, getArgU16(ip, 1)); + append_ldloca(builder, getArgU16(ip, 2)); + builder.callImport("ld_del_ptr"); + break; + } + case MintOpcode.MINT_LDTSFLDA: { + append_ldloca(builder, getArgU16(ip, 1)); + // This value is unsigned but I32 is probably right + builder.i32_const(getArgI32(ip, 2)); + builder.callImport("ldtsflda"); + break; + } + case MintOpcode.MINT_INTRINS_UNSAFE_BYTE_OFFSET: + builder.local("pLocals"); + append_ldloc(builder, getArgU16(ip, 3), WasmOpcode.i32_load); + append_ldloc(builder, getArgU16(ip, 2), WasmOpcode.i32_load); + builder.appendU8(WasmOpcode.i32_sub); + append_stloc_tail(builder, getArgU16(ip, 1), WasmOpcode.i32_store); + break; + case MintOpcode.MINT_INTRINS_GET_TYPE: + builder.block(); + // dest, src + append_ldloca(builder, getArgU16(ip, 1)); + append_ldloca(builder, getArgU16(ip, 2)); + builder.callImport("gettype"); + // bailout if gettype failed + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.NullCheck); + builder.endBlock(); + break; + case MintOpcode.MINT_INTRINS_MEMORYMARSHAL_GETARRAYDATAREF: { + const offset = cwraps.mono_jiterp_get_offset_of_array_data(); + builder.local("pLocals"); + append_ldloc_cknull(builder, getArgU16(ip, 2), ip, true); + builder.i32_const(offset); + builder.appendU8(WasmOpcode.i32_add); + append_stloc_tail(builder, getArgU16(ip, 1), WasmOpcode.i32_store); + break; + } + + case MintOpcode.MINT_CASTCLASS: + case MintOpcode.MINT_ISINST: + case MintOpcode.MINT_CASTCLASS_COMMON: + case MintOpcode.MINT_ISINST_COMMON: + case MintOpcode.MINT_CASTCLASS_INTERFACE: + case MintOpcode.MINT_ISINST_INTERFACE: { + builder.block(); + // dest, src + append_ldloca(builder, getArgU16(ip, 1)); + append_ldloca(builder, getArgU16(ip, 2)); + // klass + builder.i32_const(get_imethod_data(frame, getArgU16(ip, 3))); + // opcode + builder.i32_const(opcode); + builder.callImport("cast"); + // bailout if cast operation failed + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.CastFailed); + builder.endBlock(); + break; + } + + case MintOpcode.MINT_BOX: + case MintOpcode.MINT_BOX_VT: { + // MonoVTable *vtable = (MonoVTable*)frame->imethod->data_items [ip [3]]; + builder.i32_const(get_imethod_data(frame, getArgU16(ip, 3))); + // dest, src + append_ldloca(builder, getArgU16(ip, 1)); + append_ldloca(builder, getArgU16(ip, 2)); + builder.i32_const(opcode === MintOpcode.MINT_BOX_VT ? 1 : 0); + builder.callImport("box"); + break; + } + case MintOpcode.MINT_UNBOX: { + builder.block(); + // MonoClass *c = (MonoClass*)frame->imethod->data_items [ip [3]]; + builder.i32_const(get_imethod_data(frame, getArgU16(ip, 3))); + // dest, src + append_ldloca(builder, getArgU16(ip, 1)); + append_ldloca(builder, getArgU16(ip, 2)); + builder.callImport("try_unbox"); + // If the unbox operation succeeded, continue, otherwise bailout + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.UnboxFailed); + builder.endBlock(); + break; + } + + case MintOpcode.MINT_NEWOBJ_INLINED: { + builder.block(); + // MonoObject *o = mono_gc_alloc_obj (vtable, m_class_get_instance_size (vtable->klass)); + append_ldloca(builder, getArgU16(ip, 1)); + builder.i32_const(get_imethod_data(frame, getArgU16(ip, 2))); + // LOCAL_VAR (ip [1], MonoObject*) = o; + builder.callImport("newobj_i"); + // If the newobj operation succeeded, continue, otherwise bailout + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.AllocFailed); + builder.endBlock(); + break; + } + + case MintOpcode.MINT_NEWOBJ_VT_INLINED: { + const ret_size = getArgU16(ip, 3); + // memset (this_vt, 0, ret_size); + append_ldloca(builder, getArgU16(ip, 2)); + append_memset_dest(builder, 0, ret_size); + // LOCAL_VAR (ip [1], gpointer) = this_vt; + builder.local("pLocals"); + append_ldloca(builder, getArgU16(ip, 2)); + append_stloc_tail(builder, getArgU16(ip, 1), WasmOpcode.i32_store); + break; + } + + case MintOpcode.MINT_NEWOBJ: + case MintOpcode.MINT_NEWOBJ_VT: + case MintOpcode.MINT_CALLVIRT_FAST: + case MintOpcode.MINT_CALL: { + if (countCallTargets) { + const targetImethod = get_imethod_data(frame, getArgU16(ip, 3)); + const targetMethod = getU32(targetImethod); + const count = callTargetCounts[targetMethod]; + if (typeof (count) === "number") + callTargetCounts[targetMethod] = count + 1; + else + callTargetCounts[targetMethod] = 1; + } + if (builder.branchTargets.size > 0) { + // We generate a bailout instead of aborting, because we don't want calls + // to abort the entire trace if we have branch support enabled - the call + // might be infrequently hit and as a result it's worth it to keep going. + append_bailout(builder, ip, BailoutReason.Call); + } else { + // We're in a block that executes unconditionally, and no branches have been + // executed before now so the trace will always need to bail out into the + // interpreter here. No point in compiling more. + ip = abort; + } + break; + } + + // TODO: Verify that this isn't worse. I think these may only show up in wrappers? + // case MintOpcode.MINT_JIT_CALL: + case MintOpcode.MINT_CALLI: + case MintOpcode.MINT_CALLI_NAT: + case MintOpcode.MINT_CALLI_NAT_DYNAMIC: + case MintOpcode.MINT_CALLI_NAT_FAST: + case MintOpcode.MINT_CALL_DELEGATE: + // See comments for MINT_CALL + if (builder.branchTargets.size > 0) { + append_bailout(builder, ip, + opcode == MintOpcode.MINT_CALL_DELEGATE + ? BailoutReason.CallDelegate + : BailoutReason.Call + ); + } else { + ip = abort; + } + break; + + case MintOpcode.MINT_THROW: + // As above, only abort if this throw happens unconditionally. + // Otherwise, it may be in a branch that is unlikely to execute + if (builder.branchTargets.size > 0) { + append_bailout(builder, ip, BailoutReason.Throw); + } else { + ip = abort; + } + break; + + case MintOpcode.MINT_ENDFINALLY: + // This one might make sense to partially implement, but the jump target + // is computed at runtime which would make it hard to figure out where + // we need to put branch targets. Not worth just doing a conditional + // bailout since finally blocks always run + ip = abort; + break; + + case MintOpcode.MINT_RETHROW: + case MintOpcode.MINT_PROF_EXIT: + case MintOpcode.MINT_PROF_EXIT_VOID: + ip = abort; + break; + + // Generating code for these is kind of complex due to the intersection of JS and int64, + // and it would bloat the implementation so we handle them all in C instead and match + // the interp implementation. Most of these are rare in runtime tests or browser bench + case MintOpcode.MINT_CONV_OVF_I4_I8: + case MintOpcode.MINT_CONV_OVF_U4_I8: + case MintOpcode.MINT_CONV_OVF_I4_U8: + case MintOpcode.MINT_CONV_OVF_I4_R8: + case MintOpcode.MINT_CONV_OVF_I4_R4: + case MintOpcode.MINT_CONV_OVF_U4_I4: + builder.block(); + // dest, src + append_ldloca(builder, getArgU16(ip, 1)); + append_ldloca(builder, getArgU16(ip, 2)); + builder.i32_const(opcode); + builder.callImport("conv_ovf"); + // If the conversion succeeded, continue, otherwise bailout + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.Overflow); // could be underflow but awkward to tell + builder.endBlock(); + break; + + default: + if ( + opname.startsWith("ret") + ) { + if ((builder.branchTargets.size > 0) || trapTraceErrors || builder.options.countBailouts) + append_bailout(builder, ip, BailoutReason.Return); + else + ip = abort; + } else if (opname.startsWith("ldc")) { + if (!emit_ldc(builder, ip, opcode)) + ip = abort; + } else if (opname.startsWith("mov")) { + if (!emit_mov(builder, ip, opcode)) + ip = abort; + } else if ( + // binops + (opcode >= MintOpcode.MINT_ADD_I4) && + (opcode <= MintOpcode.MINT_CLT_UN_R8) + ) { + if (!emit_binop(builder, ip, opcode)) + ip = abort; + } else if ( + unopTable[opcode] + ) { + if (!emit_unop(builder, ip, opcode)) + ip = abort; + } else if ( + relopbranchTable[opcode] + ) { + if (!emit_relop_branch(builder, ip, opcode)) + ip = abort; + } else if ( + opname.startsWith("stfld") || + opname.startsWith("ldfld") || + opname.startsWith("stsfld") || + opname.startsWith("ldsfld") + ) { + if (!emit_fieldop(builder, frame, ip, opcode)) + ip = abort; + } else if ( + opname.startsWith("stind") || + opname.startsWith("ldind") + ) { + if (!emit_indirectop(builder, ip, opcode)) + ip = abort; + } else if ( + // math intrinsics + (opcode >= MintOpcode.MINT_ASIN) && + (opcode <= MintOpcode.MINT_MAXF) + ) { + if (!emit_math_intrinsic(builder, ip, opcode)) + ip = abort; + } else if ( + (opcode >= MintOpcode.MINT_LDELEM_I) && + (opcode <= MintOpcode.MINT_LDLEN) + ) { + if (!emit_arrayop(builder, ip, opcode)) + ip = abort; + } else if ( + (opcode >= MintOpcode.MINT_BRFALSE_I4_SP) && + (opcode <= MintOpcode.MINT_BLT_UN_I8_IMM_SP) + ) { + // NOTE: This elseif comes last so that specific safepoint branch + // types can be handled by emit_branch or emit_relop_branch, + // to only perform a conditional bailout + // complex safepoint branches, just generate a bailout + if (builder.branchTargets.size > 0) + append_bailout(builder, ip, BailoutReason.ComplexBranch); + else + ip = abort; + } else { + ip = abort; + } + break; + } + + if (ip) { + if ((trace > 1) || traceOnError || traceOnRuntimeError || dumpTraces || instrumentedTraceId) + builder.traceBuf.push(`${(ip).toString(16)} ${opname}`); + + if (!is_dead_opcode) + result++; + + ip += (info[1] * 2); + if (ip < (endOfBody - 2)) + rip = ip; + // For debugging + if (emitPadding) + builder.appendU8(WasmOpcode.nop); + } else + record_abort(traceIp, _ip, traceName, opcode); + } + + if (emitPadding) + builder.appendU8(WasmOpcode.nop); + + // Ensure that if execution runs past the end of our last branch block, we + // update eip appropriately so that we will return the right ip + builder.ip_const(rip); + builder.local("eip", WasmOpcode.set_local); + + // We need to close any open blocks before generating our closing ret, + // because wasm would allow branching past the ret otherwise + while (builder.activeBlocks > 0) + builder.endBlock(); + + // Now we generate a ret at the end of the function body so it's Valid(tm) + // When branching is enabled, we will have potentially updated eip due to a + // branch and then executed forward without ever finding it, so we want to + // return the branch target and ensure that the interpreter starts running + // from there. + builder.local("eip"); + builder.appendU8(WasmOpcode.return_); + + return result; +} + +function append_ldloc (builder: WasmBuilder, offset: number, opcode: WasmOpcode) { + builder.local("pLocals"); + builder.appendU8(opcode); + // stackval is 8 bytes, but pLocals might not be 8 byte aligned so we use 4 + // wasm spec prohibits alignment higher than natural alignment, just to be annoying + const alignment = (opcode > WasmOpcode.f64_load) ? 0 : 2; + builder.appendMemarg(offset, alignment); +} + +// You need to have pushed pLocals onto the stack *before* the value you intend to store +function append_stloc_tail (builder: WasmBuilder, offset: number, opcode: WasmOpcode) { + builder.appendU8(opcode); + // stackval is 8 bytes, but pLocals might not be 8 byte aligned so we use 4 + // wasm spec prohibits alignment higher than natural alignment, just to be annoying + const alignment = (opcode > WasmOpcode.f64_store) ? 0 : 2; + builder.appendMemarg(offset, alignment); +} + +function append_ldloca (builder: WasmBuilder, localOffset: number) { + builder.lea("pLocals", localOffset); +} + +function append_memset_local (builder: WasmBuilder, localOffset: number, value: number, count: number) { + // spec: pop n, pop val, pop d, fill from d[0] to d[n] with value val + append_ldloca(builder, localOffset); + append_memset_dest(builder, value, count); +} + +function append_memmove_local_local (builder: WasmBuilder, destLocalOffset: number, sourceLocalOffset: number, count: number) { + // spec: pop n, pop s, pop d, copy n bytes from s to d + append_ldloca(builder, destLocalOffset); + append_ldloca(builder, sourceLocalOffset); + append_memmove_dest_src(builder, count); +} + +// Loads the specified i32 value and bails out of it is null. Does not leave it on the stack. +function append_local_null_check (builder: WasmBuilder, localOffset: number, ip: MintOpcodePtr) { + builder.block(); + append_ldloc(builder, localOffset, WasmOpcode.i32_load); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.NullCheck); + builder.endBlock(); +} + +// Loads the specified i32 value and then bails out if it is null, leaving it in the cknull_ptr local. +function append_ldloc_cknull (builder: WasmBuilder, localOffset: number, ip: MintOpcodePtr, leaveOnStack: boolean) { + builder.block(); + append_ldloc(builder, localOffset, WasmOpcode.i32_load); + builder.local("cknull_ptr", WasmOpcode.tee_local); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.NullCheck); + builder.endBlock(); + if (leaveOnStack) + builder.local("cknull_ptr"); +} + +const ldcTable : { [opcode: number]: [WasmOpcode, number] } = { + [MintOpcode.MINT_LDC_I4_M1]: [WasmOpcode.i32_const, -1], + [MintOpcode.MINT_LDC_I4_0]: [WasmOpcode.i32_const, 0 ], + [MintOpcode.MINT_LDC_I4_1]: [WasmOpcode.i32_const, 1 ], + [MintOpcode.MINT_LDC_I4_2]: [WasmOpcode.i32_const, 2 ], + [MintOpcode.MINT_LDC_I4_3]: [WasmOpcode.i32_const, 3 ], + [MintOpcode.MINT_LDC_I4_4]: [WasmOpcode.i32_const, 4 ], + [MintOpcode.MINT_LDC_I4_5]: [WasmOpcode.i32_const, 5 ], + [MintOpcode.MINT_LDC_I4_6]: [WasmOpcode.i32_const, 6 ], + [MintOpcode.MINT_LDC_I4_7]: [WasmOpcode.i32_const, 7 ], + [MintOpcode.MINT_LDC_I4_8]: [WasmOpcode.i32_const, 8 ], +}; + +function emit_ldc (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + let storeType = WasmOpcode.i32_store; + + const tableEntry = ldcTable[opcode]; + if (tableEntry) { + builder.local("pLocals"); + builder.appendU8(tableEntry[0]); + builder.appendLeb(tableEntry[1]); + } else { + switch (opcode) { + case MintOpcode.MINT_LDC_I4_S: + builder.local("pLocals"); + builder.i32_const(getArgI16(ip, 2)); + break; + case MintOpcode.MINT_LDC_I4: + builder.local("pLocals"); + builder.i32_const(getArgI32(ip, 2)); + break; + case MintOpcode.MINT_LDC_I8_0: + builder.local("pLocals"); + builder.i52_const(0); + storeType = WasmOpcode.i64_store; + break; + case MintOpcode.MINT_LDC_I8: + builder.local("pLocals"); + builder.appendU8(WasmOpcode.i64_const); + builder.appendLebRef(ip + (2 * 2), true); + storeType = WasmOpcode.i64_store; + break; + case MintOpcode.MINT_LDC_I8_S: + builder.local("pLocals"); + builder.i52_const(getArgI16(ip, 2)); + storeType = WasmOpcode.i64_store; + break; + case MintOpcode.MINT_LDC_R4: + builder.local("pLocals"); + builder.appendU8(WasmOpcode.f32_const); + builder.appendF32(getArgF32(ip, 2)); + storeType = WasmOpcode.f32_store; + break; + case MintOpcode.MINT_LDC_R8: + builder.local("pLocals"); + builder.appendU8(WasmOpcode.f64_const); + builder.appendF64(getArgF64(ip, 2)); + storeType = WasmOpcode.f64_store; + break; + default: + return false; + } + } + + // spec: pop c, pop i, i[offset]=c + builder.appendU8(storeType); + // These are constants being stored into locals and are always at least 4 bytes + // so we can use a 4 byte alignment (8 would be nice if we could guarantee + // that locals are 8-byte aligned) + builder.appendMemarg(getArgU16(ip, 1), 2); + + return true; +} + +function emit_mov (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + let loadOp = WasmOpcode.i32_load, storeOp = WasmOpcode.i32_store; + switch (opcode) { + case MintOpcode.MINT_MOV_I4_I1: + loadOp = WasmOpcode.i32_load8_s; + break; + case MintOpcode.MINT_MOV_I4_U1: + loadOp = WasmOpcode.i32_load8_u; + break; + case MintOpcode.MINT_MOV_I4_I2: + loadOp = WasmOpcode.i32_load16_s; + break; + case MintOpcode.MINT_MOV_I4_U2: + loadOp = WasmOpcode.i32_load16_u; + break; + case MintOpcode.MINT_MOV_1: + loadOp = WasmOpcode.i32_load8_u; + storeOp = WasmOpcode.i32_store8; + break; + case MintOpcode.MINT_MOV_2: + loadOp = WasmOpcode.i32_load16_u; + storeOp = WasmOpcode.i32_store16; + break; + case MintOpcode.MINT_MOV_4: + break; + case MintOpcode.MINT_MOV_8: + loadOp = WasmOpcode.i64_load; + storeOp = WasmOpcode.i64_store; + break; + case MintOpcode.MINT_MOV_VT: { + const sizeBytes = getArgU16(ip, 3); + append_memmove_local_local(builder, getArgU16(ip, 1), getArgU16(ip, 2), sizeBytes); + return true; + } + case MintOpcode.MINT_MOV_8_2: + append_memmove_local_local(builder, getArgU16(ip, 1), getArgU16(ip, 2), 8); + append_memmove_local_local(builder, getArgU16(ip, 3), getArgU16(ip, 4), 8); + return true; + case MintOpcode.MINT_MOV_8_3: + append_memmove_local_local(builder, getArgU16(ip, 1), getArgU16(ip, 2), 8); + append_memmove_local_local(builder, getArgU16(ip, 3), getArgU16(ip, 4), 8); + append_memmove_local_local(builder, getArgU16(ip, 5), getArgU16(ip, 6), 8); + return true; + case MintOpcode.MINT_MOV_8_4: + append_memmove_local_local(builder, getArgU16(ip, 1), getArgU16(ip, 2), 8); + append_memmove_local_local(builder, getArgU16(ip, 3), getArgU16(ip, 4), 8); + append_memmove_local_local(builder, getArgU16(ip, 5), getArgU16(ip, 6), 8); + append_memmove_local_local(builder, getArgU16(ip, 7), getArgU16(ip, 8), 8); + return true; + default: + return false; + } + + // i + builder.local("pLocals"); + + // c = LOCAL_VAR (ip [2], argtype2) + append_ldloc(builder, getArgU16(ip, 2), loadOp); + append_stloc_tail(builder, getArgU16(ip, 1), storeOp); + + return true; +} + +let _offset_of_vtable_initialized_flag = 0; + +function get_offset_of_vtable_initialized_flag () { + if (!_offset_of_vtable_initialized_flag) { + // Manually calculating this by reading the code did not yield the correct result, + // so we ask the compiler (at runtime) + _offset_of_vtable_initialized_flag = cwraps.mono_jiterp_get_offset_of_vtable_initialized_flag(); + } + return _offset_of_vtable_initialized_flag; +} + +function append_vtable_initialize (builder: WasmBuilder, pVtable: NativePointer, ip: MintOpcodePtr) { + // TODO: Actually initialize the vtable instead of just checking and bailing out? + builder.block(); + // FIXME: This will prevent us from reusing traces between runs since the vtables can move + builder.i32_const(pVtable + get_offset_of_vtable_initialized_flag()); + builder.appendU8(WasmOpcode.i32_load8_u); + builder.appendMemarg(0, 0); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.VtableNotInitialized); + builder.endBlock(); +} + +function emit_fieldop ( + builder: WasmBuilder, frame: NativePointer, + ip: MintOpcodePtr, opcode: MintOpcode +) : boolean { + const isLoad = ( + (opcode >= MintOpcode.MINT_LDFLD_I1) && + (opcode <= MintOpcode.MINT_LDFLDA_UNSAFE) + ) || ( + (opcode >= MintOpcode.MINT_LDSFLD_I1) && + (opcode <= MintOpcode.MINT_LDSFLD_W) + ); + + const isStatic = (opcode >= MintOpcode.MINT_LDSFLD_I1) && + (opcode <= MintOpcode.MINT_LDSFLDA); + + const objectOffset = isStatic ? 0 : getArgU16(ip, isLoad ? 2 : 1), + valueOffset = getArgU16(ip, isLoad || isStatic ? 1 : 2), + offsetBytes = isStatic ? 0 : getArgU16(ip, 3), + pVtable = isStatic ? get_imethod_data(frame, getArgU16(ip, 2)) : 0, + pStaticData = isStatic ? get_imethod_data(frame, getArgU16(ip, 3)) : 0; + + if (isStatic) { + /* + if (instrumentedTraceId) { + console.log(`${instrumentedTraces[instrumentedTraceId].name} ${OpcodeInfo[opcode][0]} vtable=${pVtable.toString(16)} pStaticData=${pStaticData.toString(16)}`); + builder.i32_const(pVtable); + builder.i32_const(pStaticData); + builder.callImport("trace_args"); + } + */ + append_vtable_initialize(builder, pVtable, ip); + } else if (opcode !== MintOpcode.MINT_LDFLDA_UNSAFE) { + append_ldloc_cknull(builder, objectOffset, ip, false); + } + + let setter = WasmOpcode.i32_store, + getter = WasmOpcode.i32_load; + + switch (opcode) { + case MintOpcode.MINT_LDFLD_I1: + case MintOpcode.MINT_LDSFLD_I1: + getter = WasmOpcode.i32_load8_s; + break; + case MintOpcode.MINT_LDFLD_U1: + case MintOpcode.MINT_LDSFLD_U1: + getter = WasmOpcode.i32_load8_u; + break; + case MintOpcode.MINT_LDFLD_I2: + case MintOpcode.MINT_LDSFLD_I2: + getter = WasmOpcode.i32_load16_s; + break; + case MintOpcode.MINT_LDFLD_U2: + case MintOpcode.MINT_LDSFLD_U2: + getter = WasmOpcode.i32_load16_u; + break; + case MintOpcode.MINT_LDFLD_O: + case MintOpcode.MINT_LDSFLD_O: + case MintOpcode.MINT_STFLD_I4: + case MintOpcode.MINT_STSFLD_I4: + case MintOpcode.MINT_LDFLD_I4: + case MintOpcode.MINT_LDSFLD_I4: + // default + break; + // FIXME: These cause grisu3 to break? + case MintOpcode.MINT_STFLD_R4: + case MintOpcode.MINT_STSFLD_R4: + case MintOpcode.MINT_LDFLD_R4: + case MintOpcode.MINT_LDSFLD_R4: + getter = WasmOpcode.f32_load; + setter = WasmOpcode.f32_store; + break; + case MintOpcode.MINT_STFLD_R8: + case MintOpcode.MINT_STSFLD_R8: + case MintOpcode.MINT_LDFLD_R8: + case MintOpcode.MINT_LDSFLD_R8: + getter = WasmOpcode.f64_load; + setter = WasmOpcode.f64_store; + break; + case MintOpcode.MINT_STFLD_I1: + case MintOpcode.MINT_STSFLD_I1: + case MintOpcode.MINT_STFLD_U1: + case MintOpcode.MINT_STSFLD_U1: + setter = WasmOpcode.i32_store8; + break; + case MintOpcode.MINT_STFLD_I2: + case MintOpcode.MINT_STSFLD_I2: + case MintOpcode.MINT_STFLD_U2: + case MintOpcode.MINT_STSFLD_U2: + setter = WasmOpcode.i32_store16; + break; + case MintOpcode.MINT_LDFLD_I8: + case MintOpcode.MINT_LDSFLD_I8: + case MintOpcode.MINT_STFLD_I8: + case MintOpcode.MINT_STSFLD_I8: + getter = WasmOpcode.i64_load; + setter = WasmOpcode.i64_store; + break; + case MintOpcode.MINT_STFLD_O: + case MintOpcode.MINT_STSFLD_O: + // dest + if (isStatic) { + builder.i32_const(pStaticData); + } else { + builder.local("cknull_ptr"); + builder.i32_const(offsetBytes); + builder.appendU8(WasmOpcode.i32_add); + } + // src + append_ldloca(builder, getArgU16(ip, 2)); + builder.callImport("copy_pointer"); + return true; + case MintOpcode.MINT_LDFLD_VT: + case MintOpcode.MINT_LDSFLD_VT: { + const sizeBytes = getArgU16(ip, 4); + // dest + append_ldloca(builder, valueOffset); + // src + if (isStatic) { + builder.i32_const(pStaticData); + } else { + builder.local("cknull_ptr"); + builder.i32_const(offsetBytes); + builder.appendU8(WasmOpcode.i32_add); + } + append_memmove_dest_src(builder, sizeBytes); + return true; + } + case MintOpcode.MINT_STFLD_VT: { + const klass = get_imethod_data(frame, getArgU16(ip, 4)); + // dest = (char*)o + ip [3] + builder.local("cknull_ptr"); + builder.i32_const(offsetBytes); + builder.appendU8(WasmOpcode.i32_add); + // src = locals + ip [2] + append_ldloca(builder, valueOffset); + builder.i32_const(klass); + builder.callImport("value_copy"); + return true; + } + case MintOpcode.MINT_STFLD_VT_NOREF: { + const sizeBytes = getArgU16(ip, 4); + // dest + if (isStatic) { + builder.i32_const(pStaticData); + } else { + builder.local("cknull_ptr"); + builder.i32_const(offsetBytes); + builder.appendU8(WasmOpcode.i32_add); + } + // src + append_ldloca(builder, valueOffset); + append_memmove_dest_src(builder, sizeBytes); + return true; + } + case MintOpcode.MINT_LDFLDA_UNSAFE: + case MintOpcode.MINT_LDFLDA: + case MintOpcode.MINT_LDSFLDA: + builder.local("pLocals"); + if (isStatic) { + builder.i32_const(pStaticData); + } else { + // cknull_ptr isn't always initialized here + append_ldloc(builder, objectOffset, WasmOpcode.i32_load); + builder.i32_const(offsetBytes); + builder.appendU8(WasmOpcode.i32_add); + } + append_stloc_tail(builder, valueOffset, setter); + return true; + default: + return false; + } + + if (isLoad) + builder.local("pLocals"); + + if (isStatic) { + builder.i32_const(pStaticData); + if (isLoad) { + builder.appendU8(getter); + builder.appendMemarg(offsetBytes, 0); + append_stloc_tail(builder, valueOffset, setter); + return true; + } else { + append_ldloc(builder, valueOffset, getter); + builder.appendU8(setter); + builder.appendMemarg(offsetBytes, 0); + return true; + } + } else { + builder.local("cknull_ptr"); + + /* + if (instrumentedTraceId) { + builder.local("cknull_ptr"); + append_ldloca(builder, valueOffset); + builder.callImport("trace_args"); + } + */ + + if (isLoad) { + builder.appendU8(getter); + builder.appendMemarg(offsetBytes, 0); + append_stloc_tail(builder, valueOffset, setter); + return true; + } else { + append_ldloc(builder, valueOffset, getter); + builder.appendU8(setter); + builder.appendMemarg(offsetBytes, 0); + return true; + } + } +} + +// operator, loadOperator, storeOperator +type OpRec3 = [WasmOpcode, WasmOpcode, WasmOpcode]; +// operator, lhsLoadOperator, rhsLoadOperator, storeOperator +type OpRec4 = [WasmOpcode, WasmOpcode, WasmOpcode, WasmOpcode]; + +// thanks for making this as complex as possible, typescript +const unopTable : { [opcode: number]: OpRec3 | undefined } = { + [MintOpcode.MINT_CEQ0_I4]: [WasmOpcode.i32_eqz, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_ADD1_I4]: [WasmOpcode.i32_add, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SUB1_I4]: [WasmOpcode.i32_sub, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_NEG_I4]: [WasmOpcode.i32_sub, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_NOT_I4]: [WasmOpcode.i32_xor, WasmOpcode.i32_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_ADD1_I8]: [WasmOpcode.i64_add, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_SUB1_I8]: [WasmOpcode.i64_sub, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_NEG_I8]: [WasmOpcode.i64_sub, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_NOT_I8]: [WasmOpcode.i64_xor, WasmOpcode.i64_load, WasmOpcode.i64_store], + + [MintOpcode.MINT_ADD_I4_IMM]: [WasmOpcode.i32_add, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_MUL_I4_IMM]: [WasmOpcode.i32_mul, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_ADD_I8_IMM]: [WasmOpcode.i64_add, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_MUL_I8_IMM]: [WasmOpcode.i64_mul, WasmOpcode.i64_load, WasmOpcode.i64_store], + + [MintOpcode.MINT_NEG_R4]: [WasmOpcode.f32_neg, WasmOpcode.f32_load, WasmOpcode.f32_store], + [MintOpcode.MINT_NEG_R8]: [WasmOpcode.f64_neg, WasmOpcode.f64_load, WasmOpcode.f64_store], + + [MintOpcode.MINT_CONV_R4_I4]: [WasmOpcode.f32_convert_s_i32, WasmOpcode.i32_load, WasmOpcode.f32_store], + [MintOpcode.MINT_CONV_R8_I4]: [WasmOpcode.f64_convert_s_i32, WasmOpcode.i32_load, WasmOpcode.f64_store], + [MintOpcode.MINT_CONV_R4_I8]: [WasmOpcode.f32_convert_s_i64, WasmOpcode.i64_load, WasmOpcode.f32_store], + [MintOpcode.MINT_CONV_R8_I8]: [WasmOpcode.f64_convert_s_i64, WasmOpcode.i64_load, WasmOpcode.f64_store], + [MintOpcode.MINT_CONV_R8_R4]: [WasmOpcode.f64_promote_f32, WasmOpcode.f32_load, WasmOpcode.f64_store], + [MintOpcode.MINT_CONV_R4_R8]: [WasmOpcode.f32_demote_f64, WasmOpcode.f64_load, WasmOpcode.f32_store], + + [MintOpcode.MINT_CONV_I4_R4]: [WasmOpcode.i32_trunc_s_f32, WasmOpcode.f32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_I8_R4]: [WasmOpcode.i64_trunc_s_f32, WasmOpcode.f32_load, WasmOpcode.i64_store], + [MintOpcode.MINT_CONV_I4_R8]: [WasmOpcode.i32_trunc_s_f64, WasmOpcode.f64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_I8_R8]: [WasmOpcode.i64_trunc_s_f64, WasmOpcode.f64_load, WasmOpcode.i64_store], + + [MintOpcode.MINT_CONV_I8_I4]: [WasmOpcode.nop, WasmOpcode.i64_load32_s, WasmOpcode.i64_store], + [MintOpcode.MINT_CONV_I8_U4]: [WasmOpcode.nop, WasmOpcode.i64_load32_u, WasmOpcode.i64_store], + + [MintOpcode.MINT_CONV_U1_I4]: [WasmOpcode.i32_and, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_U2_I4]: [WasmOpcode.i32_and, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_I1_I4]: [WasmOpcode.i32_shr_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_I2_I4]: [WasmOpcode.i32_shr_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_CONV_U1_I8]: [WasmOpcode.i32_and, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_U2_I8]: [WasmOpcode.i32_and, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_I1_I8]: [WasmOpcode.i32_shr_s, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CONV_I2_I8]: [WasmOpcode.i32_shr_s, WasmOpcode.i64_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_SHL_I4_IMM]: [WasmOpcode.i32_shl, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SHL_I8_IMM]: [WasmOpcode.i64_shl, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_SHR_I4_IMM]: [WasmOpcode.i32_shr_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SHR_I8_IMM]: [WasmOpcode.i64_shr_s, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_SHR_UN_I4_IMM]: [WasmOpcode.i32_shr_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SHR_UN_I8_IMM]: [WasmOpcode.i64_shr_u, WasmOpcode.i64_load, WasmOpcode.i64_store], +}; + +const binopTable : { [opcode: number]: OpRec3 | OpRec4 | undefined } = { + [MintOpcode.MINT_ADD_I4]: [WasmOpcode.i32_add, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_ADD_OVF_I4]:[WasmOpcode.i32_add, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_ADD_OVF_UN_I4]:[WasmOpcode.i32_add, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SUB_I4]: [WasmOpcode.i32_sub, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_MUL_I4]: [WasmOpcode.i32_mul, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_MUL_OVF_I4]:[WasmOpcode.i32_mul, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_MUL_OVF_UN_I4]:[WasmOpcode.i32_mul,WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_DIV_I4]: [WasmOpcode.i32_div_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_DIV_UN_I4]: [WasmOpcode.i32_div_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_REM_I4]: [WasmOpcode.i32_rem_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_REM_UN_I4]: [WasmOpcode.i32_rem_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_AND_I4]: [WasmOpcode.i32_and, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_OR_I4]: [WasmOpcode.i32_or, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_XOR_I4]: [WasmOpcode.i32_xor, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SHL_I4]: [WasmOpcode.i32_shl, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SHR_I4]: [WasmOpcode.i32_shr_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_SHR_UN_I4]: [WasmOpcode.i32_shr_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_ADD_I8]: [WasmOpcode.i64_add, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_SUB_I8]: [WasmOpcode.i64_sub, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_MUL_I8]: [WasmOpcode.i64_mul, WasmOpcode.i64_load, WasmOpcode.i64_store], + // Overflow check is too hard to do for int64 right now + /* + [MintOpcode.MINT_DIV_I8]: [WasmOpcode.i64_div_s, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_REM_I8]: [WasmOpcode.i64_rem_s, WasmOpcode.i64_load, WasmOpcode.i64_store], + */ + [MintOpcode.MINT_DIV_UN_I8]: [WasmOpcode.i64_div_u, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_REM_UN_I8]: [WasmOpcode.i64_rem_u, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_AND_I8]: [WasmOpcode.i64_and, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_OR_I8]: [WasmOpcode.i64_or, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_XOR_I8]: [WasmOpcode.i64_xor, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_SHL_I8]: [WasmOpcode.i64_shl, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_SHR_I8]: [WasmOpcode.i64_shr_s, WasmOpcode.i64_load, WasmOpcode.i64_store], + [MintOpcode.MINT_SHR_UN_I8]: [WasmOpcode.i64_shr_u, WasmOpcode.i64_load, WasmOpcode.i64_store], + + [MintOpcode.MINT_ADD_R4]: [WasmOpcode.f32_add, WasmOpcode.f32_load, WasmOpcode.f32_store], + [MintOpcode.MINT_SUB_R4]: [WasmOpcode.f32_sub, WasmOpcode.f32_load, WasmOpcode.f32_store], + [MintOpcode.MINT_MUL_R4]: [WasmOpcode.f32_mul, WasmOpcode.f32_load, WasmOpcode.f32_store], + [MintOpcode.MINT_DIV_R4]: [WasmOpcode.f32_div, WasmOpcode.f32_load, WasmOpcode.f32_store], + + [MintOpcode.MINT_ADD_R8]: [WasmOpcode.f64_add, WasmOpcode.f64_load, WasmOpcode.f64_store], + [MintOpcode.MINT_SUB_R8]: [WasmOpcode.f64_sub, WasmOpcode.f64_load, WasmOpcode.f64_store], + [MintOpcode.MINT_MUL_R8]: [WasmOpcode.f64_mul, WasmOpcode.f64_load, WasmOpcode.f64_store], + [MintOpcode.MINT_DIV_R8]: [WasmOpcode.f64_div, WasmOpcode.f64_load, WasmOpcode.f64_store], + + [MintOpcode.MINT_CEQ_I4]: [WasmOpcode.i32_eq, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CNE_I4]: [WasmOpcode.i32_ne, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLT_I4]: [WasmOpcode.i32_lt_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGT_I4]: [WasmOpcode.i32_gt_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLE_I4]: [WasmOpcode.i32_le_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGE_I4]: [WasmOpcode.i32_ge_s, WasmOpcode.i32_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_CLT_UN_I4]: [WasmOpcode.i32_lt_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGT_UN_I4]: [WasmOpcode.i32_gt_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLE_UN_I4]: [WasmOpcode.i32_le_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGE_UN_I4]: [WasmOpcode.i32_ge_u, WasmOpcode.i32_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_CEQ_I8]: [WasmOpcode.i64_eq, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CNE_I8]: [WasmOpcode.i64_ne, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLT_I8]: [WasmOpcode.i64_lt_s, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGT_I8]: [WasmOpcode.i64_gt_s, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLE_I8]: [WasmOpcode.i64_le_s, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGE_I8]: [WasmOpcode.i64_ge_s, WasmOpcode.i64_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_CLT_UN_I8]: [WasmOpcode.i64_lt_u, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGT_UN_I8]: [WasmOpcode.i64_gt_u, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLE_UN_I8]: [WasmOpcode.i64_le_u, WasmOpcode.i64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGE_UN_I8]: [WasmOpcode.i64_ge_u, WasmOpcode.i64_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_CEQ_R4]: [WasmOpcode.f32_eq, WasmOpcode.f32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CNE_R4]: [WasmOpcode.f32_ne, WasmOpcode.f32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLT_R4]: [WasmOpcode.f32_lt, WasmOpcode.f32_load, WasmOpcode.i32_store], + // FIXME: What are these, semantically? + [MintOpcode.MINT_CLT_UN_R4]: [WasmOpcode.f32_lt, WasmOpcode.f32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGT_R4]: [WasmOpcode.f32_gt, WasmOpcode.f32_load, WasmOpcode.i32_store], + // FIXME + [MintOpcode.MINT_CGT_UN_R4]: [WasmOpcode.f32_gt, WasmOpcode.f32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLE_R4]: [WasmOpcode.f32_le, WasmOpcode.f32_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGE_R4]: [WasmOpcode.f32_ge, WasmOpcode.f32_load, WasmOpcode.i32_store], + + [MintOpcode.MINT_CEQ_R8]: [WasmOpcode.f64_eq, WasmOpcode.f64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CNE_R8]: [WasmOpcode.f64_ne, WasmOpcode.f64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLT_R8]: [WasmOpcode.f64_lt, WasmOpcode.f64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGT_R8]: [WasmOpcode.f64_gt, WasmOpcode.f64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CLE_R8]: [WasmOpcode.f64_le, WasmOpcode.f64_load, WasmOpcode.i32_store], + [MintOpcode.MINT_CGE_R8]: [WasmOpcode.f64_ge, WasmOpcode.f64_load, WasmOpcode.i32_store], + + // FIXME: unordered float comparisons +}; + +const relopbranchTable : { [opcode: number]: [comparisonOpcode: MintOpcode, immediateOpcode: WasmOpcode | false, isSafepoint: boolean] | MintOpcode | undefined } = { + [MintOpcode.MINT_BEQ_I4_S]: MintOpcode.MINT_CEQ_I4, + [MintOpcode.MINT_BNE_UN_I4_S]: MintOpcode.MINT_CNE_I4, + [MintOpcode.MINT_BGT_I4_S]: MintOpcode.MINT_CGT_I4, + [MintOpcode.MINT_BGT_UN_I4_S]: MintOpcode.MINT_CGT_UN_I4, + [MintOpcode.MINT_BLT_I4_S]: MintOpcode.MINT_CLT_I4, + [MintOpcode.MINT_BLT_UN_I4_S]: MintOpcode.MINT_CLT_UN_I4, + [MintOpcode.MINT_BGE_I4_S]: MintOpcode.MINT_CGE_I4, + [MintOpcode.MINT_BGE_UN_I4_S]: MintOpcode.MINT_CGE_UN_I4, + [MintOpcode.MINT_BLE_I4_S]: MintOpcode.MINT_CLE_I4, + [MintOpcode.MINT_BLE_UN_I4_S]: MintOpcode.MINT_CLE_UN_I4, + + [MintOpcode.MINT_BEQ_I4_SP]: [MintOpcode.MINT_CEQ_I4, false, true], + [MintOpcode.MINT_BNE_UN_I4_SP]: [MintOpcode.MINT_CNE_I4, false, true], + [MintOpcode.MINT_BGT_I4_SP]: [MintOpcode.MINT_CGT_I4, false, true], + [MintOpcode.MINT_BGT_UN_I4_SP]: [MintOpcode.MINT_CGT_UN_I4, false, true], + [MintOpcode.MINT_BLT_I4_SP]: [MintOpcode.MINT_CLT_I4, false, true], + [MintOpcode.MINT_BLT_UN_I4_SP]: [MintOpcode.MINT_CLT_UN_I4, false, true], + [MintOpcode.MINT_BGE_I4_SP]: [MintOpcode.MINT_CGE_I4, false, true], + [MintOpcode.MINT_BGE_UN_I4_SP]: [MintOpcode.MINT_CGE_UN_I4, false, true], + [MintOpcode.MINT_BLE_I4_SP]: [MintOpcode.MINT_CLE_I4, false, true], + [MintOpcode.MINT_BLE_UN_I4_SP]: [MintOpcode.MINT_CLE_UN_I4, false, true], + + [MintOpcode.MINT_BEQ_I4_IMM_SP]: [MintOpcode.MINT_CEQ_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BNE_UN_I4_IMM_SP]: [MintOpcode.MINT_CNE_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BGT_I4_IMM_SP]: [MintOpcode.MINT_CGT_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BGT_UN_I4_IMM_SP]: [MintOpcode.MINT_CGT_UN_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BLT_I4_IMM_SP]: [MintOpcode.MINT_CLT_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BLT_UN_I4_IMM_SP]: [MintOpcode.MINT_CLT_UN_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BGE_I4_IMM_SP]: [MintOpcode.MINT_CGE_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BGE_UN_I4_IMM_SP]: [MintOpcode.MINT_CGE_UN_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BLE_I4_IMM_SP]: [MintOpcode.MINT_CLE_I4, WasmOpcode.i32_const, true], + [MintOpcode.MINT_BLE_UN_I4_IMM_SP]: [MintOpcode.MINT_CLE_UN_I4, WasmOpcode.i32_const, true], + + [MintOpcode.MINT_BEQ_I8_S]: MintOpcode.MINT_CEQ_I8, + [MintOpcode.MINT_BNE_UN_I8_S]: MintOpcode.MINT_CNE_I8, + [MintOpcode.MINT_BGT_I8_S]: MintOpcode.MINT_CGT_I8, + [MintOpcode.MINT_BGT_UN_I8_S]: MintOpcode.MINT_CGT_UN_I8, + [MintOpcode.MINT_BLT_I8_S]: MintOpcode.MINT_CLT_I8, + [MintOpcode.MINT_BLT_UN_I8_S]: MintOpcode.MINT_CLT_UN_I8, + [MintOpcode.MINT_BGE_I8_S]: MintOpcode.MINT_CGE_I8, + [MintOpcode.MINT_BGE_UN_I8_S]: MintOpcode.MINT_CGE_UN_I8, + [MintOpcode.MINT_BLE_I8_S]: MintOpcode.MINT_CLE_I8, + [MintOpcode.MINT_BLE_UN_I8_S]: MintOpcode.MINT_CLE_UN_I8, + + [MintOpcode.MINT_BEQ_I8_IMM_SP]: [MintOpcode.MINT_CEQ_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BNE_UN_I8_IMM_SP]: [MintOpcode.MINT_CNE_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BGT_I8_IMM_SP]: [MintOpcode.MINT_CGT_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BGT_UN_I8_IMM_SP]: [MintOpcode.MINT_CGT_UN_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BLT_I8_IMM_SP]: [MintOpcode.MINT_CLT_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BLT_UN_I8_IMM_SP]: [MintOpcode.MINT_CLT_UN_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BGE_I8_IMM_SP]: [MintOpcode.MINT_CGE_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BGE_UN_I8_IMM_SP]: [MintOpcode.MINT_CGE_UN_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BLE_I8_IMM_SP]: [MintOpcode.MINT_CLE_I8, WasmOpcode.i64_const, true], + [MintOpcode.MINT_BLE_UN_I8_IMM_SP]: [MintOpcode.MINT_CLE_UN_I8, WasmOpcode.i64_const, true], + + [MintOpcode.MINT_BEQ_R4_S]: MintOpcode.MINT_CEQ_R4, + [MintOpcode.MINT_BNE_UN_R4_S]: MintOpcode.MINT_CNE_R4, + [MintOpcode.MINT_BGT_R4_S]: MintOpcode.MINT_CGT_R4, + [MintOpcode.MINT_BGT_UN_R4_S]: MintOpcode.MINT_CGT_UN_R4, + [MintOpcode.MINT_BLT_R4_S]: MintOpcode.MINT_CLT_R4, + [MintOpcode.MINT_BLT_UN_R4_S]: MintOpcode.MINT_CLT_UN_R4, + [MintOpcode.MINT_BGE_R4_S]: MintOpcode.MINT_CGE_R4, + // FIXME: No compare opcode for this + [MintOpcode.MINT_BGE_UN_R4_S]: MintOpcode.MINT_CGE_R4, + [MintOpcode.MINT_BLE_R4_S]: MintOpcode.MINT_CLE_R4, + // FIXME: No compare opcode for this + [MintOpcode.MINT_BLE_UN_R4_S]: MintOpcode.MINT_CLE_R4, + + [MintOpcode.MINT_BEQ_R8_S]: MintOpcode.MINT_CEQ_R8, + [MintOpcode.MINT_BNE_UN_R8_S]: MintOpcode.MINT_CNE_R8, + [MintOpcode.MINT_BGT_R8_S]: MintOpcode.MINT_CGT_R8, + [MintOpcode.MINT_BGT_UN_R8_S]: MintOpcode.MINT_CGT_UN_R8, + [MintOpcode.MINT_BLT_R8_S]: MintOpcode.MINT_CLT_R8, + [MintOpcode.MINT_BLT_UN_R8_S]: MintOpcode.MINT_CLT_UN_R8, + [MintOpcode.MINT_BGE_R8_S]: MintOpcode.MINT_CGE_R8, + // FIXME: No compare opcode for this + [MintOpcode.MINT_BGE_UN_R8_S]: MintOpcode.MINT_CGE_R8, + [MintOpcode.MINT_BLE_R8_S]: MintOpcode.MINT_CLE_R8, + // FIXME: No compare opcode for this + [MintOpcode.MINT_BLE_UN_R8_S]: MintOpcode.MINT_CLE_R8, +}; + +function emit_binop (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + // operands are popped right to left, which means you build the arg list left to right + let lhsLoadOp : WasmOpcode, rhsLoadOp : WasmOpcode, storeOp : WasmOpcode, + lhsVar = "math_lhs32", rhsVar = "math_rhs32", + info : OpRec3 | OpRec4 | undefined, + operandsCached = false; + + switch (opcode) { + case MintOpcode.MINT_REM_R4: + case MintOpcode.MINT_REM_R8: + return emit_math_intrinsic(builder, ip, opcode); + + default: { + info = binopTable[opcode]; + if (!info) + return false; + if (info.length > 3) { + lhsLoadOp = info[1]; + rhsLoadOp = info[2]; + storeOp = info[3]!; + } else { + lhsLoadOp = rhsLoadOp = info[1]; + storeOp = info[2]; + } + } + } + + switch (opcode) { + case MintOpcode.MINT_DIV_I4: + case MintOpcode.MINT_DIV_UN_I4: + case MintOpcode.MINT_DIV_UN_I8: + case MintOpcode.MINT_REM_I4: + case MintOpcode.MINT_REM_UN_I4: + case MintOpcode.MINT_REM_UN_I8: { + const is64 = (opcode === MintOpcode.MINT_DIV_UN_I8) || + (opcode === MintOpcode.MINT_REM_UN_I8); + lhsVar = is64 ? "math_lhs64" : "math_lhs32"; + rhsVar = is64 ? "math_rhs64" : "math_rhs32"; + + builder.block(); + append_ldloc(builder, getArgU16(ip, 2), lhsLoadOp); + builder.local(lhsVar, WasmOpcode.set_local); + append_ldloc(builder, getArgU16(ip, 3), rhsLoadOp); + builder.local(rhsVar, WasmOpcode.tee_local); + operandsCached = true; + // br_if requires an i32 so to do our divide by zero check on an i64 + // we do i64_eqz and then i32_eqz to invert the flag + if (is64) { + builder.appendU8(WasmOpcode.i64_eqz); + builder.appendU8(WasmOpcode.i32_eqz); + } + // If rhs is zero we want to bailout because it's a divide by zero. + // A nonzero divisor will cause us to skip past this bailout + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.DivideByZero); + builder.endBlock(); + + // Also perform overflow check for signed division operations + if ( + (opcode === MintOpcode.MINT_DIV_I4) || + (opcode === MintOpcode.MINT_REM_I4) + ) { + builder.block(); + builder.local(rhsVar); + // If rhs is -1 and lhs is MININT32 this is an overflow + builder.i32_const(-1); + builder.appendU8(WasmOpcode.i32_ne); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + // rhs was -1 since the previous br_if didn't execute. Now check lhs. + builder.local(lhsVar); + // G_MININT32 + // FIXME: Make sure the leb encoder can actually handle this + builder.i32_const(-2147483647-1); + builder.appendU8(WasmOpcode.i32_ne); + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.Overflow); + builder.endBlock(); + } + break; + } + case MintOpcode.MINT_DIV_I8: + // We have to check lhs against MININT64 which is not 52-bit safe + return false; + case MintOpcode.MINT_ADD_OVF_I4: + case MintOpcode.MINT_ADD_OVF_UN_I4: + case MintOpcode.MINT_MUL_OVF_I4: + case MintOpcode.MINT_MUL_OVF_UN_I4: + // Perform overflow check before the operation + append_ldloc(builder, getArgU16(ip, 2), lhsLoadOp); + builder.local(lhsVar, WasmOpcode.tee_local); + append_ldloc(builder, getArgU16(ip, 3), rhsLoadOp); + builder.local(rhsVar, WasmOpcode.tee_local); + builder.i32_const(opcode); + builder.callImport( + ( + (opcode === MintOpcode.MINT_ADD_OVF_UN_I4) || + (opcode === MintOpcode.MINT_MUL_OVF_UN_I4) + ) + ? "ckovr_u4" + : "ckovr_i4" + ); + builder.block(WasmValtype.void, WasmOpcode.if_); + append_bailout(builder, ip, BailoutReason.Overflow); + builder.endBlock(); + operandsCached = true; + break; + } + + // i + builder.local("pLocals"); + + // c = (lhs op rhs) + if (operandsCached) { + builder.local(lhsVar); + builder.local(rhsVar); + } else { + append_ldloc(builder, getArgU16(ip, 2), lhsLoadOp); + append_ldloc(builder, getArgU16(ip, 3), rhsLoadOp); + } + builder.appendU8(info[0]); + + append_stloc_tail(builder, getArgU16(ip, 1), storeOp); + + return true; +} + +function emit_unop (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + // operands are popped right to left, which means you build the arg list left to right + const info = unopTable[opcode]; + if (!info) + return false; + const loadOp = info[1]; + const storeOp = info[2]; + + // i + if ((opcode < MintOpcode.MINT_CONV_OVF_I1_I4) || + (opcode > MintOpcode.MINT_CONV_OVF_U8_R8)) + builder.local("pLocals"); + + // c = (op value) + switch (opcode) { + case MintOpcode.MINT_ADD1_I4: + case MintOpcode.MINT_SUB1_I4: + // We implement this as binary 'x +/- 1', the table already has i32_add so we just + // need to emit a 1 constant + append_ldloc(builder, getArgU16(ip, 2), loadOp); + builder.i32_const(1); + break; + case MintOpcode.MINT_NEG_I4: + // there's no negate operator so we generate '0 - x' + builder.i32_const(0); + append_ldloc(builder, getArgU16(ip, 2), loadOp); + break; + case MintOpcode.MINT_NOT_I4: + // there's no not operator so we generate 'x xor -1' + append_ldloc(builder, getArgU16(ip, 2), loadOp); + builder.i32_const(-1); + break; + + case MintOpcode.MINT_CONV_U1_I4: + case MintOpcode.MINT_CONV_U1_I8: + // For (unsigned char) cast of i32/i64 we do an & 255 + append_ldloc(builder, getArgU16(ip, 2), loadOp); + if (loadOp === WasmOpcode.i64_load) + builder.appendU8(WasmOpcode.i32_wrap_i64); + builder.i32_const(0xFF); + break; + case MintOpcode.MINT_CONV_U2_I4: + case MintOpcode.MINT_CONV_U2_I8: + // For (unsigned short) cast of i32/i64 we do an & 65535 + append_ldloc(builder, getArgU16(ip, 2), loadOp); + if (loadOp === WasmOpcode.i64_load) + builder.appendU8(WasmOpcode.i32_wrap_i64); + builder.i32_const(0xFFFF); + break; + case MintOpcode.MINT_CONV_I1_I4: + case MintOpcode.MINT_CONV_I1_I8: + // For (char) cast of i32 we do (val << 24) >> 24 + append_ldloc(builder, getArgU16(ip, 2), loadOp); + if (loadOp === WasmOpcode.i64_load) + builder.appendU8(WasmOpcode.i32_wrap_i64); + builder.i32_const(24); + builder.appendU8(WasmOpcode.i32_shl); + builder.i32_const(24); + break; + case MintOpcode.MINT_CONV_I2_I4: + case MintOpcode.MINT_CONV_I2_I8: + // For (char) cast of i32 we do (val << 16) >> 16 + append_ldloc(builder, getArgU16(ip, 2), loadOp); + if (loadOp === WasmOpcode.i64_load) + builder.appendU8(WasmOpcode.i32_wrap_i64); + builder.i32_const(16); + builder.appendU8(WasmOpcode.i32_shl); + builder.i32_const(16); + break; + + case MintOpcode.MINT_ADD1_I8: + case MintOpcode.MINT_SUB1_I8: + // We implement this as binary 'x +/- 1', the table already has i32_add so we just + // need to emit a 1 constant + append_ldloc(builder, getArgU16(ip, 2), loadOp); + builder.i52_const(1); + break; + case MintOpcode.MINT_NEG_I8: + // there's no negate operator so we generate '0 - x' + builder.i52_const(0); + append_ldloc(builder, getArgU16(ip, 2), loadOp); + break; + case MintOpcode.MINT_NOT_I8: + // there's no not operator so we generate 'x xor -1' + append_ldloc(builder, getArgU16(ip, 2), loadOp); + builder.i52_const(-1); + break; + + case MintOpcode.MINT_ADD_I4_IMM: + case MintOpcode.MINT_MUL_I4_IMM: + case MintOpcode.MINT_SHL_I4_IMM: + case MintOpcode.MINT_SHR_I4_IMM: + case MintOpcode.MINT_SHR_UN_I4_IMM: + append_ldloc(builder, getArgU16(ip, 2), loadOp); + builder.i32_const(getArgI16(ip, 3)); + break; + + case MintOpcode.MINT_ADD_I8_IMM: + case MintOpcode.MINT_MUL_I8_IMM: + case MintOpcode.MINT_SHL_I8_IMM: + case MintOpcode.MINT_SHR_I8_IMM: + case MintOpcode.MINT_SHR_UN_I8_IMM: + append_ldloc(builder, getArgU16(ip, 2), loadOp); + builder.i52_const(getArgI16(ip, 3)); + break; + + default: + append_ldloc(builder, getArgU16(ip, 2), loadOp); + break; + } + + if (info[0] !== WasmOpcode.nop) + builder.appendU8(info[0]); + + append_stloc_tail(builder, getArgU16(ip, 1), storeOp); + + return true; +} + +function emit_branch ( + builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode, displacement?: number +) : boolean { + const info = OpcodeInfo[opcode]; + const isSafepoint = (opcode >= MintOpcode.MINT_BRFALSE_I4_SP) && + (opcode <= MintOpcode.MINT_BLT_UN_I8_IMM_SP); + + // If the branch is taken we bail out to allow the interpreter to do it. + // So for brtrue, we want to do 'cond == 0' to produce a bailout only + // when the branch will be taken (by skipping the bailout in this block) + // When branches are enabled, instead we set eip and then break out of + // the current branch block and execution proceeds forward to find the + // branch target (if possible), bailing out at the end otherwise + switch (opcode) { + case MintOpcode.MINT_LEAVE: + case MintOpcode.MINT_LEAVE_S: + case MintOpcode.MINT_BR: + case MintOpcode.MINT_BR_S: { + displacement = ((opcode === MintOpcode.MINT_LEAVE) || (opcode === MintOpcode.MINT_BR)) + ? getArgI32(ip, 1) + : getArgI16(ip, 1); + if (traceBranchDisplacements) + console.log(`br.s @${ip} displacement=${displacement}`); + const destination = ip + (displacement * 2); + + if (displacement <= 0) { + // FIXME: If the displacement is negative, perform BACK_BRANCH_PROFILE + append_bailout(builder, destination, displacement > 0 ? BailoutReason.Branch : BailoutReason.BackwardBranch); + return true; + } + + // Simple branches are enabled and this is a forward branch. We + // don't need to wrap things in a block here, we can just exit + // the current branch block after updating eip + builder.ip_const(destination); + builder.local("eip", WasmOpcode.set_local); + builder.appendU8(WasmOpcode.br); + builder.appendULeb(0); + return true; + } + case MintOpcode.MINT_BRTRUE_I4_S: + case MintOpcode.MINT_BRFALSE_I4_S: + case MintOpcode.MINT_BRTRUE_I4_SP: + case MintOpcode.MINT_BRFALSE_I4_SP: + case MintOpcode.MINT_BRTRUE_I8_S: + case MintOpcode.MINT_BRFALSE_I8_S: { + const is64 = (opcode === MintOpcode.MINT_BRTRUE_I8_S) || + (opcode === MintOpcode.MINT_BRFALSE_I8_S); + // Wrap the conditional branch in a block so we can skip the + // actual branch at the end of it + builder.block(); + + displacement = getArgI16(ip, 2); + append_ldloc(builder, getArgU16(ip, 1), is64 ? WasmOpcode.i64_load : WasmOpcode.i32_load); + if ( + (opcode === MintOpcode.MINT_BRTRUE_I4_S) || + (opcode === MintOpcode.MINT_BRTRUE_I4_SP) + ) + builder.appendU8(WasmOpcode.i32_eqz); + else if (opcode === MintOpcode.MINT_BRTRUE_I8_S) + builder.appendU8(WasmOpcode.i64_eqz); + else if (opcode === MintOpcode.MINT_BRFALSE_I8_S) { + // do (i64 == 0) == 0 because br_if can only branch on an i32 operand + builder.appendU8(WasmOpcode.i64_eqz); + builder.appendU8(WasmOpcode.i32_eqz); + } + break; + } + default: { + // relop branches already had the branch condition loaded by the caller, + // so we don't need to load anything. After the condition was loaded, we + // treat it like a brtrue + if (relopbranchTable[opcode] === undefined) + throw new Error(`Unsupported relop branch opcode: ${opcode}`); + + if (info[1] !== 4) + throw new Error(`Unsupported long branch opcode: ${info[0]}`); + + builder.appendU8(WasmOpcode.i32_eqz); + break; + } + } + + if (!displacement) + throw new Error("Branch had no displacement"); + else if (traceBranchDisplacements) + console.log(`${info[0]} @${ip} displacement=${displacement}`); + + const destination = ip + (displacement * 2); + + // We generate a conditional branch that will skip past the rest of this + // tiny branch dispatch block to avoid performing the branch + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + + if (isSafepoint) { + // We set the high bit on our relative displacement so that the interpreter knows + // it needs to perform a safepoint after the trace exits + append_bailout(builder, destination, BailoutReason.SafepointBranchTaken, true); + } else if (displacement < 0) { + // This is a backwards branch, and right now we always bail out for those - + // so just return. + // FIXME: Why is this not a safepoint? + append_bailout(builder, destination, BailoutReason.BackwardBranch, true); + } else { + // Branching is enabled, so set eip and exit the current branch block + builder.branchTargets.add(destination); + builder.ip_const(destination); + builder.local("eip", WasmOpcode.set_local); + builder.appendU8(WasmOpcode.br); + // The branch block encloses this tiny branch dispatch block, so break + // out two levels + builder.appendULeb(1); + } + + builder.endBlock(); + return true; +} + +function emit_relop_branch (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + const relopBranchInfo = relopbranchTable[opcode]; + if (!relopBranchInfo) + return false; + + const relop = Array.isArray(relopBranchInfo) + ? relopBranchInfo[0] + : relopBranchInfo; + + const relopInfo = binopTable[relop]; + if (!relopInfo) + return false; + + // We have to wrap the computation of the branch condition inside the + // branch block because opening blocks destroys the contents of the + // wasm execution stack for some reason + builder.block(); + const displacement = getArgI16(ip, 3); + if (traceBranchDisplacements) + console.log(`relop @${ip} displacement=${displacement}`); + + append_ldloc(builder, getArgU16(ip, 1), relopInfo[1]); + // Compare with immediate + if (Array.isArray(relopBranchInfo) && relopBranchInfo[1]) { + // For i8 immediates we need to generate an i64.const even though + // the immediate is 16 bits, so we store the relevant opcode + // in the relop branch info table + builder.appendU8(relopBranchInfo[1]); + builder.appendLeb(getArgI16(ip, 2)); + } else + append_ldloc(builder, getArgU16(ip, 2), relopInfo[1]); + builder.appendU8(relopInfo[0]); + return emit_branch(builder, ip, opcode, displacement); +} + +function emit_math_intrinsic (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + let isUnary : boolean, isF32 : boolean, name: string | undefined; + let wasmOp : WasmOpcode | undefined; + const destOffset = getArgU16(ip, 1), + srcOffset = getArgU16(ip, 2), + rhsOffset = getArgU16(ip, 3); + + switch (opcode) { + // oddly the interpreter has no opcodes for abs! + case MintOpcode.MINT_SQRT: + case MintOpcode.MINT_SQRTF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_SQRTF); + wasmOp = isF32 + ? WasmOpcode.f32_sqrt + : WasmOpcode.f64_sqrt; + break; + case MintOpcode.MINT_CEILING: + case MintOpcode.MINT_CEILINGF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_CEILINGF); + wasmOp = isF32 + ? WasmOpcode.f32_ceil + : WasmOpcode.f64_ceil; + break; + case MintOpcode.MINT_FLOOR: + case MintOpcode.MINT_FLOORF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_FLOORF); + wasmOp = isF32 + ? WasmOpcode.f32_floor + : WasmOpcode.f64_floor; + break; + case MintOpcode.MINT_ABS: + case MintOpcode.MINT_ABSF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_ABSF); + wasmOp = isF32 + ? WasmOpcode.f32_abs + : WasmOpcode.f64_abs; + break; + case MintOpcode.MINT_REM_R4: + case MintOpcode.MINT_REM_R8: + isUnary = false; + isF32 = (opcode === MintOpcode.MINT_REM_R4); + name = "rem"; + break; + case MintOpcode.MINT_ATAN2: + case MintOpcode.MINT_ATAN2F: + isUnary = false; + isF32 = (opcode === MintOpcode.MINT_ATAN2F); + name = "atan2"; + break; + case MintOpcode.MINT_ACOS: + case MintOpcode.MINT_ACOSF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_ACOSF); + name = "acos"; + break; + case MintOpcode.MINT_COS: + case MintOpcode.MINT_COSF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_COSF); + name = "cos"; + break; + case MintOpcode.MINT_SIN: + case MintOpcode.MINT_SINF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_SINF); + name = "sin"; + break; + case MintOpcode.MINT_ASIN: + case MintOpcode.MINT_ASINF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_ASINF); + name = "asin"; + break; + case MintOpcode.MINT_TAN: + case MintOpcode.MINT_TANF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_TANF); + name = "tan"; + break; + case MintOpcode.MINT_ATAN: + case MintOpcode.MINT_ATANF: + isUnary = true; + isF32 = (opcode === MintOpcode.MINT_ATANF); + name = "atan"; + break; + case MintOpcode.MINT_MIN: + case MintOpcode.MINT_MINF: + isUnary = false; + isF32 = (opcode === MintOpcode.MINT_MINF); + wasmOp = isF32 + ? WasmOpcode.f32_min + : WasmOpcode.f64_min; + break; + case MintOpcode.MINT_MAX: + case MintOpcode.MINT_MAXF: + isUnary = false; + isF32 = (opcode === MintOpcode.MINT_MAXF); + wasmOp = isF32 + ? WasmOpcode.f32_max + : WasmOpcode.f64_max; + break; + default: + return false; + } + + // Pre-load locals for the stloc at the end + builder.local("pLocals"); + + if (isUnary) { + append_ldloc(builder, srcOffset, isF32 ? WasmOpcode.f32_load : WasmOpcode.f64_load); + if (wasmOp) { + builder.appendU8(wasmOp); + } else if (name) { + if (isF32) + builder.appendU8(WasmOpcode.f64_promote_f32); + builder.callImport(name); + if (isF32) + builder.appendU8(WasmOpcode.f32_demote_f64); + } else + throw new Error("internal error"); + append_stloc_tail(builder, destOffset, isF32 ? WasmOpcode.f32_store : WasmOpcode.f64_store); + return true; + } else { + append_ldloc(builder, srcOffset, isF32 ? WasmOpcode.f32_load : WasmOpcode.f64_load); + if (isF32 && name) + builder.appendU8(WasmOpcode.f64_promote_f32); + append_ldloc(builder, rhsOffset, isF32 ? WasmOpcode.f32_load : WasmOpcode.f64_load); + if (isF32 && name) + builder.appendU8(WasmOpcode.f64_promote_f32); + + if (wasmOp) { + builder.appendU8(wasmOp); + } else if (name) { + builder.callImport(name); + if (isF32) + builder.appendU8(WasmOpcode.f32_demote_f64); + } else + throw new Error("internal error"); + + append_stloc_tail(builder, destOffset, isF32 ? WasmOpcode.f32_store : WasmOpcode.f64_store); + return true; + } +} + +function emit_indirectop (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + const isLoad = (opcode >= MintOpcode.MINT_LDIND_I1) && + (opcode <= MintOpcode.MINT_LDIND_OFFSET_IMM_I8); + const isOffset = ( + (opcode >= MintOpcode.MINT_LDIND_OFFSET_I1) && + (opcode <= MintOpcode.MINT_LDIND_OFFSET_IMM_I8) + ) || ( + (opcode >= MintOpcode.MINT_STIND_OFFSET_I1) && + (opcode <= MintOpcode.MINT_STIND_OFFSET_IMM_I8) + ); + const isImm = ( + (opcode >= MintOpcode.MINT_LDIND_OFFSET_IMM_I1) && + (opcode <= MintOpcode.MINT_LDIND_OFFSET_IMM_I8) + ) || ( + (opcode >= MintOpcode.MINT_STIND_OFFSET_IMM_I1) && + (opcode <= MintOpcode.MINT_STIND_OFFSET_IMM_I8) + ); + + let valueVarIndex, addressVarIndex, offsetVarIndex = -1, constantOffset = 0; + if (isOffset) { + if (isImm) { + if (isLoad) { + valueVarIndex = getArgU16(ip, 1); + addressVarIndex = getArgU16(ip, 2); + constantOffset = getArgI16(ip, 3); + } else { + valueVarIndex = getArgU16(ip, 2); + addressVarIndex = getArgU16(ip, 1); + constantOffset = getArgI16(ip, 3); + } + } else { + if (isLoad) { + valueVarIndex = getArgU16(ip, 1); + addressVarIndex = getArgU16(ip, 2); + offsetVarIndex = getArgU16(ip, 3); + } else { + valueVarIndex = getArgU16(ip, 3); + addressVarIndex = getArgU16(ip, 1); + offsetVarIndex = getArgU16(ip, 2); + } + } + } else if (isLoad) { + addressVarIndex = getArgU16(ip, 2); + valueVarIndex = getArgU16(ip, 1); + } else { + addressVarIndex = getArgU16(ip, 1); + valueVarIndex = getArgU16(ip, 2); + } + + let getter : WasmOpcode, setter = WasmOpcode.i32_store; + switch (opcode) { + case MintOpcode.MINT_LDIND_I1: + case MintOpcode.MINT_LDIND_OFFSET_I1: + getter = WasmOpcode.i32_load8_s; + break; + case MintOpcode.MINT_LDIND_U1: + case MintOpcode.MINT_LDIND_OFFSET_U1: + getter = WasmOpcode.i32_load8_u; + break; + case MintOpcode.MINT_LDIND_I2: + case MintOpcode.MINT_LDIND_OFFSET_I2: + getter = WasmOpcode.i32_load16_s; + break; + case MintOpcode.MINT_LDIND_U2: + case MintOpcode.MINT_LDIND_OFFSET_U2: + getter = WasmOpcode.i32_load16_u; + break; + case MintOpcode.MINT_LDIND_OFFSET_IMM_I1: + getter = WasmOpcode.i32_load8_s; + break; + case MintOpcode.MINT_LDIND_OFFSET_IMM_U1: + getter = WasmOpcode.i32_load8_u; + break; + case MintOpcode.MINT_LDIND_OFFSET_IMM_I2: + getter = WasmOpcode.i32_load16_s; + break; + case MintOpcode.MINT_LDIND_OFFSET_IMM_U2: + getter = WasmOpcode.i32_load16_u; + break; + case MintOpcode.MINT_STIND_I1: + case MintOpcode.MINT_STIND_OFFSET_I1: + case MintOpcode.MINT_STIND_OFFSET_IMM_I1: + getter = WasmOpcode.i32_load; + setter = WasmOpcode.i32_store8; + break; + case MintOpcode.MINT_STIND_I2: + case MintOpcode.MINT_STIND_OFFSET_I2: + case MintOpcode.MINT_STIND_OFFSET_IMM_I2: + getter = WasmOpcode.i32_load; + setter = WasmOpcode.i32_store16; + break; + case MintOpcode.MINT_LDIND_I4: + case MintOpcode.MINT_LDIND_OFFSET_I4: + case MintOpcode.MINT_LDIND_OFFSET_IMM_I4: + case MintOpcode.MINT_STIND_I4: + case MintOpcode.MINT_STIND_OFFSET_I4: + case MintOpcode.MINT_STIND_OFFSET_IMM_I4: + case MintOpcode.MINT_STIND_REF: + getter = WasmOpcode.i32_load; + break; + case MintOpcode.MINT_LDIND_R4: + case MintOpcode.MINT_STIND_R4: + getter = WasmOpcode.f32_load; + setter = WasmOpcode.f32_store; + break; + case MintOpcode.MINT_LDIND_R8: + case MintOpcode.MINT_STIND_R8: + getter = WasmOpcode.f64_load; + setter = WasmOpcode.f64_store; + break; + case MintOpcode.MINT_LDIND_I8: + case MintOpcode.MINT_LDIND_OFFSET_I8: + case MintOpcode.MINT_LDIND_OFFSET_IMM_I8: + case MintOpcode.MINT_STIND_I8: + case MintOpcode.MINT_STIND_OFFSET_I8: + case MintOpcode.MINT_STIND_OFFSET_IMM_I8: + getter = WasmOpcode.i64_load; + setter = WasmOpcode.i64_store; + break; + default: + return false; + } + + append_ldloc_cknull(builder, addressVarIndex, ip, false); + + // FIXME: ldind_offset/stind_offset + + if (isLoad) { + // pre-load pLocals for the store operation + builder.local("pLocals"); + // Load address + builder.local("cknull_ptr"); + // For ldind_offset we need to load an offset from another local + // and then add it to the null checked address + if (isOffset && offsetVarIndex >= 0) { + append_ldloc(builder, offsetVarIndex, WasmOpcode.i32_load); + builder.appendU8(WasmOpcode.i32_add); + } else if (constantOffset < 0) { + // wasm memarg offsets are unsigned, so do a signed add + builder.i32_const(constantOffset); + builder.appendU8(WasmOpcode.i32_add); + constantOffset = 0; + } + // Load value from loaded address + builder.appendU8(getter); + builder.appendMemarg(constantOffset, 0); + + append_stloc_tail(builder, valueVarIndex, setter); + } else if (opcode === MintOpcode.MINT_STIND_REF) { + // Load destination address + builder.local("cknull_ptr"); + // Load address of value so that copy_managed_pointer can grab it + append_ldloca(builder, valueVarIndex); + builder.callImport("copy_pointer"); + } else { + // Pre-load address for the store operation + builder.local("cknull_ptr"); + // For ldind_offset we need to load an offset from another local + // and then add it to the null checked address + if (isOffset && offsetVarIndex >= 0) { + append_ldloc(builder, offsetVarIndex, WasmOpcode.i32_load); + builder.appendU8(WasmOpcode.i32_add); + } else if (constantOffset < 0) { + // wasm memarg offsets are unsigned, so do a signed add + builder.i32_const(constantOffset); + builder.appendU8(WasmOpcode.i32_add); + constantOffset = 0; + } + // Load value and then write to address + append_ldloc(builder, valueVarIndex, getter); + builder.appendU8(setter); + builder.appendMemarg(constantOffset, 0); + } + return true; +} + +function append_getelema1 ( + builder: WasmBuilder, ip: MintOpcodePtr, + objectOffset: number, indexOffset: number, elementSize: number +) { + builder.block(); + + // Preload the address of our temp local - we will be tee-ing the + // element address to it because wasm has no 'dup' instruction + builder.local("temp_ptr"); + + // (array, size, index) -> void* + append_ldloca(builder, objectOffset); + builder.i32_const(elementSize); + append_ldloc(builder, indexOffset, WasmOpcode.i32_load); + builder.callImport("array_address"); + builder.local("temp_ptr", WasmOpcode.tee_local); + + // If the operation failed it will return 0, so we bail out to the interpreter + // so it can perform error handling (there are multiple reasons for a failure) + builder.appendU8(WasmOpcode.br_if); + builder.appendULeb(0); + append_bailout(builder, ip, BailoutReason.ArrayLoadFailed); + + builder.endBlock(); + + // The operation succeeded and the null check consumed the element address, + // so load the element address back from our temp local + builder.local("temp_ptr"); +} + +function emit_arrayop (builder: WasmBuilder, ip: MintOpcodePtr, opcode: MintOpcode) : boolean { + const isLoad = ( + (opcode <= MintOpcode.MINT_LDELEMA_TC) && + (opcode >= MintOpcode.MINT_LDELEM_I) + ) || (opcode === MintOpcode.MINT_LDLEN), + objectOffset = getArgU16(ip, isLoad ? 2 : 1), + valueOffset = getArgU16(ip, isLoad ? 1 : 3), + indexOffset = getArgU16(ip, isLoad ? 3 : 2); + + let elementGetter: WasmOpcode, + elementSetter = WasmOpcode.i32_store, + elementSize: number, + isPointer = false; + + switch (opcode) { + case MintOpcode.MINT_LDLEN: + append_local_null_check(builder, objectOffset, ip); + builder.local("pLocals"); + append_ldloca(builder, objectOffset); + builder.callImport("array_length"); + append_stloc_tail(builder, valueOffset, WasmOpcode.i32_store); + return true; + case MintOpcode.MINT_LDELEMA1: { + // Pre-load destination for the element address at the end + builder.local("pLocals"); + + elementSize = getArgU16(ip, 4); + append_getelema1(builder, ip, objectOffset, indexOffset, elementSize); + + append_stloc_tail(builder, valueOffset, WasmOpcode.i32_store); + return true; + } + case MintOpcode.MINT_LDELEM_REF: + case MintOpcode.MINT_STELEM_REF: + elementSize = 4; + elementGetter = WasmOpcode.i32_load; + isPointer = true; + break; + case MintOpcode.MINT_LDELEM_I1: + elementSize = 1; + elementGetter = WasmOpcode.i32_load8_s; + break; + case MintOpcode.MINT_LDELEM_U1: + elementSize = 1; + elementGetter = WasmOpcode.i32_load8_u; + break; + case MintOpcode.MINT_STELEM_U1: + case MintOpcode.MINT_STELEM_I1: + elementSize = 1; + elementGetter = WasmOpcode.i32_load; + elementSetter = WasmOpcode.i32_store8; + break; + case MintOpcode.MINT_LDELEM_I2: + elementSize = 2; + elementGetter = WasmOpcode.i32_load16_s; + break; + case MintOpcode.MINT_LDELEM_U2: + elementSize = 2; + elementGetter = WasmOpcode.i32_load16_u; + break; + case MintOpcode.MINT_STELEM_U2: + case MintOpcode.MINT_STELEM_I2: + elementSize = 2; + elementGetter = WasmOpcode.i32_load; + elementSetter = WasmOpcode.i32_store16; + break; + case MintOpcode.MINT_LDELEM_U4: + case MintOpcode.MINT_LDELEM_I4: + case MintOpcode.MINT_STELEM_I4: + elementSize = 4; + elementGetter = WasmOpcode.i32_load; + break; + case MintOpcode.MINT_LDELEM_R4: + case MintOpcode.MINT_STELEM_R4: + elementSize = 4; + elementGetter = WasmOpcode.f32_load; + elementSetter = WasmOpcode.f32_store; + break; + case MintOpcode.MINT_LDELEM_I8: + case MintOpcode.MINT_STELEM_I8: + elementSize = 8; + elementGetter = WasmOpcode.i64_load; + elementSetter = WasmOpcode.i64_store; + break; + case MintOpcode.MINT_LDELEM_R8: + case MintOpcode.MINT_STELEM_R8: + elementSize = 8; + elementGetter = WasmOpcode.f64_load; + elementSetter = WasmOpcode.f64_store; + break; + case MintOpcode.MINT_LDELEM_VT: { + const elementSize = getArgU16(ip, 4); + // dest + builder.local("pLocals"); + builder.i32_const(getArgU16(ip, 1)); + builder.appendU8(WasmOpcode.i32_add); + // src + append_getelema1(builder, ip, objectOffset, indexOffset, elementSize); + // memcpy (locals + ip [1], src_addr, size); + append_memmove_dest_src(builder, elementSize); + return true; + } + default: + return false; + } + + if (isPointer) { + // Copy pointer to/from array element + if (isLoad) + append_ldloca(builder, valueOffset); + append_getelema1(builder, ip, objectOffset, indexOffset, elementSize); + if (!isLoad) + append_ldloca(builder, valueOffset); + builder.callImport("copy_pointer"); + } else if (isLoad) { + // Pre-load destination for the value at the end + builder.local("pLocals"); + + // Get address of the element, then load it + append_getelema1(builder, ip, objectOffset, indexOffset, elementSize); + builder.appendU8(elementGetter); + builder.appendMemarg(0, 0); + + append_stloc_tail(builder, valueOffset, elementSetter); + } else { + // Get address of the element first as our destination + append_getelema1(builder, ip, objectOffset, indexOffset, elementSize); + append_ldloc(builder, valueOffset, elementGetter); + + builder.appendU8(elementSetter); + builder.appendMemarg(0, 0); + } + return true; +} + +function append_bailout (builder: WasmBuilder, ip: MintOpcodePtr, reason: BailoutReason, highBit?: boolean) { + builder.ip_const(ip, highBit); + if (builder.options.countBailouts) { + builder.i32_const(reason); + builder.callImport("bailout"); + } + builder.appendU8(WasmOpcode.return_); +} + +const JITERPRETER_TRAINING = 0; +const JITERPRETER_NOT_JITTED = 1; +let mostRecentOptions : JiterpreterOptions | undefined = undefined; + +export function mono_interp_tier_prepare_jiterpreter ( + frame: NativePointer, method: MonoMethod, ip: MintOpcodePtr, + startOfBody: MintOpcodePtr, sizeOfBody: MintOpcodePtr +) : number { + mono_assert(ip, "expected instruction pointer"); + if (!mostRecentOptions) + mostRecentOptions = getOptions(); + + // FIXME: We shouldn't need this check + if (!mostRecentOptions.enableTraces) + return JITERPRETER_NOT_JITTED; + + let info = traceInfo[ip]; + + if (!info) + traceInfo[ip] = info = new TraceInfo(ip); + else + info.hitCount++; + + if (info.hitCount < minimumHitCount) + return JITERPRETER_TRAINING; + else if (info.hitCount === minimumHitCount) { + counters.traceCandidates++; + let methodFullName: string | undefined; + if (trapTraceErrors || mostRecentOptions.estimateHeat || (instrumentedMethodNames.length > 0)) { + const pMethodName = cwraps.mono_wasm_method_get_full_name(method); + methodFullName = Module.UTF8ToString(pMethodName); + Module._free(pMethodName); + } + const methodName = Module.UTF8ToString(cwraps.mono_wasm_method_get_name(method)); + info.name = methodFullName || methodName; + const fnPtr = generate_wasm( + frame, methodName, ip, startOfBody, sizeOfBody, methodFullName + ); + if (fnPtr) { + counters.tracesCompiled++; + // FIXME: These could theoretically be 0 or 1, in which case the trace + // will never get invoked. Oh well + info.fnPtr = fnPtr; + return fnPtr; + } else { + return mostRecentOptions.estimateHeat ? JITERPRETER_TRAINING : JITERPRETER_NOT_JITTED; + } + } else if (!mostRecentOptions.estimateHeat) + throw new Error("prepare should not be invoked at this point"); + else + return JITERPRETER_TRAINING; +} + +export function jiterpreter_dump_stats (b?: boolean) { + if (!mostRecentOptions || (b !== undefined)) + mostRecentOptions = getOptions(); + + if (!mostRecentOptions.enableStats && (b !== undefined)) + return; + + console.log(`// jiterpreter produced ${counters.tracesCompiled} traces from ${counters.traceCandidates} candidates (${(counters.tracesCompiled / counters.traceCandidates * 100).toFixed(1)}%), ${counters.jitCallsCompiled} jit_call trampolines, and ${counters.entryWrappersCompiled} interp_entry wrappers`); + console.log(`// time spent: ${elapsedTimes.generation | 0}ms generating, ${elapsedTimes.compilation | 0}ms compiling wasm`); + if (mostRecentOptions.countBailouts) { + for (let i = 0; i < BailoutReasonNames.length; i++) { + const bailoutCount = cwraps.mono_jiterp_get_trace_bailout_count(i); + if (bailoutCount) + console.log(`// traces bailed out ${bailoutCount} time(s) due to ${BailoutReasonNames[i]}`); + } + } + + if (mostRecentOptions.estimateHeat) { + const counts : { [key: string] : number } = {}; + const traces = Object.values(traceInfo); + + for (let i = 0; i < traces.length; i++) { + const info = traces[i]; + if (!info.abortReason) + continue; + else if (info.abortReason === "end-of-body") + continue; + + if (counts[info.abortReason]) + counts[info.abortReason] += info.hitCount; + else + counts[info.abortReason] = info.hitCount; + } + + if (countCallTargets) { + console.log("// hottest call targets:"); + const targetPointers = Object.keys(callTargetCounts); + targetPointers.sort((l, r) => callTargetCounts[Number(r)] - callTargetCounts[Number(l)]); + for (let i = 0, c = Math.min(20, targetPointers.length); i < c; i++) { + const targetMethod = Number(targetPointers[i]) | 0; + const pMethodName = cwraps.mono_wasm_method_get_full_name(targetMethod); + const targetMethodName = Module.UTF8ToString(pMethodName); + const hitCount = callTargetCounts[targetMethod]; + Module._free(pMethodName); + console.log(`${targetMethodName} ${hitCount}`); + } + } + + traces.sort((l, r) => r.hitCount - l.hitCount); + console.log("// hottest failed traces:"); + for (let i = 0, c = 0; i < traces.length && c < 20; i++) { + // this means the trace has a low hit count and we don't know its identity. no value in + // logging it. + if (!traces[i].name) + continue; + // Filter out noisy methods that we don't care about optimizing + if (traces[i].name!.indexOf("Xunit.") >= 0) + continue; + // FIXME: A single hot method can contain many failed traces. This creates a lot of noise + // here and also likely indicates the jiterpreter would add a lot of overhead to it + // Filter out aborts that aren't meaningful since it is unlikely to ever make sense + // to fix them, either because they are rarely used or because putting them in + // traces would not meaningfully improve performance + if (traces[i].abortReason && traces[i].abortReason!.startsWith("mono_icall_")) + continue; + switch (traces[i].abortReason) { + case "trace-too-small": + case "call": + case "callvirt.fast": + case "calli.nat.fast": + case "calli.nat": + case "call.delegate": + case "newobj": + case "newobj_vt": + case "intrins_ordinal_ignore_case_ascii": + case "intrins_marvin_block": + case "intrins_ascii_chars_to_uppercase": + case "switch": + case "call_handler.s": + case "rethrow": + case "endfinally": + case "end-of-body": + continue; + } + c++; + console.log(`${traces[i].name} @${traces[i].ip} (${traces[i].hitCount} hits) ${traces[i].abortReason}`); + } + + const tuples : Array<[string, number]> = []; + for (const k in counts) + tuples.push([k, counts[k]]); + + tuples.sort((l, r) => r[1] - l[1]); + + console.log("// heat:"); + for (let i = 0; i < tuples.length; i++) + console.log(`// ${tuples[i][0]}: ${tuples[i][1]}`); + } else { + for (let i = 0; i < MintOpcode.MINT_LASTOP; i++) { + const opname = OpcodeInfo[i][0]; + const count = cwraps.mono_jiterp_adjust_abort_count(i, 0); + if (count > 0) + abortCounts[opname] = count; + else + delete abortCounts[opname]; + } + + const keys = Object.keys(abortCounts); + keys.sort((l, r) => abortCounts[r] - abortCounts[l]); + for (let i = 0; i < keys.length; i++) + console.log(`// ${keys[i]}: ${abortCounts[keys[i]]} abort(s)`); + } + + if ((typeof(globalThis.setTimeout) === "function") && (b !== undefined)) + setTimeout( + () => jiterpreter_dump_stats(b), + 15000 + ); +} diff --git a/src/mono/wasm/runtime/net6-legacy/export-types.ts b/src/mono/wasm/runtime/net6-legacy/export-types.ts index 5de5931a17fd58..0a13bc1cf39a9c 100644 --- a/src/mono/wasm/runtime/net6-legacy/export-types.ts +++ b/src/mono/wasm/runtime/net6-legacy/export-types.ts @@ -252,4 +252,4 @@ export type MONOType = { getF64: (offset: MemOffset) => number; }; -export { MonoArray, MonoObject, MonoString }; \ No newline at end of file +export { MonoArray, MonoObject, MonoString }; diff --git a/src/mono/wasm/runtime/run.ts b/src/mono/wasm/runtime/run.ts index 1e224837222ca0..612bca2111261a 100644 --- a/src/mono/wasm/runtime/run.ts +++ b/src/mono/wasm/runtime/run.ts @@ -8,6 +8,7 @@ import cwraps from "./cwraps"; import { assembly_load } from "./class-loader"; import { mono_assert } from "./types"; import { consoleWebSocket, mono_wasm_stringify_as_error_with_stack } from "./logging"; +import { jiterpreter_dump_stats } from "./jiterpreter"; /** * Possible signatures are described here https://docs.microsoft.com/en-us/dotnet/csharp/fundamentals/program-structure/main-command-line @@ -73,7 +74,7 @@ export function mono_exit(exit_code: number, reason?: any): void { set_exit_code_and_quit_now(exit_code, reason); } })(); - // we need to throw, rather than let the caller continue the normal execution + // we need to throw, rather than let the caller continue the normal execution // in the middle of some code, which expects this to stop the process throw runtimeHelpers.ExitStatus ? new runtimeHelpers.ExitStatus(exit_code) @@ -144,6 +145,8 @@ function appendElementOnExit(exit_code: number) { } function logErrorOnExit(exit_code: number, reason?: any) { + jiterpreter_dump_stats(false); + if (runtimeHelpers.config.logExitCode) { if (exit_code != 0 && reason) { if (reason instanceof Error) diff --git a/src/mono/wasm/runtime/types/emscripten.ts b/src/mono/wasm/runtime/types/emscripten.ts index 5ddd3f6ceb6223..5dfef8981cbda4 100644 --- a/src/mono/wasm/runtime/types/emscripten.ts +++ b/src/mono/wasm/runtime/types/emscripten.ts @@ -52,6 +52,8 @@ export declare interface EmscriptenModule { FS_readFile(filename: string, opts: any): any; removeRunDependency(id: string): void; addRunDependency(id: string): void; + addFunction(fn: Function, signature: string): number; + getWasmTableEntry(index: number): any; stackSave(): VoidPtr; stackRestore(stack: VoidPtr): void; stackAlloc(size: number): VoidPtr; diff --git a/src/mono/wasm/wasm.proj b/src/mono/wasm/wasm.proj index d0ad0dde5501c3..a69c76b6d09f2d 100644 --- a/src/mono/wasm/wasm.proj +++ b/src/mono/wasm/wasm.proj @@ -122,6 +122,8 @@ + + @@ -157,6 +159,8 @@ <_EmccLinkFlags Condition="'$(MonoWasmThreads)' == 'true'" Include="-s PTHREAD_POOL_SIZE=0" /> <_EmccLinkFlags Condition="'$(MonoWasmThreads)' == 'true'" Include="-s PTHREAD_POOL_SIZE_STRICT=2" /> <_EmccLinkFlags Include="-s ALLOW_MEMORY_GROWTH=1" /> + + <_EmccLinkFlags Include="-s ALLOW_TABLE_GROWTH=1" /> <_EmccLinkFlags Include="-s NO_EXIT_RUNTIME=1" /> <_EmccLinkFlags Include="-s FORCE_FILESYSTEM=1" /> <_EmccLinkFlags Include="-s EXPORTED_RUNTIME_METHODS=$(_EmccExportedRuntimeMethods)" /> From 6b9da71512f18e2a69ed463ff98af126715bd184 Mon Sep 17 00:00:00 2001 From: Katelyn Gadd Date: Tue, 15 Nov 2022 11:57:07 -0800 Subject: [PATCH 2/3] Address PR feedback --- src/mono/mono/mini/interp/jiterpreter.c | 6 ++++-- src/mono/mono/mini/interp/jiterpreter.h | 11 +++++------ src/mono/mono/mini/interp/transform.c | 12 +++++++++++- src/mono/mono/mini/interp/transform.h | 5 +++-- 4 files changed, 23 insertions(+), 11 deletions(-) diff --git a/src/mono/mono/mini/interp/jiterpreter.c b/src/mono/mono/mini/interp/jiterpreter.c index 7075302d54cbea..1f5c75396d2dd3 100644 --- a/src/mono/mono/mini/interp/jiterpreter.c +++ b/src/mono/mono/mini/interp/jiterpreter.c @@ -41,7 +41,6 @@ void jiterp_preserve_module (void); #include #include -#define JITERPRETER_IMPLEMENTATION #include "jiterpreter.h" static gint32 jiterpreter_abort_counts[MINT_LASTOP + 1] = { 0 }; @@ -910,6 +909,9 @@ should_generate_trace_here (InterpBasicBlock *bb, InterpInst *last_ins) { return FALSE; } +InterpInst* +mono_jiterp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode); + /* * Insert jiterpreter entry points at the correct candidate locations: * The first basic block of the function, @@ -957,7 +959,7 @@ jiterp_insert_entry_points (void *_td) if (enabled && should_generate) { td->cbb = bb; - interp_insert_ins (td, NULL, MINT_TIER_PREPARE_JITERPRETER); + mono_jiterp_insert_ins (td, NULL, MINT_TIER_PREPARE_JITERPRETER); // Note that we only clear enter_at_next here, after generating a trace. // This means that the flag will stay set intentionally if we keep failing // to generate traces, perhaps due to a string of small basic blocks diff --git a/src/mono/mono/mini/interp/jiterpreter.h b/src/mono/mono/mini/interp/jiterpreter.h index 55b3be429d63d7..c21960d2154356 100644 --- a/src/mono/mono/mini/interp/jiterpreter.h +++ b/src/mono/mono/mini/interp/jiterpreter.h @@ -1,3 +1,6 @@ +#ifndef __MONO_MINI_JITERPRETER_H__ +#define __MONO_MINI_JITERPRETER_H__ + #ifdef HOST_BROWSER // enables specialized mono_llvm_cpp_catch_exception replacement (see jiterpreter-jit-call.ts) @@ -81,16 +84,12 @@ mono_jiterp_do_jit_call_indirect ( gpointer cb, gpointer arg, gboolean *out_thrown ); - -#ifndef JITERPRETER_IMPLEMENTATION - extern gboolean jiterpreter_traces_enabled; extern gboolean jiterpreter_interp_entry_enabled; extern gboolean jiterpreter_jit_call_enabled; extern gboolean jiterpreter_wasm_eh_enabled; extern WasmDoJitCall jiterpreter_do_jit_call; -#endif - +#endif // HOST_BROWSER -#endif +#endif // __MONO_MINI_JITERPRETER_H__ diff --git a/src/mono/mono/mini/interp/transform.c b/src/mono/mono/mini/interp/transform.c index 5167810ecfdfe5..89a611b1713764 100644 --- a/src/mono/mono/mini/interp/transform.c +++ b/src/mono/mono/mini/interp/transform.c @@ -260,7 +260,7 @@ interp_insert_ins_bb (TransformData *td, InterpBasicBlock *bb, InterpInst *prev_ } /* Inserts a new instruction after prev_ins. prev_ins must be in cbb */ -InterpInst* +static InterpInst* interp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode) { return interp_insert_ins_bb (td, td->cbb, prev_ins, opcode); @@ -10317,3 +10317,13 @@ mono_interp_transform_method (InterpMethod *imethod, ThreadContext *context, Mon // FIXME: Add a different callback ? MONO_PROFILER_RAISE (jit_done, (method, imethod->jinfo)); } + +#if HOST_BROWSER + +InterpInst* +mono_jiterp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode) +{ + return interp_insert_ins (td, prev_ins, opcode); +} + +#endif diff --git a/src/mono/mono/mini/interp/transform.h b/src/mono/mono/mini/interp/transform.h index 88387b2dc1170e..7ad00ab0b2618d 100644 --- a/src/mono/mono/mini/interp/transform.h +++ b/src/mono/mono/mini/interp/transform.h @@ -269,9 +269,10 @@ mono_test_interp_generate_code (TransformData *td, MonoMethod *method, MonoMetho void mono_test_interp_method_compute_offsets (TransformData *td, InterpMethod *imethod, MonoMethodSignature *signature, MonoMethodHeader *header); -/* used by jiterpreter */ +#if HOST_BROWSER InterpInst* -interp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode); +mono_jiterp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode); +#endif /* debugging aid */ void From 1c576179d4cd2e5b4a725567af6082591b9ffff9 Mon Sep 17 00:00:00 2001 From: Katelyn Gadd Date: Tue, 15 Nov 2022 14:00:49 -0800 Subject: [PATCH 3/3] Code cleanup; Disable jiterpreter features by default for initial merge --- src/mono/mono/mini/interp/jiterpreter.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/src/mono/mono/mini/interp/jiterpreter.c b/src/mono/mono/mini/interp/jiterpreter.c index 1f5c75396d2dd3..6d6972bf63b14b 100644 --- a/src/mono/mono/mini/interp/jiterpreter.c +++ b/src/mono/mono/mini/interp/jiterpreter.c @@ -54,11 +54,11 @@ gboolean jiterpreter_traces_enabled = FALSE, jiterpreter_jit_call_enabled = FALSE, #else // traces_enabled controls whether the jiterpreter will JIT individual interpreter opcode traces -gboolean jiterpreter_traces_enabled = TRUE, +gboolean jiterpreter_traces_enabled = FALSE, // interp_entry_enabled controls whether specialized interp_entry wrappers will be jitted - jiterpreter_interp_entry_enabled = TRUE, + jiterpreter_interp_entry_enabled = FALSE, // jit_call_enabled controls whether do_jit_call will use specialized trampolines for hot call sites - jiterpreter_jit_call_enabled = TRUE, + jiterpreter_jit_call_enabled = FALSE, #endif // if enabled, we will insert trace entry points at backwards branch targets, so that we can // JIT loop bodies @@ -909,9 +909,6 @@ should_generate_trace_here (InterpBasicBlock *bb, InterpInst *last_ins) { return FALSE; } -InterpInst* -mono_jiterp_insert_ins (TransformData *td, InterpInst *prev_ins, int opcode); - /* * Insert jiterpreter entry points at the correct candidate locations: * The first basic block of the function,