ARM64: JIT does not do TailCall when it requires more stack space than the caller has its incoming argument space

[rahku]

For x64 it uses JIT_TailCall jithelper to implement the above which is not getting used for arm64 yet.

category:cq
theme:tail-call
skill-level:intermediate
cost:medium

[jkotas]

This should be taken care of by tail-call design that @janvorli is working on

[sdmaclea]

@janvorli Is this still an open issue?

[sandreenko]

I have added a check that if we have a prefix tail call then we have to do fast or slow tail call.
It showed that there are many tests that just ignored tail. opcode if they were not able to do a fast tail call:
JIT\jit64\mcc\interop\mcc_i06\mcc_i06.cmd
JIT\jit64\mcc\interop\mcc_i16\mcc_i16.cmd
JIT\jit64\mcc\interop\mcc_i36\mcc_i36.cmd
JIT\jit64\mcc\interop\mcc_i56\mcc_i56.cmd
JIT\jit64\mcc\interop\mcc_i66\mcc_i66.cmd
JIT\jit64\mcc\interop\mcc_i76\mcc_i76.cmd
JIT\Methodical\Invoke\25params\25param2c_il_d\25param2c_il_d.cmd
and some others.

[janvorli]

@sandreenko we support tail calls with helpers only on Windows x86, x64 and ARM at the moment.
I have created a design for a new way of handling tailcalls with helpers that we plan to use for Windows ARM64 and all targets on Unix two years ago (https://github.com/dotnet/coreclr/blob/master/Documentation/design-docs/tailcalls-with-helpers.md). But it was not implemented yet due to higher priority issues keeping pushing that further to the future.
While it seems we could easily make the current way work on ARM64 Windows, it would not work on ARM64 Unix.

[AndyAyersMS]

@erozenfeld we can close this now, right?

[erozenfeld]

Fixed by #341.