AOT support

[stevefan1999-personal]

I wonder if we can use the emitted code statically, so we can save the JIT code into an object file for caching (for example). This means the codegen should not emit any position-dependent code (I think it should be PIC by default) and do not reference any runtime-dependent address as well. Is it possible for SLJIT to do this?

(Out of context but as a query) Far as I can see the stackless in SLJIT means there aren't any push/pop operation directly on the stack pointer, and you have to maintain an external stack created on a heap right? How does that translate in terms of assembly?

[zherczeg]

All code is PIC except branches / calls.

• Calls to other pic code must use indirect calls. This is slower than normal calls.
• The code must not be too big. This is especially complicated on MIPS. Branches only have 16 bit offset, so +-128K is the maximum range. There is a J instruction, but the target address must be in the same 256MB region, which is not exactly PIC friendly. For larger branches, the absolute address is encoded in the instruction stream.

You can allocate a stack area for local data for every function. You need to manage your variables manually there (hence the sljit name comes). See the last argument of sljit_emit_enter. The SLJIT_MEM1(SLJIT_SP) is the base address of the local data.

[nacho00112]

So I actually can use the stack (without heap) but have to manage its usage manually?
I've confused with this theme.

[zherczeg]

Yes. SLJIT_MEM1(SLJIT_SP), offset is the way to read/write the stack. The stack is just a linear memory area.

[nacho00112]

This guy seems to have made a stackful version of the project
https://github.com/GregorR/sfjit
what do you think about it?

[zherczeg]

It seems the author added alloca support, but still no local management. Btw building a higher level generic compiler based on sljit is possible. It can manage locals. E.g. the WebAssembly compilers based on sljit do this.
https://github.com/partic2/pwart
https://github.com/Samsung/walrus

[zherczeg]

Back to the topic of AOT, I have an idea. A serialize/de-serialize interface could save/restore the internal representation of the compiler. The loaded code could be compiled with generate code. This is not as fast as loading PIC, but the compiler can optimize branches, and still much faster than a full compilation.

[zherczeg]

I have added serialization to the code:
https://github.com/zherczeg/sljit/blob/master/sljit_src/sljitLir.h#L2288

Since this is a new feature, there might be bugs in the code. With this interface the majority of the compilation can be skipped without giving up performance.

[stevefan1999-personal]

I have added serialization to the code: https://github.com/zherczeg/sljit/blob/master/sljit_src/sljitLir.h#L2288

Since this is a new feature, there might be bugs in the code. With this interface the majority of the compilation can be skipped without giving up performance.

Is that also backward/forward compatible? I might consider saving the AOT results and cache to local filesystem (similar to GAC)

[zherczeg]

It has a version number, so you can detect if your cache is not valid anymore, and can recompile the code:
https://github.com/zherczeg/sljit/blob/master/sljit_src/sljitSerialize.c#L46