vmhook : user defined vmops: proc fn(): int {.vmhook.} = impl to JIT-compile and dlopen fn at CT (using regular backend, not VM)

[timotheecour]

goal

• compile a proc at CT using its own compilation context/defines etc, in much same way as nimscript is a separate compilation context than subsequent compilation.

motivation

• fix instantiationInfo give wrong path separator when cross compiling nim-lang/Nim#16702 and all related issues where host differs from target (eg cross compilation). This includes nim js where some issues are currently hard to fix because const n = int.sizeof is 4 in js, even during VM even though host may be 64 bit.
• generalizes vmops (kind of a user defined vmops)
• would fix issues like one mentioned in Add setutils.complement, setutils.fullSet nim-lang/Nim#17066 (comment) (selectively disable a block in js mode if host is bsd; the issue being that defined(bsd) is false on bsd if in js mode)
• speed key routines at CT instead of relying on VM, and enabling this for 3rd party code like nim-regex (refs regex very slow at CT · Issue #104 · nitely/nim-regex); note that for stdlib, we can have vmops for that (eg hashes.hashVmImplByte) but that doesn't help for 3rd party code

example

when not defined js:
  import posix

proc fn(a: int): int {.vmhook.} = # {.vmhook: "-d:bar".} is also possible
  doAssert not defined(js)
  doAssert int.sizeof == 8
  doAssert declared(posix)

doAssert no declared(posix)
doAssert defined(js)
doAssert int.sizeof == 8

doAssert defined(js)
doAssert int.sizeof == 4
const a = fn(3)
doAssert a == 8

implementation

• a vmhook proc is compiled at CT (in its own separate compilation context using regular backend)
• reuse similar logic as --compiletimeFFI for conversion to/from PNode to value
• see also vmops, vmconv, vmhooks
• I've already implemented this a while back in a private branch

bikeshedding

import or importc could also be used by importc is ambiguous because a proc with no body could mean either a regular importc proc, or a ctfe proc with empty body
another option is vmhost or vmhook or ctfe, jit

proc fn(): int {.vmhook.} =
  result = 12

links

• this provides a more general alternative to refs #16702 make DirSep + friends work with cross-compilation and VM by timotheecour · Pull Request #16916 · nim-lang/Nim (which would have drawback of require lots of conditionals in stdlib) exp_fix_16702_DirSep_crosscompile
• NimVM eventually get replaced? - Nim forum