Renames JIT to Compiler and notes it is AOT (#564)

This notably changes NewRuntimeJIT to NewRuntimeCompiler as well renames
packages from jit to compiler.

This clarifies the implementation is AOT, not JIT, at least when
clarified to where it occurs (Runtime.CompileModule). In doing so, we
reduce any concern that compilation will happen during function
execution. We also free ourselves to create a JIT option without
confusion in the future via CompileConfig or otherwise.

Fixes #560

Signed-off-by: Adrian Cole <adrian@tetrate.io>

.github/CODEOWNERS

@@ -1,6 +1,6 @@
 # https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
 
 /internal/asm/ @mathetake
-/internal/wasm/jit/ @mathetake
+/internal/engine/compiler/ @mathetake
 
 * @codefromthecrypt @mathetake

Makefile

@@ -3,19 +3,19 @@ golangci_lint := github.com/golangci/golangci-lint/cmd/golangci-lint@v1.46.0
 # sync this with netlify.toml!
 hugo          := github.com/gohugoio/hugo@v0.98.0
 
-ensureJITFastest := -ldflags '-X github.com/tetratelabs/wazero/internal/integration_test/vs.ensureJITFastest=true'
+ensureCompilerFastest := -ldflags '-X github.com/tetratelabs/wazero/internal/integration_test/vs.ensureCompilerFastest=true'
 .PHONY: bench
 bench:
 	@go test -run=NONE -benchmem -bench=. ./internal/integration_test/bench/...
-	@go test -benchmem -bench=. ./internal/integration_test/vs/... $(ensureJITFastest)
+	@go test -benchmem -bench=. ./internal/integration_test/vs/... $(ensureCompilerFastest)
 
 .PHONY: bench.check
 bench.check:
 	@go build ./internal/integration_test/bench/...
 	@# Don't use -test.benchmem as it isn't accurate when comparing against CGO libs
 	@for d in vs/wasmedge vs/wasmer vs/wasmtime ; do \
 		cd ./internal/integration_test/$$d ; \
-		go test -bench=. . -tags='wasmedge' $(ensureJITFastest) ; \
+		go test -bench=. . -tags='wasmedge' $(ensureCompilerFastest) ; \
 		cd - ;\
 	done
 

RATIONALE.md

@@ -248,9 +248,8 @@ If later, we have demand for multiple stores, that can be accomplished by overlo
 `Runtime.Store(name) Store`.
 
 ## wazeroir
-wazero's intermediate representation (IR) is called `wazeroir`. Compiling into an IR provides us a faster interpreter
-and a building block for a future JIT compilation engine. Both of these help answer demands for a more performant
-runtime vs interpreting Wasm directly (the `naivevm` interpreter).
+wazero's intermediate representation (IR) is called `wazeroir`. Lowering into an IR provides us a faster interpreter
+and a closer to assembly representation for used by our compiler.
 
 ### Intermediate Representation (IR) design
 `wazeroir`'s initial design borrowed heavily from the defunct `microwasm` format (a.k.a. LightbeamIR). Notably,
@@ -407,15 +406,15 @@ means that we have 1 GiB size of slice which seems large enough for most applica
 
 While the the spec says that a module can have up to 2^32 tables, wazero limits this to 2^27 = 134,217,728.
 One of the reasons is even that number would occupy 1GB in the pointers tables alone. Not only that, we access tables slice by
-table index by using 32-bit signed offset in the JIT implementation, which means that the table index of 2^27 can reach 2^27 * 8 (pointer size on 64-bit machines) = 2^30 offsets in bytes.
+table index by using 32-bit signed offset in the compiler implementation, which means that the table index of 2^27 can reach 2^27 * 8 (pointer size on 64-bit machines) = 2^30 offsets in bytes.
 
 We _believe_ that all use cases are fine with the limitation, but also note that we have no way to test wazero runtimes under these unusual circumstances.
 
 If a module reaches this limit, an error is returned at the compilation phase.
 
-## JIT engine implementation
+## Compiler engine implementation
 
-See [wasm/jit/RATIONALE.md](internal/wasm/jit/RATIONALE.md).
+See [wasm/compiler/RATIONALE.md](internal/compiler/RATIONALE.md).
 
 ## Golang patterns
 

README.md

@@ -103,9 +103,9 @@ they may answer them for you!
 
 ## Runtime
 
-There are two runtime configurations supported in wazero: _JIT_ is default:
+There are two runtime configurations supported in wazero: _Compiler_ is default:
 
-If you don't choose, ex `wazero.NewRuntime()`, JIT is used if supported. You can also force the interpreter like so:
+If you don't choose, ex `wazero.NewRuntime()`, Compiler is used if supported. You can also force the interpreter like so:
 ```go
 r := wazero.NewRuntimeWithConfig(wazero.NewRuntimeConfigInterpreter())
 ```
@@ -116,11 +116,12 @@ machine. Its implementation doesn't have any platform (GOARCH, GOOS) specific
 code, therefore _interpreter_ can be used for any compilation target available
 for Go (such as `riscv64`).
 
-### JIT
-JIT (Just In Time) compiles WebAssembly modules into machine code during
-`Runtime.CompileModule` so that they are executed natively at runtime. JIT is
-faster than Interpreter, often by order of magnitude (10x) or more. This is
-done while still having no host-specific dependencies.
+### Compiler
+Compiler compiles WebAssembly modules into machine code ahead of time (AOT),
+during `Runtime.CompileModule`. This means your WebAssembly functions execute
+natively at runtime. Compiler is faster than Interpreter, often by order of
+magnitude (10x) or more. This is done while still having no host-specific
+dependencies.
 
 If interested, check out the [RATIONALE.md][8] and help us optimize further!
 
@@ -131,7 +132,7 @@ Both runtimes pass [WebAssembly 1.0 spectests][7] on supported platforms:
 | Runtime     | Usage| amd64 | arm64 | others |
 |:---:|:---:|:---:|:---:|:---:|
 | Interpreter|`wazero.NewRuntimeConfigInterpreter()`|✅ |✅|✅|
-| JIT |`wazero.NewRuntimeConfigJIT()`|✅|✅ |❌|
+| Compiler |`wazero.NewRuntimeConfigCompiler()`|✅|✅ |❌|
 
 ## Support Policy
 
@@ -159,7 +160,7 @@ For example, once Go 1.29 is released, wazero may use a Go 1.28 feature.
 
 ### Platform
 
-wazero has two runtime modes: Interpreter and JIT. The only supported operating
+wazero has two runtime modes: Interpreter and Compiler. The only supported operating
 systems are ones we test, but that doesn't necessarily mean other operating
 system versions won't work.
 
@@ -169,7 +170,7 @@ We currently test Linux (Ubuntu and scratch), MacOS and Windows as packaged by
 * Interpreter
   * Linux is tested on amd64 (native) as well arm64 and riscv64 via emulation.
   * MacOS and Windows are only tested on amd64.
-* JIT
+* Compiler
   * Linux is tested on amd64 (native) as well arm64 via emulation.
   * MacOS and Windows are only tested on amd64.
 
@@ -190,7 +191,7 @@ wazero is a registered trademark of Tetrate.io, Inc. in the United States and/or
 [5]: https://github.com/WebAssembly/WASI
 [6]: https://pkg.go.dev/golang.org/x/sys/unix
 [7]: https://github.com/WebAssembly/spec/tree/wg-1.0/test/core
-[8]: ./internal/wasm/jit/RATIONALE.md
+[8]: internal/engine/compiler/RATIONALE.md
 [9]: https://github.com/tetratelabs/wazero/issues/506
 [10]: https://go.dev/doc/devel/release
 [11]: https://github.com/actions/virtual-environments

config.go

@@ -9,9 +9,9 @@ import (
 	"math"
 
 	"github.com/tetratelabs/wazero/api"
+	"github.com/tetratelabs/wazero/internal/engine/compiler"
+	"github.com/tetratelabs/wazero/internal/engine/interpreter"
 	"github.com/tetratelabs/wazero/internal/wasm"
-	"github.com/tetratelabs/wazero/internal/wasm/interpreter"
-	"github.com/tetratelabs/wazero/internal/wasm/jit"
 )
 
 // RuntimeConfig controls runtime behavior, with the default implementation as NewRuntimeConfig
@@ -145,13 +145,22 @@ var engineLessConfig = &runtimeConfig{
 	enabledFeatures: wasm.Features20191205,
 }
 
-// NewRuntimeConfigJIT compiles WebAssembly modules into runtime.GOARCH-specific assembly for optimal performance.
+// NewRuntimeConfigCompiler compiles WebAssembly modules into
+// runtime.GOARCH-specific assembly for optimal performance.
 //
-// Note: This panics at runtime the runtime.GOOS or runtime.GOARCH does not support JIT. Use NewRuntimeConfig to safely
-// detect and fallback to NewRuntimeConfigInterpreter if needed.
-func NewRuntimeConfigJIT() RuntimeConfig {
+// The default implementation is AOT (Ahead of Time) compilation, applied at
+// Runtime.CompileModule. This allows consistent runtime performance, as well
+// the ability to reduce any first request penalty.
+//
+// Note: While this is technically AOT, this does not imply any action on your
+// part. wazero automatically performs ahead-of-time compilation as needed when
+// Runtime.CompileModule is invoked.
+// Note: This panics at runtime the runtime.GOOS or runtime.GOARCH does not
+// support Compiler. Use NewRuntimeConfig to safely detect and fallback to
+// NewRuntimeConfigInterpreter if needed.
+func NewRuntimeConfigCompiler() RuntimeConfig {
 	ret := *engineLessConfig // copy
-	ret.newEngine = jit.NewEngine
+	ret.newEngine = compiler.NewEngine
 	return &ret
 }
 

config_supported.go

@@ -2,9 +2,9 @@
 
 package wazero
 
-const JITSupported = true
+const CompilerSupported = true
 
-// NewRuntimeConfig returns NewRuntimeConfigJIT
+// NewRuntimeConfig returns NewRuntimeConfigCompiler
 func NewRuntimeConfig() RuntimeConfig {
-	return NewRuntimeConfigJIT()
+	return NewRuntimeConfigCompiler()
 }

config_unsupported.go

@@ -2,7 +2,7 @@
 
 package wazero
 
-const JITSupported = false
+const CompilerSupported = false
 
 // NewRuntimeConfig returns NewRuntimeConfigInterpreter
 func NewRuntimeConfig() RuntimeConfig {

internal/asm/amd64/assembler.go

@@ -4,7 +4,7 @@ import (
 	"github.com/tetratelabs/wazero/internal/asm"
 )
 
-// Assembler is the interface used by amd64 JIT compiler.
+// Assembler is the interface used by amd64 compiler.
 type Assembler interface {
 	asm.AssemblerBase
 

internal/asm/amd64/consts.go

@@ -23,7 +23,7 @@ const (
 
 // AMD64-specific instructions.
 //
-// Note: This only defines amd64 instructions used by wazero's JIT compiler.
+// Note: This only defines amd64 instructions used by wazero's compiler.
 // Note: Naming conventions intentionally match the Go assembler: https://go.dev/doc/asm
 // See https://www.felixcloutier.com/x86/index.html
 const (

internal/asm/arm64/consts.go

@@ -340,7 +340,7 @@ func RegisterName(r asm.Register) string {
 
 // Arm64-specific instructions.
 //
-// Note: This only defines arm64 instructions used by wazero's JIT compiler.
+// Note: This only defines arm64 instructions used by wazero's compiler.
 // Note: Naming conventions intentionally match the Go assembler: https://go.dev/doc/asm
 const (
 	NOP asm.Instruction = iota

internal/asm/arm64/impl.go

@@ -684,8 +684,8 @@ func (a *AssemblerImpl) EncodeRelativeBranch(n *NodeImpl) (err error) {
 			const maxSignedInt19 int64 = 1<<19 - 1
 			const minSignedInt19 int64 = -(1 << 19)
 			if offset < minSignedInt19 || offset > maxSignedInt19 {
-				// This should be a bug in our JIT compiler as the conditional jumps are only used in the small offsets (~a few bytes),
-				// and if ever happens, JIT compiler can be fixed.
+				// This should be a bug in our compiler as the conditional jumps are only used in the small offsets (~a few bytes),
+				// and if ever happens, compiler can be fixed.
 				return fmt.Errorf("BUG: relative jump offset %d/4(=%d)must be within %d and %d", offset, imm19, minSignedInt19, maxSignedInt19)
 			}
 			// https://developer.arm.com/documentation/ddi0596/2021-12/Base-Instructions/B-cond--Branch-conditionally-?lang=en
@@ -1522,7 +1522,7 @@ func (a *AssemblerImpl) encodeLoadOrStoreWithRegisterOffset(
 
 // validateMemoryOffset validates the memory offset if the given offset can be encoded in the assembler.
 // In theory, offset can be any, but for simplicity of our homemade assembler, we limit the offset range
-// that can be encoded enough for supporting JIT compiler.
+// that can be encoded enough for supporting compiler.
 func validateMemoryOffset(offset int64) (err error) {
 	if offset > 255 && offset%8 != 0 {
 		// This is because we only have large offsets for load/store with Wasm value stack, and its offset

internal/asm/assembler.go

@@ -120,7 +120,7 @@ type AssemblerBase interface {
 // JumpTableMaximumOffset represents the limit on the size of jump table in bytes.
 // When users try loading an extremely large WebAssembly binary which contains a br_table
 // statement with approximately 4294967296 (2^32) targets. Realistically speaking, that kind of binary
-// could result in more than ten gigabytes of native JITed code where we have to care about
+// could result in more than ten gigabytes of native compiled code where we have to care about
 // huge stacks whose height might exceed 32-bit range, and such huge stack doesn't work with the
 // current implementation.
 const JumpTableMaximumOffset = math.MaxUint32

internal/engine/compiler/RATIONALE.md

@@ -0,0 +1,67 @@
+# Compiler engine
+
+This package implements the Compiler engine for WebAssembly *purely written in Go*.
+In this README, we describe the background, technical difficulties and some design choices.
+
+## General limitations on pure Go Compiler engines
+
+In Go program, each Goroutine manages its own stack, and each item on Goroutine
+stack is managed by Go runtime for garbage collection, etc.
+
+These impose some difficulties on compiler engine purely written in Go because
+we *cannot* use native push/pop instructions to save/restore temporary
+variables spilling from registers. This results in making it impossible for us
+to invoke Go functions from compiled native codes with the native `call`
+instruction since it involves stack manipulations.
+
+*TODO: maybe it is possible to hack the runtime to make it possible to achieve
+function calls with `call`.*
+
+## How to generate native codes
+
+wazero uses its own assembler, implemented from scratch in the
+[`internal/asm`](../../asm/) package. The primary rationale are wazero's zero
+dependency policy, and to enable concurrent compilation (a feature the
+WebAssembly binary format optimizes for).
+
+Before this, wazero used [`twitchyliquid64/golang-asm`](https://github.com/twitchyliquid64/golang-asm).
+However, this was not only a dependency (one of our goals is to have zero
+dependencies), but also a large one (several megabytes added to the binary).
+Moreover, any copy of golang-asm is not thread-safe, so can't be used for
+concurrent compilation (See [#233](https://github.com/tetratelabs/wazero/issues/233)).
+
+The assembled native codes are represented as `[]byte` and the slice region is
+marked as executable via mmap system call.
+
+## How to enter native codes
+
+Assuming that we have a native code as `[]byte`, it is straightforward to enter
+the native code region via Go assembly code. In this package, we have the
+function without body called `compilercall`
+
+```go
+func compilercall(codeSegment, engine, memory uintptr)
+```
+
+where we pass `codeSegment uintptr` as a first argument. This pointer is to the
+first instruction to be executed. The pointer can be easily derived from
+`[]byte` via `unsafe.Pointer`:
+
+```go
+code := []byte{}
+/* ...Compilation ...*/
+codeSegment := uintptr(unsafe.Pointer(&code[0]))
+compilercall(codeSegment, ...)
+```
+
+And `compilercall` is actually implemented in [arch_amd64.s](./arch_amd64.s)
+as a convenience layer to comply with the Go's official calling convention.
+We delegate the task to jump into the code segment to the Go assembler code.
+
+## How to achieve function calls
+
+Given that we cannot use `call` instruction at all in native code, here's how
+we achieve the function calls back and forth among Go and (compiled) Wasm
+native functions.
+
+TODO:

internal/wasm/jit/arch.go → internal/engine/compiler/arch.go

@@ -1,14 +1,14 @@
-package jit
+package compiler
 
 var (
 	// newArchContext returns a new archContext which is architecture-specific type to be embedded in callEngine.
 	// This must be initialized in init() function in architecture-specific arch_*.go file which is guarded by build tag.
 	newArchContext func() archContext
 )
 
-// jitcall is used by callEngine.execWasmFunction and the entrypoint to enter the JITed native code.
+// compilercall is used by callEngine.execWasmFunction and the entrypoint to enter the compiled native code.
 // codeSegment is the pointer to the initial instruction of the compiled native code.
 // ce is "*callEngine" as uintptr.
 //
 // Note: this is implemented in per-arch Go assembler file. For example, arch_amd64.s implements this for amd64.
-func jitcall(codeSegment, ce uintptr, moduleInstanceAddress uintptr)
+func compilercall(codeSegment, ce uintptr, moduleInstanceAddress uintptr)

internal/wasm/jit/arch_amd64.go → internal/engine/compiler/arch_amd64.go

@@ -1,4 +1,4 @@
-package jit
+package compiler
 
 import (
 	"github.com/tetratelabs/wazero/internal/wazeroir"

internal/wasm/jit/arch_amd64.s → internal/engine/compiler/arch_amd64.s

@@ -1,8 +1,8 @@
 #include "funcdata.h"
 #include "textflag.h"
 
-// jitcall(codeSegment, ce, moduleInstanceAddress)
-TEXT ·jitcall(SB),NOSPLIT|NOFRAME,$0-24
+// compilercall(codeSegment, ce, moduleInstanceAddress)
+TEXT ·compilercall(SB),NOSPLIT|NOFRAME,$0-24
         MOVQ ce+8(FP),R13                     // Load the address of *callEngine. into amd64ReservedRegisterForCallEngine.
         MOVQ moduleInstanceAddress+16(FP),R12 // Load the address of *wasm.ModuleInstance into amd64CallingConventionModuleInstanceAddressRegister.
         MOVQ codeSegment+0(FP),AX             // Load the address of native code.

internal/wasm/jit/arch_arm64.go → internal/engine/compiler/arch_arm64.go

@@ -1,4 +1,4 @@
-package jit
+package compiler
 
 import (
 	"math"
@@ -13,12 +13,12 @@ func init() {
 
 // archContext is embedded in callEngine in order to store architecture-specific data.
 type archContext struct {
-	// jitCallReturnAddress holds the absolute return address for jitcall.
-	// The value is set whenever jitcall is executed and done in jit_arm64.s
+	// compilerCallReturnAddress holds the absolute return address for compilercall.
+	// The value is set whenever compilercall is executed and done in compiler_arm64.s
 	// Native code can return back to the ce.execWasmFunction's main loop back by
 	// executing "ret" instruction with this value. See arm64Compiler.exit.
-	// Note: this is only used by JIT code so mark this as nolint.
-	jitCallReturnAddress uint64 //nolint
+	// Note: this is only used by Compiler code so mark this as nolint.
+	compilerCallReturnAddress uint64 //nolint
 
 	// Loading large constants in arm64 is a bit costly, so we place the following
 	// consts on callEngine struct so that we can quickly access them during various operations.

internal/wasm/jit/arch_arm64.s → internal/engine/compiler/arch_arm64.s

@@ -1,12 +1,12 @@
 #include "funcdata.h"
 #include "textflag.h"
 
-// jitcall(codeSegment, ce, moduleInstanceAddress)
-TEXT ·jitcall(SB),NOSPLIT|NOFRAME,$0-24
+// compilercall(codeSegment, ce, moduleInstanceAddress)
+TEXT ·compilercall(SB),NOSPLIT|NOFRAME,$0-24
         // Load the address of *callEngine into arm64ReservedRegisterForCallEngine.
         MOVD ce+8(FP),R0
         // In arm64, return address is stored in R30 after jumping into the code.
-        // We save the return address value into archContext.jitReturnAddress in Engine.
+        // We save the return address value into archContext.compilerReturnAddress in Engine.
         // Note that the const 136 drifts after editting Engine or archContext struct. See TestArchContextOffsetInEngine.
         MOVD R30,136(R0)
         // Load the address of *wasm.ModuleInstance into arm64CallingConventionModuleInstanceAddressRegister.