Wat to Asm linker ?

[ljulliar]

As far as i can see, wat2wasm can only take a single WAT file as input. Are you aware of any tool that would take several wasm file as input (each of them being a module) and assemble them into a single wasm file ?

Or would concatenating all the wat files together before invoking wat2wasm do the trick ?

[binji]

Are you aware of any tool that would take several wasm file as input (each of them being a module) and assemble them into a single wasm file ?

If you generate wasm object files, you can link them with LLVM's wasm-ld tool. wat2wasm can generate object files with wat2wasm -r, though it doesn't support all the features of the object format.

You may also want to look at Lobster's single-file wasm binary writer, which can generate object files too.

And LLVM can generate wasm object files too, if you compile your code with --target=wasm32.

Or would concatenating all the wat files together before invoking wat2wasm do the trick ?

Concatenating all the files might work, as long as you are careful that you don't reuse names, and that you don't reference anything by index (since those will need to be remapped).

[ljulliar]

I went to wasm-ld straigth ahead :-) and got the following error when trying to link wasm object file

;;mod_a.wat
(module $mod_a
  (import "mod_b" "square" (func $square (param i32) (result i32)))

  (start $start)

  ;; _start entry point expected by wasm-ld
  (func $start
    (drop (call $square (i32.const 16)))
  )
)

;;mod_b.wat
(module $mod_b
  (export "square" (func $square))

  (func $square (param $value i32) (result i32)
    (i32.mul (local.get $value) (local.get $value))
  )
)


$ wat2wasm -r ./mod_a.wat
$ wat2wasm -r ./mod_b.wat
$ wasm-ld-8 ./mod_a.wasm ./mod_b.wasm -o main.wasm
wasm-ld-8: error: entry symbol not defined (pass --no-entry to supress): _start
wasm-ld-8: error: ./mod_a.wasm: undefined symbol: square

I don't quite understand why those 2 symbols are undefined (note: whether I name the start function $start or $_start doesn't change a thing). And it certainly doesn't explain why the square function is not recognized.

Any idea ?

[binji]

Hm, it looks like the -r flag doesn't do enough to create an object file. The linker expects there to be a symbol table it can use (see here), but this isn't currently generated.

You may want to try using the LLVM assembler (or clang) to generate object files instead. It would be nice to add this functionality to wat2wasm, but I won't be able to work on this for a while. Maybe someone else will pick it up, however.

@sbc100 @aardappel

[ljulliar]

I have tried using clang already but it doesn' seem to recognize the wat format:

$ clang-8 --target=wasm32 ./mod_b.wat -o mod_b.wasm
wasm-ld: error: cannot open crt1.o: No such file or directory
wasm-ld: error: unknown file type: ./mod_b.wat
wasm-ld: error: unable to find library -lc
wasm-ld: error: cannot open /usr/lib/llvm-8/lib/clang/8.0.1/lib/libclang_rt.builtins-wasm32.a: No such file or directory
clang: error: lld command failed with exit code 1 (use -v to see invocation)

May be I should try llvm-as

[tlively]

LLVM and its associated tools do not recognize the standard WebAssembly text formats. They instead use their own .s format that is much closer to traditional assembly languages.

Binaryen used to have a wasm-merge tool, but it was deleted in WebAssembly/binaryen#1969 because we didn't see much use of it. Perhaps you could build an older version of binaryen that includes it?

[aardappel]

There's a fair bit of complexity in Wasm modules that are linkable, in particular as it relates to relocation, since so much in the module format is indices that all need to change once merged. So using LLD to do this for you is the way to go, trying to merge in any other way will likely end in tears :)

If you want to write small linkable modules by hand, using llvm .s files is currently one of the few options that should work. You'll need to be using tip of tree LLVM as I've been improving this format a fair bit recently, any older LLVM will likely not work. This format follows the general conventions LLVM has for any assembly files, you can see Wasm specific examples here: https://github.com/llvm/llvm-project/tree/master/llvm/test/MC/WebAssembly

Note that assembly assumes you know what you're doing, for example it won't stop you from writing instruction sequences that produce illegal arguments on the stack. You'll need to run the result thru wasm-validate to ensure you've done the right things.

Frankly, you're better off writing your linkable modules in C.

I agree that we should add features to the .wat format to exactly represent the sections in https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md

[wingo]

At this point I think this bug can be closed. With the latest bug-fixes in #1537 in theory wat2wasm --relocatable does everything needed to make .o files, and we can punt linking to the linker; LLD works fine.

[binji]

Sounds good, we likely will want to add other features to make relocations nicer later (e.g. WebAssembly/design#1368), but that's unrelated to this issue.