Add a wasmtime-specific wasmtime_wat2wasm C API

[alexcrichton]

This commit implements a wasmtime-specific C API for converting the text
format to the binary format. An upstream spec issue exists for adding
this to the C API, but in the meantime we can experiment with our own
version of this API and use it in the C# extension, for example!

Closes #1000

[peterhuene]

Looks great! Just the two comments below.

[peterhuene]

👍 Thanks for fixing this so we could remove all the .wasm files, they were bugging me too.

[rene-fonseca]

Question: Would it be possible to support line number and column info for parse failure - ie. separate from error message? Or maybe just prepare API for now - and then add info later if it becomes possible to support. Just to avoid having to change API later.

[peterhuene]

I believe the wat crate already provides line and column, along with contextual information of where the parse error was.

[rene-fonseca]

EDITED:

Looking at https://webassembly.github.io/spec/core/text/lexical.html - it seems that null-terminator is allowed in comments. Or am I misunderstanding something here.

Hmm - thought I saw it using const char* - but looking again it is using a wasm_byte_vec_t which is good for the API itself.

But maybe double check if null-terminator is supported internally.

let wat = match str::from_utf8((*wat).as_slice()) {

https://github.com/bytecodealliance/wasmtime/pull/1206/files#diff-c69f02cd99a3d2831c084254572074e6R98

[peterhuene]

Oh sorry, I misread that. As separate pieces of information, not how we're doing it currently, no.

@alexcrichton is there an error type we can extract the line and column from?

[alexcrichton]

Yes the wat/wast crates have enough information to return line numbers of errors. The C API in general doesn't do super well with errors and adding rich information like filename/line number (as opposed to just a string) would likely require us to invent new idioms/machinery/etc. I don't think there's any reason why we can't do that, but I would imagine that if we were to focus on errors in the C API we should take a look at them as a whole instead of only situation-solving one API.

[rene-fonseca]

Maybe add some note in API documentation about if UTF-8 BOM is allowed or not. Not sure if Rust runtime will strip it. If it is supposed to work then it would be good to have a wat test file with a BOM.

[alexcrichton]

Oh I forgot to mention earlier, but your comment about the nul byte was previously correct when it was using char* but now that it's using wasm_byte_vec_t* that should be handled because it's a pointer/length.

For BOM handling Rust doesn't do anything special there, it would be up to the lexer, if anything, to skip it. Currently the wat parser doesn't skip it and will fail on any file that contains the BOM marker. Looking at the lexical structure of the text format I don't think the BOM is allowed, so it sounds like this may be a spec question/update/tweak to bring up with the official repo.

[rene-fonseca]

Thanks @alexcrichton. But one note - BOM, if handled, should only be done at the decoding/encoding layer and not passed on to the internal Unicode representation. So not direct impact here for lexer.

[alexcrichton]

Ok I'm gonna go ahead and land this and we can continue to iterate on specifics in-tree.