Add `wasmer create-exe` #1693

MarkMcCaskey · 2020-10-08T21:17:24Z

This adds the wasmer create-exe subcommand. This subcommand is a combination of wasmer compile --object-file and linking that compiled Wasm with a main function and libwasmer. Put more plainly: it lets us transform Wasm modules into native executables in one step.

In order for this to work we need:

Ship wasm.h with Wasmer or use different mechanism to find it
Ship wasmer_wasm.h with Wasmer
Requires up to date libwasmer... had to build one and copy it over, may fail in CI because of this... will be fixed with next release though
More gracefully handle wasmer installed without WASMER_DIR, etc (even if just error messages, should be tested)
Add tests

Review

Add a short description of the the change to the CHANGELOG.md file

lib/cli/src/commands/create_exe.rs

lib/cli/src/commands/wasmer_create_exe_main.c

syrusakbary · 2020-10-08T23:09:36Z

lib/cli/src/commands/create_exe.rs

+fn get_wasmer_include_directory() -> anyhow::Result<PathBuf> {
+    let mut path = PathBuf::from(env::var("WASMER_DIR")?);
+    path.push("include");
+    Ok(path)
+}
+
+fn get_libwasmer_path() -> anyhow::Result<PathBuf> {
+    let mut path = PathBuf::from(env::var("WASMER_DIR")?);
+    path.push("lib");
+
+    #[cfg(not(windows))]
+    path.push("libwasmer.a");
+    #[cfg(windows)]
+    path.push("libwasmer.lib");
+
+    Ok(path)
+}


It would be a good idea to abstract this out, and also reuse them on the lib/cli/src/commands/config.rs file

Yeah it looks like the config logic isn't tested right now... changes Ivan and I have been making to the C API are going to break it I think...

The config code implies we can just do -lwasmer which is interesting that would make the code have fewer cfgs in it.

The code in wasmer config for linking against wasmer seems to default to the dynamic libwasmer 🤔

lib/cli/src/store.rs

Hywan · 2020-10-12T07:47:23Z

I'm not sure what create-exe does. Can you elaborate please?

syrusakbary · 2020-10-12T22:14:43Z

lib/cli/src/commands/wasmer_create_exe_main.c

+  wasm_engine_t* engine = wasm_engine_new_with_config(config);
+  wasm_store_t* store = wasm_store_new(engine);
+
+  wasm_module_t* module = wasmer_object_file_engine_new(store, "qjs.wasm");


I think this is a typo. Can we use the first argument in argv (or "wasmer-exe" in case argc == 0)

argc should never be 0 I think, there's always supposed to be 1 arg which is the name of the program... We could handle it but I'm not aware of a situation in which that can happen

lib/cli/src/commands/create_exe.rs

tests/integration/cli/src/link_code.rs

syrusakbary · 2020-10-12T22:20:01Z

tests/integration/cli/src/link_code.rs

+        #[cfg(windows)]
+        let command = command.arg("-luserenv").arg("-lWs2_32").arg("-ladvapi32");


It seems this logic is duplicated?

Yeah, this is test code. I'm not sure where a good place to share this code would be.

We could maybe make it public in the wasmer-cli crate and then depend on it, but I think it's probably fine to duplicate

Those flags are required to link with wasmer compile, but since we do that in create-exe we also do that

MarkMcCaskey · 2020-10-12T22:59:53Z

bors try

bors · 2020-10-12T23:08:02Z

try

Build failed:

Test on macos

MarkMcCaskey · 2020-10-12T23:18:10Z

bors try

bors · 2020-10-12T23:57:40Z

Build failed:

Test on macos

MarkMcCaskey · 2020-10-13T00:21:59Z

bors r+

bors · 2020-10-13T00:30:59Z

Canceled.

MarkMcCaskey · 2020-10-13T00:31:04Z

bors r-

MarkMcCaskey · 2020-10-13T00:31:10Z

bors r+

1693: Add `wasmer create-exe` r=MarkMcCaskey a=MarkMcCaskey This adds the `wasmer create-exe` subcommand. This subcommand is a combination of `wasmer compile --object-file` and linking that compiled Wasm with a main function and libwasmer. Put more plainly: it lets us transform Wasm modules into native executables in one step. In order for this to work we need: - [x] Ship wasm.h with Wasmer or use different mechanism to find it - [x] Ship wasmer_wasm.h with Wasmer - [x] Requires up to date libwasmer... had to build one and copy it over, may fail in CI because of this... will be fixed with next release though - [x] More gracefully handle wasmer installed without WASMER_DIR, etc (even if just error messages, should be tested) - [x] Add tests # Review - [ ] Add a short description of the the change to the CHANGELOG.md file Co-authored-by: Mark McCaskey <mark@wasmer.io> Co-authored-by: Mark McCaskey <5770194+MarkMcCaskey@users.noreply.github.com>

bors · 2020-10-13T00:55:43Z

Build failed:

Test on macos

MarkMcCaskey · 2020-10-13T01:02:42Z

bors r+

1693: Add `wasmer create-exe` r=MarkMcCaskey a=MarkMcCaskey This adds the `wasmer create-exe` subcommand. This subcommand is a combination of `wasmer compile --object-file` and linking that compiled Wasm with a main function and libwasmer. Put more plainly: it lets us transform Wasm modules into native executables in one step. In order for this to work we need: - [x] Ship wasm.h with Wasmer or use different mechanism to find it - [x] Ship wasmer_wasm.h with Wasmer - [x] Requires up to date libwasmer... had to build one and copy it over, may fail in CI because of this... will be fixed with next release though - [x] More gracefully handle wasmer installed without WASMER_DIR, etc (even if just error messages, should be tested) - [x] Add tests # Review - [ ] Add a short description of the the change to the CHANGELOG.md file Co-authored-by: Mark McCaskey <mark@wasmer.io> Co-authored-by: Mark McCaskey <5770194+MarkMcCaskey@users.noreply.github.com>

bors · 2020-10-13T01:35:10Z

Build failed:

Test on windows

MarkMcCaskey · 2020-10-13T18:12:43Z

bors r+

1693: Add `wasmer create-exe` r=MarkMcCaskey a=MarkMcCaskey This adds the `wasmer create-exe` subcommand. This subcommand is a combination of `wasmer compile --object-file` and linking that compiled Wasm with a main function and libwasmer. Put more plainly: it lets us transform Wasm modules into native executables in one step. In order for this to work we need: - [x] Ship wasm.h with Wasmer or use different mechanism to find it - [x] Ship wasmer_wasm.h with Wasmer - [x] Requires up to date libwasmer... had to build one and copy it over, may fail in CI because of this... will be fixed with next release though - [x] More gracefully handle wasmer installed without WASMER_DIR, etc (even if just error messages, should be tested) - [x] Add tests # Review - [ ] Add a short description of the the change to the CHANGELOG.md file Co-authored-by: Mark McCaskey <mark@wasmer.io> Co-authored-by: Mark McCaskey <5770194+MarkMcCaskey@users.noreply.github.com>

bors · 2020-10-13T18:19:15Z

Build failed:

Audit
Code lint

MarkMcCaskey · 2020-10-13T18:21:27Z

bors r+

1693: Add `wasmer create-exe` r=MarkMcCaskey a=MarkMcCaskey This adds the `wasmer create-exe` subcommand. This subcommand is a combination of `wasmer compile --object-file` and linking that compiled Wasm with a main function and libwasmer. Put more plainly: it lets us transform Wasm modules into native executables in one step. In order for this to work we need: - [x] Ship wasm.h with Wasmer or use different mechanism to find it - [x] Ship wasmer_wasm.h with Wasmer - [x] Requires up to date libwasmer... had to build one and copy it over, may fail in CI because of this... will be fixed with next release though - [x] More gracefully handle wasmer installed without WASMER_DIR, etc (even if just error messages, should be tested) - [x] Add tests # Review - [ ] Add a short description of the the change to the CHANGELOG.md file Co-authored-by: Mark McCaskey <mark@wasmer.io> Co-authored-by: Mark McCaskey <5770194+MarkMcCaskey@users.noreply.github.com>

bors · 2020-10-13T18:28:34Z

Build failed:

Audit
Code lint

MarkMcCaskey · 2020-10-13T20:47:22Z

bors r+

1693: Add `wasmer create-exe` r=MarkMcCaskey a=MarkMcCaskey This adds the `wasmer create-exe` subcommand. This subcommand is a combination of `wasmer compile --object-file` and linking that compiled Wasm with a main function and libwasmer. Put more plainly: it lets us transform Wasm modules into native executables in one step. In order for this to work we need: - [x] Ship wasm.h with Wasmer or use different mechanism to find it - [x] Ship wasmer_wasm.h with Wasmer - [x] Requires up to date libwasmer... had to build one and copy it over, may fail in CI because of this... will be fixed with next release though - [x] More gracefully handle wasmer installed without WASMER_DIR, etc (even if just error messages, should be tested) - [x] Add tests # Review - [ ] Add a short description of the the change to the CHANGELOG.md file Co-authored-by: Mark McCaskey <mark@wasmer.io> Co-authored-by: Mark McCaskey <5770194+MarkMcCaskey@users.noreply.github.com>

bors · 2020-10-13T21:15:13Z

Build failed:

Test on windows

MarkMcCaskey · 2020-10-13T21:46:00Z

bors r+

bors · 2020-10-13T22:22:59Z

Build succeeded:

jaykrell · 2021-06-26T03:38:08Z

Sounds great, not very turnkey or well documented from what I see so far..

Hywan · 2021-07-05T07:34:13Z

It is true. We are going to document that more. Thanks!

Add first draft of wasmer create-exe

65265db

syrusakbary reviewed Oct 8, 2020

View reviewed changes

lib/cli/src/commands/create_exe.rs Outdated Show resolved Hide resolved

syrusakbary reviewed Oct 8, 2020

View reviewed changes

lib/cli/src/commands/wasmer_create_exe_main.c Outdated Show resolved Hide resolved

syrusakbary reviewed Oct 8, 2020

View reviewed changes

lib/cli/src/store.rs Show resolved Hide resolved

Mark McCaskey added 2 commits October 8, 2020 16:33

Add test for wasmer create-exe

4b665ad

Add --dir and --mapdir support to native executable

ba7b24f

Mark McCaskey added 3 commits October 12, 2020 13:48

Add target support to wasmer create-exe

c6460d1

Improve error message when WASMER_DIR is not set

21c1770

Add test using mapdir and dir, fix misc bugs

a0b237c

MarkMcCaskey changed the title ~~Add first draft of wasmer create-exe~~ Add wasmer create-exe Oct 12, 2020

MarkMcCaskey marked this pull request as ready for review October 12, 2020 22:07

MarkMcCaskey requested a review from Hywan as a code owner October 12, 2020 22:07