Emscripten: emcc not found on win32 cross-compile

[vvanders]

Saw this on Nightly a few months back and just thought it was a stability issue but looks like with the recent 1.20 release it's made it into stable:

(note some portion of the paths are sanitized with 'zzz')

error: failed to run custom build command for `lua v0.0.11 (file:///C:/zzz/rust-lua53)`
process didn't exit successfully: `C:\zzz\target\release\build\lua-91f996492c3978b1\build-script-build` (exit code: 101)
--- stdout
TARGET = Some("asmjs-unknown-emscripten")
OPT_LEVEL = Some("3")
TARGET = Some("asmjs-unknown-emscripten")
HOST = Some("x86_64-pc-windows-msvc")
TARGET = Some("asmjs-unknown-emscripten")
TARGET = Some("asmjs-unknown-emscripten")
HOST = Some("x86_64-pc-windows-msvc")
CC_asmjs-unknown-emscripten = None
CC_asmjs_unknown_emscripten = None
TARGET_CC = None
CC = None
TARGET = Some("asmjs-unknown-emscripten")
HOST = Some("x86_64-pc-windows-msvc")
CFLAGS_asmjs-unknown-emscripten = None
CFLAGS_asmjs_unknown_emscripten = None
TARGET_CFLAGS = None
CFLAGS = None
PROFILE = Some("release")
running: "emcc.bat" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-DLUA_32BITS" "-DLUA_COMPAT_LOADSTRING" "-o" "C:\\zzz\target\\asmjs-unknown-emscripten\\release\\build\\lua-7f6f6329781534a1\\out\\lua-source\\src\\lapi.o" "-c" "lua-source\\src\\lapi.c"
cargo:warning=python: can't open file 'C:\zzz\rust-lua53\\emcc': [Errno 2] No such file or directory
exit code: 2

When I was debugging this before the issue came down to how Command was now pulling in certain environment variables.

Specifically the way emcc is invoked through emcc.bat on windows:

@echo off
python "%~dp0\emcc" %*

Here "%~dp0" is supposed to be the folder of where emcc.bat exists, however when run from the rust Command it ends up being the folder that the crate is being build it.

Let me know if this makes better sense as an issue in the core Rust project. Right now I can't build emscripten since 1.20 fails and I can't get 1.19 to install the emscripten component when pinned to 1.19.

[alexcrichton]

Thanks! I believe that the intention here is that cmd is used as the binary rather than emcc.bat itself (which has sketch semantics apparently?)

I can try to work on something locally to see if we can get that working.

[alexcrichton]

Mind testing out bb68302 and seeing if it works?

[vvanders]

Yeah, gave it a shot and no dice. It resets the environment and emscripten has to run from a custom command prompt so it can't find emcc/emar.

It seems like the fundamental issue here is that the new Command from 1.20 doesn't correctly execute bat files with %~dp0.

[vvanders]

Actually, take that back. I just build a simple repro case and I'm seeing what I'd expect. Let me dig a bit more and see what's going on.

[vvanders]

Pinned it down. It has to do with the batch file being searchable in the PATH. If you call the bat file directly it works fine but if call with just the name.

Will put up a git repo with exact repro here in a little bit.

[vvanders]

Okay, repro up here: https://github.com/vvanders/rust_command_repro

Should I open a second issue against the Rust stdlib and reference it here or is this the right place for it?

[alexcrichton]

Hm I think I missed a flag by accident, mind trying again? I'm not sure if cmd searches PATH by default? This was rust-lang/rust#42791 upstream which we decided to close.

[vvanders]

Is there a new commit to try?

I'm a little worried about 42791 being closed because it feels like this is a regression in a stable API.

[alexcrichton]

Oh sorry yes the commit is 510d34f

Unfortunately this was a regression for some cases and not a regression for others. It's not clear that it should be reverted and/or fixed.

[vvanders]

Got it, I'll give that a shot here and let you know how it works.

[vvanders]

So, same error :/

I think the fundamental issue is that on win32 emscripten expects to be run within in Emscripten Command Prompt and so any invocation of Cmd is going to change the environment in ways that it doesn't expect.

I'm having a hard time seeing how Emscripten works on win32 without batch files running with the correct behavior that's expected. Right now I'm going to have to pin all my projects to 0.19.0 because 0.20.0 is a breaking change for us.

[vvanders]

Also here's the specific error at the end:

cargo:warning=python: can't open file 'C:\zzz\\emar': [Errno 2] No such file or directory

Where C:\zzz is the dir that's running gcc-rs from a build.rs script.

[vvanders]

Okay, worked around the emar issue with cmd /c. However what I thought was a simple gcc-rs break is actually looking like a full breakage of the entire Emscripten toolchain on win32.

I'm seeing:

• libc::c_char is changing definition from *const i8 to *const u8 meaning rust bindings to C things fail.
• Base emscritpen support in cargo/rustc is failing the exact same way(my sample repo above fails to compile with --target=asmjs-unknown-emscripten not finding emcc in the same way).

All of these work fine on 1.19.0. I'll open up an issue shortly against the main Rust repo(unless you think it's worth revisiting the closed issue above).

Also totally realize that emscripten isn't Tier 1 support so I understand if this is going to take a little bit of time to resolve correctly. I'm unblocked now that I've found out how to pin to 1.19.0 for the meantime.

[alexcrichton]

Hm ok so to confirm a few things...

libc::c_char is changing definition from *const i8 to *const u8 meaning rust bindings to C things fail.

This was unfortunate, but expected, fallout from rust-lang/libc#742. You may wish to temporarily pin to older versions of libc until the change propagates to rustc stable.

Base emscritpen support in cargo/rustc is failing the exact same way

I'll comment on #44443...

[vvanders]

This was unfortunate, but expected, fallout from rust-lang/libc#742. You may wish to temporarily pin to older versions of libc until the change propagates to rustc stable.

Just to understand this a bit more, does this mean that things will eventually converge? The problem is the project we build goes to ~5 different targets(win32, osx, linux, android and emscripten) so we need to libc/C ABI to be stable across all the platforms for us.

[alexcrichton]

does this mean that things will eventually converge

Indeed! The standard library's definitions were updated in rust-lang/rust#44116. Looks like this didn't make the cut for beta though so I'll nominate it for a backport.

[vvanders]

Okay, great! We'll just stay pinned on 1.19.0 in the short term.

Sorry for being a bit of a pain about all of this, apprecaite all the support.

[alexcrichton]

No not at all! I'm hoping that we can get emscripten to tier 1 in the near-ish future anyway to hopefully smooth out this experience!

[vvanders]

That would be awesome.

I will say that I've been floored with the cross-platform support in Rust. I've got a ~15 year background in doing C/C++ that runs on a wide range of targets and Rust has been impressively smooth.

[alexcrichton]

I believe this has since been fixed, so closing.

[vvanders]

So just had a chance to pull latest nightly + gcc-rs(now cc?) and still seeing an issue with emar.bat:

=python: can't open file 'C:\zzz\\emar': [Errno 2] No such file or directory

I'm guessing that the same cmd trick needs to happen for emar as well?

[alexcrichton]

Oops indeed! Does 4f73673 fix the issue?

[vvanders]

🎉 Yes!

Have a completely working win32 host with that change! Thanks for all the back & forth, I hope I wasn't too much of a pain. If you ever need a win32 guinea pig don't hesitate to reach out.

[alexcrichton]

No problem! Glad I could help out to get things fixed again :)