llvm-nm met The end of the file was unexpectedly encountered error

[vvakame]

I'm playing Rust + WebAssembly + emscripten sdk-incoming-64bit.
I encounter compiling issue after #5004 merged.

My environment

$ rustc -V
rustc 1.17.0-nightly (b1e31766d 2017-03-03)
$ emcc -v
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 1.37.3
clang version 3.9.0 (https://github.com/kripken/emscripten-fastcomp-clang/ 5725e3dd8325570ac9d92924fb1af5db9e879e58) (https://github.com/kripken/emscripten-fastcomp/ 9735be7b8bee9c207f33244fe3c5f85f45094470) (emscripten 1.37.3 : 1.37.3)
Target: x86_64-apple-darwin16.4.0
Thread model: posix
InstalledDir: /Users/vvakame/work/emsdk_portable/clang/fastcomp/build_incoming_64/bin
INFO:root:(Emscripten: Running sanity checks)

reproduction

$ source ~/work/emsdk_portable/emsdk_env.sh
$ echo "pub fn main() {}" > main.rs
$ rustc --target wasm32-unknown-emscripten main.rs
error: linking with `emcc` failed: exit code: 1
  |
  = note: "emcc" "-s" "DISABLE_EXCEPTION_CATCHING=0" "-L" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib" "main.0.o" "-o" "main.js" "-s" "EXPORTED_FUNCTIONS=[\"_main\",\"_rust_eh_personality\"]" "-O0" "--memory-init-file" "0" "-g0" "-s" "DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=[]" "-L" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-9a66b6a343d52844.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/librand-6bc49e032a89c77d.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libcollections-a2a467c3ca3b6479.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libstd_unicode-e54225ff8f33e08f.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libpanic_unwind-9d79f761aa668a33.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libunwind-2beb731af7a6faec.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc-ce7b9706e1719f27.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc_system-5636d8d1255715e9.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/liblibc-95af4192ed69a1c8.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libcore-cd0ca85e71f914ca.rlib" "/Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libcompiler_builtins-0bf24067248742a8.rlib" "-l" "c" "-s" "BINARYEN=1" "-s" "ERROR_ON_UNDEFINED_SYMBOLS=1"
  = note: Traceback (most recent call last):
            File "/Users/vvakame/work/emsdk_portable/emscripten/incoming/emcc", line 13, in <module>
              emcc.run()
            File "/Users/vvakame/work/emsdk_portable/emscripten/incoming/emcc.py", line 1540, in run
              extra_files_to_link += system_libs.calculate([f for _, f in sorted(temp_files)] + extra_files_to_link, in_temp, stdout_=None, stderr_=None, forced=forced_stdlibs)
            File "/Users/vvakame/work/emsdk_portable/emscripten/incoming/tools/system_libs.py", line 304, in calculate
              symbolses = shared.Building.parallel_llvm_nm(map(os.path.abspath, temp_files))
            File "/Users/vvakame/work/emsdk_portable/emscripten/incoming/tools/shared.py", line 1623, in parallel_llvm_nm
              raise Exception('llvm-nm failed on file ' + files[i] + ': return code ' + str(object_contents[i].returncode) + ', error: ' + object_contents[i].output)
          Exception: llvm-nm failed on file /Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-9a66b6a343d52844.rlib: return code 1, error:
          std-9a66b6a343d52844.0.o:
                   U _Unwind_Backtrace
                   U _Unwind_FindEnclosingFunction
                   U _Unwind_GetIPInfo
          -------- T _ZN101_$LT$$RF$$u27$a$u20$$u5b$std..net..addr..SocketAddr$u5d$$u20$as$u20$std..net..addr..ToSocketAddrs$GT$15to_socket_addrs17h5114a9a05148274dE
          -------- T _ZN101_$LT$std..ffi..os_str..OsString$u20$as$u20$core..convert..From$LT$collections..string..String$GT$$GT$4from17h5b6e8ea4a4507a33E
          -------- T _ZN101_$LT$std..io..cursor..Cursor$LT$$RF$$u27$a$u20$mut$u20$$u5b$u8$u5d$$GT$$u20$as$u20$std..io..Write$GT$5flush17h4852463c3395cc24E

too loooooong errors...

          -------- s vtable.i
          -------- s vtable.j
          -------- s vtable.k
          -------- s vtable.l
          -------- s vtable.m
                   U waitpid
                   U write
          /Users/vvakame/work/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-9a66b6a343d52844.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered

error: aborting due to previous error

workaround

$ cd ~/work/emsdk_portable/emscripten/incoming
$ git rev-parse HEAD
800c110c1dd8cbc138ddb255e35076eb3fd94ab0
$ git checkout eac8e0b0142ae5466a79b8aec070521dfd75ea76
$ cd -
/tmp/test
$ rustc --target wasm32-unknown-emscripten main.rs
$ echo $?
0

Is this emscripten issue or rust compiler's latent error?

NOTE Rust environment setup

$ curl https://sh.rustup.rs -sSf | sh
$ source $HOME/.cargo/env
$ rustup install nightly
$ rustup target add wasm32-unknown-emscripten --toolchain nightly
$ rustup default nightly

[juj]

That commit added error detection to the llvm-nm calls, which were previously missing. I believe the root issue is this line:

/Users/vvakame/work/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-9a66b6a343d52844.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered

Perhaps the issue was always there with rust, but the compiler was fine to produce functional output executables when the error got ignore.

Initially it looks like perhaps llvm-nm is being called on the file "rust.metadata.bin" which is not an object file. Does that sound plausible? That suggests that the fix would be to make the rust compiler toolchain not attempt to llvm-nm files that are not object files.

[juj]

Further thinking, I realize that it can be fine that an .a file contains input files that are not object files, and it is too rushy to abort compilation by throwing an exception if an object file cannot be processed, so demoted those exceptions to debug messages instead. Errors are still propagated to caller by passing the returncode attribute, so error detection is not lost. That commit should fix up the issue.

[vvakame]

To be honest, my knowledge of the domain is not enough. 😿

I was guessed, rustc passed parameter only *.rlib files.
It was expected arguments by rustc.

Sorry, I can't found rlib's spec...
rust-lang/rust#21482 (comment)
rust-lang/rust#25820 (comment)

[vvakame]

93a082b solved my issue. thanks!

[vvakame]

I tried above command.
I got same error by llvm-nm that installed via Homebrew (macOS).

$ /usr/local/Cellar/llvm/3.9.1/bin/llvm-nm /Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-9a66b6a343d52844.rlib

std-9a66b6a343d52844.0.o:
         U _Unwind_Backtrace
         U _Unwind_FindEnclosingFunction
         U _Unwind_GetIPInfo
-------- T _ZN101_$LT$$RF$$u27$a$u20$$u5b$std..net..addr..SocketAddr$u5d$$u20$as$u20$std..net..addr..ToSocketAddrs$GT$15to_socket_addrs17h5114a9a05148274dE
-------- T _ZN101_$LT$std..ffi..os_str..OsString$u20$as$u20$core..convert..From$LT$collections..string..String$GT$$GT$4from17h5b6e8ea4a4507a33E
-------- T _ZN101_$LT$std..io..cursor..Cursor$LT$$RF$$u27$a$u20$mut$u20$$u5b$u8$u5d$$GT$$u20$as$u20$std..io..Write$GT$5flush17h4852463c3395cc24E
-------- T _ZN101_$LT$std..path..Iter$LT$$u27$a$GT$$u20$as$u20$core..convert..AsRef$LT$std..ffi..os_str..OsStr$GT$$GT$6as_ref17h50d2fc7cbc78b1e1E
-------- T _ZN101_$LT$std..sys..imp..process..process_common..ExitStatus$u20$as$u20$core..convert..From$LT$i32$GT$$GT$4from17h42f94b98e7a8e17dE
-------- T _ZN102_$LT$std..net..addr..SocketAddr$u20$as$u20$core..convert..From$LT$std..net..addr..SocketAddrV4$GT$$GT$4from17hbc8a5d95224bc3c9E

...

-------- s vtable.l
-------- s vtable.m
         U waitpid
         U write
/usr/local/Cellar/llvm/3.9.1/bin/llvm-nm: /Users/vvakame/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-9a66b6a343d52844.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered

[vvakame]

I reported this issue to rust repository. rust-lang/rust#40352
thank you your supporting! please feel free to close.

[stale]

This issue has been automatically marked as stale because there has been no activity in the past 2 years. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.