Undefined symbol _emscripten_memcpy_js linking no object files with `-lc` #22758

hoodmane · 2024-10-17T13:15:41Z

Edit: To reproduce doesn't require any rust at all:

$ emcc -lc -sDISABLE_EXCEPTION_CATCHING=0 -sWASM_BIGINT
error: undefined symbol: _emscripten_memcpy_js (referenced by root reference (e.g. compiled C/C++ code))
warning: To disable errors for undefined symbols use `-sERROR_ON_UNDEFINED_SYMBOLS=0`
warning: __emscripten_memcpy_js may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors
emcc: error: '/home/rchatham/Documents/programming/emsdk/node/18.20.3_64bit/bin/node /home/rchatham/Documents/programming/emsdk/upstream/emscripten/src/compiler.mjs /tmp/tmpc06i0z65.json' failed (returned 1)

Similar bug to #22742. If you remove either -lc or -sWASM_BIGINT from the link command the error goes away.

Version of emscripten/emsdk:

$ emcc -v
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.69 (c55ff404dbe1eb9f263c48b584d5ba627224763a)
clang version 20.0.0git (https:/github.com/llvm/llvm-project 50866e84d1da8462aeb96607bf6d9e5bbd5869c5)
Target: wasm32-unknown-emscripten
Thread model: posix

Failing command line in full:

echo "fn main(){}" > a.rs
rustc -Clink-arg=-sWASM_BIGINT a.rs -o a.js --target=wasm32-unknown-emscripten

Full link command and output with -v appended:
Link command:

emcc \
a.a.5e989e15ed0bb312-cgu.0.rcgu.o a.0l3w7uzzjgtykfn4cqm4q6yup.rcgu.o \
/home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-1d3ccab4f7990fa9.rlib \
/home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libpanic_unwind-7c13b9baf1486a91.rlib \
/home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc-6b694a8bc75de25c.rlib \
/home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcore-99daaba8bfbbfe7e.rlib \
-o a.js \
-sWASM_BIGINT -lc -v --reproducer=repro.tar

Output:

 /home/rchatham/Documents/programming/emsdk/upstream/bin/clang --version
 /home/rchatham/Documents/programming/emsdk/upstream/bin/wasm-ld -o a.wasm a.a.5e989e15ed0bb312-cgu.0.rcgu.o a.0l3w7uzzjgtykfn4cqm4q6yup.rcgu.o -L/home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-1d3ccab4f7990fa9.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libpanic_unwind-7c13b9baf1486a91.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_demangle-fb333a2ac0a82ceb.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd_detect-dacc3810688bd02e.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libhashbrown-c5a0e9d03eccf5a2.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_alloc-81ea19ddb8698fb5.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libminiz_oxide-940a4059390b281a.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libadler-3449d83794adc734.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libunwind-41f0d2fd6457e3c4.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcfg_if-dbc33fcbc9032cd3.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liblibc-aaba565dd73ad2b0.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc-6b694a8bc75de25c.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_core-499c8f9444f2d48c.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcore-99daaba8bfbbfe7e.rlib /home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcompiler_builtins-592839b635ef17b6.rlib -lc-debug -L/home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib -L/home/rchatham/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/self-contained -L/home/rchatham/Documents/programming/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten -lGL-getprocaddr -lal -lhtml5 -lbulkmemory -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++ -lc++abi-debug -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr /tmp/tmp924vfcwvlibemscripten_js_symbols.so --strip-debug --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_get_current --export=emscripten_stack_init --export=_emscripten_stack_alloc --export=__cxa_can_catch --export=__cxa_increment_exception_refcount --export=__cxa_decrement_exception_refcount --export=setThrew --export=__cxa_free_exception --export=__wasm_call_ctors --export=_emscripten_stack_restore --export=__get_exception_message --export=free --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=main --export-if-defined=__main_argc_argv --export-if-defined=fflush --export-table -z stack-size=65536 --no-growable-memory --initial-heap=16777216 --no-entry --stack-first --table-base=1
 /home/rchatham/Documents/programming/emsdk/upstream/bin/llvm-objcopy a.wasm a.wasm --remove-section=.debug* --remove-section=producers
 /home/rchatham/Documents/programming/emsdk/node/18.20.3_64bit/bin/node /home/rchatham/Documents/programming/emsdk/upstream/emscripten/src/compiler.mjs /tmp/tmprtajj_6r.json
error: undefined symbol: _emscripten_memcpy_js (referenced by root reference (e.g. compiled C/C++ code))
warning: To disable errors for undefined symbols use `-sERROR_ON_UNDEFINED_SYMBOLS=0`
warning: __emscripten_memcpy_js may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors
emcc: error: '/home/rchatham/Documents/programming/emsdk/node/18.20.3_64bit/bin/node /home/rchatham/Documents/programming/emsdk/upstream/emscripten/src/compiler.mjs /tmp/tmprtajj_6r.json' failed (returned 1)

The text was updated successfully, but these errors were encountered:

hoodmane · 2024-10-17T15:48:44Z

It seems that it fails with "BULK_MEMORY": true and works with "BULK_MEMORY": false in the json input to compiler.mjs.

hoodmane · 2024-10-17T16:01:14Z

So together -sDISABLE_EXCEPTION_CATCHING=1 -lc puts _emscripten_memcpy_js in the imports, -sWASM_BIGINT increases the minimum safari version to 15000 which implies BULK_MEMORY. The reproduction also works with:

emcc -lc -sDISABLE_EXCEPTION_CATCHING=0 -sMIN_SAFARI_VERSION=150000

I guess the bug is the fact that _emscripten_memcpy_js shows up in the import set?

hoodmane · 2024-10-17T16:08:59Z

Clearing out emscripten_memcpy.c and replacing it with a dummy definition fixes the error, so it has something to do with that file. Passing -lc somehow is getting it to hit the case where __wasm_bulk_memory__ is false?

sbc100 · 2024-10-17T18:58:58Z

Clearing out emscripten_memcpy.c and replacing it with a dummy definition fixes the error, so it has something to do with that file. Passing -lc somehow is getting it to hit the case where __wasm_bulk_memory__ is false?

Yes, it seems that library.js is being processed with BULK_MEMORY set which mean it doesn't need to include _emscripten_memcpy_js. However the bulkmemory variant of memcpy is not being linked as a native library.

I believe what is happening is that libbulkmemory is designed to always come before libc so that it can override functiosn in libc, but when you explicitly add libc its putting libc first:

emscripten/tools/system_libs.py

Lines 2361 to 2369 in f6ed386

    
           # C libraries that override libc must come before it 
        
           if settings.PRINTF_LONG_DOUBLE: 
        
             add_library('libprintf_long_double') 
        
           # See comment in libc_optz itself 
        
           if settings.SHRINK_LEVEL >= 2 and not settings.LINKABLE and \ 
        
              not os.environ.get('EMCC_FORCE_STDLIBS'): 
        
             add_library('libc_optz') 
        
           if settings.BULK_MEMORY: 
        
             add_library('libbulkmemory')

Using this technique of overriding libc function always us to ship fewer versions of libc as a whole, but does run into this kind of issue. I'm not sure what the best fix is yet...

sbc100 · 2024-10-17T19:00:32Z

Should we perhaps just a very hacky and ignore explictly -lc on the command line, right now we map it the appropriate libc variant, but that messes up the ordering we want to enforce

hoodmane · 2024-10-18T08:56:07Z

Right, so injecting the system libs ahead of the other link flag makes the error go away:

--- a/tools/link.py
+++ b/tools/link.py
@@ -1850,7 +1850,7 @@ def phase_calculate_system_libraries(linker_arguments, newargs):
     # Ports are always linked into the main module, never the side module.
     extra_files_to_link += ports.get_libs(settings)
   extra_files_to_link += system_libs.calculate(newargs)
-  linker_arguments.extend(extra_files_to_link)
+  linker_arguments[0:0] = extra_files_to_link

Also this works:

--- a/tools/link.py
+++ b/tools/link.py
@@ -2761,6 +2761,8 @@ def process_libraries(state, linker_inputs):
       new_flags.append((i, flag))
       continue
     lib = removeprefix(flag, '-l')
+    if lib == "c":
+      continue
 
     logger.debug('looking for library "%s"', lib)

What about if someone passes -nodefaultlibs though?

There have been a lot of bugs when the caller passes `-lc` over the years. For example it crashes if we do: ``` emcc -lc -sDISABLE_EXCEPTION_CATCHING=0 -sMIN_SAFARI_VERSION=150000 ``` Rust likes to pass `-lc`. Let's drop it and stop causing trouble for Rust. Resolves emscripten-core#22758, emscripten-core#22742 and would have resolved emscripten-core#16680 if it hadn't disappeared first.

Hack: drop -lc to work around buggy Emscripten behaviors when -lc is passed. This can be removed when the problems are resolved in Emscripten and we drop support for the currently existing versions of Emscripten. See emscripten-core/emscripten#22758 Unblocks rust-lang#131736

hoodmane mentioned this issue Oct 17, 2024

Emscripten: link with -sWASM_BIGINT rust-lang/rust#131736

Open

hoodmane changed the title ~~Undefined symbol _emscripten_memcpy_js when linking rust code with -sWASM_BIGINT~~ Undefined symbol _emscripten_memcpy_js from emcc -lc -sDISABLE_EXCEPTION_CATCHING=0 -sWASM_BIGINT Oct 17, 2024

hoodmane changed the title ~~Undefined symbol _emscripten_memcpy_js from emcc -lc -sDISABLE_EXCEPTION_CATCHING=0 -sWASM_BIGINT~~ Undefined symbol _emscripten_memcpy_js linking empty file with -lc Oct 17, 2024

hoodmane changed the title ~~Undefined symbol _emscripten_memcpy_js linking empty file with -lc~~ Undefined symbol _emscripten_memcpy_js linking no object files with -lc Oct 17, 2024

hoodmane linked a pull request Oct 18, 2024 that will close this issue

Drop explicitly passed -lc unless -nodefaultlibs or similar is passed #22765

Open

hoodmane mentioned this issue Oct 18, 2024

Do not pass -lc to the Emscripten linker rust-lang/rust#131885

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Undefined symbol _emscripten_memcpy_js linking no object files with `-lc` #22758

Undefined symbol _emscripten_memcpy_js linking no object files with `-lc` #22758

hoodmane commented Oct 17, 2024 •

edited

Loading

hoodmane commented Oct 17, 2024 •

edited

Loading

hoodmane commented Oct 17, 2024

hoodmane commented Oct 17, 2024

sbc100 commented Oct 17, 2024

sbc100 commented Oct 17, 2024

hoodmane commented Oct 18, 2024

Undefined symbol _emscripten_memcpy_js linking no object files with -lc #22758

Undefined symbol _emscripten_memcpy_js linking no object files with -lc #22758

Comments

hoodmane commented Oct 17, 2024 • edited Loading

hoodmane commented Oct 17, 2024 • edited Loading

hoodmane commented Oct 17, 2024

hoodmane commented Oct 17, 2024

sbc100 commented Oct 17, 2024

sbc100 commented Oct 17, 2024

hoodmane commented Oct 18, 2024

Undefined symbol _emscripten_memcpy_js linking no object files with `-lc` #22758

Undefined symbol _emscripten_memcpy_js linking no object files with `-lc` #22758

hoodmane commented Oct 17, 2024 •

edited

Loading

hoodmane commented Oct 17, 2024 •

edited

Loading