New LLVM backend: Issue while linking globals

[gabrielcuvillier]

While trying to compile the Doom 3 port with the new LLVM backend, there is a problem at link time:

error: undefined symbol: g$SIMDProcessor
error: undefined symbol: g$_ZN11idHashIndex13INVALID_INDEXE
error: undefined symbol: g$_ZN5idLib10cvarSystemE
error: undefined symbol: g$_ZN5idLib10fileSystemE
error: undefined symbol: g$_ZN5idLib3sysE
error: undefined symbol: g$_ZN5idLib6commonE
error: undefined symbol: g$_ZN6idCVar10staticVarsE
error: undefined symbol: g$_ZN6idMatX7tempPtrE
error: undefined symbol: g$_ZN6idMath10SQRT_THREEE
error: undefined symbol: g$_ZN6idMath11FLT_EPSILONE
error: undefined symbol: g$_ZN6idMath11SQRT_1OVER2E
error: undefined symbol: g$_ZN6idMath12ONEFOURTH_PIE
error: undefined symbol: g$_ZN6idMath2PIE
error: undefined symbol: g$_ZN6idMath5iSqrtE
error: undefined symbol: g$_ZN6idMath6TWO_PIE
error: undefined symbol: g$_ZN6idMath7HALF_PIE
error: undefined symbol: g$_ZN6idMath8INFINITYE
error: undefined symbol: g$_ZN6idMath9M_DEG2RADE
error: undefined symbol: g$_ZN6idMath9M_RAD2DEGE
error: undefined symbol: g$_ZN6idVecX7tempPtrE
error: undefined symbol: g$_ZN6idVecX9tempIndexE
error: undefined symbol: g$_ZTV14idSIMD_Generic
error: undefined symbol: g$common
error: undefined symbol: g$cvarSystem
error: undefined symbol: g$mat3_identity
error: undefined symbol: g$vec3_origin
error: undefined symbol: __memory_base
error: undefined symbol: __table_base

Most of the undefined symbols are global variables or constants in D3 code. I am not sure what is the root cause of the problem.
However, "__table_base" and "__memory_base" undefined symbols looks quite suspect.

Any hints ?
This is working fine with fastcomp backend

Note that I am not using dynamic linking (no MAIN_MODULE or SIDE_MODULE). There is static libs though.

To reproduce the problem (I did not isolate the problem, sorry) :
git clone https://github.com/gabrielcuvillier/d3wasm.git
cd d3wasm.git
git checkout -b NoEmterpretify origin/NoEmterpretify
mkdir build-wasm
cd build-wasm
emcmake cmake ../neo -d CMAKE_BUILD_TYPE=Release
emmake make

• issue at link time *

[blockspacer]

Do you include in build files:

• Simd_Generic.cpp
• HashIndex.cpp (https://github.com/search?q=%22idHashIndex%22&type=Code)
  e.t.c.
  ?

Also check simd related build defines/flags (both cmake related and emscripten related)

(Files not found in you repo, but can be found in other github repos, for some reason. Maybe you need version update?)

Also where is declared SQRT_THREE? I can`t find it anywhere.

[blockspacer]

Also you can try to build for linux/mac/win with latest llvm to get better messages about linking errors.

For emcc - build with
-s ERROR_ON_MISSING_LIBRARIES=1
-s ERROR_ON_UNDEFINED_SYMBOLS=1

[gabrielcuvillier]

Yes, both Simd_Generic.cpp and HashIndex.cpp are included in the CMakeFile.txt
SQRT_THREE is declared in neo/idlib/math/Math.h and defined in Math.cpp

[kripken]

cc @sbc100 - those g$ functions make me think dynamic linking is involved somehow (although oddly the project doesn't build a main or a side module?)

[sbc100]

It looks like you are perhaps building some/all of your objects with -fPIC? We should really fix this do that that becomes a linker error, or "just works".

[gabrielcuvillier]

@sbc100 good catch! There is the -fPIC option that I did not spotted previously.
When I remove it, everything works fine!

Good!

[gabrielcuvillier]

By the way, the final wasm binary size is 15% smaller with the LLVM backend than with fastcomp, which is very good

[msabwat]

It looks like you are perhaps building some/all of your objects with -fPIC? We should really fix this do that that becomes a linker error, or "just works".

Hi !

I had the same issue with "__table_base" and "__memory_base" coming up as unresolved symbols. while linking with zlib and openjpeg.

When -fPIC is disabled, linking works fine.

I couldn't reproduce the issue with a small example. Is the issue because of -fPIC behavior while linking statically?

[sbc100]

Yes currently you cant link -fPIC objects into static executables. We are working on a fix. This sounds like it was consitent with what you are seeing, right?

[msabwat]

Yes, I could not reproduce it, so I wanted to be sure :D after your explanation, I can.

you cant link -fPIC objects into static executables

I was trying to link a main.c with "-fPIC enabled libs", without the -fPIC option which worked fine.

$ emcc -fPIC libhello.c -o libhello.bc
$ emcc -fPIC main.c libhello.bc -o test.html
error: undefined symbol: __memory_base
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
Error: Aborting compilation due to previous errors
shared:ERROR: '/home/b1ue/b1ue/workdir_vlc/emsdk/node/8.9.1_64bit/bin/node /home/b1ue/b1ue/workdir_vlc/emsdk/upstream/emscripten/src/compiler.js /tmp/tmpyw15c8zq.txt /home/b1ue/b1ue/workdir_vlc/emsdk/upstream/emscripten/src/library_pthread_stub.js' failed (1)

Thanks,

[kripken]

@sbc100 has this been fixed? We got another bug report on this in #8986. Maybe this was closed by mistake?

[sbc100]

I mean this specific issues was fixed for the OP :)

In general we do still have any issue here which is that -fPIC objects can't be linked into regular executables. In general this shouldn't be a problem but native developers expect that this works because it happens to work on other plaforms so we should make it work here too.

I opened this llvm issue to track the underlying issue: https://bugs.llvm.org/show_bug.cgi?id=42656, and this corresponding emscripten issue: #9013

[kripken]

Cool, thanks @sbc100

[ghost]

I've narrowed it down in my project to just the errors

error: undefined symbol: __memory_base
error: undefined symbol: __table_base

And when I switch from MAIN_MODULE=2 to MAIN_MODULE=0 they went away. Obviously, I do want this to be my main, but I supposed I don't need to MODULIZE it. Here are my 2 sets of LDFLAGS from another build I had working, nothing really pops out at me as being wrong with config except the CFLAGS being included on one and not the other, but that shouldn't be relevant during dynamic linking is it?

https://gist.github.com/briancullinan/19c6589f54e7aae49fadd35389ff6a9d

EDIT: MAIN_MODULE=1 appears to still be working. I rewrote my entire build system just so I could help test and take advantage of these new features.

EDIT: Tried to compile again and now it's back up to 40MB with a bunch of AAAAAA in the middle.

I don't see why it needs to generate this, why not use a virtual filesystem to create this empty space before the WebAssembly.create() so this giant file doesn't need to be downloaded? It should be possibly to include a library without MAIN_MODULE, I don't add a special MAIN_MODULE specifier when I am building natively with a .dylib or .so

[sbc100]

I'm not convinced this is that same issue. Perhaps it worth opening another issue if you are still seeing: error: undefined symbol: __memory_base.

You are right that in theory we don't need to make the main program relocatable and we could effectively statically link it. I'm tried to do this in the past but ran into a few issues so its not there yet. For now you need at lease MAIN_MODULE=2 (which should give you a small binaryen compared to MAIN_MODULE=1). As for all those 41414141 bytes I'm not sure what they could be but I would guess its static data of some kind. Using wasm-object -x should help you figure it out.

(Sadly I can't find my PR that tried to allow the main executable to linked without RELOCATABLE/-pie but I should perhaps try to revive it)

[ghost]

I think I was misusing the options. I'm just trying to understand it better so maybe I can fix some of this. Like if RDL_LAZY sets the lazyLoad flag I can avoid the WebAssembly.Compile (versus WebAssembly.compile) error. I think I can also make some solid demos for people looking for a more complete example. I want to open issues but out of 900 I need to learn this stuff so I can help close some before whining about design decisions. I'm copying webgl functions into my own lib because Q3 stacks the version bindings on top of each other and emjs doesn't seem to support mixing opengl1 and 2. This is all still very strange to me. You might be more interested, I tried to make a MODULARIZED .js that packages its own library_webgl with it. So loading the new library functions as a module and then loading that module's wasm like a regular dylink. Kind of best of both worlds. Is late calling of dlopen at runtime after startup what emjs refers to as "weak linking is unsupported"? Like dlopen only works at startup? Because otherwise I can't get an automatic list of Export_Functions. I think this is all starting to make sense. Amazing how many thousands of lines of code I have to memorize. Ha!

…

On Tue, May 25, 2021, 13:51 Sam Clegg ***@***.***> wrote: I'm not convinced this is that same issue. Perhaps it worth opening another issue if you are still seeing: error: undefined symbol: __memory_base. You are right that in theory we don't need to make the main program relocatable and we could effectively statically link it. I'm tried to do this in the past but ran into a few issues so its not there yet. For now you need at lease MAIN_MODULE=2 (which should give you a small binaryen compared to MAIN_MODULE=1). As for all those 41414141 bytes I'm not sure what they could be but I would guess its static data of some kind. Using wasm-object -x should help you figure it out. (Sadly I can't find my PR that tried to allow the main executable to linked without RELOCATABLE/-pie but I should perhaps try to revive it) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8761 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF7FO3WRWWCB7OPF4MYW2ETTPQEV5ANCNFSM4HVR2UTA> .

[sbc100]

dlopen is supported, and you can load libraries at runtime. (See the tests that use dlopen in test_core.py and test_other.py). The caveat is that you will will need to ensure that your MAIN_MODULE includes enough of the native and JS standard libraries to support the library when it is loaded.

You have two choices for how to do that:

1. use MAIN_MODULE=1 to conservatively include the entire standard library.
2. Explicitly export all the symbols needed for you side module by adding them to -sEXPORTED_FUNCTIONS (you can use llvm-nm on your side module to help you build this list).