New LLVM backend: Issue while linking globals

[gabrielcuvillier]

While trying to compile the Doom 3 port with the new LLVM backend, there is a problem at link time:

error: undefined symbol: g$SIMDProcessor
error: undefined symbol: g$_ZN11idHashIndex13INVALID_INDEXE
error: undefined symbol: g$_ZN5idLib10cvarSystemE
error: undefined symbol: g$_ZN5idLib10fileSystemE
error: undefined symbol: g$_ZN5idLib3sysE
error: undefined symbol: g$_ZN5idLib6commonE
error: undefined symbol: g$_ZN6idCVar10staticVarsE
error: undefined symbol: g$_ZN6idMatX7tempPtrE
error: undefined symbol: g$_ZN6idMath10SQRT_THREEE
error: undefined symbol: g$_ZN6idMath11FLT_EPSILONE
error: undefined symbol: g$_ZN6idMath11SQRT_1OVER2E
error: undefined symbol: g$_ZN6idMath12ONEFOURTH_PIE
error: undefined symbol: g$_ZN6idMath2PIE
error: undefined symbol: g$_ZN6idMath5iSqrtE
error: undefined symbol: g$_ZN6idMath6TWO_PIE
error: undefined symbol: g$_ZN6idMath7HALF_PIE
error: undefined symbol: g$_ZN6idMath8INFINITYE
error: undefined symbol: g$_ZN6idMath9M_DEG2RADE
error: undefined symbol: g$_ZN6idMath9M_RAD2DEGE
error: undefined symbol: g$_ZN6idVecX7tempPtrE
error: undefined symbol: g$_ZN6idVecX9tempIndexE
error: undefined symbol: g$_ZTV14idSIMD_Generic
error: undefined symbol: g$common
error: undefined symbol: g$cvarSystem
error: undefined symbol: g$mat3_identity
error: undefined symbol: g$vec3_origin
error: undefined symbol: __memory_base
error: undefined symbol: __table_base

Most of the undefined symbols are global variables or constants in D3 code. I am not sure what is the root cause of the problem.
However, "__table_base" and "__memory_base" undefined symbols looks quite suspect.

Any hints ?
This is working fine with fastcomp backend

Note that I am not using dynamic linking (no MAIN_MODULE or SIDE_MODULE). There is static libs though.

To reproduce the problem (I did not isolate the problem, sorry) :
git clone https://github.com/gabrielcuvillier/d3wasm.git
cd d3wasm.git
git checkout -b NoEmterpretify origin/NoEmterpretify
mkdir build-wasm
cd build-wasm
emcmake cmake ../neo -d CMAKE_BUILD_TYPE=Release
emmake make

• issue at link time *

[blockspacer]

Do you include in build files:

• Simd_Generic.cpp
• HashIndex.cpp (https://github.com/search?q=%22idHashIndex%22&type=Code)
  e.t.c.
  ?

Also check simd related build defines/flags (both cmake related and emscripten related)

(Files not found in you repo, but can be found in other github repos, for some reason. Maybe you need version update?)

Also where is declared SQRT_THREE? I can`t find it anywhere.

[blockspacer]

Also you can try to build for linux/mac/win with latest llvm to get better messages about linking errors.

For emcc - build with
-s ERROR_ON_MISSING_LIBRARIES=1
-s ERROR_ON_UNDEFINED_SYMBOLS=1

[gabrielcuvillier]

Yes, both Simd_Generic.cpp and HashIndex.cpp are included in the CMakeFile.txt
SQRT_THREE is declared in neo/idlib/math/Math.h and defined in Math.cpp

[kripken]

cc @sbc100 - those g$ functions make me think dynamic linking is involved somehow (although oddly the project doesn't build a main or a side module?)

[sbc100]

It looks like you are perhaps building some/all of your objects with -fPIC? We should really fix this do that that becomes a linker error, or "just works".

[gabrielcuvillier]

@sbc100 good catch! There is the -fPIC option that I did not spotted previously.
When I remove it, everything works fine!

Good!

[gabrielcuvillier]

By the way, the final wasm binary size is 15% smaller with the LLVM backend than with fastcomp, which is very good

[msabwat]

It looks like you are perhaps building some/all of your objects with -fPIC? We should really fix this do that that becomes a linker error, or "just works".

Hi !

I had the same issue with "__table_base" and "__memory_base" coming up as unresolved symbols. while linking with zlib and openjpeg.

When -fPIC is disabled, linking works fine.

I couldn't reproduce the issue with a small example. Is the issue because of -fPIC behavior while linking statically?

[sbc100]

Yes currently you cant link -fPIC objects into static executables. We are working on a fix. This sounds like it was consitent with what you are seeing, right?

[msabwat]

Yes, I could not reproduce it, so I wanted to be sure :D after your explanation, I can.

you cant link -fPIC objects into static executables

I was trying to link a main.c with "-fPIC enabled libs", without the -fPIC option which worked fine.

$ emcc -fPIC libhello.c -o libhello.bc
$ emcc -fPIC main.c libhello.bc -o test.html
error: undefined symbol: __memory_base
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
Error: Aborting compilation due to previous errors
shared:ERROR: '/home/b1ue/b1ue/workdir_vlc/emsdk/node/8.9.1_64bit/bin/node /home/b1ue/b1ue/workdir_vlc/emsdk/upstream/emscripten/src/compiler.js /tmp/tmpyw15c8zq.txt /home/b1ue/b1ue/workdir_vlc/emsdk/upstream/emscripten/src/library_pthread_stub.js' failed (1)

Thanks,

[kripken]

@sbc100 has this been fixed? We got another bug report on this in #8986. Maybe this was closed by mistake?

[sbc100]

I mean this specific issues was fixed for the OP :)

In general we do still have any issue here which is that -fPIC objects can't be linked into regular executables. In general this shouldn't be a problem but native developers expect that this works because it happens to work on other plaforms so we should make it work here too.

I opened this llvm issue to track the underlying issue: https://bugs.llvm.org/show_bug.cgi?id=42656, and this corresponding emscripten issue: #9013