Syscall eventfd and getgrgid symbols missing

[aliasdevelopment]

I am trying to compile libzmq based on master branch and run it on Firefox Nighly.
I have managed to get phtread compiled and linked, but get linker errors with respect to eventfd and getgrgid symbols.

I have read through all the documentation and can not find anything that explains this issue of how to avoid it. I can see that musl provides these function calls, but I have been unable to compile musl and have found no obvious configuration flags which solves this.

I am using CMake with -DCMAKE_TOOLCHAIN_FILE="/home/test/emsdk/emscripten/master/cmake/Modules/Platform/Emscripten.cmake"

I have made a simple example that show the issue.

What am I missing?

#include <sys/eventfd.h>
#include <sys/types.h>
#include <grp.h>


unsigned int main_loop()
{
  int efd = eventfd(0, 0);
  if (efd == -1)
    return 1; 

  group* test = getgrgid(0);    
  return 0;
}

$ cmake --build . Scanning dependencies of target zmq_sender [ 50%] Building CXX object CMakeFiles/zmq_sender.dir/applications/platform/emscripten_main.cpp.o INFO:root:(Emscripten: Running sanity checks) clang version 3.9.0 (https://github.com/kripken/emscripten-fastcomp-clang.git 60a7e9a9c22b67309e5b1258d38fadfa481a25d3) (https://github.com/kripken/emscripten-fastcomp.git 881bd352731d21c7117ad7e2ece347aacae83965) (emscripten 1.37.0 : 1.37.0) Target: asmjs-unknown-emscripten Thread model: posix InstalledDir: /home/test/emsdk/clang/fastcomp/build_master_64/bin "/home/test/emsdk/clang/fastcomp/build_master_64/bin/clang-3.9" -cc1 -triple asmjs-unknown-emscripten -emit-llvm-bc -emit-llvm-uselists -disable-free -disable-llvm-verifier -discard-value-names -main-file-name emscripten_main.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -no-integrated-as -mconstructor-aliases -v -dwarf-column-info -debugger-tuning=gdb -coverage-file /home/test/testbuild/CMakeFiles/zmq_sender.dir/applications/platform/emscripten_main.cpp.o -nostdsysteminc -nobuiltininc -resource-dir /home/test/emsdk/clang/fastcomp/build_master_64/bin/../lib/clang/3.9.0 -D __EMSCRIPTEN_major__=1 -D __EMSCRIPTEN_minor__=37 -D __EMSCRIPTEN_tiny__=0 -D _LIBCPP_ABI_VERSION=2 -D "MAIN_LOOP=\"eventfd.cpp\"" -D ZMQ_STATIC -I /home/test/test/externals -I /home/test/test/applications/libzmq_sender -I /home/test/test/applications -I /home/test/test/applications/platform -I /home/test/INSTALL/Release_Unix_Makefiles/include -D NDEBUG -O2 -Werror=implicit-function-declaration -Wall -std=gnu++11 -fdeprecated-macro -fno-dwarf-directory-asm -fdebug-compilation-dir /home/test/testbuild -ferror-limit 19 -fmessage-length 105 -fobjc-runtime=gnustep -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -nobuiltininc -nostdsysteminc -isystem/home/test/emsdk/emscripten/master/system/include/libcxx -isystem/home/test/emsdk/emscripten/master/system/lib/libcxxabi/include -isystem/home/test/emsdk/emscripten/master/system/include/compat -isystem/home/test/emsdk/emscripten/master/system/include -isystem/home/test/emsdk/emscripten/master/system/include/SSE -isystem/home/test/emsdk/emscripten/master/system/include/libc -isystem/home/test/emsdk/emscripten/master/system/lib/libc/musl/arch/emscripten -isystem/home/test/emsdk/emscripten/master/system/local/include -isystem/home/test/emsdk/emscripten/master/system/include/SDL -disable-llvm-optzns -o CMakeFiles/zmq_sender.dir/applications/platform/emscripten_main.cpp.o -x c++ /home/test/test/applications/platform/emscripten_main.cpp clang -cc1 version 3.9.0 based upon LLVM 3.9.0 default target x86_64-unknown-linux-gnu #include "..." search starts here: #include <...> search starts here: /home/test/test/externals /home/test/test/applications/libzmq_sender /home/test/test/applications /home/test/test/applications/platform /home/test/INSTALL/Release_Unix_Makefiles/include /home/test/emsdk/emscripten/master/system/include/libcxx /home/test/emsdk/emscripten/master/system/lib/libcxxabi/include /home/test/emsdk/emscripten/master/system/include/compat /home/test/emsdk/emscripten/master/system/include /home/test/emsdk/emscripten/master/system/include/SSE /home/test/emsdk/emscripten/master/system/include/libc /home/test/emsdk/emscripten/master/system/lib/libc/musl/arch/emscripten /home/test/emsdk/emscripten/master/system/local/include /home/test/emsdk/emscripten/master/system/include/SDL End of search list. In file included from /home/test/test/applications/platform/emscripten_main.cpp:1: /home/test/test/applications/eventfd.cpp:12:10: warning: unused variable 'test' [-Wunused-variable] group* test = getgrgid(0); ^ 1 warning generated. [100%] Linking CXX executable zmq_sender.html INFO:root:generating system library: libc-mt.bc... (this will be cached in "/home/dude/.emscripten_cache/asmjs/libc-mt.bc" for subsequent builds) INFO:root: - ok INFO:root:generating system library: pthreads.bc... (this will be cached in "/home/dude/.emscripten_cache/asmjs/pthreads.bc" for subsequent builds) INFO:root: - ok INFO:root:generating system library: dlmalloc_threadsafe.bc... (this will be cached in "/home/dude/.emscripten_cache/asmjs/dlmalloc_threadsafe.bc" for subsequent builds) INFO:root: - ok warning: unresolved symbol: getgrgid warning: unresolved symbol: eventfd [100%] Built target zmq_sender

[kripken]

Those aren't implemented in our system support. It's not clear we can support them on the web platform, at best we could try to "fake" an implementation. Does the app actually need these? Perhaps it can be compiled without them?

[aliasdevelopment]

I have reviewed the CMake of the libzmq project and seen that there exists defines which will work around eventfd and getgrgid. I will try work around these two and update the libzmq CMakeLists file to be more robust against this.

One thought for webscripten:
Would it make sense to add some pragma error reporting on incomplete implementations and not supported features. In my case and probably in other cases it would be nice to be notified about usage of incomplete/not supported features.

Note:
I am impressed by your work and all contributors of this projects. Keep up the good work

[aliasdevelopment]

Get the following when running the compiled test application.

`unsupported socketcall syscall 8
abort("unsupported socketcall syscall 8") at jsStackTrace@http://localhost:8080/zmq_sender.js:1:17771
stackTrace@http://localhost:8080/zmq_sender.js:1:17942
abort@http://localhost:8080/zmq_sender.js:19:11303
___syscall102@http://localhost:8080/zmq_sender.js:1:164414
ZC@http://localhost:8080/zmq_sender.js:7:115525
wi@http://localhost:8080/zmq_sender.js:6:109600
vi@http://localhost:8080/zmq_sender.js:6:109440
te@http://localhost:8080/zmq_sender.js:6:48844
_c@http://localhost:8080/zmq_sender.js:6:9553
Fc@http://localhost:8080/zmq_sender.js:6:901
Cc@http://localhost:8080/zmq_sender.js:6:466
Dc@http://localhost:8080/zmq_sender.js:6:692
callMain@http://localhost:8080/zmq_sender.js:19:9556
doRun@http://localhost:8080/zmq_sender.js:19:10259
run/<@http://localhost:8080/zmq_sender.js:19:10427

If this abort() is unexpected, build with -s ASSERTIONS=1 which can give more information.`

And I tried to add the -s ASSERTIONS=1 without more information.

I can see from library_syscall.js that the case 8 for the syscall102 is not supported. Unsure if this is because it is not done yet or if it is impossible to support in the emscripten web platform.

[atrosinenko]

AFAIK eventfd is a Linux-specific syscall. In general, it may help to persuade the program to be compiled to use POSIX APIs and not Linux ones, so it will use pipe syscall instead that is implemented. I have no idea on thread-safety of pipe in Emscripten, though. Are syscalls proxied to single thread?

getgrgid, on the other hand, looks like a regular library function that processes some file. Why are these functions not found: are they explicitly thrown away to reduce JS size or they really cannot be compiled (for example, they absent in musl)?

Meanwhile, in some projects there can be many such unresolved symbol warnings but these functions are not called at run-time, so this seem to make no problem for me, at least in asmjs, I don't know how it will be handled in wasm, though.

[juj]

The eventfd mechanism is something we can implement and support fully with wasm multithreading. The intent in native linux world is to have a system-wide semaphore-like mechanism to notify about events (across multiple processes, rather than threads). In our case, eventfd will be just a regular semaphore that lives in the filesystem.

However it's currently not too easy to add proper support for this, since it likely needs a paired implementation of select, poll or epoll for Emscripten first. That is a lot of work to do proper, and needs ASMFS to mature first. You may be able to get something simple hacked in quickly, but a complete implementation needs dev cycles and testing.

The getgrgid function seems to find information about an user group on the system. There is no concept of user groups for Emscripten, I think we fake user ID of 100 and group ID of 100 or something similar, and username and group are both root, but that's that. It probably makes most sense to take a close look at why this library needs this function to evaluate what conceptually would make most sense in Emscripten environment. There is some benefit to fake stubbing these kinds of functions out, but whenever we do so, that means that code will compile silently, and developers of other codebases will not then be notified about these kinds of code locations in their own tree, so it's a tradeoff.

[sigiesec]

While it would be nice to have eventfd support, actually, neither getgrgid nor eventfd are required to build libzmq. Somehow the feature detection gets this wrong, and sets ZMQ_HAVE_EVENTFD and ZMQ_HAVE_SO_PEERCRED though there is no (full) support (and at least for me also ZMQ_HAVE_PTHREAD_SET_AFFINITY). When I disabled these manually, I can successfully compile some of the tests to JavaScript. See also my notes in zeromq/libzmq#3437

[sigiesec]

Maybe I was a bit too optimistic. When eventfd is not supported, socketpair is used. This exists as a symbol, but it is not supported: #1060

There is another implementation for OpenVMS (and one for Windows using the Windows-flavoured Socket API), maybe that could be used, but it uses TCP/IP, so eventfd would really be great.

[hortonelectric]

Is anyone planning to work on this? It's a showstopper for my project.

[hortonelectric]

@juj

[trashhalo]

I was curious if this was now possible via WASI but I ran into this issue. bytecodealliance/wasmtime#70

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.