wasi-sdk-20+threads sysroot does not include libc++/libc++abi

[m4c0]

Hi! I downloaded the wasi-sdk-20+threads pre-release sysroot file and it does not contain libc++.a or libc++abi.a in its subfolder:

$ tar ztf wasi-sysroot-20.0.threads.tar.gz | grep libc++
wasi-sysroot/lib/wasm32-wasi/libc++experimental.a
wasi-sysroot/lib/wasm32-wasi/libc++abi.a
wasi-sysroot/lib/wasm32-wasi/libc++.a

By comparison, this is the result for libc:

$ tar ztf wasi-sysroot-20.0.threads.tar.gz | grep libc.a
wasi-sysroot/lib/wasm32-wasi-threads/libc.a
wasi-sysroot/lib/wasm32-wasi/libc.a

Is this intentional? If so, should we expect C++ support eventually?

[abrown]

Will #301 resolve this?

[m4c0]

That would bring support for C++ threads - which is a great addition to C++/WASM community.

But it would be nice to have some level of C++ support until we have #301 done (such as a non-threaded version). At least, it would enable WASM with C++ and shared-memory.

I was able to copy C++'s libs from wasm32-wasi to wasm32-wasi-threads and they kinda worked. I believe it is possible to build a C++ STL without <threads> support if needed.

[abrown]

I was interested in how #301 might change things and indeed, when I retrieve the artifacts from #301 using ci/download-workflow-artifacts.sh and unpack them I see the C++ libraries that were missing.

$ tar ztf wasi-sysroot-19.5g1e4440799c2d.tar.gz | grep libc++
wasi-sysroot/lib/wasm32-wasi/libc++abi.a
wasi-sysroot/lib/wasm32-wasi/libc++experimental.a
wasi-sysroot/lib/wasm32-wasi/libc++.a
wasi-sysroot/lib/wasm32-wasi-threads/libc++abi.a
wasi-sysroot/lib/wasm32-wasi-threads/libc++experimental.a
wasi-sysroot/lib/wasm32-wasi-threads/libc++.a

I think this is good evidence that we should merge #301, but let me try to answer some comments first.

But it would be nice to have some level of C++ support until we have #301 done (such as a non-threaded version). At least, it would enable WASM with C++ and shared-memory.

Unless I'm missing something, the C++ support you're looking for is already present in the wasm32-wasi target. You should be able to compile C++ programs with that target unless they have anything to do with threads. To compile any C++ program that has anything to do with threads, you will need #301 and to use the wasm32-wasi-threads target — it's a bit of an "all or nothing" deal right now. The reason for this is that WebAssembly atomic instructions, shared memory, and wait/notify are all feature-gated behind the atomics flag (some tools call this threads) and due to how WebAssembly compilation in LLVM works, all of the object files must have the same feature flags. Hence the need for a new target, wasm32-wasi-threads, in which both the library files and your program code are compiled with the atomics flag.

I was able to copy C++'s libs from wasm32-wasi to wasm32-wasi-threads and they kinda worked

I think we should try to test out the atomics-enabled libc++ libraries from #301 with whatever you are trying to do. Obviously the C++ support for wasm32-wasi-threads is very experimental so any feedback here would be good. E.g., you mention that things "kinda worked" — did something actually break? If you would like I can upload the sysroot you need here and you can test things out? Feel free to contact me on Zulip if you want to coordinate in real-time about this.

[toyobayashi]

Any plan to publish a new pre-release that includes libc++?

[abrown]

I think the release itself which needs #313 and #314 to merge is probably closer to happening--they need a review before I do anything (cc: @sunfishcode, @sbc100).

[m4c0]

Sorry for the delay on my feedback and thanks for the prompt responses!

But it would be nice to have some level of C++ support until we have #301 done (such as a non-threaded version). At least, it would enable WASM with C++ and shared-memory.

Unless I'm missing something, the C++ support you're looking for is already present in the wasm32-wasi target. You should be able to compile C++ programs with that target unless they have anything to do with threads. To compile any C++ program that has anything to do with threads, you will need #301 and to use the wasm32-wasi-threads target — it's a bit of an "all or nothing" deal right now. The reason for this is that WebAssembly atomic instructions, shared memory, and wait/notify are all feature-gated behind the atomics flag (some tools call this threads) and due to how WebAssembly compilation in LLVM works, all of the object files must have the same feature flags. Hence the need for a new target, wasm32-wasi-threads, in which both the library files and your program code are compiled with the atomics flag.

The C++ support I need is in wasm32-wasi, yes. But I need atomics in order to share memory between WebWorkers. It is a classic producer-consumer: one worker producing audio samples, another worker consuming it (honestly, only because WebAudio forces me to use workers).

In other words, I was trying to link a WASM file that can accept a SharedArrayBuffer from its Memory when instantiated. But the link flag complains about the atomics feature in one of libc's .o file.

In theory, I would be able to share a Memory between workers using postMessage, then instantiate one container per worker using a shared memory - effectively using JS's environment to manage the threads while C/C++ takes care of locks/thread-safety.

I was able to copy C++'s libs from wasm32-wasi to wasm32-wasi-threads and they kinda worked

I think we should try to test out the atomics-enabled libc++ libraries from #301 with whatever you are trying to do. Obviously the C++ support for wasm32-wasi-threads is very experimental so any feedback here would be good. E.g., you mention that things "kinda worked" — did something actually break? If you would like I can upload the sysroot you need here and you can test things out? Feel free to contact me on Zulip if you want to coordinate in real-time about this.

The "kinda-work" part is the short of: my code compiles and runs on a specific browser, using shared memory, without any visible weirdness but I can't properly run my spike tests because I'm facing different browser-specific issues.

I had all sort of issues from different browsers when trying to actually use shared memory. Safari can't postMessage a WebAssembly.Memory. Chrome fails on different audio issues. Firefox I can't recall... 😅

But my code is C++ without STL or threads inside the WASM container, so I don't need libc++ from #301 - I only need really basic stuff (like cxx_personality, etc).

[sbc100]

If you are targeting the web, and audio workers in particular, you might want to consider using emscripten. Its has builtin support for threads and workers and audio worklets: https://emscripten.org/docs/api_reference/wasm_audio_worklets.html

[abrown]

Ok, I think this issue can be closed now that the wasi-sdk-20 release is published. We can reopen if needed--feel free to try it out and see!