Best way to build from source

[jakirkham]

Have been reading over the docs and some related discussions. Based on the docs my understanding is one builds the patched versions of LLVM and Clang (called Fastcomp and Fastcomp-Clang) then one downloads Emscripten somewhere (no build steps here?). However recent conversations make it sound like LLVM has taken on the role of Fastcomp and Fastcomp-Clang. Is that correct or am I misunderstanding something? If so, does that mean the build steps are now just download Emscripten? If not, will Fastcomp and Fastcomp-Clang clash with an existing LLVM and Clang install or are they renamed in such a way to avoid clashing?

ref: emscripten-core/emscripten-fastcomp#193 (comment)
ref: #5488
ref: #6350
ref: #6830

[kripken]

The misunderstanding here might be with the LLVM wasm backend. When that is used, fastcomp (both LLVM and clang) are not needed, as we use the LLVM wasm backend instead of the fastcomp asm.js backend.

Emscripten supports both backends currently, but the asm.js one is the default as it is more stable. I'd still recommend using that, and it's what the docs talk about. But we hope to switch the default to the wasm backend eventually.

[jakirkham]

Thanks for the clarification. That's very helpful.

As we are interested in using Conda ( conda/conda#7619 ) for handling the building and packaging of WebAssembly libraries, trying to figure out the best way to create and package the build toolchain first and then figure out what the packaging story should look like for WebAssembly.

Have just recently rebuilt the LLVM stack in conda-forge to support the WebAssembly backend. ( conda-forge/llvmdev-feedstock#42 ) That seems to work reasonably well in simple programs.

Was thinking about going the fastcomp route, but that seems like a lot of work if it is soon going to be deprecated and replaced. Is there a timetable for that?

Currently it looks like the libc story in WebAssembly is incomplete. Does that sound correct? Was able to find a musl fork that seemed to be trying to solve this (though admittedly appears to be experimental at this stage), but ran into some issues building. ( jfbastien/musl#50 ) IIUC Emscripten includes it's own libc, which is a bit more robust. Do you have any advice here?

Was able to package Binaryen in conda without too much trouble crossplatform. Seems that is needed for Emscripten to work correctly. Are there other tools that we should be including the build toolchain? Am particularly interested in the MVP toolchain here.

Looking at emsdk am a bit unclear how best to include this. It seems like this is a tool used by developers to select their toolchain. So it may not make sense in a package manager, but I'm not very familiar with it. Expect I'm probably missing many things here. Clarification would be helpful.

Also am trying to figure out what the best story is for an Emscripten package. Is there some build step somewhere (particularly for the libc parts)? Are we safe to copy scripts into a bin directory or do they need to be symlinked or patched to do that? It looks like there is a configuration directory that sits in the users home directory. Am debating whether to supply our own (e.g. in share) or whether we continue with the one in the user's home directory.

Thoughts on any/all of this would be greatly appreciated.

[kripken]

Was thinking about going the fastcomp route, but that seems like a lot of work if it is soon going to be deprecated and replaced. Is there a timetable for that?

No timetable yet. But I expect we'll support fastcomp for a long time, for asm.js support, even after the wasm backend is the default.

I'd say that if the wasm backend path has all the features you need (i.e., you don't need asm.js or any fastcomp-specific feature) then no need for fastcomp.

Currently it looks like the libc story in WebAssembly is incomplete. Does that sound correct?

Emscripten's libc is complete and very stable at this point. I'd just use that, unless you have very custom needs and can afford to do extra work.

Are there other tools that we should be including the build toolchain?

For the wasm backend path, you also need lld, which is in a separate repo from the other LLVM tools.

Binaryen is also necessary, as you mentioned.

Looking at emsdk am a bit unclear how best to include this.

The emsdk can do most of all of the above for you. I'm not familiar with conda, so not sure if a comparison is relevant or not. But look at the emscripten tutorial - the idea with the emsdk is that you get it, then it gets everything else you need: LLVM, clang, lld, binaryen, etc. etc., including binary builds when available. In general, projects using emscripten often use the emsdk to do the work for them, or they tell their users to just grab the emsdk.

Emscripten itself will create .emscripten and some .emscripten_* directories in the user's home dir, for the cache, ports, etc. You can also specify a different location for those files (EM_CACHE, EM_CONFIG env vars).

[jakirkham]

Thanks for the advice.

So I started building Emscripten's libc using embuilder.py with LLVM and had some issues in the linking phase. wasm-ld stated the following options were unknown: --export and --lto-O0. Also there were a few lines like wasm-ld: error: cannot open .... Wondering if there was something I missed building lld. Am using LLVM 6.0.1. Any ideas?

Edit: Think this was due to incompatible versions of LLVM and Emscripten. Changing to an older version of Emscripten worked around this issue.

[sbc100]

If you want to use the llvm backend with Emscripten you really want to be using the latest tip of tree version of LLVM. In the future we may support older/stable LLVM releases but right now we are still working on the backed so you want the very latest version.

[jakirkham]

That makes sense.

Is it reasonable to use slightly older versions of Emscripten along with a stable version of LLVM or is that not really viable?

[sbc100]

Sure that should work, just remember that there only a very vague (major version only) link between the LLVM version and the emscripten version, so you are probably going to have select an emscripten version that roughly matches (in time) the LLVM version you have.

The upstream LLVM churn should get getting less frequent going forward so incompatibilities should become less frequent.

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.