Size regression in wasm builds from 1.39.1-fastcomp to 1.39.1 due to pthreads passive data segments not optimized for zeros

[niklasf]

There's a huge increase in binary size when building https://github.com/niklasf/stockfish.wasm (v0.5.11) with the latest emscripten.

Any information I could provide that would help to narrow this down?

1.38.48 (last good):

392K	stockfish.wasm
144K	stockfish.js
12K	stockfish.worker.js
40K	stockfish.js.mem
592K	total

1.39.0 (first bad):

1,3M	stockfish.wasm
144K	stockfish.js
12K	stockfish.worker.js
40K	stockfish.js.mem
1,5M	total

1.39.1 (still bad).

Flags: https://github.com/niklasf/stockfish.wasm/blob/bb648dd021e3e585def8b86050800f47001524a5/src/Makefile#L243

[niklasf]

Much better with 1.39.1-fastcomp:

132K	stockfish.js
40K	stockfish.js.mem
392K	stockfish.wasm
12K	stockfish.worker.js
576K	total

[sbc100]

Can you post the linker command line options you use? There are some tools for analyzing the size of WebAssembly binaries such as https://github.com/rustwasm/twiggy. In this case the size it quite significant so I suspect that something is preventing the GC of dead function in llvm upstream case. Seeing you command line options will help figure it out.

…

On Mon, Nov 4, 2019 at 6:14 AM Niklas Fiekas ***@***.***> wrote: Much better with 1.39.1-fastcomp: 132K stockfish.js 40K stockfish.js.mem 392K stockfish.wasm 12K stockfish.worker.js 576K total — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[niklasf]

Thanks. The Makefile linked above runs the following command for the source files:

em++ -Wall -Wcast-qual -std=c++11 -s MODULARIZE=1 -s EXPORT_NAME="Stockfish" -s NO_EXIT_RUNTIME=1 -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=1 -s TOTAL_MEMORY=109051904 -s EXPORTED_FUNCTIONS="['_main', '_uci_command']" -s "EXTRA_EXPORTED_RUNTIME_METHODS=['ccall']" -s "INCOMING_MODULE_JS_API=['print', 'postRun']" --pre-js pre.js -DNDEBUG -O3 -DIS_64BIT -DNO_PREFETCH -msse3 -mpopcnt -DUSE_POPCNT -flto -c -o main.o main.cpp

The final binary is linked with basically the same flags:

em++ -o stockfish.js benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o -s MODULARIZE=1 -s EXPORT_NAME="Stockfish" -s NO_EXIT_RUNTIME=1 -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=1 -s TOTAL_MEMORY=109051904 -s EXPORTED_FUNCTIONS="['_main', '_uci_command']" -s "EXTRA_EXPORTED_RUNTIME_METHODS=['ccall']" -s "INCOMING_MODULE_JS_API=['print', 'postRun']" --pre-js pre.js -lpthread -Wall -Wcast-qual -std=c++11 -s MODULARIZE=1 -s EXPORT_NAME="Stockfish" -s NO_EXIT_RUNTIME=1 -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=1 -s TOTAL_MEMORY=109051904 -s EXPORTED_FUNCTIONS="['_main', '_uci_command']" -s "EXTRA_EXPORTED_RUNTIME_METHODS=['ccall']" -s "INCOMING_MODULE_JS_API=['print', 'postRun']" --pre-js pre.js -DNDEBUG -O3 -DIS_64BIT -DNO_PREFETCH -msse3 -mpopcnt -DUSE_POPCNT -flto

I'll try twiggy now.

[niklasf]

Unfortunately twiggy panics on the version compiled with the llvm backend:

$ twiggy garbage 1.39.1-llvm-stockfish.wasm 
thread 'main' panicked at 'should not parse the same key into multiple items', /home/niklas/.cargo/registry/src/github.com-1ecc6299db9ec823/twiggy-ir-0.6.0/./ir.rs:53:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

---

Binaries attached, gzipped to be able to upload on GitHub:

1.39.1-fastcomp-stockfish.wasm.gz
1.39.1-llvm-stockfish.wasm.gz

[kripken]

Bloaty might be helpful too. Or just running wasm-dis and running emscripten's tools/find_bigfuncs.py.

It's possible the newer llvm in upstream inlines differently, which affects code size, but a difference that big sounds like something else. Should be easy to debug though given it's so big.

How much do things change when removing -flto from all the commands?

[niklasf]

I noticed something else while looking at the gzipped files:

136K	1.39.1-fastcomp-stockfish.wasm.gz
128K	1.39.1-llvm-stockfish.wasm.gz

So the difference goes away after compression. In fact LLVM is now better!

Will run bloaty and try removing -flto soon.

[kripken]

Any chance you can upload builds with names (--profiling-funcs at link)? That would make comparing them easier.

edit: nevermind :) see next comment

[kripken]

Comparing the existing builds, wasm-opt --metrics shows basically the entire difference is

- [memory-data]  : 0       
+ [memory-data]  : 1013251 

So the fastcomp build has no memory in the wasm, but the upstream one does.

I believe that makes sense, as in upstream the data is in the wasm, and we use passive segments so load them only in the main thread. In fastcomp, we separate them out, and load them once in JS.

What is odd is that you wrote you have a 40K .mem file in both cases. Did the upstream build not emit one, but there was one already in that dir perhaps?

I see there are many many zeros in the upstream build's data segments. I guess we don't do the usual tricks to skip zeros in that case as they are passive @tlively ?

[niklasf]

Thank you very much, this explains the difference! I can confirm that it's just the .mem file from the previous build.

[tlively]

The MemoryPacking pass currently bails out in the presence of memory.init and data.drop segments, so zeros in non-bss segments will not be eliminated with pthread builds at the moment. This could be fixed by duplicating and updating bulk memory operations when the segments they reference are split or otherwise changed.

I've noted this on my backlog, but I'm not sure how to prioritize it. @kripken how critical do you think this is?

[kripken]

Good question. If this were for non-pthreads builds too I'd be much more concerned. As it's pthreads-only, it's still important but less I guess. I put it in the middle column on https://github.com/emscripten-core/emscripten/projects/1 now fwiw.

[xiyuyizhi]

same problem , when switch 1.39.0-fastcomp to 1.39.0-upstream, wasm size(ffmpeg libx) from 1.1M to 2.1M, use thread also

[tlively]

The fix for this has been merged upstream in WebAssembly/binaryen#2426 and should make its way into the next release of Emscripten (1.39.7).