Bazel: @emsdk cannot pick up SDL2 headers and libraries.

[sohanjg]

Bazel build with emsdk (for wasm_cc_binary) is unable to fetch and discover the SDL2 file on specifying them in linkopts ("-sUSE_SDL=2", "-sUSE_SDL_IMAGE=2", "-sUSE_SDL_TTF=2", "-sUSE_SDL_NET=2") and coopts ("-DUSE_SDL=2", "-DUSE_SDL_IMAGE=2", "-DUSE_SDL_TTF=2", "-DUSE_SDL_NET=2",) as mentioned in https://emscripten.org/docs/compiling/Building-Projects.html?#emscripten-ports
Compilation fails with "SDL2/SDL.h" headers not found errors.

Note : SDL1 headers and libs/API, work out of the box. We are experimenting in Skia bazel environment.

[sbc100]

The -s options are needed at compile time too. For example, you should use -sUSE_SDL=2 and not -DUSE_SDL2

[sohanjg]

Thanks ! Now it fails with some permission error, about RO file system. I am on Ubuntu 22.04 with sudo access.
Pasting the build log.

`Starting local Bazel server and connecting to it...
INFO: Analyzed target //example:hello_world_wasm (133 packages loaded, 12128 targets configured).
INFO: Found 1 target...
ERROR: /home/ubuntu/skia/tools/timer/BUILD.bazel:7:16: Compiling tools/timer/Timer.cpp failed: (Exit 1): emcc.sh failed: error executing command external/emsdk/emscripten_toolchain/emcc.sh '--sysroot=external/emscripten_bin_linux/emscripten/cache/sysroot' -fdiagnostics-color -fno-exceptions -fno-strict-aliasing -funsigned-char ... (remaining 155 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Traceback (most recent call last):
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/4/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 4010, in
sys.exit(main(sys.argv))
File "/usr/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/4/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 4003, in main
ret = run(args)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/4/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 1137, in run
linker_inputs = phase_compile_inputs(options, state, newargs, input_files)
File "/usr/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/4/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 2697, in phase_compile_inputs
compile_source_file(i, input_file)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/4/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 2677, in compile_source_file
cmd = get_clang_command(input_file)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/4/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 2618, in get_clang_command
return get_compiler(src_file) + get_cflags(state.orig_args) + compile_args + [src_file]
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/4/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 889, in get_cflags
ports.add_cflags(cflags, settings)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/ports/init.py", line 364, in add_cflags
port.get(Ports, settings, shared)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/ports/freetype.py", line 19, in get
ports.fetch_project('freetype', 'https://github.com/emscripten-ports/FreeType/archive/' + TAG + '.zip', 'FreeType-' + TAG, sha512hash=HASH)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/ports/init.py", line 159, in fetch_project
fullname = os.path.join(Ports.get_dir(), name)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/ports/init.py", line 140, in get_dir
shared.safe_ensure_dirs(dirname)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/utils.py", line 28, in safe_ensure_dirs
os.makedirs(dirname, exist_ok=True)
File "/usr/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/cache/ports'
Target //example:hello_world_wasm failed to build
`

[sbc100]

I think this is a limitation of the bazel setup for emsdk. IIRC you can't build new ports that were not already built when the emsdk was created/built. Is that right @walkingeyerobot ?

I though that we set FROZEN_CACHE which would make the error message a little more readable I think.

[sohanjg]

Thanks ! Is there any workaround for the limitation in bazel setup ?
If not, does emsdk plan to upgrade built-in SDL1 to SDL2 anytime soon ?

[sbc100]

I don't know of any way to add new ports to the bazel SDK today. Presumably the right solution would be to convert the ports builds into bazel BUILD files? Perhaps this could be done automatically, but I don't know of anyone that is working on it.

[walkingeyerobot]

The right solution is indeed to convert the ports builds into bazel BUILD files. This cannot be done automatically, but I am actively working on it. It's quite a large undertaking though, and I don't expect to finish for some time (on the order of months).

I'm not exactly sure how to workaround, but unsetting FROZEN_CACHE in bazel/emscripten_toolchain/emscripten_config would be my first stop. I don't expect this to immediately work, but the error message that gives would help inform the next step. You may also be able to build the cache outside of bazel and then copy the appropriate files to wherever bazel is looking for them.

[sbc100]

Can we automatically generate BUILD files from tools/system_libc.py? I would have through that would be the simplest way to keep them up-to-date.. I was thinking of generating ninja or cmake files from that code too.. in order to speed up out builds.

[walkingeyerobot]

Yes, we can generate most of the rules required. However, there's some logic which will make it tricky to generate (i.e. switching what files are part of the input based on certain features such as threading). Generating these rules seems much preferable to keeping them up to date by hand. Also, there's a lot of infrastructure that needs to be figured out and written to make sure the new library rules are built and integrated correctly.

[sohanjg]

Thank you for checking this. With FROZEN_CACHE as None/False, doesn't give us much more info except that it fails while creating directory because of permission issues.

`ERROR: /home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/spirv_cross/BUILD.bazel:4:11: Compiling spirv_cross_parsed_ir.cpp failed: (Exit 1): emcc.sh failed: error executing command external/emsdk/emscripten_toolchain/emcc.sh '--sysroot=external/emscripten_bin_linux/emscripten/cache/sysroot' -fdiagnostics-color -fno-exceptions -fno-strict-aliasing -funsigned-char ... (remaining 33 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Traceback (most recent call last):
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/9/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 40, in
import emscripten
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/emscripten.py", line 23, in
from tools import building
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/building.py", line 21, in
from . import shared
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/shared.py", line 714, in
Cache = cache.Cache(config.CACHE)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/cache.py", line 35, in init
self.ensure()
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/cache.py", line 81, in ensure
utils.safe_ensure_dirs(self.dirname)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/utils.py", line 28, in safe_ensure_dirs
os.makedirs(dirname, exist_ok=True)
File "/usr/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/home/ubuntu/.emscripten_cache'
ERROR: /home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/spirv_cross/BUILD.bazel:4:11: Compiling spirv_cross.cpp failed: (Exit 1): emcc.sh failed: error executing command external/emsdk/emscripten_toolchain/emcc.sh '--sysroot=external/emscripten_bin_linux/emscripten/cache/sysroot' -fdiagnostics-color -fno-exceptions -fno-strict-aliasing -funsigned-char ... (remaining 33 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Traceback (most recent call last):
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/sandbox/linux-sandbox/10/execroot/skia/external/emscripten_bin_linux/emscripten/emcc.py", line 40, in
import emscripten
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/emscripten.py", line 23, in
from tools import building
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/building.py", line 21, in
from . import shared
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/shared.py", line 714, in
Cache = cache.Cache(config.CACHE)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/cache.py", line 35, in init
self.ensure()
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/cache.py", line 81, in ensure
utils.safe_ensure_dirs(self.dirname)
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/756dee633784ef959e49802380c6f32b/external/emscripten_bin_linux/emscripten/tools/utils.py", line 28, in safe_ensure_dirs
os.makedirs(dirname, exist_ok=True)
File "/usr/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/home/ubuntu/.emscripten_cache'
`

[achoum]

Hi @sohanjg,

I am currently trying to compile a bazel+emscripten+sdl2 binary and hitting some issues (#1229), and I saw your bug :).

Have you been able to solve your situation and compile your project with bazel+emscripten+sdl2?

Cheers,

[sohanjg]

Hey ! No, we went for a workaround and included sdl2 as a separate 3rd party library in our project, instead of relying on emscripten.
Here is a WIP version of it, if it helps, https://skia-review.googlesource.com/c/skia/+/573643

[achoum]

Sorry to hear about the workaround, and thanks a lot for the change link :).
This is a great help.