new workflow: static linux binary

[azimut]

https://github.com/daewok/sbcl/blob/static-executable-v2/README.static-executable
https://github.com/daewok/sbcl/blob/static-executable-v2/tools-for-build/Dockerfile.static-executable-example

Depends on #1

[azimut]

No ODE/Csound/OpenAL/SdlMixer on Alpine packages. But there is Bullet. Likely not for this project then. Although, I might try compiling it for it.

[azimut]

Wondering if libstd would need to be build statically too.

https://github.com/richfelker/musl-cross-make/issues/64

[azimut]

Alpine packages https://pkgs.alpinelinux.org/packages
Some packages don't come with their static .a versions by policy (?) https://gitlab.alpinelinux.org/alpine/aports/-/issues/12663

[azimut]

https://gitlab.alpinelinux.org/alpine/aports/-/issues/12893

the main issue with static linking linux graphical applications is that libgl cannot be statically linked and cannot be loaded in a statically linked binary. theoretically you can create a statically linked software rendering x11 program, but that's an sdl2 issue, not an alpine issue. looks like there is a SDL_LOADSO_DLOPEN in this vein, but i would recommend asking sdl2, not alpine.

[azimut]

8 years ago

Question about if it will be possible statically compile SDL with musl but dload() OpenGL compiled with glibc

Answer was no (I think)

https://musl.openwall.narkive.com/ODEqT9jH/dlopen-ing-glibc-linked-libraries
https://www.openwall.com/lists/musl/2012/12/08/4

[azimut]

https://blog.gibson.sh/2017/11/26/creating-portable-linux-binaries/
https://lobste.rs/s/qx5zuo/how_create_portable_linux_binaries_even

In general

Unless they’re directly interacting with the system (like SDL or OpenAL), you can even link them statically if the license allows it.

About SDL2

Don’t link SDL or OpenAL or similar libs statically, so your users can easily replace your bundled libs in 5 years to get better support for the latest fads in window management, audio playback, …
Linux distributions tend to build SDL2 in a way that explicitly links against all those libs, so indeed build SDL2 yourself; make sure to have the relevant development headers installed

About libstc++ and libc

Don’t try to statically link libc or libstdc++, this may cause trouble with other system libs you’ll use (because the system libs dynamically link libc and maybe libstdc++ - probably in versions different from what you linked statically. That causes conflicts.)
So what you want is to only use your bundled libs IF they’re newer than the ones on the system - and this decision has to be made per lib. This means that rpath $ORIGIN is not an option (because you want to make the decision which library path to use - system or your own - at runtime).

Uses Debian 7. Which (GCC 4.7) can cause issues due being unable to compile newer c++ code.

[azimut]

Trying this on azimut/water. Current problem both with SDL and ImageMagick is on the "step 3" when using the static-sbcl and the .core. SBCL tries to load again the libraries.

This might be a problem with CFFI.

https://github.com/spwhitton/consfigurator/blob/419781a056dd888c57e76a86e4dc0d3a5da96c6d/doc/pitfalls.rst#dumping-and-reinvoking-lisp

This second issue could be partly mitigated using CFFI's STATIC-PROGRAM-OP ASDF operation, as described in the "Static Linking" section of the CFFI manual ... CFFI currently tries to reopen all shared libraries upon reinvocation regardless of the use of STATIC-PROGRAM-OP.

| Merged | github.com/cffi/cffi/pull/160 |
| Open | github.com/cffi/cffi/pull/163 |
| ^163 | github.com/cffi/cffi/pull/162 |
| ^163 | github.com/cffi/cffi/pull/157 |

EDIT: sidestepped the issue. Forking the lisp libraries (sdl,magickwand) and wrapping it with a (eval-when) and now it seems to work as long as STEP 1 (dumping .core) compiles these libraries for the first time. Only reason it works now is due actions/cache (lol)

[azimut]

Error now is the same they have here, SDL can't initialize.

utdemir/ghc-musl#14 (comment)
nh2/static-haskell-nix#108

Which leads back the alpine gitlab issue linked above (lol) where they tried compiling a C++ app too.

I wonder if the issue is MUSL and not SDL (see musl.openwall.narkive.com link above)

[azimut]

gdb'ing into this. I have 2 binaries:

1. imagemagick only, static compiled, which works in calling the library and generating a PNG
2. imagemagick+sdl2, static compiled with sdl2 having debug symbols, does not work

Both SIGSEGV a lot:

1. does it 14 times before a LWP (thread) is started
2. does it 14 times before a LWP (thread) is started and many times after
   SDL Debug symbols work, but do not provide help on the SIGSEGV
   15th SIGSEGV is

0x000000000040e717 in cull_weak_hash_table_bucket (rehash=0, save_culled_values=0, fix_pointers=0x409b40 <pair_follow_fps>, alivep_test=0x40a540 <survived_gc_yet_VALUE>, hash_vector=0x1000448010, next_vector=0x1000440020, kv_vector=0x1000420010, index=9, bucket=99, hash_table=0x100017cdc0) at gc-common.c:1553

Which is part of SBCL (!)
This happens before any call to anything SDL_* happens.
Many other SIGSEGV happen before SDL_INIT but with no detail like above.
After SDL seems to attempt to start...

  SDL_TLSSet_REAL (id=1, value=0xffffffffffffffff, destructor=0x0) at /github/home/build/SDL2-2.0.12/src/thread/SDL_thread.c:57
  0x000000000095fdf1 in SDL_GetErrBuf () at /github/home/build/SDL2-2.0.12/src/thread/SDL_thread.c:247
  0x0000000000a78f2d in SDL_ClearError_REAL () at /github/home/build/SDL2-2.0.12/src/SDL_error.c:142
  0x0000000000a77aa3 in SDL_InitSubSystem_REAL (flags=65535) at /github/home/build/SDL2-2.0.12/src/SDL.c:150
  0x0000000000a77c9d in SDL_Init_REAL (flags=65535) at /github/home/build/SDL2-2.0.12/src/SDL.c:291
  0x0000000000920d59 in SDL_Init (a=65535) at /github/home/build/SDL2-2.0.12/src/dynapi/SDL_dynapi_procs.h:85

And finally this is how the callstack looks more or less...Initializes mouse/keyboard/main thread...but when it comes time to setup video...it knows it is X11...it tries to call dlopen("libX11.so.6") (the correct name for my system) but it fails

SDL_StartEventLoop+4
  SDL_TimerInit
    SDL_CreateThreadInternal+48 (cretes a 3rd thread)
SDL_InitSubSystem_REAL+238
  SDL_VideoInit_REAL+13
      SDL_InitSubSystem_REAL+11
         SDL_MouseInit+143
         SDL_getenv_REAL+12 SDL_VIDEODRIVER
         SDL_VideoInit_REAL+364        /github/home/build/SDL2-2.0.12/src/video/SDL_video.c:504
             X11_Available+21          /github/home/build/SDL2-2.0.12/src/video/x11/SDL_x11video.c:99
                 SDL_X11_LoadSymbols+8 /github/home/build/SDL2-2.0.12/src/video/x11/SDL_x11dyn.c:155
                     SDL_LoadObject_REAL+12
                     arg sofile = 0xd8d8d5 "libX11.so.6": 108 'l'
                         [0] from 0x0000000000bde71f in stub_dlopen+0 at src/ldso/dlopen.c:6
                     SDL_SetError_REAL+89 at /github/home/build/SDL2-2.0.12/src/SDL_error.c:46

Which if we look at the source code...https://git.musl-libc.org/cgit/musl/tree/src/ldso/dlopen.c

#include <dlfcn.h>
#include "dynlink.h"

static void *stub_dlopen(const char *file, int mode)
{
	__dl_seterr("Dynamic loading not supported");
	return 0;
}

weak_alias(stub_dlopen, dlopen);

LMAO

[azimut]

Due above. I will move to Ubuntu for now, later to an older ubuntu that 18 on a container.

Something related I found is that SDL supports an override variable when is compiled statically. That is to dynamically load a different version of itself.
https://sdl-mirror.readthedocs.io/en/latest/README-dynapi.html

[azimut]

Ubuntu (18/20) have some missing static .a of some libraries (seems like Alpine was more generous that I though). Confirmed lcms2 and lqr. So is on Fedora for lcms2. And also on Arch.

/usr/bin/ld: cannot find -llcms2
/usr/bin/ld: cannot find -llqr-1
/usr/bin/ld: cannot find -lImath
/usr/bin/ld: cannot find -lHalf
/usr/bin/ld: cannot find -lIex
/usr/bin/ld: cannot find -lIexMath
/usr/bin/ld: cannot find -lIlmThread

🙃

I don't want to be a OS mantainer. But...I might try Gentoo(kubler) which would also reduce the size of the binary/compilation time?

---

Edit: another bad idea is using alpine for a DYNAMIC compiled binary. Raises 2 problems:

1. sdl2(musl) would dlopen() X11/GL(glibc) which might or not need libgcompat and might be some undefined behaviour
2. I would also need to ship libc.musl.so

[azimut]

What a wild ride! I still need to learn more about daewok patch and linking in general.

But there is already support for what I want from this in sbcl already https://gitlab.com/ralt/static-program-op

I know less about what dark magic they use for it (they don't seem to use the .a version of the libraries!)

Ultimately you end up with a binary that is only dynamically linked against LIBC and a few more libraries

 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

While embedding (?) other .so libraries, which is good enough for me.

If I build in in ubuntu 18.04 (it has glibc 2.27, circa 2018).