Support musl-based AppImages

[rofl0r]

basically appimages are dynamically linked programs and require the GLIBC dynlinker.
they don't work on distros using e.g. musl libc, uclibc, or any other libc. also as can be seen by the open issues, they fail even to load between ubuntu 16 and 20.
even when having the right glibc version installed, they then fail because they also require a libfuse.so installed on the host.

the only way to create a really cross-linux binary is to statically link it.
i'd propose the loader is changed to use musl libc to create a static linked binary that has fuse built in (i.e. also statically linked) and then execs the dynlinker inside the squashfs with the right arguments to run the application entrypoint.

[ghost]

While I respect the problem, I personally have problems with the solution, most notably the static linking part.

my point of view on the security aspect of statically linking is that, older versions of libfuse.so may include security vulnerabilities, and so older packages may not inherit the new version of libfuse.so from the system. and besides, some linux distributions may provide customized versions of libfuse.so to match with their kernel, and that may be considered more platform incompatible to some. one of my other considerations is size, as a x86_64 fuse library for OpenSUSE Tumbleweed, as described by pkgs.org, is as big as 124.34 KB s, which is almost the same size of the current continuous build's runtime-x86_64 executable, lightweight my arse.

I'm not saying its best to still use dynamic linking, I'm just describing my problems with the solution. and besides, lets not ignore the fact that whilst it doesn't run "everywhere" per se, it runs on all the Linux platforms I've ever used, and that's a win for me.

I still feel bad that musl users are left out though. is there any other solution one can think of?

[rofl0r]

older versions of libfuse.so may include security vulnerabilities

they indeed may. however that's not an issue. the libfuse.a that's statically linked to the loader has only one purpose (and will only be used once for that): mount the appended squashfs. supposedly the fs is from a trusted source (the one shipping the appimage), so there's no exploitable scenario at hand. if an evil hacker can modify your appimage file, he already has access to your box.

[rofl0r]

one of my other considerations is size, as a x86_64 fuse library for OpenSUSE Tumbleweed, as described by pkgs.org, is as big as 124.34 KB

i can see that you never used static linking at all, or if you did then at least not with musl. thanks for raising those concerns though, so i can play mythbuster here.
so, whereas a .so must always provide all symbols the API provides, a static link can selectively link only the required functions. since you dont link all functions libfuse provides, your binary will only grow a few kb when using it (at least when the linking is done with certain options gcc/ld provides - one possibilty is to use LTO, the other -ffunction-sections -fdata-sections -Wl,-gc-sections).

as an example, here is a statically linked (with musl) bubblewrap binary: http://ftp.barfooze.de/pub/sabotage/bin/bubblewrap-static_x86_64.xz . it's only 70KB, yet includes selected parts of libc and libcap2.

[TheAssassin]

I still feel bad that musl users are left out though. is there any other solution one can think of?

On Alpine for instance, there's this glibc shim. I was once told it would work with some AppImages.

[rofl0r]

On Alpine for instance, there's this glibc shim.

that's basically just a hack, not a solution, and it works only to a limited extent for x86_64/i386.

[probonopd]

"Package once and run everywhere" is simplified for "properly made AppImages will run on most mainstream desktop distributions".

With "everywhere" we mean Ubuntu, Debian, openSUSE, and other mainstream desktop distributions.

If you like certain sentences to be changed in the documentation, then please point us towards the sentences in question.

[rofl0r]

... and other mainstream distributions.

isn't alpine linux mainstream too meanwhile ? i heard it's the number #1 used distro for docker containers. void linux seems to also have a huge following these days, judging from the number of commits and authors to their package repo.

but yeah, mainstream and everywhere isn't quite the same thing.

after reading the slogans on the appimage homepage (and linus' testimony) i thought "hey, finally someone got this right".

but my actual usage experiment was quite disappointing:

1. requires GLIBC, and a somewhat recent version at that
2. installed an ubuntu rootfs , but there it failed because no libfuse available
3. installed libfuse, now it complains libGL.so is missing.

at this point i gave up. (ftr, this is the appimage in question https://github.com/rizinorg/cutter/releases/download/v1.12.0/Cutter-v1.12.0-x64.Linux.appimage )

i actually have my own "appimage" project, it's a 23 line shellscript: https://codeberg.org/rofl0r/hardcore-utils/src/branch/master/packapp.sh

given, it doesn't try to stuff everything in a single file but into a single dir. though thinking about the use of squashfs some more, the contents need to be extracted for execution, so they either waste ram or additional temporary space. so in the end one requires the space for the compressed PLUS the uncompressed app.

[ghost]

i can see that you never used static linking at all, or if you did then at least not with musl.

Ah, my bad! You're right, I never tried explicitly experimenting with static linking. And, I'm happy that you have an alternative application to pack "app" "images"!

( Have you tried --appimage-extract on the appimage you linked to? Maybe that's more non-hacky than using a GLIBC shim. )

If there's nothing else you would like to point about this sentence, I think its best if you close this issue. Though, I'd like to help employ more cross-platform practices for AppImage, I'm thinking its better to create a new github issue for that.

I'm sorry if I'm assuming that you're done here,

Best regards,
Jaye!
( don't mind if I use your hardcore-utils repo you just linked to, nice stuff )

[ghost]

OH and BTW, --appimage-extract-and-run is also a neat command-line argument. I've also realized that #1081 , to my perspective, points to the issue of libfuse.so not working well on some distributions.

[probonopd]

isn't alpine linux mainstream too meanwhile

You are right, it should say "mainstream desktop distributions", not things that run on servers or in containers.

installed an ubuntu rootfs , but there it failed because no libfuse available

The easiest way is to run a Ubuntu Live ISO. What is missing in the Linux world is a definition of what packages can be taken for granted in a "desktop system", so we are using the Live ISOs as an approximation.

requires GLIBC, and a somewhat recent version at that

Would love to get rid of it, but that would require a) all dependencies to be bundled inside the AppImage for each application (go-appimage appimagetool -s deploy and appimage-builder can do this by the way), and b) the AppImage runtime to be statically linked against FUSE. Not impossible...

i actually have my own "appimage" project
given, it doesn't try to stuff everything in a single file but into a single dir.

That's a great first step. If you make this dir an AppDir, then you can even convert it into an AppImage by running it through appimagetool...

[rofl0r]

OH and BTW, --appimage-extract-and-run is also a neat command-line argument. I've also realized that #1081 , to my perspective, points to the issue of libfuse.so not working well on some distributions.

as mentioned in my last reply, after fixing the fuse problem, it then wanted libGL.so next.
thanks for the hint about --appimage-extract-and-run though, maybe this should be printed by default when libfuse isn't found.

Would love to get rid of it, but that would require a) all dependencies to be bundled inside the AppImage for each application

can't see anything bad about that. the above mentioned appimage, which is roughly 100MB, extracts to about 700MB of files, so whether there's 5 more MB (libc/libstdc++/fuse...) in it doesn't really matter imo. with the compression factor of about 7x the archive would be less an a meg bigger. i think people who choose to use something like appimage do so because there's lots of hard-to-build dependencies involved, and they really do want to have the app working everywhere.

and b) the AppImage runtime to be statically linked against FUSE. Not impossible...

*nod*. and if you use musl libc to do the static linking, the loader binary/stub will be almost certainly less than 200KB.

The easiest way is to run a Ubuntu Live ISO.

i understand, but i've chosen the rootfs approach because it has a lot of advantages imo. for one, you don't need to fragment your resources for use in a virtual machine (pre-allocated harddisk image, pre-allocated RAM), it makes it annoying to access other files outside the VM, you need to configure a separate networking subsys, etc.

[probonopd]

I agree that the option of being able to do this would be beneficial.

[ShalokShalom]

@rofl0r In case you write a tutorial on doing it like that, please let me know. Putting all dependencies into one package was for me what AppImage (and others) promised in the first place and I am still looking for a solution like this.

Sadly, this will not repackage all the already available packages, but at least those, I do.

Thanks a lot

[probonopd]

Putting all dependencies into one package was for me what AppImage (and others) promised in the first place and I am still looking for a solution like this.

Check out go-appimage appimagetool -s deploy by me and/or AppImageBuilder by @azubieta.

[ShalokShalom]

And that also includes musl libc and rootfs instead Ubuntu?
I forgot to mention that, while its equally promising to me.
Thanks a lot for the support

[probonopd]

And that also includes musl libc and rootfs instead Ubuntu?

It bundles whatever files it finds on the system it is running on.

I have not tested it with musl libc iirc, but i'd be nice to see if someone would take the time to make it work on such systems as well (in case it doesn't already). I remember I had to do some special ticks for ld-linux that comes with glibc, so possibly one would need to add some special cases for musl, although I don't think I had looked into this subject yet.

[rofl0r]

@ShalokShalom my rootfs approach is described here:
https://github.com/sabotage-linux/sabotage/wiki/Running-a-minimal-ubuntu-rootfs-as-regular-user
however that doesn't give you a complete app pack.

you can compile static linked applications using the cross-compile approach described in the README of sabotage linux. after bootstrapping stage1 with a crosscompiler, you build the libraries required, and then finally the application using STATICBUILD=1 butch rebuild foo where foo is the name of the program (after having a build recipe for that program in the pkg directory). this gives you a completely standalone binary like the bubblewrap binary i linked to earlier.

there's a caveat however: some programs (e.g. weechat) have a plug-in system using dlopen, so statically linking them fails. also libGL is designed to load its drivers dynamically (so one can load proprietary stuff like nvidia drivers), so statically linking desktop programs isn't feasible. there's been an attempt of the musl community to create an openGL daemon that links to the driver dynamically and serves requests from a libGL that can be statically linked against an app and communicates with the daemon, however it's unfinished as the contributor lost interest.

if you have any musl-based rootfs, say from alpine linux, you can however also use my app-pack shell script to bundle the app including all dynamically linked dependencies (except dlopen()ed ones, which you may copy manually into the app directory though).

[ShalokShalom]

Seeing this in the official documentation is in our all interest?

Then I suggest renaming this issue into something that approaches this, just so we have a reminder.

[probonopd]

the only way to create a really cross-linux binary is to statically link it.
i'd propose the loader is changed to use musl libc to create a static linked binary that has fuse built in (i.e. also statically linked) and then execs the dynlinker inside the squashfs with the right arguments to run the application entrypoint.

After the fuse debacle (libfuse2 no longer being shipped by Ubuntu) and lenthy discussions with some Ubuntu developers I tend to agree. We should strive to have a statically linked runtime. Any library we depend on might be removed from distributions at any time, because distributions are unwilling to commit to long-term compatibilitiy guarantees. Building the runtime statically with musl libc is probably the way to go.

@rofl0r do you know how to do it?

[rofl0r]

do you know how to do it?

certainly. you can either do it by compiling on a distro using musl already, or build a cross-toolchain using musl-cross-make, and then build all the required library dependencies from source, such that a static library is generated (a package using autoconf would typically provide --disable-shared --enable-static options to do so), and install that library into the musl-cross-make prefix. after you've prepared all dependencies, you then build the appimage launcher with said cross-compiler and LDFLAGS=-static.
my build from source distro sabotage linux actually provides a quite comfortable way to cross-compile the packages (recipes) it ships into a cross-compile rootfs prefix, which could be used for such a purpose.

if you need help with that you can come to my channel #sabotage on irc.libera.chat or #musl for general help with using musl.

[probonopd]

Thank you @rofl0r. Here you can see my slow trial and error "progress": probonopd/static-tools#17

Any hints appreciated. Thanks!

[ShalokShalom]

Wonderful, I love this decision.

As someone, who values independence per se, I always wondered why AppImage depends so heavily on libraries, who are shipped on some distributions, maintained on some distributions, and regularly failed to load on some distributions.

I think the way to statically link everything, and doing so with musl, is a HUGE improvement to AppImage as a whole.

Congrats 🥳

[probonopd]

Thanks for your encouragement @ShalokShalom

[probonopd]

Discussion continues at

• Statically linked runtime #877