Add Alpine Linux support

[azubieta]

Alpine Linux is well known for being a highly optimized solution for containerized software distribution. This also applies to AppImages. Therefore it will be a great opportunity to reduce the final AppImages size.

[probonopd]

To be more specific, this is about using musl libc applications (such as those that come with Alpine Linux) as the payload inside AppImages. https://github.com/probonopd/go-appimage can do it, if you run with the -s option (standalone = bundle everything), like in the example at mumble-voip/mumble#3959 (comment).

Another issue is about making the AppImage runtime run on musl libc based distributions like Alpine Linux, too: AppImage/AppImageKit#1015. Any help on that issue is highly appreciated, too.

[azubieta]

appimage-builder relies on the base system package manager to resolve dependencies, so it's also about supporting apk.

[lslvr]

Another issue is about making the AppImage runtime run on musl libc based distributions like Alpine Linux

@probonopd: Why not statically linking the runtime? Or is there anything stopping you from doing it? As I see, doing so would solve that problem.

@azubieta: I'd like to help with this issue. I've used Alpine in the past.

[lslvr]

Oh, I just found this: AppImage/AppImageKit#877 (and related issues).

[azubieta]

To make an Alpine base AppImage we need a muslc bound or static runtime. Which is not currently available.

@probonopd are there any plans for it?

[xordspar0]

I made a prototype of this from scratch using witchery, a tool made by one of the Alpine devs for making filesystem images that contain all dependencies for an app and nothing else. Witchery is only ~150 lines of POSIX shell, so appimage-builder could either use it or re-implement it from scratch. abuild, the Alpine package building tool, and apk, the Alpine package manager, do most of the heavy lifting.

Here's what my prototype does:

1. The app has to be build on Alpine or in a chroot. Installing Alpine in a chroot is very easy (manual instructions, automated script).
2. make install the app to a staging area.
3. Use witcher-buildapk to package the app itself with all its assets. It puts the whole staging directory into an apk file. abuild auto-detects all dependencies by tracing the dynamic library requirements with scanelf.
4. Use witcher-compose to build the AppDir. witcher-compose basically shells out to the normal Alpine package manager to install the apk file we just made to the AppDir with all its dependencies. At this point I had to use a flag on witcher-compose to say "Please also install these libraries that my application is going to dlopen at runtime". appimage-builder is better at this because of the strategy it uses of grabbing every file that the app touches at runtime.
5. Finally, write a 2 line shell script to use as the AppRun to set the LD_LIBRARY_PATH and run the app.

It's just a prototype, and the process is a bit messy now, but it's exciting to see a /lib/ld-musl-x86_64.so.1 binary running on Ubuntu. 😃 There is still plenty of engineering work needed here to iron out how this will work with appimage-builder, but the solution is within reach.

[probonopd]

@xordspar0 do you have a downloadable example of an AppImage created with your prototype? I'd like to try it out. Thanks!

[xordspar0]

Sure! I uploaded it here: https://github.com/xordspar0/nxengine-evo/releases/tag/alpine-inside

This version isn't any smaller than the Ubuntu-based AppImage, but I think that's because I installed more dependencies than necessary in the "add libraries until all the dlopens succeed" step. There's more investigation to do here.

[azubieta]

Hello @xordspar0 this looks really interesting!
I tried the AppImage you shared and it still requires /lib/ld-musl-x86_64.so.1 which means that you need use it to launch the application in your AppRun. It would be something like this:

LD_LIBRARY_PATH=$HERE/lib:$HERE/usr/lib LIBGL_DRIVERS_PATH=$HERE/usr/lib/xorg/modules/dri $HERE/lib/ld-musl-x86_64.so.1 $HERE/usr/bin/nxengine-evo

After that fix I got the following error, which is something I was fearing. The bundled graphic libraries are not compatible with my system. This is why AppImages are a bit "permeable" and use some of the system resources. So to make this work we should be able to allow the application load the system graphics libraries.

[xordspar0]

it still requires /lib/ld-musl-x86_64.so.1

Oh, that's weird. I didn't realize I had ld-musl installed on my system. Thanks for the easy fix.

The bundled graphic libraries are not compatible with my system.

Oops, that it my mistake. You may be right that it's a good idea to make the app use system graphics libraries, but that's actually not the issue you're running into. When this app prints that message, it actually means that it couldn't find the graphics assets. I forgot to compile with position-independent resource paths.

Thanks for taking a look! I fixed these two issues and uploaded another image to https://github.com/xordspar0/nxengine-evo/releases/tag/alpine-inside.

[probonopd]

Very interesting! The basics seem to be in place - I can run the AppImage even on FreeBSD helloSystem and it doesn't explode 👍

It prints this message:

FreeBSD% runappimage ~/Downloads/NXEngine-Evo-v2.6.5+138a3e2-Linux-x86_64+musl+deps.AppImage 
AppRun                                  org.nxengine.nxengine_evo.png
dev                                     proc
etc                                     tmp
home                                    usr
lib                                     var
org.nxengine.nxengine_evo.desktop
terminate called after throwing an instance of 'spdlog::spdlog_ex'
  what():  Failed opening file debug.log for writing: Read-only file system
Abort trap

Seems to suggest that the musl part is working just fine, but something wants to write inside the AppImage, which obviously is not going to work by design. Hope this is easy to fix.

When I extract the AppImage and run the AppRun in the AppDir, I get

I'd call this a resounding success. Speaking of sound, /etc/alsa/conf.d/10-samplerate.conf does not exist on FreeBSD but I'd say this is an issue of the application rather than your packaing method.

Congrats, this is awesome. Awesome beyond belief! Might be a good way to low-overhead truly compatible (even with FreeBSD!) AppImages. My logic being, if it can run on FreeBSD, it should run on pretty much all Linuxes.

[azubieta]

Now it works on my end. With audio and input 👍 Awesome job!

Let's get our hands dirty with code, It would be great if you could put in a small doc the whole process to create this bundle so we can add it to appimage-builder.

[azubieta]

Now we have a runtime that can be executed in alpine linux (see: https://github.com/AppImageCrafters/appimage-runtime). But the AppRun is now the one that is dynamically linked to glibc because it uses gnu_get_libc_version to find the version of that lib in the system.

We need to find another way of doing it. ref: AppImageCrafters/AppRun#61

[eyduh]

Running postmarketOS, which is built on alpine, on my laptop, leaving comment here to get pinged for development of this issue and can test stuff if needed.

[xordspar0]

Here's another example of my weird Alpine-based approach. https://git.sr.ht/~xordspar0/packages/tree/main/item/AppImage/srb2/build.sh

• Works great on my Linux+ARM handheld.
• Uses the tiny witchery-compose script (which uses apk magic to get all dynamically linked dependencies automatically) and go-appimage (which takes care of ld-musl stuff for you).
• I took a shortcut by using a pre-built version of the app from the Alpine repos, but that led to some hacks around issues like absolute data directories, so I'm not sure I came out ahead. In any case, it's useful to know that this works with pre-built software thanks to go-appimage's patchelf trickery.

[probonopd]

Very interesting @xordspar0. I like the approach to use Alpine.

# This old version of patchelf doesn't work with binaries over a certain size
# https://github.com/NixOS/patchelf/issues/305
rm squashfs-root/usr/bin/patchelf

At some point the newer versions of patchelf crippled binaries; if you are sure that this is no longer the case then we may consider to use the latest version in go-appimage.

# Demangle ld-musl, which appimagetool messes up with patchelf.
cp /lib/ld-musl-$(uname -m).so.1 "$appdir"/lib/

I'd like to have this fixed in go-appimage, opened an issue here:

• Do not run patchelf on ld-musl probonopd/go-appimage#310

[xordspar0]

At some point the newer versions of patchelf crippled binaries; if you are sure that this is no longer the case then we may consider to use the latest version in go-appimage.

I'm sure it's working in my case! I was going to submit a PR to update patchelf, but I saw in the git blame that it was held back for a reason. I'm using the latest patchelf from Alpine edge, 0.18. I believe go-appimage is using 0.9, if my memory is correct? I just checked if Alpine has any downstream patches. Technically there is one but it wouldn't matter for this case. https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/main/patchelf

I'd like to have this fixed in go-appimage, opened an issue here:

Thank you for opening those two issues, I meant to do that but I got tired and went to bed. And for fixing them! That was quick. I tested my script again without those two hacks and everything works perfectly.