Musl Support #1151
Comments
|
Hi, maybe you misunderstood here 👇
No, AM does not build anything at all. AM downloads from the source (sites, official repositories or not) the packages distributed by others. I will tell you more, when I have the opportunity to choose between GLIBC and Musl, I always prefer the latter because it is more widely compatible. The only AppImage package among the over 2100 available that is assembled "on the fly" is I also specified it in the README
you can download any installation script (command Finally, precisely because it is based on BASH/SHELL, it can work on all architectures, regardless of whether the system is based on GLIBC or Musl. So, as already stated in the README, if a program doesn't work, you should contact the upstream developer, or at least whoever packaged that program. It's not AM's responsibility to make them work, but to put you (the user) and the developer/packager in contact. |
|
@CrazyAce25 It is now possible to make appimages that work on musl systems, see these examples and I have more recent ones as well. And I'm making PRs that fix these issues, but for example this one is stalled for some reason.
the former* 😄 |
I'm not talking about the packages I build in my repositories, but the ones others distribute, when they create multiple builds and specify GLIBC or Musl. I tend to choose Musl in general. My "packaging" skills are limited, you know. |
|
@CrazyAce25, is unrelated to AM, but this may help you in your quest: You can use If you want examples of it, you can check the AppBundleHUB store & repo. Here are some:
Well, you get the gist of it...
The Alpine Linux rootfs is used by default, but you could just as easily choose Void Linux, Arch, CachyOS, etc. |
|
Guys, it doesn't take much to make better packages than mine, be it AppBundle or AppImage. Mine are just "proofs of concept", and @Samueru-sama has improved them a lot, just look at his GIMP and MPV Appimages. If my amateur work as a packager has served to spur one of you to say "hey, but I can do better"... for me it's already a big victory. |
|
Also @CrazyAce25 the Steam appimage and all the conty appimages that Ivan has should work on your system as well. |
and I emphasize that 80% of the work done on Steam AppImage is due to @Samueru-sama |
Thank you for the explanation. So its basically just a way to pull appimages from any place it can reliably source them. I honestly had not really looked into how AM/Appman worked under the hood and just recalled reading in the documentation somewhere that it was an AUR style repository of build scripts which made me think it was actually building the applications for some applications by pulling the binaries and creating an appimage from said binaries. Anyway, this makes much more sense and explains why musl isnt supported across the board. Hopefully @Samueru-samas efforts via helix-editor/helix#11243 get integrated sooner rather than later. Its seems like quite an oversight that appimage wasn't created with musl in mind to begin with (maybe musl wasnt really used at the time?). Given that it was overlooked it seems strange that its taken so long for this functionality to be integrated. |
You might want to make some noise at the helix issue about this if that's something that you want as well btw. Anyways, if you want help/need getting a specific appimage to work on musl let me know on the discussion I have about that. |
|
So I was thinking about how AM/Appman actually works compared to how I thought it worked and was wondering if something could be implemented to extend its functionality. Since you have demonstrated that there is a way to convert arch packages to appimage (https://github.com/ivan-hc/ArchImage), would it not be possible to have AM/Appman pull an arch package from the chaotic AUR (https://aur.chaotic.cx/) and then build it into an appimage that would be universally usable on all Linux versions, including musl based distros? If that worked, the same could be tried for debians https://docs.makedeb.org/prebuilt-mpr/introduction/ packages? I know your page shows that the deb conversion isnt quite as successful yet, but maybe could be improved or we could test which worked and which didnt and make use of those that do that arent already provided via the chaotic-AUR? This would be astoundingly useful for a large number of people, effectively extending a majority of the AUR (and possibly the Debian Prebuilt-MPR) to the entire Linux eco-system at large. |
Anything specifically that you think I could/should say that would actually make a difference in getting them to move things forward? |
|
@CrazyAce25 my "Archimages" are just a desperate (like me) attempt to create AppImage packages in a simple and alternative way. That is, nothing that @Samueru-sama and others like him can't do 10 times better. In fact, the popularity my packages are aiming to achieve is meant to "convince upstream developers to promote the AppImage format". In fact, I will close/archive my unofficial repositories as soon as upstream developers decide to take on all that work. Also, I have built over 70 AppImage packages in all my repositories. That's too much for me to handle. I'm trying to get rid of them, one at a time, as better projects than mine come along. My only goal is to convince developers that if an ignoramus like me can build AppImage packages for free, they can do it too. Unfortunately, many say they "don't have the resources to maintain them". That's bullshit. I'm the proof that they can do it themselves, and without much effort. |
|
Alright, it just seems like these large binary repositories that already exist offer a huge opportunity to build appimages on the fly at install time that would make life much better for the Linux eco system at large. Chaotic AUR, Deb MakePKG and Nix packages could all be utilized. I may pitch the idea to Dbin and see what they think. Also, I dont think that go-appimage builds in a way that is compatible with musl based distros does it? Seems like it should probably be done with Nix-Appimage if it doesnt. Is there any downside to using Nix-Appimage instead of go-appimage to create AppImages or do we simply get musl compatibility when utilizing Nix but the rest is basically the same? I used go-appimage for the Cromite appimage build script, but that was my first attempt at building an appimage but Im now thinking I should have used Nix-Appimage. |
Dbin's author here, I have the AppBundleHUB, if you're open to contributing more recipes to the repo, they'll appear on It currently uses Alpine & ArchLinux(cachyOS because it has smaller packages, somehow), it converts their packages into portable AppBundles. (I'm a Musl user btw). Support for other systems can be added, you just need to create a pelfCreator "extension" (like this: https://github.com/xplshn/AppBundleHUB/blob/master/baseSystems/arch_x86_64) You can check it out here:
Yes, the size of the produced AppImages is gianormous, and also... You can use Ajam's NixAppImage fork, which fixes some of the shortcomings NixAppImages have: https://github.com/pkgforge/nix-appimage But the generated file will still be gigantic |
"Portable AppBundles! For everyone! I'm just trying to make sure Im clearly understanding how your system works. On your AppBundle page it makes it seem like you are essentially utilizing the ROOTFS from Alpine Linux combined with the chosen program in a container together a bit like Docker (basically pelfCreator & something like Conty working together?) and calling this an "AppBundle." You mention above that you are also utilizing Cachy as well, so is this done in the same manner where you are utilizing the Cachy ROOTFS combined with the given program and packaging it together in a container that you then call an "AppBundle" Do all of your AppBundles end up supporting musl due to the way you package things together in the contaner system you utilize, thus other Linux ROOTFS systems and packages can be added and utilized with your packaging system and they would all still have support for musl? Also, how large are the resulting files utilizing your system vs Nix-Appimage & Go-Appimage and are there any compatibility issues like Nix-Appimage experiences with OpenGL? Thanks for taking the time to explain, I greatly appreciate it. |
The AppBundle format is not related in any way here to what pelfCreator.
That's it. pelfCreator could be used to create AppImages too. And it ALWAYS parts from a rootfs, but it can then ditch it, or ditch only certain, unnecessary parts of it.
Examples:
NOTE: NOTE 2: When I say container you may be thinking of fancy sucky stuff, well, I'm just using |
|
Thank you for the explanation. Im new to these various package creation techniques and so Im just trying to wrap my head around what each of them offers and how they can work together to understand the best way to get near native performance with the smallest footprint while ensuring the highest level of compatibility (MUSL, GLIBC, networking, Graphics stack) and avoiding the installation spaghetti of FlatPak & maybe Soar (untested). I think Im beginning to better grasp how your system is setup now. I wasnt aware that bwrap could be used as a container to run apps like a low level version of docker that just requires a little more manual configuration. This system of Appbundle utilizing pelfCreator for the creation of the AppDir and bwrap as the container system seems to be the best available for packaging near native performance applications while being able to ensure full MUSL/ GLIBC etc compatibility in a single executable file like AppImage. |
|
Does this sound feasible? Creating a script capable of creating a ROOTFS using tools like debootstrap for Debian/Ubuntu or pacstrap for Arch etc etc to create a minimal root filesystem, then chrooting into the newly created ROOTFS to install whatever packages required that you are wanting to AppBundle (e.g., Chrome or whatever), then use pelfCreator to create an AppDir from the ROOTFS, then configures bwrap to run the application in an isolated environment for the programs that bwrap is capable of functioning well with (or it could be left out of the binary to be run by the user at their discretion @ runtime) and lastly use ssc (Simple Script Compiler) to create the binary or something else if it fits the situation. I only say ssc because you can target different interpreters based on the environment where the binary will be utilized and has features that allow for embedding an interpreter or archive into the output binary which reduces reliance on the availability of a specific interpreter on the target system and thus enhances portability, which is the entire point of this exercise. Not that bwrap cant be utilized like has been mentioned, but it seems like using SSC would bypass the issue of name spaces and provide maximum portability and speed. Id like to be able to automate this process via interactive script to the highest degree possible. |
That you are a musl user and that you need this, do it at the issue that the PR closes and not at the PR itself though.
Flatpak works this way as well more or less, same with NixOS, bubblewrap is used as a way to "swap" It does have a slight performance impact though, and also these solutions depends on access to user namespaces which may be disabled on your distro. (specially ubuntu is doing this now from 24.04, but fixing the issue is easy though). I recently migrated all my anylinux appimages to this repo and explained the goals here. They do not use containers so far, but of course they take more time and effort to make. |
|
Everything described here, @xplshn's PelfCreator already does.
The only right answer here is @Samueru-sama's hardwork at pkgforge-dev/Anylinux-AppImages, all other solutions either rely on containers, are gigantic in size or all of these.
I am really curious how you got to this comparison. Dbin & Soar both use the same source: METADATA.md |
It was my understanding that Soar actually installs files at various locations on the system such as FlatPak and not simply a single executable such as AppImage. I have never installed Soar, only read about it and so its possible I may be mistaken. If it does indeed only install single binary files (1 file per program like appimage), I'll install and take a look at it. |
It's a package manager Just like AM/Dbin. It does the same things as what AM/Dbin do. |
Interesting. I thought there were some package files it installed that were single file appimage style packages and some that were binary style packages that it would install at various locations on the filesystem. I will definitely take a look at it since it seems I was mistaken about its mode of operation. How is the Soar package compatibility with musl? |
It depends on the catalog. If you install stuff from, There's a new repo: https://github.com/pkgforge/soarpkgs Since, this isn't related to the original issue, and conflicts with AM's interest, a better place to discuss would be creating an Issue/Discussion at any of these repo: Or better, if you use telegram: https://t.me/official_loonix/63949 |
pelfCreator does all of this already, its automatic. About And yes, making fully self-contained AppBundles is possible. |
@Azathothas pelfCreator offers a mode where the rootfs is ditched and it only keeps the necessary files, its the |
Wow, that's impressive.
Good to know, thank you.
You've done some truly amazing work & I simply didn't know enough to grasp the extent to which you're already traveled these roads. Thank you very much for taking the time to educate me, Ive learned quite a bit in a short time here. |
Well it seems quite the waste to have Cromite being packaged with go-appimage when it apparently isn't going to be compatible with musl systems as that was the entire reason I wrote that script for packaging it to begin with (didnt know what I didnt know). Any chance you might be interested in modifying the script to build it the way you specify here (if possible) so we end up with an appimage thats compatible with musl? Or maybe go-appimage would be interested in integrating your modifications so that the output file built with their system results in a musl compatible appimage? Or is it just going to be easier to get an AppBundle built that could then be accessible via AM/Appman or Dbin on Chimera_Linux since Chimera doesnt utilize the linux userspace tools required for AM/Appman functionality as far as I know? |
You can but you need to make an FHS file structure in the AppDir and run go-appimagetool with the But I can tell you already it is not going to work, go-appimage just has too many bugs.
That's already being considered, they are not "my modifications" most of the time though, it is thanks to sharun which was made by @VHSgunzo. In fact if you check the issues at sharun you will see probono (the author of go-appimage) using it to build an appimage of orcaslicer that works on musl systems.
AM has to be updated to be POSIX shell, I already did this on a few parts of it though.
Going for the big fish eh, ok will take a look at it. |
This is why I asked you before trying to jump back in and modify the Cromite build script myself to try to make it functional on musl. I simply do not know enough about it. That Cromite build script was the first appimage build script I have ever made and I know little of go-appimage internal workings.
Thanks for the clarification. The fact that the go-appimage creator is already working on this sounds promising. Hopefully it works out and we get a go-appimage build that ends up building musl compatible appimages in the near future.
Thats wonderful news. I was not aware that this was even on the radar and being worked on. I think thats a really smart move as there are inevitably going to be more distros popping up that will be making use of the BSD userspace utils that Chimera ported (or improved the port of and is now up-keeping). People wont be limited to building with busy-box / toy-box now and I think thats a wonderful thing. I dont know about you or others here, but I can't stand systemd, so I solely make use of systems that avoid that catastrophe, my favorites being Obarun (which has future aspirations of adopting musl), EasyOS and Chimera. Im loving S6/66 and just starting to learn Dinit.
Thank you for taking a look and seeing what you might be able to do. You know a ton more about it than I do so I thought I would at least ask. If you are able to make it work, I'll be able to study your build script modifications and learn something. With the number of systems running musl now and the number that are inevitably going to pop up, I hate to keep building software in a way that is only glibc compatible when the biggest issue holding back musl OS adoption right now is that very lack of easily accessible compatible software. |
|
cromite is almost ready btw: https://github.com/Samueru-sama/cromite-AppImage/releases/tag/continuous You will notice that it crashes when you try to load any website, that's because nss is a disgrace and I need to apply this fix: JonathonReinhart/staticx#129 |
It is still in the experimental stage. Some commands, like I would like to test AM in FreeBSD indeed, but I do not know how feasible it would be to use it. |
|
@CrazyAce25 Test now at the cromite AppImage repo. |
The Cromite Appimage you created works great on Chimera-Linux. EasyOS gave the error, "root without --no-sandbox is not supported" but launched just fine using the --no-sandbox flag. For their built in AppImage system it generally runs AppImages under their own user, but its only automatic for their integrated AppImage files I believe and Im simply not aware of how to launch it that way yet myself as its the first time Ive been able to get a non integrated appimage to launch on that system at all. Either way, doesnt look to be an issue with the appimage you created. Congats on a job well done and thank you. So Ive got a few questions now. First, is this something that is going to be auto built when new versions of Cromite get released so that it stays up to date? Also, is your build script going to be something the creator of Cromite could easily use or should we just let him build with the script I created for now and hope that go-appimage eventually integrates changes to implement full musl support or is there another option? @xplshn The next question is can we get this integrated with Dbin & AppBundle Web Store so that its accessible on Chimera-Linux (and whatever/whoever else may need it) since AM doesnt function there (yet)? https://easyos.org/about/how-and-why-easyos-is-different.html |
Sure! I can add this program to the AppBundleHUB |
Even though it is an unofficial release, you can still use the or Since this is an ongoing request to the upstream developer, I'd rather wait for the guys to give me the go-ahead. |
|
@ivan-hc Is there anyway to do a search through the entire catalog of all AM packages offered for those that offer support for musl? If this isnt currently possible, is it a feature that could be added somehow? I'd like to pitch AM to EasyOS for integration but Im rather certain they arent going to integrate a mixed collection for their user base that can't easily be sorted by packages that function on their OS. @xplshn Is every package offered by AppBundleHub and/or Dbin supported on musl systems? |
Yes, that's the plan. If there's anything that you find does not work on glibc (since I'm a Musl user) do tell me. They're all made with portability in mind. And AppBundleHUB uses Alpine as its host to build stuff, so... Yeah... Things are supposed to work in both MUSL & GLIBC. But you can think of Musl as having first-rate support, since that's what I use. |
|
@CrazyAce25 I distribute other people's packages, I don't know which ones actually support Musl and which ones don't... in that case, I think Dbin is ideal for your use case, instead of AM. |
Id be happy to test them all 1 by 1 on a musl system for support if it was something that you were interested in adding, but of course the issue being there is no way to keep track of ones that may change support in the future. Anyway, I just wanted to check with you to make sure I wasnt overlooking anything. |
|
@CrazyAce25 from the Toolpacks repository I already manage everything except the AppImage packages, those are already in the AM database. I think that the same ones, from the Toolpacks repository are certainly more compatible, having additional changes that should guarantee compatibility... but I think that it should be the developers' or upstream packagers' task to improve their packages. It is with user feedback that we improve the products, not by taking on the work of others (I have over 70 AppImage packages to maintain, and believe me, it's not worth it... in fact... I regret having made so many of them). |
|
@CrazyAce25 its on |
Do your AppBundleHUB packages auto populate to the AppBundleHUB store @ https://fatbuffalo.neocities.org/AppBundleHUBStore? |
Yes, I'm lazy, so I made sure to make all of the infra scalable, and by scalable, I mean, automatic. Packages which have an |
Cromite has never had a FlatPak made from it so I doubt it is going to be in that database (if Im understanding this correctly). If that's the only way its able to auto populate, it will likely have to be added manually or some other way. |
That is really wonderful to know and will likely be quite useful at some point. You might consider adding that info to your github description about how it can be used via FreeBSD, even if its not officially supported yet (or may never be) as Im sure there are people who would be more than happy to utilize it on FreeBSD, officially supported or not. |
That would be an issue with the namespaces, which is weird since this is usually an issue that affects non privileged users. Chromium sandbox used to rely on a SUID binary and they later moved to namespaces, both of which may not be available to regular users but you sure as hell should be able to use when root. In any case this would be an issue with Cromite and not the appimage itself, since the appimage doesn't actually use containers to work.
The repo is set to build automatically every 15 days, although I will lower this to every week. (EDIT nvm it was already every week). I want to improve the CI of all my appimages to only build when a new upstream version is released and also preserve the old version, haven't had the time to look well into it.
Yes
Most AppImages of web browsers and also electron apps just rely on the binaries they already compile, which target most glibc systems on their own and that seems to be good enough for most people, but not for your case of course. I will post the AppImage at the issue anyway.
go-appimage already has "full musl" support, just like how ffmpeg supports h265 hardware accelerated encoding with amd gpus on linux? I couldn't get it to work but ffmpeg does support it 😁 |
|
@CrazyAce25 I have decided to only allow truly portable packages for https://github.com/pkgforge/pkgcache In the meantime, come hangout with us (@Samueru-sama @xplshn and rest of @pkgforge) at: https://t.me/official_loonix/63954 |
Sounds like good news to me, I'd be interested to hear/learn more about it.
Unfortunately Obaruns live usb is having problems installing on my new hardware and won't boot properly (without the use of Ventoy with grub2 boot) for some reason unknown to me as of yet. Even when I get it to boot utilizing Grub2 via Ventoy, the installation never actually boots upon completion. I'll see if I can make my way over there once Im able to get these issues sorted out. |
|
Hi, @CrazyAce25 , hopefully you have fixed your boot issue. All future means & links to communicate can be found at: https://docs.pkgforge.dev/contact/chat Feedback is appreciated, as work moves ahead to only contain truly portable packages in our prebuilt cache |
@CrazyAce25 Progress has been made at #1209 to improve compatibility with the bsd userland, now the only requirement is having bash wget and curl. |
|
Sorry @xplshn I tried to keep the conversation more on topic, removing our "private" conversation. |
Is your feature request related to a problem? Please describe.
For those that wish to be able to use AM/Appman on musl based systems, yes.
https://discourse.nixos.org/t/nix-bundle-nix-bundle-appimages-glibc-questions/22827
AppImage/AppImageKit#1112
Describe the solution you'd like
A way to add musl support when the package is built.
Describe alternatives you've considered
Since AM/Appman is a system of build scripts and not simply a repository of binary packages, I would love to see a way to add musl support to the builds. If it adds too much to the size of the packages, maybe a flag to add musl support for those that need it. There are at least 4 systems (Chimera-Linux, Alpine Linux, EasyOS, Void Linux), 2 I personally use that rely on musl and its painful to not have access to the wonderful system you have built via AM/Appman. Two of these independent distros, Chimera-Linux (which utilizes bsd userspace, so may need a bit more help to function correctly) & EasyOS (which supports a very small collection of Appimages natively) specifically, are hurting for compatible software that is easily accessible and AM/Appman would be the perfect solution if there was a way to have musl support added to the builds, possibly through a flag at install time? Having FlatPak functioning on these systems but not AppImage which is suppose to be the answer to universal package distribution doesnt put AppImage in a good light and leaves these innovative distributions using less than optimal solutions. It seems like a real opportunity for AM/Appman to shine if its possible to adapt to this use case.
The text was updated successfully, but these errors were encountered: