Portability and future-proofness

[phil294]

I would like ahk_x11 (and consequently, all compiled scripts made from it) to be eternally compatible with all sorts of Linux distributions, for as long as possible, while also staying as small as possible in terms of size. I'd like to think of Win32 API being a role model here: An AutoHotkey script compiled in 2002 on Win95 will still run perfectly fine on a Windows 11 machine. For this very project, I consider this as a notable goal too because it will often not be shipped from your distribution, but as a locally shared executable file, and thus everything needs to be future-proof as much as possible.

Now in the Linux world, things are much more complicated, thanks to the fragmentation of modules. As far as I can see, there are two options:

1. Statically link everything. This is the current approach.
   1. Upside: As small as possible
   2. Primary downside: Need to predict the future wrt which libraries to dynamically link. I tried to take Flatpak's excludelist as a guideline, but that list is probably also useless for any prediction further than > 5 years ahead into the future ("This is a working document; expect it to change over time."). Somehow I even missed some dependencies already as users have reported missing dependency libssl (libssl.so.1.1: cannot open shared object file #18). Not a great start...
   3. Not yet statically linked:
      1. X11. Xorg has been a deprecated and unmaintained beast for decades now and only exists out of a sheer lack of successful successor. It's now finally on its way out, with Wayland taking over, but that is a big problem because the latter misses critical features, and it means breakage of dependencies for AHK_X11. Once we support Wayland protocol Add Wayland support #2 (the sooner the better), the binary should be able to run on either system. Is it possible for a binary to just optionally dynamically link to X11? I assume not, so at some point statically linking X11 will be necessary either way.
      2. Gtk 3. Right now, Gtk 3 is installed everywhere. I'd like to link it statically let alone to escape Gnome devs' inane fetishism for breaking changes, but alas, they seem to have made this nearly impossible (?). In a few years time, Gtk 3 will inevitably be deprecated and omitted from certain distributions, so something needs to happen here. Also: the sooner the better. I'm not even against ditching Gtk entirely. It was an obvious choice because it is ubiquitous and somewhat similar to windows api. However, when there is something smaller (read: statically linkable and stable) that fulfills all requirements by AHK spec, I would like to switch, but preferrably only if it properly integrates with OS theming and supports atspi. So something like Gtk or Qt can work, but Webview-based technologies can not. Needs more research, but for now I'm staying with Gtk...
         1. This also includes At-Spi, the interface for assistive technologies, now used for all sorts of Control* commands (release coming soon). This can be decoupled though, not sure about static linking once again
2. Flatpak
   1. Upside: Gtk can be dynamically linked but still shipped alongside
   2. Downside 1: Much larger bundle size (?)
   3. Downside 2: Script compilation probably needs to work differently. Right now, ahk scripts are simply put at the end of the binary, but since flatpak relies on its own weird extraction step and dependency management at runtime, I'm not sure this can all still work
   4. Flatpak adds bloat and complicates everything and is a general headache I would like to avoid... maybe it's still the right solution though (https://ludocode.com/blog/flatpak-is-not-the-future is a good article)

Right now I would prefer to build one big binary where everything (including x11 and gtk) is linked statically so that the only dynamic dependencies are the kernel and libc, apparently the only dependencies led by sane people. I don't have much notable experience with all of this, however (as you may have been able to figure out), so this really needs some more thought input and/or preferable some advice too. The fact that AHK_X11 is built with Crystal and not C/C++ should not matter, the usual constraints are all the same, and external libraries can be bound to Crystal code.

[Drugoy]

Hi.

I would like ahk_x11 ... to be ..., while also staying as small as possible in terms of size

Could you, please, provide some rough numbers for comparison?
Also, don't you think that AppImage format would suit here (well, IMO everywhere else too) better than Flatpak?

My opinion (not that I tell you what to do) is that I'd gladly accept better compatibility (wrt to future) at the cost of, say, 100mb on the disk for a program that I'd use heavily (and I know I would if AHK_X11 would be as comfy to use and as functional as AHK was for me on Windows).

Recommended articles:
https://github.com/AppImage/AppImageKit/wiki/Similar-projects - a good comparison of AppImage against other similar projects (mainly Snap and Flatpak).
https://github.com/AppImage/AppImageKit/wiki#for-application-developers - small FAQ on 'AppImage for application developers'

[phil294]

Hello @Drugoy, thanks for your input :-)

Could you, please, provide some rough numbers for comparison?

I think below 10 MB would be very much desirable but I am starting to think this might not be possible, unless I recompile all 89 lib dependencies myself while meticulously eliminating dead third party code. I kind of want to try this, at least with the larger dependencies: libcrypto takes 2.9 MiB, libxml 1.8 MiB and so on. The largest one is libicudata with 25.7 MiB, not yet sure what can be done about it. libgtk alone eats up 7.2 MiB as a dynamic library (statically linking it isn't even documented/supported). This looks like a major, never-ending undertaking, but an interesting challenge to cut it down as much as possible, e.g. by putting all dependencies as sub projects in this repository, setting up tests and a build pipeline and then aggressively removing code. I wonder if this is an approach that has ever been taken by another Gtk application author before, or maybe my understanding of it all is flawed anyway. But some sub-dependencies such as libxml may be completely removable after all. It seems ahk_x11 is a rather special case due to the desired script compilation / portability feature. By all means, there's more important things to do such as wayland support or adding the remaining missing features to the language.

Also, don't you think that AppImage format would suit here (well, IMO everywhere else too) better than Flatpak?

You are right, I actually meant (read: confused it with) AppImage. I read more about it such as the links you provided and it seems like the best solution for now. I got it t work with linuxdeploy ./linuxdeploy-x86_64.AppImage --appdir AppDir --output appimage --desktop-file assets/ahk_x11.desktop --icon-file assets/logo.png --executable ./bin/ahk_x11 --library libxdo.so.3 --plugin gtk and this results in 28 MiB in size, and it seems to generally work. Without the Gtk plugin, some parts such as libicudata are omitted, resulting in 10 MiB, but then most things don't work.

28 MiB seems like an okay compromise for now, even though it's only this small because the internal components are compressed.

I'm still not sure how installer (without appimaged daemon) and script compilation will fit in there, but surely somehow they will, so I'm expecting to file a single ahk_x11.AppImage for the next release, for which I have no ideas or plan yet (no pressing issues right now)

[phil294]

Migration to AppImage now done. Everything went reasonably well, even though there can apparently be problems with some custom GTK themes? linuxdeploy/linuxdeploy-plugin-gtk#39

My plan would now be to stay with Gtk 3.x forever, it's mostly preferrable to Gtk 4 imo

Reducing the binary size is still on my far-off to do list, but since the main problem here is mostly resolved, I'll close this