How to add fixes for MacOS universal builds

[NaLiJa]

While exploring possibilities to add glfm as a mobile backend to rmlui, I noticed that the universal builds for macOS are difficult at the moment.
freetye is usually not universal, so you need to build it yourself and include it. The rlottie plugin needs a fix for arm64 build (and thus universal)
Since these are dependencies, what would be the preferred way to fix this in rmlui?
For example the freetype fix would require a small change to CMakeLists as well to prefer a build in „Dependencies“ over builds via Homebrew or system.

[mikke89]

I don't really have any experience with universal builds. This sounds a bit like a situation where different users might have different needs? Providing one specific approach might not be suitable for most users, so then I wonder if this is something we should provide support for at all?

In any case, I don't think this is something I can contribute meaningfully to. I'll take pull requests to patch our current solution, if it can be demonstrated that it improves it. But otherwise there have been discussions about completely removing any built-in support for universal builds in #446. Do you believe there are some simple things we can do there to be helpful to a number of users?

[NaLiJa]

RmlUi is almost not directly affected, the RmlUi parts can be built universal as they are. You only need to supply the required architectures in the corresponding cmake option. The area of work are the dependencies. For an universal build they need to be universal as well and this is often not the case. For example if you install freetype via homebrew, you get the lib for your platform, but they do not offer an universal version. So, you have to compile the dependencies yourself in universal form (freetype, rlottie, lunasvg,…).
to make this work, we would need to change the cmake file to prefer builds in the Dependencies folder over other locations in the system.
The question is how to build the dependencies. In the Dependencies folder there is an old shell script for freetype. We could enhance this script to cope with all dependencies. With this solution we would need to build the dependencies out-band and simply link. If you need the dependencies in a different form, you would need to recompile with the shell script.
Another option would be to trigger a recompile of the dependencies directly from the cmake file. Which would of course be more comfortable.

I personally think, the capability of having universal builds is very valuable. From the developer view it makes things a lot easier, because you can work with one single universal buildchain instead of messing around with libs for different architectures. There are pro and cons of universal build, but except for some special cases it mainly comes down to the size increase of universal builds. But I think in many (most?) cases, the additional size does not outweigh the advantages. And universal builds are optional. If your project is really so large that binary size becomes a major concern you can still build explicitly for architectures. If you have a project in a small team which is targeted at Win, Linux, MacOS, Android, iOS, you are probably happy about every opportunity to slim down the build chain.

And speaking about the mobile platforms, I think this is an area of great interest. If RmlUi can support mobile platforms as well, it would be a huge step in getting a viable alternative to Qt/QML. However with the mobile platforms, you are dealing with „universal builds“ again quickly. With arm and x64 for iPhone and simulator and x64 and arm abi for Android. So supporting universal building might pave way for mobile platforms as well. I read in the cmake PR that mobile building should work, but I haven’t tried this yet, and I assume it only works with SDL. That was the main reason why I wanted to investigate glfm which supports GLES3 and Metal.

I could try to write scripts or work on the cmake file to make universal builds possible (we could also include an updated Mac/iOS cmake toolchain), the question is if it might be better to build outband or rebuild the dependencies directly from the cmake build?!

Hope, it is possible to follow my thoughts (otherwise sorry, English is not my mother tongue)

[mikke89]

I appreciate the input. I don't question the value of universal builds, rather, I wonder how much we should modify our library to accompany such use cases, rather than simply let users make the necessary additions and integrate it how they want.

The main reasons I'm saying this is that:

1. The current Apple framework implementation in our CMakeLists is very invasive. This is something I'd very much like to avoid when modernizing our CMake code.
2. Such a feature is not something I'm able to test much, and therefore not able to maintain myself.

I think it would make sense to add this feature under the following conditions:

1. It is not invasive to our existing build code. Especially with our new CMake modernization, we would want the code additions to be almost completely separate from our main code.
2. This is considered a community contributed feature, and maintained by users.
3. It actually has value for several users. In particular, there should be one obvious way to do this, so that other users agree and don't decide to do it a completely different way.

It really comes down to a cost/benefit-ratio in terms of maintainability and usefulness, and I admittedly know too little about either part of that equation here. An actual prototype/pull request would be very helpful to inform the cost-part, especially on top of the CMake modernization pull request. And input from more users who are interested in this would be very helpful to better understand the benefit-part.

I would also like to add that we have plenty of people using the library on mobile, so clearly this is not a blocker for that.

[NaLiJa]

Yes, I agree, it is surely a good idea to keep the build scripts compact and "maintainable". I cannot yet fully say how many changes would be needed for universal support. At the moment I would guess that not much has to be changed or added, but perhaps I am overlooking stuff.

It is also a little bit a matter of philosophy. Should the build scripts only provide the necessary bare bones or should they try to provide as much guiding as possible? I do not know what be might better from the users' perspective.

Considering how I usually get to know new libraries, I am often quickly heading to the examples, just to get an idea how things work and what is possible. If this is something that many people do, it would be definitely good to have an selection of nice examples (which you already have), which can be rather effordlessly built for the the supported platforms - to try things out and have a working example of a build chain.

So what I could try (and we could verify after each step if it is worth pursuing):

• Fetch the commits for the cmake rewrite
• See what I need to do to make the mac demos self comtained like we applied recently to the current cmake file
• Try to adapt the cmake scripts to support universal builds
• Try to add a working example for mobile builds (although I need to investigate here further), perhaps with special demos to fit mobile platforms

To make customizing the build scripts easier, theoretically one could also split up the cmake files for different platforms. But it is hard to tell out of the blue, if the benefits would outweigh the additional efforts for maintenance.

Concerning mobile platforms: Do you perhaps have a lead to examples which implement rmlui in mobile solutions? It would surely help to get a glance at working implementations, especially concerning the "special" mobile features like touch support, device orientation, save drawing areas and so on...

[mikke89]

Yeah, this sounds good! I completely agree with this plan, and looking forward to seeing the result. It would certainly be nice with some concrete examples/profiles for mobile too.

I haven't seen any open source mobile examples that I can remember, only closed source, maybe you can find something through #186 with some of the engines there. Touch support is discussed in this old issue here: #60. From the ones I've tried, it seems most implementations simply use the mouse API, it could be useful to implement something more specific for touch. For device orientation, I believe that should only be a matter of setting the resolution, and then one can use media queries for more specific behavior differences.

[hobyst]

CMake already seems to support Apple universal binaries via CMAKE_OSX_ARCHITECTURES. Passing -DCMAKE_OSX_ARCHITECTURES=x86_64;arm64 when calling CMake at configure time should be enough to make universal binaries for macOS and the same principle can be applied to any other Apple platform. With the little to no intrusion the CMake script from #446 makes into the system via literal flags, universal binaries should work without any modification to the CMake project. The concept of a "universal binary" doesn't exist outside the Apple ecosystem, as the rest require binaries to have a specific target architecture.

About building other dependencies, it's outside of the scope of the project in my opinion. We are talking about getting access to pre-built binaries that fit a specific configuration, and therefore talking about the job of a package manager. Getting access to libraries and binaries that fit a specific case scenario has always been a challenge when it comes to any software that compiles to native code. If any change is needed, it should be a matter of contributing to the conflicting libraries and looking into making things work with package managers like Conan or vcpkg, not into changing the RmlUi CMake build project.

The rewrite happening on #446 is made with dependency management in mind and makes consumer-provided dependencies the priority above the ones found in the system, as CMake intends. Additional search paths can be specified by setting CMAKE_PREFIX_PATH and CMAKE_MODULE_PATH when calling CMake at configure time to specify custom search paths. The project automatically adds the Dependencies folder as well, but does so after the ones specified by the consumer. However, regardless of the dependency, CMake will still need some kind of way to know the details of the dependency, via either CMake package config files for dependencies that also support CMake in their build system and find modules for those which don't. It might be good to create simple find module templates that consumers can then use to specify where their dependencies are.

If what you want is automation, it is way more trivial to do simple automated build scripts written using something like Python than it would be to reinvent the wheel and force CMake to do something it isn't intended for. I agree they could be useful, but they may end up being a burden to maintain.

[NaLiJa]

RmlUi can immediately built with both architectures, but it doesn’t help, because you need to build all dependencies universal as well. Most users will have them preinstalled on the system or fetched via homebrew or macports, but those are usually not universal. And this affects all dependencies from SDL, glfw to freetype to the plugins.
So this especially not about pre-built binaries but compiled from source binaries which get rebuilt when it is necessary. At least on my System the cmake does not prefer the Dependencies folder, but of course this might be a matter of configuration. Have to look at this.

I think, it all comes down to the question how easy it should be to start and get along. Having only the bare bones it obviously easier to maintain, but it might makes things a lot harder for beginners, it might cause problems when upgrading to a newer RmlUi versions because the users might use heavily customised cmake scripts and it might cause users reinvent things all the time that are more or less „standard tasks“. Please don’t get me wrong, this is really only meant as an exchange of thoughts. The matter of maintainability is a very valid concern. Not sure what might be the best way. I think somehow it would be good to provide solutions/demos for common tasks in building and deploying RmlUi apps. Providing shell scripts could be an alternative as well.

[hobyst]

Following Apple's documentation, if you already have access to compiled arch-specific binary files, it should be possible to generate a universal binary without recompiling them using the lipo tool.

[NaLiJa]

If you have both binaries, you could use it, but normally you still need to compile it yourself. For example using homebrew, you cannot install both the x64 and the arm64 binaries afaik. And if you need to build anyway, you can build universal either way.

[hobyst]

At least on my System the cmake does not prefer the Dependencies folder, but of course this might be a matter of configuration.

If the dependencies CMake is trying to find are being located via find modules, and those modules don't provide a way for users to point out their custom builds, that might be the issue. In such case, the solution would be to create a simple find module that sets up your custom build of the library as an IMPORTED target. It's not trivial for someone that doesn't know about CMake, but that's how things are and the reason it might be good to create simple find module templates for consumers to use in case they want to use custom builds of the dependencies.

I think somehow it would be good to provide solutions/demos for common tasks in building and deploying RmlUi apps.

Making tutorials like that as part of the documentation would be good, but they would never go as deep into things as many users might expect them to. Depending on which dependencies are being built, getting to know how to build them exactly the way consumers expect will actually take them some time to learn the build system the dependencies use. This is something that package managers like Conan and vcpkg try to solve, but unfortunately many of the dependencies RmlUi depends on aren't available using them.

So this especially not about pre-built binaries but compiled from source binaries which get rebuilt when it is necessary.
[...]
I think, it all comes down to the question how easy it should be to start and get along. Having only the bare bones it obviously easier to maintain, but it might makes things a lot harder for beginners, it might cause problems when upgrading to a newer RmlUi versions because the users might use heavily customised cmake scripts and it might cause users reinvent things all the time that are more or less „standard tasks“.

Following what you said about the compiled from source binaries, I'm assuming that with "users reinventing the wheel" you mean consumers trying to integrate the build of third-party dependencies into their build chain. Doing such a thing isn't easy as you need to know how to interface with the build chain of every single dependency, and it's exactly the problem that package managers try to solve. Because of this, I think the best way to fix this would be to provide recipes for package managers.

And if you need to build anyway, you can build universal either way.

Some build systems don't support Apple universal binaries, so you would still have to build a binary for every single architecture you want to support and use the lipo tool to merge them into a universal binary.

[NaLiJa]

Hmm, don’t know to proceed. If the top priority is on keeping the cmake files as unchanged as possible, I guess something like universal builds cannot be integrated. Perhaps I will follow this the other way around. I will try to configure build chains and see if anything can be generalised for a wider use. I have experimented at bit and it might be possible to bake universal libs with the lipo tool from homebrew binaries, but at least when targeting iOS there is probably no way around compiling from source.

[NaLiJa]

I have added a small commit to the cmake branch to port over the last small mac os changes, so the demo can be run. hey are not deployable, but this is not in scope for the demos

[mikke89]

Hm, it's hard for me to say anything useful before I know how invasive it will be. I do generally agree that this sounds more like a job for a package manager / (meta) build system, rather than a package configuration. But if we could make some simple changes, maybe some CMake presets that could help, perhaps some side-scripts. Or something like that... Then it sounds acceptable.

Have you looked into how, or if, vcpkg and Conan deal with universal builds?

[NaLiJa]

I will investigate vcpkg and conan next. I have built a way around homebrew with prebuilding universal versions of the dependencies with the lipo tool, but I am not fully happy with it. First there is no iOS support in homebrew so this would require a completely different solution and all homebrew libs have the standard prefix applied which make them unsuitable for deployment without further fixes.
will see how far I can get with the above package managers. At least for vcpkg there seems to be a recent commit which fixes the lib path.
I guess, either way, some of this stuff should go into helper scripts, because it would blow up the cmake files.
Will report…

[NaLiJa]

So, here are some results from my testing:
the universal builds can be done with the help of vcpkg. What is nice about vcpkg is that - in contrast to homebrew - it supports ios (and other targets as well). The building however is not straight forward. An immediate universal build of the dependencies does not work. While most libraries build fine for x64 and arm64 in one go, for example libpng fails to build because it contains assembly code. There is no rlottie port in vcpkg, although it was quite straight forward to create one.
To create universal versions of the dependencies I wrote a small script that merges the x64 and arm64 into universal libraries with the lipo tool.
In the next step I had to apply a couple of changes to the cmake file to cope with the vcpkg dependencies. The package configurations are different, so you need different ways to include the packages in the cmake file. Plus some of the packages have a strange naming scheme that I needed to adapt - although perhaps there is an easier way that I didn't see because of my lack in experience with vcpkg.
Actually vcpkg can copy the required libs to the bundle, but this didn't work in all cases, neither. For example, when copying libpng to the bundle, vcpkg misses the brotli libs. So, in the end, I wrote another small script which copies all libs to the bundle and fixes the paths with install_name_tool.

So, what needs to be done in brief:

• Add a rlottie port for vcpkg
• Install the arm64 and x64 libs separately
• Merge the libs with a small helper script into universal versions
• Make some small modifications to the rmlui cmake file to some find_package commands. After that you can build for archs "x64;arm64" together
• To make the apps deployable run another small helper script to copy the dependencies to the bundle and fix the rpaths.

Since the helper scripts are needed anyway, I guess applying the cmake changes directly to the main cmake file is not a good idea since the changes only apply to the macos universal with vcpkg build and I am not sure if those changes would cause conflicts with other platforms. Perhaps the above stuff could be put into a small guide, together with the build scripts ?!?

[mikke89]

Very nice overview of the steps needed to make it work, thanks for sharing this with everyone.

Yeah, unfortunately, with vcpkg one sometimes need to modify how packages are found. We don't want to modify our CMakeLists for this specifically, this is just one of those things the user has to deal with. At least I have heard about efforts within CMake to let package managers themselves better control how packages are found, so that this is no longer needed. That work can't come soon enough.

I think your guide here alone gives a very good understanding of how to go ahead with this for other users. However, I'd be very excited to see a PR for a small guide like that with some more details, plus build scripts!