Prebuilt vcpkg-tool+cmake binaries for ARM64 Linux

[russelltg]

It would be pretty cool if arm64 linux was a fully supported platform for downloading prebuilt binaries. I'm willing to help make this happen, and have a machine for testing.

CMake already has prebuilt arm64 linux binaries, so that part should be easy at least. I assume steps are something like:

1. Get arm64 Linux CI/releases working for vcpkg-tool
2. Fix bootstrap and vcpkgTools.xml

Anything else? Is this interesting?

[Osyotr]

#27905 (comment)

[russelltg]

@BillyONeal could you clarify if in that comment you meant you need hardware AND cross compilation workflow or hardware OR cross compilation instructions?

I can absolutely set up cross compilation for you, and probably even running tests in a docker container. Would that be acceptable?

Can't help you with hardware :)

[BillyONeal]

@BillyONeal could you clarify if in that comment you meant you need hardware AND cross compilation workflow or hardware OR cross compilation instructions?

I don't know. Is cross compilation how folks who do this normally do it? If so I don't think we need hardware, but we do need someone with the hardware to check and make sure it actually works.

[russelltg]

There are pretty much 3 primary ways of doing this to my knowledge:

1. Have real hardware
2. Cross-compile using a cross-toolchain and a sysroot
3. Emulate real hardware (via docker or full VM, docker being much simpler)

Nice thing about 3) is you can run a testsuite as well as you already have a full emulated environment, but compilation is slower as you're also emulating the compiler itself. A blend can work well too (use a cross-toolchain then run tests in an emulated environment).

1 is preferred when possible, but often isn't (especially for 32-bit targets which can't build some larger projects these days anyways....)

Since vcpkg-tool has few dependencies, I don't think creating a sysroot would be very difficult--may even be able to get away with just installing the cross-stdlib that ubuntu ships if you don't need any other host packages.

[BillyONeal]

@russelltg The other bit is the RHEL 'devtoolset' games we play in order to make sure our bits work on older boxes: https://github.com/microsoft/vcpkg-tool/blob/7b53f45badf690bc18a33446a202cde841234d60/azure-pipelines/signing.yml#L227

[russelltg]

Yeah, that makes sense and is good practice when shipping linux binaries. Of course the other option is to statically link libc but that isn't without caveats too. (but nice thing about that is it will run on Alpine, as only dep is the kernel).

Fwiw, it seems like devtoolset is to bring in a newer compiler to and old RHEL verison (which has an old glibc, which is needed for it to be compatible with old systems).

One thing I learned about recently is zig c++, which lets you just specify a glibc version right on the commandline, as well as transparently handling cross-compiling. It's just wrapping clang under the hood. Unfortuantely, it doesn't seem to support precompiled headers, but I was able to trivally cross compile it against an old glibc (2.17, same as RHEL 7):

$ /bin/cat zig-toolchain-aarch64.cmake
set(CMAKE_SYSTEM_NAME "Linux")
set(CMAKE_SYSTEM_PROCESSOR "aarch64")
set(CMAKE_C_COMPILER "zig" cc -target aarch64-linux-gnu.2.17)
set(CMAKE_CXX_COMPILER "zig" c++ -target aarch64-linux-gnu.2.17)
$ cmake -Bbuild -GNinja -DCMAKE_TOOLCHAIN_FILE=zig-toolchain-aarch64.cmake -DCMAKE_DISABLE_PRECOMPILE_HEADERS=ON
$ ninja -C build
$ file build/vcpkg
build/vcpkg: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 2.0.0, with debug_info, not stripped
$ docker run --rm -v `pwd`:/vcpkg-tool arm64v8/ubuntu:jammy ldd -r -v /vcpkg-tool/build/vcpkg
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v4) and no specific platform was requested
	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000400000810000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000400000830000)
	libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x00004000009f0000)
	/lib/ld-linux-aarch64.so.1 (0x00007fbd9d1c1000)

	Version information:
	/vcpkg-tool/build/vcpkg:
		libpthread.so.0 (GLIBC_2.17) => /lib/aarch64-linux-gnu/libpthread.so.0
		libc.so.6 (GLIBC_2.17) => /lib/aarch64-linux-gnu/libc.so.6
		libdl.so.2 (GLIBC_2.17) => /lib/aarch64-linux-gnu/libdl.so.2
		ld-linux-aarch64.so.1 (GLIBC_2.17) => /lib/ld-linux-aarch64.so.1
	/lib/aarch64-linux-gnu/libpthread.so.0:
		libc.so.6 (GLIBC_2.17) => /lib/aarch64-linux-gnu/libc.so.6
	/lib/aarch64-linux-gnu/libc.so.6:
		ld-linux-aarch64.so.1 (GLIBC_PRIVATE) => /lib/ld-linux-aarch64.so.1
		ld-linux-aarch64.so.1 (GLIBC_2.17) => /lib/ld-linux-aarch64.so.1
	/lib/aarch64-linux-gnu/libdl.so.2:
		libc.so.6 (GLIBC_2.17) => /lib/aarch64-linux-gnu/libc.so.6 
$ VCPKG_FORCE_SYSTEM_BINARIES=1 QEMU_LD_PREFIX=/usr/aarch64-linux-gnu ./build/vcpkg --version
vcpkg package management program version 2999-12-31-unknownhash-debug

This statically links in libc++, so similar to what your existing glibc builds do (except I assume those statically link in libstdc++). Since vcpkg works on macOS I assume libc++ is an acceptable standard library for vcpkg.

Since I had to disable precompiled headers, I had to patch vcpkg to add a bunch of missing includes, which would be a slight maintenance burden (although I consider having proper includes good practice, but ymmv). I'm happy to open a PR with those missing includes

[russelltg]

Also, I'm realizing that not requiring VCPKG_FORCE_SYSTEM_BINARIES would be the most important part for me and also the easiest, considering CMake ships arm64 binaries....

[github-actions]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

[russelltg]

I want this to stay open

[joiskash]

Are there any updates on this issue?

[thegecko]

Is there a way for the community to contribute this support or is the build system outside of the repository?

Using GitHub actions, Arm linux builds can be compiled using cross-compilation docker containers such as used in https://github.com/prebuild/prebuildify-cross or https://github.com/dockcross/dockcross

[thegecko]

Is there a way for the community to contribute this support or is the build system outside of the repository?

Ah, I see there's another repo. I've taken a stab at this here: microsoft/vcpkg-tool#1478