[arrow] 14.0.2 unable to detect the active compiler's information

[davesworking]

Operating system

Linux

Compiler

glibc 2.28

Steps to reproduce the behavior

14.0.1 SHA 2d8202f06d45f4220fa8aa0fc222d1d8221fd144, builds correctly on Linux-ARM64 armv8a-glibc2.28 (
Linux, version 6.1.8-v8.1.el8.altarch.1)

14.0.2 SHA d6341bc4de5c5f71ffcd129b952a42e823b21bb4 has error:

error: vcpkg was unable to detect the active compiler's information. See above for the CMake failure output.

Failure logs

-- Configuring arm64-linux-rel
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:112 (message):
Command failed: /bin/cmake /Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/scripts/detect_compiler -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/packages/detect_compiler_arm64-linux -DCMAKE_SYSTEM_NAME=Linux -DBUILD_SHARED_LIBS=OFF -DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/scripts/toolchains/linux.cmake -DVCPKG_TARGET_TRIPLET=arm64-linux -DVCPKG_SET_CHARSET_FLAG=ON -DVCPKG_PLATFORM_TOOLSET=external -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON -DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE -DCMAKE_VERBOSE_MAKEFILE=ON -DVCPKG_APPLOCAL_DEPS=OFF -DCMAKE_TOOLCHAIN_FILE=/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/scripts/buildsystems/vcpkg.cmake -DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON "-DVCPKG_CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1" -DVCPKG_CXX_FLAGS_RELEASE= -DVCPKG_CXX_FLAGS_DEBUG= "-DVCPKG_C_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1" -DVCPKG_C_FLAGS_RELEASE= -DVCPKG_C_FLAGS_DEBUG= -DVCPKG_CRT_LINKAGE=dynamic -DVCPKG_LINKER_FLAGS= -DVCPKG_LINKER_FLAGS_RELEASE= -DVCPKG_LINKER_FLAGS_DEBUG= -DVCPKG_TARGET_ARCHITECTURE=arm64 -DCMAKE_INSTALL_LIBDIR:STRING=lib -DCMAKE_INSTALL_BINDIR:STRING=bin -D_VCPKG_ROOT_DIR=/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg -DZ_VCPKG_ROOT_DIR=/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg -D_VCPKG_INSTALLED_DIR=/Developer/teamcity/work/e7a9578bd49a8fa6/Components/Library/Version/Linux-ARM64 -DVCPKG_MANIFEST_INSTALL=OFF
Working Directory: /Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/buildtrees/detect_compiler/arm64-linux-rel
Error code: 1
See logs for more information:
/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-CMakeCache.txt.log
/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-out.log
/Dev/Linux-ARM64/armv8a-glibc2.28/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-err.log

Call Stack (most recent call first):
scripts/cmake/vcpkg_configure_cmake.cmake:344 (vcpkg_execute_required_process)
scripts/detect_compiler/portfile.cmake:18 (vcpkg_configure_cmake)
scripts/ports.cmake:170 (include)

Additional context

config-arm64-linux-rel-err.log:

CMake Error at CMakeLists.txt:11 (enable_language):
  The CMAKE_C_COMPILER:

    aarch64-linux-gnu-gcc

  is not a full path and was not found in the PATH.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.

[azolotko]

I'm observing a similar issue on a different project:

[cmake] Detecting compiler hash for triplet arm64-linux...
[cmake] error: while detecting compiler information:
[cmake] The log file content at "/tmp/vcpkg/buildtrees/detect_compiler/stdout-arm64-linux.log" is:
[cmake] -- Configuring arm64-linux-rel
[cmake] CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:112 (message):
[cmake]     Command failed: /usr/bin/cmake /tmp/vcpkg/scripts/detect_compiler -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/tmp/vcpkg/packages/detect_compiler_arm64-linux -DCMAKE_SYSTEM_NAME=Linux -DBUILD_SHARED_LIBS=OFF -DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/toolchains/linux.cmake -DVCPKG_TARGET_TRIPLET=arm64-linux -DVCPKG_SET_CHARSET_FLAG=ON -DVCPKG_PLATFORM_TOOLSET=external -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON -DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE -DCMAKE_VERBOSE_MAKEFILE=ON -DVCPKG_APPLOCAL_DEPS=OFF -DCMAKE_TOOLCHAIN_FILE=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON -DVCPKG_CXX_FLAGS= -DVCPKG_CXX_FLAGS_RELEASE= -DVCPKG_CXX_FLAGS_DEBUG= -DVCPKG_C_FLAGS= -DVCPKG_C_FLAGS_RELEASE= -DVCPKG_C_FLAGS_DEBUG= -DVCPKG_CRT_LINKAGE=dynamic -DVCPKG_LINKER_FLAGS= -DVCPKG_LINKER_FLAGS_RELEASE= -DVCPKG_LINKER_FLAGS_DEBUG= -DVCPKG_TARGET_ARCHITECTURE=arm64 -DCMAKE_INSTALL_LIBDIR:STRING=lib -DCMAKE_INSTALL_BINDIR:STRING=bin -D_VCPKG_ROOT_DIR=/tmp/vcpkg -DZ_VCPKG_ROOT_DIR=/tmp/vcpkg -D_VCPKG_INSTALLED_DIR=/workspaces/abc/build/vcpkg_installed -DVCPKG_MANIFEST_INSTALL=OFF
[cmake]     Working Directory: /tmp/vcpkg/buildtrees/detect_compiler/arm64-linux-rel
[cmake]     Error code: 1
[cmake]     See logs for more information:
[cmake]       /tmp/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-CMakeCache.txt.log
[cmake]       /tmp/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-out.log
[cmake]       /tmp/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-err.log
[cmake] 
[cmake] Call Stack (most recent call first):
[cmake]   scripts/cmake/vcpkg_configure_cmake.cmake:344 (vcpkg_execute_required_process)
[cmake]   scripts/detect_compiler/portfile.cmake:18 (vcpkg_configure_cmake)
[cmake]   scripts/ports.cmake:170 (include)
[cmake] 
[cmake] 
[cmake] error: vcpkg was unable to detect the active compiler's information. See above for the CMake failure output.
[cmake] -- Running vcpkg install - failed
[cmake] CMake Error at /tmp/vcpkg/scripts/buildsystems/vcpkg.cmake:899 (message):
[cmake]   vcpkg install failed.  See logs for more information:
[cmake]   /workspaces/abc/build/vcpkg-manifest-install.log
[cmake] Call Stack (most recent call first):
[cmake]   /usr/share/cmake/Modules/CMakeDetermineSystem.cmake:148 (include)
[cmake]   CMakeLists.txt:3 (project)
[cmake] 
[cmake] 
[cmake] CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
[cmake] -- Configuring incomplete, errors occurred!
[proc] The command: /usr/bin/cmake --no-warn-unused-cli -DCMAKE_BUILD_TYPE:STRING=Debug -DCMAKE_TOOLCHAIN_FILE:STRING=/tmp/vcpkg/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET:STRING=x64-linux-release -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE -DCMAKE_C_COMPILER:FILEPATH=/usr/bin/aarch64-alpine-linux-musl-gcc -DCMAKE_CXX_COMPILER:FILEPATH=/usr/bin/aarch64-alpine-linux-musl-g++ -S/workspaces/abc -B/workspaces/abc/build -G Ninja exited with code: 1

As you can see, the compiler detection fails on arm64/aarch64, even when CMAKE_C_COMPILER and CMAKE_CXX_COMPILER are explicitly specified.

$ cat /tmp/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-out.log
-- Cross compiling arm64 on host aarch64, use cross compiler: aarch64-linux-gnu-g++/aarch64-linux-gnu-gcc
-- The C compiler identification is unknown
-- Configuring incomplete, errors occurred!

$ cat /tmp/vcpkg/buildtrees/detect_compiler/config-arm64-linux-rel-err.log 
CMake Error at CMakeLists.txt:11 (enable_language):
  The CMAKE_C_COMPILER:

    aarch64-linux-gnu-gcc

  is not a full path and was not found in the PATH.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.

I suspect that the regression was introduced in #35084.

[davesworking]

If you install Ninja, the error will probably go away

[azolotko]

If you install Ninja, the error will probably go away

Ninja is installed. The same build succeeds when vcpkg is switched from HEAD to the 2023.07.21 branch.

Ok, I take this back. The problem with ninja is system-specific.

Compiler detection is still an issue, however.

[dg0yt]

So you have an aarch64 host, but no aarch64-linux-gnu-gcc executable in the PATH?

[azolotko]

So you have an aarch64 host, but no aarch64-linux-gnu-gcc executable in the PATH?

Yes, that’s an alpine container image.

[dg0yt]

As you can see, the compiler detection fails on arm64/aarch64, even when CMAKE_C_COMPILER and CMAKE_CXX_COMPILER are explicitly specified.

This was the ninja error. Did you retry?
scripts/toolchains/linux.cmake doesn't override CMAKE_xxx_COMPILER if it is already defined.

[azolotko]

Ok, now it makes sense. detect_compiler failed because I didn't have aarch64-linux-gnu-gcc, but it shouldn't matter because I specify CMAKE_xxx_COMPILER explicitly anyway. And then, I got that error about ninja which I mistakenly correlated with the detect_compiler failure. Sorry for hijacking the issue, it looked relevant at the first sight.

[jorisvandenbossche]

I suspect that the regression was introduced in #35084.

I also ran into this error for compiling GDAL in a docker container (for python wheel builds), and applying a patch that essentially reverts #35084 seems to have fixed the issue.

[JulianGro]

The real question seems to be: "Why does this fail?"

if(NOT DEFINED CMAKE_C_COMPILER)
    set(CMAKE_C_COMPILER "aarch64-linux-gnu-gcc")
endif()

It is supposed to only change the compiler if it is not already set. Does this run before CMake checks for the compiler?

[dg0yt]

The real question seems to be: "Why does this fail?"

Which failure do you refer to? The variable is set successfully. However, configuration must fail if ninja or that compiler are missing.

It is supposed to only change the compiler if it is not already set. Does this run before CMake checks for the compiler?

Yes. On first run. This is how CMake toolchain file processing works.

---

The real question is why not just use gcc? The default guess is not a good guess when the image needs aarch64-alpine-linux-musl-gcc.
I have an idea what to change for hosts which are reported as aarch64. But it would be nice to not have to touch it again for arm32, and I don't those systems' identifier.

[JulianGro]

I think just fixing aarch64 is good already.
It is way more used nowadays, and a fix for that would be a good example for someone wanting to fix 32 bit arm.

Just for reference, relatively weird systems like Alpine aren't the only ones with the problem. Fedora also doesn't have aarch64-linux-gnu-gcc in its PATH. Pretty sure that is a Debian (and therefore also Ubuntu) specific thing.

[github-actions]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

[jorisvandenbossche]

I think this has been fixed by #38113 in the meantime (in any case, with a recent vcpkg checkout, the issue I mentioned above about installing GDAL is resolved)