Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GAP_pkg_browse to 1.8.13 #3931

Merged
merged 2 commits into from
Nov 24, 2021

Conversation

fingolfin
Copy link
Member

No description provided.

@fingolfin
Copy link
Member Author

Hmmm... getting this for x86_64-unknown-freebsd -- any idea, @giordano ? This used to be build a couple weeks ago (about 2021-10-23)

[22:20:20] In file included from src/ncurses.c:26:
[22:20:20] /workspace/destdir/include/ncurses/ncurses.h:61:10: fatal error: 'ncursesw/ncurses_dll.h' file not found
[22:20:20] #include <ncursesw/ncurses_dll.h>
[22:20:20]          ^~~~~~~~~~~~~~~~~~~~~~~~
[22:20:20] 1 error generated.

@giordano
Copy link
Member

giordano commented Nov 24, 2021

We haven't touched ncurses in a while and the header is there:

% tar tzvf Ncurses.v6.2.0.x86_64-unknown-freebsd.tar.gz|grep 'include/'             
lrwxr-xr-x 0/0               0 1970-01-01 01:00 include/ncurses -> ncursesw
-rw-r--r-- 0/0          100226 1970-01-01 01:00 include/ncursesw/curses.h
-rw-r--r-- 0/0            2969 1970-01-01 01:00 include/ncursesw/eti.h
-rw-r--r-- 0/0           18829 1970-01-01 01:00 include/ncursesw/form.h
-rw-r--r-- 0/0           12293 1970-01-01 01:00 include/ncursesw/menu.h
-rw-r--r-- 0/0            4249 1970-01-01 01:00 include/ncursesw/nc_tparm.h
lrwxr-xr-x 0/0               0 1970-01-01 01:00 include/ncursesw/ncurses.h -> curses.h
-rw-r--r-- 0/0            4522 1970-01-01 01:00 include/ncursesw/ncurses_dll.h
-rw-r--r-- 0/0            4210 1970-01-01 01:00 include/ncursesw/panel.h
-rw-r--r-- 0/0           41387 1970-01-01 01:00 include/ncursesw/term.h
-rw-r--r-- 0/0            9114 1970-01-01 01:00 include/ncursesw/term_entry.h
-rw-r--r-- 0/0            3480 1970-01-01 01:00 include/ncursesw/termcap.h
-rw-r--r-- 0/0           14839 1970-01-01 01:00 include/ncursesw/tic.h
-rw-r--r-- 0/0            3186 1970-01-01 01:00 include/ncursesw/unctrl.h

I guess adding

CPPFLAGS="-I${includedir}"

would help? I'd note you have

 make -j${nproc} CFLAGS="-I$prefix/include/ncurses"

which looks wrong if they include <ncursesw/*>?

@fingolfin
Copy link
Member Author

Ahh, I missed that it is ncursesw not ncurses. Thanks

@fingolfin
Copy link
Member Author

Ah, no, wait: the code does not use ncursesw; it does

#include        <ncurses.h>

and then the include for ncursesw happens in ncurses.h -- which is odd. Anyway, I think it's clear how to workaround it.

@giordano
Copy link
Member

Note that

lrwxr-xr-x 0/0               0 1970-01-01 01:00 include/ncurses -> ncursesw

ncurses is a symlink to ncursesw

@fingolfin
Copy link
Member Author

I recently also had to add -I${includedir} for FreeBSD. While I don't mind, this is strange: no other OS requires this; and I would have thought that the cross compiler automatically has this on the search list. And, indeed, it used to a few weeks ago. This smells like a regression in the FreeBSD builder?

@giordano
Copy link
Member

This smells like a regression in the FreeBSD builder?

We don't add automatically add directories under ${prefix} to search paths (headers nor binary libraries), it's up to build systems to figure that out, for example by passing --prefix=${prefix} to Autoconf's configure.

We do help CMake and Meson because they use toolchain files (and Meson in particular is hard to otherwise configure, when cross-compiling it refuses to read most environment variables and command line options, unlike CMake).

@giordano giordano merged commit f322e48 into JuliaPackaging:master Nov 24, 2021
@fingolfin fingolfin deleted the mh/GAP_pkg_browse branch November 24, 2021 22:16
@fingolfin
Copy link
Member Author

But autoconf/automake donot automatically add e.g. -I$includedir to CPPFLAGS, nor do most build systems implemented with them (well, of those I am familiar with / have seen). OTOH, on all systems other than FreeBSD, this is not currently necessary, nor was it necessary 4 weeks ago with FreeBSD. So something changed...?

@giordano
Copy link
Member

I feel like we've had this discussion before 😜

@fingolfin
Copy link
Member Author

Indeed, but we never finished it. And I don't buy this "on some platforms you have to add -I$includedir if it is not picked up automatically": my point is that on those other platforms, it isn't "picked up either" -- nothing in the build system adds it, and indeed, inspecting the build logs, it is not passed to the compiler.

E.g. on https://dev.azure.com/JuliaPackaging/Yggdrasil/_build/results?buildId=14664&view=logs&j=e134b759-2408-588b-d47e-e02e51d77b28&t=3b7ce12a-2937-578c-7203-17eee6ef538c you can see that mpfr.h, mpfi.h etc. are picked up without -I$includedir. On all platforms, except FreeBSD. And again, FreeBSD picked it up until a couple weeks ago.

@fingolfin
Copy link
Member Author

(I mean, most likely I am missing something, but I don't see what, and the change in behavior makes me wary -- why did the FreeBSD builder change its behavior?!?)

@giordano
Copy link
Member

These are the default search paths for GCC and LLVM on Linux vs FreeBSD:

julia> using BinaryBuilderBase

julia> BinaryBuilderBase.runshell(Platform("x86_64", "linux"));
sandbox:${WORKSPACE} # export SUPER_VERBOSE=1
sandbox:${WORKSPACE} # gcc -v -E - < /dev/null
ccache /opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=0 -frandom-seed=0x412a7d32 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -Wl,-rpath-link,/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -march=x86-64 -mtune=generic -v -E -
Using built-in specs.
COLLECT_GCC=/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc
Target: x86_64-linux-gnu
Configured with: /workspace/srcdir/gcc-4.8.5/configure --prefix=/workspace/destdir --target=x86_64-linux-gnu --host=x86_64-linux-musl --build=x86_64-linux-musl --disable-multilib --disable-werror --enable-shared --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-linux-gnu/sys-root --program-prefix=x86_64-linux-gnu- --disable-bootstrap --with-arch=core2 --enable-languages=c,c++,fortran,objc,obj-c++
Thread model: posix
gcc version 4.8.5 (GCC) 
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=0' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'
 /opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/4.8.5/cc1 -E -quiet -v -iprefix /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/ -isysroot /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root -D _GLIBCXX_USE_CXX11_ABI=0 - -march=x86-64 -mtune=generic -frandom-seed=0x412a7d32
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/4.8.5/include"
ignoring nonexistent directory "/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/local/include"
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/4.8.5/include-fixed"
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/4.8.5/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/include
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/include-fixed
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/../../../../x86_64-linux-gnu/include
 /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"
COMPILER_PATH=/opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/4.8.5/:/opt/x86_64-linux-gnu/bin/../libexec/gcc/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/../../../../x86_64-linux-gnu/bin/
LIBRARY_PATH=/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/:/opt/x86_64-linux-gnu/bin/../lib/gcc/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/../../../../x86_64-linux-gnu/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/../../../../x86_64-linux-gnu/lib/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=0' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'
sandbox:${WORKSPACE} # clang -v -E - < /dev/null
ccache /opt/x86_64-linux-musl/bin/clang -target x86_64-linux-gnu --sysroot=/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root --gcc-toolchain=/opt/x86_64-linux-gnu -rtlib=libgcc -stdlib=libstdc++ -march=x86-64 -mtune=generic -v -E -
clang version 12.0.0 (/home/mose/.julia/dev/BinaryBuilderBase/deps/downloads/llvm-project.git d28af7c654d8db0b68c175db5ce212d74fb5e9bc)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
Found candidate GCC installation: /opt/x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/4.8.5
Selected GCC installation: /opt/x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/4.8.5
Candidate multilib: .;@m64
Selected multilib: .;@m64
clang-12: warning: argument unused during compilation: '-rtlib=libgcc' [-Wunused-command-line-argument]
clang-12: warning: argument unused during compilation: '-stdlib=libstdc++' [-Wunused-command-line-argument]
 (in-process)
 "/opt/x86_64-linux-musl/bin/clang-12" -cc1 -triple x86_64-unknown-linux-gnu -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -tune-cpu generic -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /opt/x86_64-linux-musl/lib/clang/12.0.0 -isysroot /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root -internal-isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include -internal-isystem /opt/x86_64-linux-musl/lib/clang/12.0.0/include -internal-externc-isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/include -internal-externc-isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/include -fdebug-compilation-dir /workspace -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -o - -x c -
clang -cc1 version 12.0.0 based upon LLVM 12.0.0 default target x86_64-linux-musl
ignoring nonexistent directory "/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include"
ignoring nonexistent directory "/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-linux-musl/lib/clang/12.0.0/include
 /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 341 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2
sandbox:${WORKSPACE} # logout

julia> BinaryBuilderBase.runshell(Platform("x86_64", "freebsd"));
sandbox:${WORKSPACE} # export SUPER_VERBOSE=1
sandbox:${WORKSPACE} # gcc -v -E - < /dev/null
ccache /opt/x86_64-unknown-freebsd12.2/bin/x86_64-unknown-freebsd12.2-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -frandom-seed=0x412a7d32 -L/opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/lib64 -Wl,-rpath-link,/opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/lib64 -march=x86-64 -mtune=generic -v -E -
Using built-in specs.
COLLECT_GCC=/opt/x86_64-unknown-freebsd12.2/bin/x86_64-unknown-freebsd12.2-gcc
Target: x86_64-unknown-freebsd12.2
Configured with: /workspace/srcdir/gcc-6.1.0/configure --prefix=/workspace/destdir --target=x86_64-unknown-freebsd12.2 --host=x86_64-linux-musl --build=x86_64-linux-musl --disable-multilib --disable-werror --enable-shared --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-unknown-freebsd12.2/sys-root --program-prefix=x86_64-unknown-freebsd12.2- --disable-bootstrap --with-arch=x86-64 --enable-languages=c,c++,fortran
Thread model: posix
gcc version 6.1.0 (GCC) 
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=1' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'
 /opt/x86_64-unknown-freebsd12.2/bin/../libexec/gcc/x86_64-unknown-freebsd12.2/6.1.0/cc1 -E -quiet -v -iprefix /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/ -isysroot /opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root -D _GLIBCXX_USE_CXX11_ABI=1 - -march=x86-64 -mtune=generic -frandom-seed=0x412a7d32
ignoring duplicate directory "/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/../../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/include"
ignoring nonexistent directory "/opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/usr/local/include"
ignoring duplicate directory "/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/../../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/include-fixed"
ignoring duplicate directory "/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/../../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/../../../../x86_64-unknown-freebsd12.2/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/include
 /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/include-fixed
 /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/../../../../x86_64-unknown-freebsd12.2/include
 /opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"
COMPILER_PATH=/opt/x86_64-unknown-freebsd12.2/bin/../libexec/gcc/x86_64-unknown-freebsd12.2/6.1.0/:/opt/x86_64-unknown-freebsd12.2/bin/../libexec/gcc/:/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/../../../../x86_64-unknown-freebsd12.2/bin/
LIBRARY_PATH=/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/:/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/:/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/6.1.0/../../../../x86_64-unknown-freebsd12.2/lib/:/opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/lib/:/opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/usr/lib/
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=1' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'
sandbox:${WORKSPACE} # clang -v -E - < /dev/null
ccache /opt/x86_64-linux-musl/bin/clang -target x86_64-unknown-freebsd12.2 --sysroot=/opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root -march=x86-64 -mtune=generic -v -E -
clang version 12.0.0 (/home/mose/.julia/dev/BinaryBuilderBase/deps/downloads/llvm-project.git d28af7c654d8db0b68c175db5ce212d74fb5e9bc)
Target: x86_64-unknown-freebsd12.2
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
 (in-process)
 "/opt/x86_64-linux-musl/bin/clang-12" -cc1 -triple x86_64-unknown-freebsd12.2 -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mframe-pointer=all -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -tune-cpu generic -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /opt/x86_64-linux-musl/lib/clang/12.0.0 -isysroot /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root -fdebug-compilation-dir /workspace -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -o - -x c -
clang -cc1 version 12.0.0 based upon LLVM 12.0.0 default target x86_64-linux-musl
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-linux-musl/lib/clang/12.0.0/include
 /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 338 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

They look consistent to me across different platforms? The difference is that cc is GCC on Linux and LLVM on FreeBSD (as is on macOS), but in any case the default search paths only include the sys-root and compiler's include directories.

@fingolfin
Copy link
Member Author

I suspect the answer is related to this line:

ignoring nonexistent directory "/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include"

Because I am seeing this for a FreeBSD sandbox:

sandbox:${WORKSPACE}/srcdir # cpp -v /dev/null -o /dev/nulll
Using built-in specs.
COLLECT_GCC=/opt/x86_64-unknown-freebsd12.2/bin/x86_64-unknown-freebsd12.2-cpp
Target: x86_64-unknown-freebsd12.2
Configured with: /workspace/srcdir/gcc-7.1.0/configure --prefix=/workspace/destdir --target=x86_64-unknown-freebsd12.2 --host=x86_64-linux-musl --build=x86_64-linux-musl --disable-multilib --disable-werror --enable-shared --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-unknown-freebsd12.2/sys-root --program-prefix=x86_64-unknown-freebsd12.2- --disable-bootstrap --with-arch=x86-64 --enable-languages=c,c++,fortran
Thread model: posix
gcc version 7.1.0 (GCC) 
COLLECT_GCC_OPTIONS='-E' '-v' '-o' '/dev/nulll' '-mtune=generic' '-march=x86-64'
 /opt/x86_64-unknown-freebsd12.2/bin/../libexec/gcc/x86_64-unknown-freebsd12.2/7.1.0/cc1 -E -quiet -v -iprefix /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/ -isysroot /opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root /dev/null -o /dev/nulll -mtune=generic -march=x86-64
ignoring duplicate directory "/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/../../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/include"
ignoring duplicate directory "/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/../../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/include-fixed"
ignoring duplicate directory "/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/../../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/../../../../x86_64-unknown-freebsd12.2/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/include
 /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/include-fixed
 /opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/../../../../x86_64-unknown-freebsd12.2/include
 /opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/usr/local/include
 /opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/usr/include
End of search list.
COMPILER_PATH=/opt/x86_64-unknown-freebsd12.2/bin/../libexec/gcc/x86_64-unknown-freebsd12.2/7.1.0/:/opt/x86_64-unknown-freebsd12.2/bin/../libexec/gcc/:/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/../../../../x86_64-unknown-freebsd12.2/bin/
LIBRARY_PATH=/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/:/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/:/opt/x86_64-unknown-freebsd12.2/bin/../lib/gcc/x86_64-unknown-freebsd12.2/7.1.0/../../../../x86_64-unknown-freebsd12.2/lib/:/opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/lib/:/opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/usr/lib/
COLLECT_GCC_OPTIONS='-E' '-v' '-o' '/dev/nulll' '-mtune=generic' '-march=x86-64'

And looking closer:

sandbox:${WORKSPACE}/srcdir # ls -al /opt/x86_64-unknown-freebsd12.2/bin/../x86_64-unknown-freebsd12.2/sys-root/usr/
total 36
drwxrwxr-x    1 root     root          4096 Nov 12 22:30 .
drwxrwxr-x    1 root     root          4096 Nov 12 22:30 ..
lrwxrwxrwx    1 root     root            10 Nov 12 22:30 include -> ../include
drwxrwxr-x    9 root     root         28672 Nov 12 22:31 lib
lrwxrwxrwx    1 root     root            18 Nov 12 22:30 local -> /workspace/destdir

So, that local dir is a symlink to /workspace/destdir and so that include dir is in the default include search list for the compiler after all!

Of course that does not yet explain why it doesn't actually work on the FreeBSD builder.

@fingolfin
Copy link
Member Author

fingolfin commented Nov 24, 2021

Ah OK: My mistake was that I checked the search paths of cpp resp. gcc. But I should have checked cc, and that is indeed missing that ../sys-root/usr/local/include path:

# cc -v -E - < /dev/null
clang version 12.0.0 (/home/mose/.julia/dev/BinaryBuilderBase/deps/downloads/llvm-project.git d28af7c654d8db0b68c175db5ce212d74fb5e9bc)
Target: x86_64-unknown-freebsd12.2
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
 (in-process)
 "/opt/x86_64-linux-musl/bin/clang-12" -cc1 -triple x86_64-unknown-freebsd12.2 -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mframe-pointer=all -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -tune-cpu generic -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /opt/x86_64-linux-musl/lib/clang/12.0.0 -isysroot /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root -fdebug-compilation-dir /workspace/srcdir/float-0.9.9 -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -o - -x c -
clang -cc1 version 12.0.0 based upon LLVM 12.0.0 default target x86_64-linux-musl
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-linux-musl/lib/clang/12.0.0/include
 /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 338 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

@fingolfin
Copy link
Member Author

Meanwhile on aarch64-apple-darwin:

# cc -v -E - < /dev/null
clang version 12.0.0 (/home/mose/.julia/dev/BinaryBuilderBase/deps/downloads/llvm-project.git d28af7c654d8db0b68c175db5ce212d74fb5e9bc)
Target: arm64-apple-darwin20
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
 (in-process)
 "/opt/x86_64-linux-musl/bin/clang-12" -cc1 -triple arm64-apple-macosx10.10.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -fno-rounding-math -munwind-tables -faligned-alloc-unavailable -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu apple-a12 -target-feature +neon -target-abi darwinpcs -fallow-half-arguments-and-returns -debugger-tuning=lldb -v -nostdinc++ -resource-dir /opt/x86_64-linux-musl/lib/clang/12.0.0 -isystem /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/usr/include/c++/v1 -isysroot /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root -internal-isystem /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/usr/local/include -internal-isystem /opt/x86_64-linux-musl/lib/clang/12.0.0/include -internal-externc-isystem /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/usr/include -Wno-unused-command-line-argument -fdebug-compilation-dir /workspace/srcdir -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -o - -x c -
clang -cc1 version 12.0.0 based upon LLVM 12.0.0 default target x86_64-linux-musl
ignoring nonexistent directory "/opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/usr/include/c++/v1
 /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/usr/local/include
 /opt/x86_64-linux-musl/lib/clang/12.0.0/include
 /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/usr/include
 /opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root/System/Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 374 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

So here ..sys-root/usr/local/include is being searched.

In conclusion, I really think something is off with the FreeBSD clang configuration.

@giordano
Copy link
Member

giordano commented Nov 24, 2021

Ok, if I create ${includedir}, for macOS I get:

julia> BinaryBuilderBase.runshell(Platform("x86_64", "macos"));
sandbox:${WORKSPACE} # cc -v -E - < /dev/null
clang version 12.0.0 (/home/mose/.julia/dev/BinaryBuilderBase/deps/downloads/llvm-project.git d28af7c654d8db0b68c175db5ce212d74fb5e9bc)
Target: x86_64-apple-darwin14
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
 (in-process)
 "/opt/x86_64-linux-musl/bin/clang-12" -cc1 -triple x86_64-apple-macosx10.10.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mframe-pointer=all -fno-rounding-math -munwind-tables -faligned-alloc-unavailable -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu x86-64 -tune-cpu generic -debugger-tuning=lldb -v -nostdinc++ -resource-dir /opt/x86_64-linux-musl/lib/clang/12.0.0 -isystem /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/usr/include/c++/v1 -isysroot /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root -internal-isystem /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/usr/local/include -internal-isystem /opt/x86_64-linux-musl/lib/clang/12.0.0/include -internal-externc-isystem /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/usr/include -Wno-unused-command-line-argument -fdebug-compilation-dir /workspace -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -o - -x c -
clang -cc1 version 12.0.0 based upon LLVM 12.0.0 default target x86_64-linux-musl
ignoring nonexistent directory "/opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/usr/include/c++/v1
 /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/usr/local/include
 /opt/x86_64-linux-musl/lib/clang/12.0.0/include
 /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/usr/include
 /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/System/Library/Frameworks (framework directory)
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 347 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

which does include ${sys-root}/usr/local/include, which is is a symlink to ${inlcudedir}, while on FreeBSD I still get

julia> BinaryBuilderBase.runshell(Platform("x86_64", "freebsd"));
sandbox:${WORKSPACE} # cc -v -E - < /dev/null
clang version 12.0.0 (/home/mose/.julia/dev/BinaryBuilderBase/deps/downloads/llvm-project.git d28af7c654d8db0b68c175db5ce212d74fb5e9bc)
Target: x86_64-unknown-freebsd12.2
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
 (in-process)
 "/opt/x86_64-linux-musl/bin/clang-12" -cc1 -triple x86_64-unknown-freebsd12.2 -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mframe-pointer=all -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -tune-cpu generic -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /opt/x86_64-linux-musl/lib/clang/12.0.0 -isysroot /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root -fdebug-compilation-dir /workspace -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -o - -x c -
clang -cc1 version 12.0.0 based upon LLVM 12.0.0 default target x86_64-linux-musl
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-linux-musl/lib/clang/12.0.0/include
 /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 338 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

But I wouldn't know where to look at in the setup, in 0_RootFS/llvm_common.jl I don't see anything even loosely related to header search paths at all? Also, last time we touched the FreeBSD toolchain was in August, not last month.

@giordano
Copy link
Member

Also GCC for musl doesn't include ${includedir} in its search paths:

julia> BinaryBuilderBase.runshell(Platform("x86_64", "linux"; libc="musl"));
sandbox:${WORKSPACE} # cc -v -E - < /dev/null
Using built-in specs.
COLLECT_GCC=/opt/x86_64-linux-musl/bin/x86_64-linux-musl-gcc
Target: x86_64-linux-musl
Configured with: /workspace/srcdir/gcc-4.8.5/configure --prefix=/workspace/destdir --target=x86_64-linux-musl --host=x86_64-linux-musl --build=x86_64-linux-musl --disable-multilib --disable-werror --enable-shared --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-linux-musl/sys-root --program-prefix=x86_64-linux-musl- --disable-bootstrap --with-arch=x86-64 --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --disable-symvers --enable-languages=c,c++,fortran,objc,obj-c++
Thread model: posix
gcc version 4.8.5 (GCC) 
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=0' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-linux-musl/x86_64-linux-musl/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'
 /opt/x86_64-linux-musl/bin/../libexec/gcc/x86_64-linux-musl/4.8.5/cc1 -E -quiet -v -iprefix /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/ -isysroot /opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root -D _GLIBCXX_USE_CXX11_ABI=0 - -march=x86-64 -mtune=generic -frandom-seed=0x412a7d32
ignoring duplicate directory "/opt/x86_64-linux-musl/bin/../lib/gcc/../../lib/gcc/x86_64-linux-musl/4.8.5/../../../../x86_64-linux-musl/include"
ignoring duplicate directory "/opt/x86_64-linux-musl/bin/../lib/gcc/../../lib/gcc/x86_64-linux-musl/4.8.5/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/../../../../x86_64-linux-musl/include
 /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/include
 /opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/opt/x86_64-linux-musl/x86_64-linux-musl/sys-root/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "<stdin>"
COMPILER_PATH=/opt/x86_64-linux-musl/bin/../libexec/gcc/x86_64-linux-musl/4.8.5/:/opt/x86_64-linux-musl/bin/../libexec/gcc/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/../../../../x86_64-linux-musl/bin/
LIBRARY_PATH=/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/:/opt/x86_64-linux-musl/bin/../lib/gcc/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/../../../../x86_64-linux-musl/lib/../lib64/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/../../../../x86_64-linux-musl/lib/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/4.8.5/../../../:/opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root/lib/:/opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root/usr/lib/
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=0' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-linux-musl/x86_64-linux-musl/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'

@fingolfin
Copy link
Member Author

To be clear the bit about "changed in the last month" is based on that fact that this very recipe was built last around 2021-10-23 (based on commit/author date), and then it built fine on FreeBSD. In this update, there were no changes to the buildsystem. But I also went and tried to rebuild the previous version, and this also fails now.

Since then, BB and BBBase were updated, but I guess that can't be of relevance.

This is quite mysterious.

Also, interesting about musl. I'll investigate there, too

@fingolfin
Copy link
Member Author

BTW I think searching SYSROOT/usr/include and SYSROOT/usr/local/include is just automatic default of gcc and clang.

And on musl, I do get SYSROOT/usr/local/include, too, albeit with a different GCC version (the one selected by the GAP_pkg_*, and hence also by this builder here):

# cc -v -E - < /dev/null
Using built-in specs.
COLLECT_GCC=/opt/x86_64-linux-musl/bin/x86_64-linux-musl-gcc
Target: x86_64-linux-musl
Configured with: /workspace/srcdir/gcc-7.1.0/configure --prefix=/workspace/destdir --target=x86_64-linux-musl --host=x86_64-linux-musl --build=x86_64-linux-musl --disable-multilib --disable-werror --enable-shared --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-linux-musl/sys-root --program-prefix=x86_64-linux-musl- --disable-bootstrap --with-arch=x86-64 --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --disable-symvers --enable-languages=c,c++,fortran,objc,obj-c++
Thread model: posix
gcc version 7.1.0 (GCC) 
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=1' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-linux-musl/x86_64-linux-musl/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'
 /opt/x86_64-linux-musl/bin/../libexec/gcc/x86_64-linux-musl/7.1.0/cc1 -E -quiet -v -iprefix /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/ -isysroot /opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root -D _GLIBCXX_USE_CXX11_ABI=1 - -march=x86-64 -mtune=generic -frandom-seed=0x412a7d32
ignoring duplicate directory "/opt/x86_64-linux-musl/bin/../lib/gcc/../../lib/gcc/x86_64-linux-musl/7.1.0/../../../../x86_64-linux-musl/include"
ignoring duplicate directory "/opt/x86_64-linux-musl/bin/../lib/gcc/../../lib/gcc/x86_64-linux-musl/7.1.0/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/../../../../x86_64-linux-musl/include
 /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/include
 /opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root/usr/local/include
 /opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root/usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/opt/x86_64-linux-musl/x86_64-linux-musl/sys-root/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "<stdin>"
COMPILER_PATH=/opt/x86_64-linux-musl/bin/../libexec/gcc/x86_64-linux-musl/7.1.0/:/opt/x86_64-linux-musl/bin/../libexec/gcc/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/../../../../x86_64-linux-musl/bin/
LIBRARY_PATH=/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/:/opt/x86_64-linux-musl/bin/../lib/gcc/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/../../../../x86_64-linux-musl/lib/../lib64/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/../../../../x86_64-linux-musl/lib/:/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/7.1.0/../../../:/opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root/lib/:/opt/x86_64-linux-musl/bin/../x86_64-linux-musl/sys-root/usr/lib/
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=1' '-frandom-seed=0x412a7d32' '-L/opt/x86_64-linux-musl/x86_64-linux-musl/lib64' '-march=x86-64' '-mtune=generic' '-v' '-E'

@giordano
Copy link
Member

BTW I think searching SYSROOT/usr/include and SYSROOT/usr/local/include is just automatic default of gcc and clang.

Well, everywhere except LLVM on FreeBSD apparently 😬

@fingolfin
Copy link
Member Author

Aha, but now that makes me realize that I have to be careful about which compilers I look at (of course I should have realized that early when you reminded me once again that clang is default on FreebSD, not gcc). And that made me realize that I made a change not so long ago: I changed G/GAP_pkg/bundled/gac to use cc instead of gcc!

So, that solves the mystery of why "it stopped working"!

But it leaves the mystery why some of the various GCC/clang variants look into SYSROOT/usr/local/include (which I BTW think is a crucial feature!) and why some don't. Perhaps there is a bug here, perhaps not; but now it's clear that whatever it is, it may have been around for quite a bit longer

@fingolfin
Copy link
Member Author

BTW I think searching SYSROOT/usr/include and SYSROOT/usr/local/include is just automatic default of gcc and clang.

Well, everywhere except LLVM on FreeBSD apparently 😬

Well and GCC 4.8.5 on musl, too.... As it is, the default config can of course be adjusted when building the cross compilers. Since "we" build them (right?), I am hopeful this could be adjusted/unified again.

@fingolfin
Copy link
Member Author

Aha, in GCC, the default include paths can be tweaked via INCLUDE_DEFAULTS. And GCCBootstrap@4/bundled/patches/gcc494_musl.patch overrides that (I first thought the name says it is only used for GCC 4.9.4, not 4.8.5, but it really seems to be applied to 4.8.5, as far as I could tell). This might explain why that gcc is different but only on musl. It also gives a hint how one could "fix" this

simeonschaub pushed a commit to simeonschaub/Yggdrasil that referenced this pull request Feb 23, 2022
* Update GAP_pkg_browse to 1.8.13

* Workaround FreeBSD issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants