fcntl.h is not detected by configure #2
Comments
|
please could you share your configure line - since we regularly build and test with macOS SDKs - this is quite unexpected. Note: https://github.com/iains/gcc-12-branch/tree/gcc-12-1-darwin-pre-r1 |
After this, I end up in |
|
You do not need --disable-multilib unless you are building on <= 10.13 and you really want to avoid the 32b multiline. I'm not clear on what the build-time rpaths are supposed to do ? If you are using this branch, it will DTRT with rpaths during build. If you want to add a 'distribution rpath` (i.e. outside of the normal install tree) - there's a new option for that to be added to release 1. I usually have the Xcode command line tools installed in their default position then something like: should work. edit: the |
|
for reference, here's an example configure line used for testing (on macOS 12, but similar lines elsewhere) - I build GMP et. al. in-tree (but adding the I'm using GCC to bootstrap here, because of building Ada and D .. |
The dependencies are edited to be relocatable and as such have
The CLT is skipped because it is a unique resource and version controlling it is impossible. Basically, we have N Xcode installs and which one the CLT uses is always the "latest" which makes "use Xcode 13.1" in CI problematic. So, we just avoid installing it altogether and instead use
Without the CLT, the build could not find standard headers. This was the only way I found to get that to work. I suppose I should mention that I'm also targeting |
The branch(es) here [11.3 and 12.1 - darwin] implement @rpath support in the compiler automatically for OSX >= 10.11 since that is needed for a correct build where DYLD_LIBRARY_PATH is filtered out by system tools. The @rpath support is not upstreamed yet - but hopefully will be in GCC-13 (meanwhile it's in the -darwin branches and necessary for arm64). So .. you should not need any magic, the compiler will build its runtimes with The only time you need to do something extra is if you play a trick that references the dylibs from outside the compiler's install tree [which some of the distros do]. e.g. I'm adding a configure option to support that in 11.3r3 and 12.1r1.
The issue you could run into is that Apple change the layout of SDK headers from time to time, and that layout can be visible in the stdc++ library ..
it's the wrong configure option ;) The built compiler will honour: or fall back to the configured sysroot if neither of the above is present - actually this is the same as the behaviour of clang with the exception that clang does not have the fallback.
no difference from x86_64 at the configuration level .. more experimental at the compiler level tho ;) |
Ah, that would be the difference; I last got 11.1 working on x86_64 and started looking at arm64 now that it seems to work well enough for numpy/scipy.
Yes, that they do :( .
Alright, I'll try that. |
addendum: I do not think this is going to work, the build-time paths need to be absolute. The mechanism by which macOS usually controls this is using This should work (if it does not it's a bug, please report) for GCC with one important point: since Xcode hijacks the names you have to use |
I'm really only interested in gfortran to be honest; the gcc and g++ binaries are largely ignored (so far). But I'll keep that in mind. |
|
or - you set SDKROOT from your use-case (which is all that keeping things consistent is probably a good plan, even if you are not using |
|
Oh, I suppose I missed these settings which are done before building mpc, gmp, and mpfr: The |
|
ISTR I will be updating the 11.3 and 12.1 releases (hopefully this weekend) to include a couple of tweaks to the rpath handling and a critical fix for Arm64. meanwhile the -pre-rx branches have those changes if you want to test. Yes, 11.0 is the min for arm64 - I do build cross compilers from x86_64 => arm64 on macOS 11 and 12. |
So that sets the build conditions for the libraries, and for executables that run on the build system (which might, of course, also be the host). The actual compiler bootstrap uses STAGE1_xxxx and BOOT_CFLAGS to control things but, honestly, you do not need to be messing with this stuff ... unless you have some very special need, the regular configure should DTRT - and I'd rather debug finding a solution to a special need than a wrongly-configured compiler ;) |
The problem, IIRC, was that the library ids were just plain filenames. I have a post-install step for the deps that changes them to be These all came from what I needed to do to get a gfortran-11.2 that could be extracted anywhere for use in CI. I didn't add anything that didn't make progress (some might not be strictly necessary anymore, but better spellings of what I'm trying to do would be appreciated). I'll try and get x86_64 to do what it needs to at least before forging ahead on arm64 any further. |
|
OK. Well, that should all "just work" now with the branches here if it doesn't then let's debug why. |
|
It seems that Xcode 13.0 headers just aren't supported yet (they all fail to detect |
|
hmm that does not make sense to me - I've tested with Xcode 13.x (from 13.0 to 13.2.1) .. please post your exact amended configure line - I will take a look on various versions (although the tests in issue #1 here cover OSX 10.5 ... macOS 12. so it worked not long ago :D ) |
|
typically, folks who do not use the CLT do things like:
|
|
Attached is the script I'm using. Expects to be run as |
|
what sets DEVELOPER_DIR? ... I don't see anything in the script. |
|
That comes from the CI configuration (it lives there because the CI also has tags which need synced to say "run on a machine with this Xcode version") |
|
The configuration still looks wrong to me - which is why the build is failing (GCC configures and builds fine on darwin9 ... 21). To fix the configuration script - we need to know what the objective is .... So (inference mode engaged), in summary, is the requirement: " I want to build a compiler on macOS latest and greatest, to run on macOS/OSX something else, as if it had been built there using the tools and SDK in DEVELOPER_DIR "? some assumptions:
Do you do a batch of these for a given compiler version, or is it done 'on demand' when there's a support problem? (that could make a difference to the best way to do this). There is an "OSX way" (which was already there before I joined the project, so please don't shoot me) of doing some things which is somewhat different from the "Linux way" .. so the build recipes need to account for that; depending on circumstances (and the gap between system revisions) the OSX-way does have some neat tricks tho. |
|
FWIW, I configured and built a Darwin17 (10.13) compiler on Darwin20 (11) without any particular problem - so at least one mechanism works even with that sized gap. Unfortunately, $dayjob is very hectic so not able to do much more at the moment - what is needed next is a clear description of the constraints for your build case(s). For example, can we assume that $DEVELOPER_DIR is an identical path on the build and deployment system? |
|
So I have gotten back to this. Updates:
With all of that, things get further. Current errors: On arm64: On x86_64: Currently trying to reproduce outside CI so I can investigate without having CI upload absolutely everything to dig around in. |
|
Ok, so reading the docs on the It does end up with a relocatable installation that I don't have to worry about where it ends up as well :) . |
|
I've been OOO so not quite picked up on everything yet - I guess what would be ideal from my PoV is to understand exactly what you are trying to achieve... even if that requires patches to make it happen. It is not generally necessary to add any magic flags to build a relocatable compiler from these branches. However, building for a different host OS version (i.e. $build != $host) does require some care, because there are dependencies on the SDK. |
|
The full script I'm using to build the compiler can be seen here: https://gitlab.kitware.com/utils/ci-utilities/-/merge_requests/8 The goal is to have a This is all because macOS CI from virtual machines is a true PITA when you also need to test things like OpenGL rendering, so we're stuck on bare metal without things like containers to help out. Nevermind that slinging multi-GB VM images around for CI purposes would be…cumbersome to say the least. We only need |
|
This is a long reply - what you are trying to do (and what I have to do for my own CI) has various constraints that a typical end-user for home-brew or macports does not face (intentionally, of course, they are insulating the users from these complexities). The branches here for GCC10+ support a relocatable compiler; upstream GCC (for Darwin) does not (yet). FAOD: these are development branches, not forks, I intend to upstream all the work here - but it is currently $hobby so, unless someone decides to step forward and sponsor it to be $dayjob, it all takes an arbitrarily long time ... More-or-less all the constraints we face (for someone trying to create a flexible deployment) arise from two factors:
Of course, many (maybe most) of the macOS users of GCC are self-building for own use and not distributing exes to 3rd parties, so they do not see the packaging issues we do.
Thanks, that is helpful. The comments here relate to the expectation of what $DEVELOPER_DIR can actually achieve in practice. sysroot:
The compilers here are, by default, relocatable on
You should not need that with these branches; if you do, that's a bug - please report it.
There are three other relatively easy options (and numerous less-easy, no doubt):
For my use-cases, I build for
What I do is to have N versions of Xcode CLTs installed in /XC/NN.MM/CommandLineTools and then use
Well, (my opinion) Similar issues apply to Ada, which is supported by GCC and by the branches here (actually probably even more so, since Ada is intentionally very constrained). ===== OK - so there are some opportunities noted above. The one constraint that you have to cater for, no matter how you package this is that; if you intend to deploy the compiler on macOS NN.MM, then you must build it with an appropriate SDK for NN.MM (it will not, in general, work to use $LATEST_SDK) .. this is because of the issue noted that the SDKs include features and sometimes bugs that GCC does not support / break GCC's assumptions. The GCC build includes a phase called (note that fixincludes is not a Darwin-specific thing, there are other OSs with buggy headers) The sad thing (from my PoV) is that GCC for Darwin itself supports building for any version from OSX10.4 => macOS 12 but the SDK issues (and references to SDK headers in runtimes) currently prevent us from deploying that way. It is on the (very long) $hobby TODO to figure out some way to build an SDK from the public sources that does not have these constraints .. |
|
addendum: Of course, providing that the build system meets the criteria for the SDK license, you can obviously extract SDKs from many versions of Xcode/CLT and have them available on the build system so that you can select the right one for the case you are trying to deal with. edit: |
|
I suspect that some of the things you are specifying are defeating the automatic configuration provisions and making it much more complex than necessary. This is why I am still asking exactly what you want to achieve - I suspect we can simplify this quite a bit. initial additional comments: I have not yet found a case where it necessary specify cpp, the configuration process is supposed to discover "clang -E" or "gcc -E" as appropriate, you are probably overriding this. See the notes on SDK above for one part of this... but I think you do not need to add this - the configuration sysroot flags should handle it. When bootstrapping the compiler the stage1 is built with minimum constraints - it needs to run on $build (as does stage2)... so that does set the limit on the range of systems for which you can bootstrap for macOS N on macOS N+M. If you want to add the -mmacosx-version-min then it would need to be added to STAGE1_CLFLAGS, STAGE1_CXXFLAGS, BOOT_CFLAGS and CFLAGS/CXXFLAGS_FOR_TARGET but it's probable that it is better to export MACOSX_DEPLOYMENT_TARGET= instead. I have never encountered this problem, I think that somehow the recipes here are defeating some part of the configuration process. exactly which Xcode version and SDK version, which $build OS version and which $target OS version - I'd like to try to reproduce it. I do not understand the reason for these options; (1) The branches here will automatically build with embedded rpaths, (2) GCC defaults to building with static libstdc++/libgcc on systems that support it. This means that if you use clang as the bootstrap compiler, GCC will still be statically linked at stage 2 and 3. I am concerned that there is partial overriding of configuration happening which is causing stuff to get out of sync. Once again, I think we need to wind back to exactly what the requirement is; i.e. "I want to build a compiler to run on $host OS version X on $build OS version Y using clang/xcode as the bootstrap" .. because I suspect we can simplify this some more. the configure and build are supposed to support MACOSX_DEPLOYMENT_TARGET (but we do need to check that we can bootstrap for X on Y). If we cannot bootstrap for X on Y (because the editions are too far apart, or different architectures) then the build process is somewhat different, we need to do build a native cross. It is unlikely to work properly to attempt to a single-stage build when the bootstrap compiler is anything other than a close version of GCC. We should be able to do what you want to do - so if some part fails then that's a bug, but at the moment I cannot disentangle the unusual configure options from the requirement and what the actual bugs might be :) |
|
So I did a quick test of what I expect to work... $build = macOS 11.7 x86_64 I am using the gcc-12-2-darwin-pre-release-1 (but the process should be the same for any of the darwin branches here that support Arm64). precondition: If you have something different for that, then the configure might require adjustment. On $build: This mechanism for configuring a bootstrap relies on knowing that it is valid to bootstrap for Darwin17 on Darwin20 .. there is no easy way to automate that (but, of course, if it's not valid to do so then we can expect it to fail). NOTE: headerpad_max_install_names should not really be required, we are not expecting to edit the install IDs of any libraries now (I only put it in for comparison with your script). On $host: So we have relocated the compiler from NOTE: @rpaths are the default for 10.11+ for these branches So we have built the libraries with the correct MACOSX versions and we have We can see the original configure line, but it is working quite happily at the relocated position. I smoke-tested C, C++ (with exception handing) and Fortran - which all work as expected (not a formal run of the testsuite, but certainly a reasonable kick of the tires). |
|
additional notes:
These are the versions I used here: |
The eliminate reg-reg move by inverting the condition of a cmove #2 peephole2 converts the following sequence: 473: bx:DI=[r14:DI*0x8+r12:DI] 960: r15:DI=r8:DI 485: {flags:CCC=cmp(r15:DI+bx:DI,bx:DI);r15:DI=r15:DI+bx:DI;} 737: r15:DI={(geu(flags:CCC,0))?r15:DI:bx:DI} to: 1110: {flags:CCC=cmp(r8:DI+bx:DI,bx:DI);r8:DI=r8:DI+bx:DI;} 1111: r15:DI=[r14:DI*0x8+r12:DI] 1112: r15:DI={(geu(flags:CCC,0))?r8:DI:r15:DI} Please note that(insn 1110) uses register BX, but its initialization was eliminated. Avoid conversion if eliminated move intialized a register, used in the moved instruction. 2022-11-03 Uroš Bizjak <ubizjak@gmail.com> gcc/ChangeLog: PR target/107404 * config/i386/i386.md (eliminate reg-reg move by inverting the condition of a cmove #2 peephole2): Check if eliminated move initialized a register, used in the moved instruction. gcc/testsuite/ChangeLog: PR target/107404 * g++.target/i386/pr107404.C: New test. (cherry picked from commit 553b1d3dd5b9253ebdf66ee3260c717d5b807dd1)
While looking at PR 105549, which is about fixing the ABI break introduced in GCC 9.1 in parameter alignment with bit-fields, we noticed that the GCC 9.1 warning is not emitted in all the cases where it should be. This patch fixes that and the next patch in the series fixes the GCC 9.1 break. We split this into two patches since patch #2 introduces a new ABI break starting with GCC 13.1. This way, patch #1 can be back-ported to release branches if needed to fix the GCC 9.1 warning issue. The main idea is to add a new global boolean that indicates whether we're expanding the start of a function, so that aarch64_layout_arg can emit warnings for callees as well as callers. This removes the need for aarch64_function_arg_boundary to warn (with its incomplete information). However, in the first patch there are still cases where we emit warnings were we should not; this is fixed in patch #2 where we can distinguish between GCC 9.1 and GCC.13.1 ABI breaks properly. The fix in aarch64_function_arg_boundary (replacing & with &&) looks like an oversight of a previous commit in this area which changed 'abi_break' from a boolean to an integer. We also take the opportunity to fix the comment above aarch64_function_arg_alignment since the value of the abi_break parameter was changed in a previous commit, no longer matching the description. 2022-11-28 Christophe Lyon <christophe.lyon@arm.com> Richard Sandiford <richard.sandiford@arm.com> gcc/ChangeLog: * config/aarch64/aarch64.cc (aarch64_function_arg_alignment): Fix comment. (aarch64_layout_arg): Factorize warning conditions. (aarch64_function_arg_boundary): Fix typo. * function.cc (currently_expanding_function_start): New variable. (expand_function_start): Handle currently_expanding_function_start. * function.h (currently_expanding_function_start): Declare. gcc/testsuite/ChangeLog: * gcc.target/aarch64/bitfield-abi-warning-align16-O2.c: New test. * gcc.target/aarch64/bitfield-abi-warning-align16-O2-extra.c: New test. * gcc.target/aarch64/bitfield-abi-warning-align32-O2.c: New test. * gcc.target/aarch64/bitfield-abi-warning-align32-O2-extra.c: New test. * gcc.target/aarch64/bitfield-abi-warning-align8-O2.c: New test. * gcc.target/aarch64/bitfield-abi-warning.h: New test. * g++.target/aarch64/bitfield-abi-warning-align16-O2.C: New test. * g++.target/aarch64/bitfield-abi-warning-align16-O2-extra.C: New test. * g++.target/aarch64/bitfield-abi-warning-align32-O2.C: New test. * g++.target/aarch64/bitfield-abi-warning-align32-O2-extra.C: New test. * g++.target/aarch64/bitfield-abi-warning-align8-O2.C: New test. * g++.target/aarch64/bitfield-abi-warning.h: New test. (cherry picked from commit 3df1a115be22caeab3ffe7afb12e71adb54ff132)
Here the ahead-of-time overload set pruning in finish_call_expr is unintentionally returning a CALL_EXPR whose (pruned) callee is wrapped in an ADDR_EXPR, despite the original callee not being wrapped in an ADDR_EXPR. This ends up causing a bogus declaration mismatch error in the below testcase because the call to min in #1 gets expressed as a CALL_EXPR of ADDR_EXPR of FUNCTION_DECL, whereas the level-lowered call to min in #2 gets expressed instead as a CALL_EXPR of FUNCTION_DECL. This patch fixes this by stripping the spurious ADDR_EXPR appropriately. Thus the first call to min now also gets expressed as a CALL_EXPR of FUNCTION_DECL, matching the behavior before r12-6075-g2decd2cabe5a4f. PR c++/107461 gcc/cp/ChangeLog: * semantics.cc (finish_call_expr): Strip ADDR_EXPR from the selected callee during overload set pruning. gcc/testsuite/ChangeLog: * g++.dg/template/call9.C: New test. (cherry picked from commit 59e0376f607805ef9b67fd7b0a4a3084ab3571a5)
After r13-5684-g59e0376f607805 the (pruned) callee of a non-dependent CALL_EXPR is a bare FUNCTION_DECL rather than ADDR_EXPR of FUNCTION_DECL. This innocent change revealed that cp_tree_equal doesn't first check dependence of a CALL_EXPR before treating a FUNCTION_DECL callee as a dependent name, which leads to us incorrectly accepting the first two testcases below and rejecting the third: * In the first testcase, cp_tree_equal incorrectly returns true for the two non-dependent CALL_EXPRs f(0) and f(0) (whose CALL_EXPR_FN are different FUNCTION_DECLs) which causes us to treat #2 as a redeclaration of #1. * Same issue in the second testcase, for f<int*>() and f<char>(). * In the third testcase, cp_tree_equal incorrectly returns true for f<int>() and f<void(*)(int)>() which causes us to conflate the two dependent specializations A<decltype(f<int>()(U()))> and A<decltype(f<void(*)(int)>()(U()))>. This patch fixes this by making called_fns_equal treat two callees as dependent names only if the overall CALL_EXPRs are dependent, via a new convenience function call_expr_dependent_name that is like dependent_name but also checks dependence of the overall CALL_EXPR. PR c++/107461 gcc/cp/ChangeLog: * cp-tree.h (call_expr_dependent_name): Declare. * pt.cc (iterative_hash_template_arg) <case CALL_EXPR>: Use call_expr_dependent_name instead of dependent_name. * tree.cc (call_expr_dependent_name): Define. (called_fns_equal): Adjust to take two CALL_EXPRs instead of CALL_EXPR_FNs thereof. Use call_expr_dependent_name instead of dependent_name. (cp_tree_equal) <case CALL_EXPR>: Adjust call to called_fns_equal. gcc/testsuite/ChangeLog: * g++.dg/cpp0x/overload5.C: New test. * g++.dg/cpp0x/overload5a.C: New test. * g++.dg/cpp0x/overload6.C: New test. (cherry picked from commit 31924665c86d47af6b1f22a74f594f2e1dc0ed2d)
|
Ok, so I'm updating to 13.2 and would like to try and resolve things here. I had been using Xcode 14.2 and expect to continue that for x86_64 but migrate to 14.3 for arm64. I first tried to strip everything down as much as possible (the only flags for GCC are: Note that the dependencies are
I add With this, we get much further until: To fix this, I pass This is why the So it looks like I can remove:
|
You should be able to build a relocatable GCC "out of the tin" with no special additions.. once I understand the disjoint installation scheme for the dependent libs, hopefully I should be able to make some suggestions. edit: I actually do not know if GMP et. al support building as "@rpath/xxxxxx.dylib" by default, but I presume that this can be coerced - the GCC configuration does not influence the GMP/MPFR/MPC one, except to ensure the code is built PIC where needed (so building as shared libs in-tree is not going to work) |
There is a different mechanism for determining the execution-time SDK using However, it does work fine for x86_64-apple-darwinNN-gcc etc. (and I would expect it would work for edit: it is however, in any case at the present, a requirement that the same SDK version is used at compiler build and execution time (the SDKPATH can allow it to be somewhere different). We are working on making that more flexible (but even with clang the range of SDK version support is usually limited to one or two OS revs.) |
I agree. It's just that the flags needed to do so seem to differ so much.
No, they're installed with
They're all built in one CI job, but I can try this.
No, there's a post-install step to rewrite them. The IDs are otherwise absolute paths to |
This is easily set up by saying
What I do is -- from a different build directory -- where What it doesn’t do is insert proper |
I think that this is one problem - you need a runpath for the installation - but you cannot supply a relative one. There could be some clever way around this - but .. first ...
... I was about to suggest that we start from a known working case and then add the extra requirements. So.. if you
The rule for sysroot at execution time is: use |
|
Is there a mapping of macOS versions to |
for macOS < 11 for macOS 11+ Note toolchains should really just use the major number (otherwise you'll get linker complaints about mismatched versions if you try to link for earlier) |
Try Wikipedia - the third column, "Darwin version" |
|
Ok, so I am getting further now by having GCC bootstrap libraries. I can remove a lot of the rpath shenanigans it seems. However, |
|
JFTR, isl 0.20-24 work for me (I'm currently using 0.24) .. probably should test something newer at some point. I am not sure, off hand, if fortran is making heavy use of isl (or the status of graphite). |
unless, you are doing something special (if you are using the branches here) you should not need any extra configury magic. The intention of starting with the simple case was then to add back in any extra requirements - and figure out the best way to handle them. We have a fundamental difference from clang, in that we have to carry the runtimes with the installation - where clang can assume that the ones it needs (c++, c++abi, compiler_rt, libunwind) are already installed. It is also possible to configure GCC to use libc++ (but that will not help you in a case where you want to move a fortran compiler to a CI box). |
|
I am down to just I've dropped |
Great! ..
I've not tried 0.25 or 0.26 - but 0.24 works everywhere from darwin9 (10.5) - darwin23 (14.0, Sonoma beta), depends if anything you are testing depends on graphite etc. |
|
I don't think those libraries being shared matters that much. I'll also note that I've had to add |
Well, I agree (actually think it's safer to make sure one is using the one the toolchain was checked with - so built int) - but just checking.
Hmm.. we already (for some versions of macOS) automatically disable multilibs when the target does not support them ... ... I guess you can run into an issue if you want to build for a target that does support them, but then try to test on a later one. In that case --disable-multilib seems a reasonable solution. |
|
On 29 Aug 2023, at 15:44, Iain Sandoe ***@***.***> wrote:
I've not tried 0.25 or 0.26 - but 0.24 works everywhere from darwin9 (10.5) - darwin23 (14.0, Sonoma beta), depends if anything you are testing depends on graphite etc.
As of 2023-08-23, contrib/download_prerequisites installs
gmp-6.2.1
mpfr-4.1.0
mpc-1.2.1
isl-0.24
|
|
I'll close this in the meantime. Further issues can be dealt with on the list (I've emailed about isl 0.26 support) or in the GCC 13 repo as needed. Thanks for your help! |
The
configurecheck forfcntl.hfails because gcc errors about#if defined(__has_attribute)not having a(after it in an SDK header. Same thing fordefined(__has_feature)on the same line. Tested with thegcc-12.1-darwin-r0tag. Manually editinglibiberty/config.hto say it has the header lets me get further.The text was updated successfully, but these errors were encountered: