macOS: LD version detection is unable to detect linker for compiler gfortran with Xcode CLT 15.0 #12282

wmutschl · 2023-09-22T13:29:45Z

Describe the bug
We have recently switched to meson for building dynare, an industry-leading software used by many central banks, economic policy and research institutions as well as universities.

All was good with Xcode Command Line Tools (CLT) 14.3.1, but recently I updated to Xcode CLT 15.0 and now I get an error ld: unknown options: --version when detecting linker for compiler gfortran -Wl,--version`:

The Meson build system
Version: 1.2.1
Source dir: /Users/wmutschl/dynare/irf-matching/irf-matching
Build dir: /Users/wmutschl/dynare/irf-matching/irf-matching/build-matlab
Build type: native build
Project name: dynare
Project version: 6-unstable

meson.build:12:0: ERROR: Unable to detect linker for compiler `gfortran -Wl,--version -L/var/folders/b0/t0gqszsj5qg8mvpk7fqy0f7h0000gn/T/tmp.cH3j0LwS`
stdout: 
stderr: collect2 version 13.2.0
/usr/bin/ld -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/ -dynamic -arch x86_64 -platform_version macos 13.0.0 0.0 -o a.out -L/var/folders/b0/t0gqszsj5qg8mvpk7fqy0f7h0000gn/T/tmp.cH3j0LwS -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc/x86_64-apple-darwin22/13 -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc/x86_64-apple-darwin22/13/../../.. --version -lemutls_w -lgcc -lSystem -no_compact_unwind -rpath @loader_path -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current/gcc/x86_64-apple-darwin22/13 -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current/gcc -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current
ld: unknown options: --version 
collect2: error: ld returned 1 exit status

Here is the output of meson-log.txt:

# cat /Users/wmutschl/dynare/unstable/build-matlab/meson-logs/meson-log.txt 

Build started at 2023-09-21T08:24:25.638781
Main binary: /usr/local/opt/python@3.11/bin/python3.11
Build Options: -Dmatlab_path=/Applications/MATLAB/x86_64/MATLAB_R2023a.app -Dbuildtype=debugoptimized '-Dfortran_args=['"'"'-B'"'"','"'"'/Users/wmutschl/dynare/slicot/lib'"'"']' --native-file=scripts/homebrew-native.ini
Python system: Darwin
The Meson build system
Version: 1.2.1
Source dir: /Users/wmutschl/dynare/unstable
Build dir: /Users/wmutschl/dynare/unstable/build-matlab
Build type: native build
Project name: dynare
Project version: 6-unstable
-----------
Detecting compiler via: `gfortran --version` -> 0
stdout:
GNU Fortran (Homebrew GCC 13.2.0) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-----------
Running command: gfortran -E -dM -
-----
-----------
Detecting linker via: `gfortran -Wl,--version` -> 1
stderr:
collect2 version 13.2.0
/usr/bin/ld -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/ -dynamic -arch x86_64 -platform_version macos 13.0.0 0.0 -o a.out -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc/x86_64-apple-darwin22/13 -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc/x86_64-apple-darwin22/13/../../.. --version -lemutls_w -lgcc -lSystem -no_compact_unwind -rpath @loader_path -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current/gcc/x86_64-apple-darwin22/13 -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current/gcc -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current
ld: unknown options: --version 
collect2: error: ld returned 1 exit status
-----------

meson.build:12:0: ERROR: Unable to detect linker for compiler `gfortran -Wl,--version`
stdout: 
stderr: collect2 version 13.2.0
/usr/bin/ld -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk/ -dynamic -arch x86_64 -platform_version macos 13.0.0 0.0 -o a.out -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc/x86_64-apple-darwin22/13 -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc -L/usr/local/Cellar/gcc/13.2.0/bin/../lib/gcc/current/gcc/x86_64-apple-darwin22/13/../../.. --version -lemutls_w -lgcc -lSystem -no_compact_unwind -rpath @loader_path -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current/gcc/x86_64-apple-darwin22/13 -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current/gcc -rpath /usr/local/Cellar/gcc/13.2.0/lib/gcc/current
ld: unknown options: --version 
collect2: error: ld returned 1 exit status

To Reproduce
Update to Xcode Command Line Tools 15.0 and follow the instructions to build for macos, you will get the error. Downgrade to Xcode Command Line Tools 14.3.1 and there is no error.
Here is a minimal version of the build process to replicate:

# brew packages
brew install meson bison flex boost gcc gsl libmatio veclibfort octave sphinx-doc docutils wget pkg-config git-lfs
# compile Dynare from source
git clone --recurse-submodules https://git.dynare.org/Dynare/dynare.git $HOME/dynare/unstable
cd $HOME/dynare/unstable
meson setup --native-file scripts/homebrew-native.ini -Dbuild_for=octave -Dbuildtype=debugoptimized -Dfortran_args="['-B']" build-octave

homebrew-native.ini looks like this:

[binaries]
cpp = 'g++-13'
flex = '/usr/local/opt/flex/bin/flex'
bison = '/usr/local/opt/bison/bin/bison'

Expected behavior
No error.

system parameters

MacOS Ventura 13.5.2 or 13.6
Python 3.11.5
meson 1.2.1
ninja 1.11.1

The text was updated successfully, but these errors were encountered:

eli-schwartz · 2023-09-22T13:34:17Z

Hmm, weird.

What options does /usr/bin/ld accept? Is there a different spelling for this particular linker to emit its version number?

(I thought we try both -v and --version, but I'd need to double check.)

wmutschl · 2023-09-22T13:46:24Z

Many thanks for the quick reply! I just updated to Xcode CLT 15.0 again:

# /usr/bin/ld -v
@(#)PROGRAM:ld  PROJECT:dyld-1015.7
BUILD 18:48:48 Aug 22 2023
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
will use ld-classic for: armv6 armv7 armv7s arm64_32 i386 armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.0.12.3)
Library search paths:
Framework search paths:

and

#/usr/bin/ld --version
ld: unknown option: --version

wmutschl · 2023-09-22T13:50:39Z

However, the same with Xcode CLT 14.3.1:

# /usr/bin/ld -v
@(#)PROGRAM:ld  PROJECT:ld64-857.1
BUILD 23:13:29 May  7 2023
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 14.0.3, (clang-1403.0.22.14.1) (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 14.0.3 (tapi-1403.0.5.1)

and

# /usr/bin/ld --version
ld: unknown option: --version

eli-schwartz · 2023-09-22T13:54:53Z

This looks like the difference between "PROJECT:ld64" and "PROJECT:dyld" is tripping up the detection. I think there was a recent discussion about this, hold on...

wmutschl · 2023-09-22T13:56:42Z

Yes, but I checked, I have the fix #11958 in my /usr/local/opt/meson/lib/python3.11/site-packages/mesonbuild/linkers/detection.py.
Line 189 and following:

    elif e.endswith('(use -v to see invocation)\n') or 'macosx_version' in e or 'ld: unknown option:' in e:
        if isinstance(comp_class.LINKER_PREFIX, str):
            cmd = compiler + [comp_class.LINKER_PREFIX + '-v,-r'] + extra_args
        else:
            cmd = compiler + comp_class.LINKER_PREFIX + ['-v'] + extra_args
        _, newo, newerr = Popen_safe_logged(cmd, msg='Detecting Apple linker via')

        for line in newerr.split('\n'):
            if 'PROJECT:ld' in line or 'PROJECT:dyld' in line:
                v = line.split('-')[1]
                break
        else:
            __failed_to_detect_linker(compiler, check_args, o, e)
        linker = linkers.AppleDynamicLinker(compiler, for_machine, comp_class.LINKER_PREFIX, override, version=v)

eli-schwartz · 2023-09-22T13:59:11Z

Okay then, I'm out of ideas that I can think of and suggest from my phone while I'm on the go. I'll try to investigate in greater depth later today or perhaps on Sunday.

wmutschl · 2023-09-22T14:01:26Z

I am very grateful for any idea, thank you so much for taking the time!

wmutschl · 2023-09-22T14:19:48Z

I have also updated the first post which has minimal instructions to reproduce the error.

jpakkane · 2023-09-22T19:36:20Z

Does it fail if you use Meson from Git trunk?

wmutschl · 2023-09-22T21:59:36Z

I am sorry, but I don't know how to install the most recent git version. I tried:

git clone https://github.com/mesonbuild/meson.git $HOME/meson
brew uninstall meson
brew install bison flex boost gcc gsl libmatio veclibfort octave sphinx-doc docutils wget pkg-config git-lfs
git clone --recurse-submodules https://git.dynare.org/Dynare/dynare.git $HOME/dynare/unstable
cd $HOME/dynare/unstable
$HOME/meson/meson.py setup --native-file scripts/homebrew-native.ini -Dbuild_for=octave -Dbuildtype=debugoptimized -Dfortran_args="['-B']" build-octave

But now I get:

meson.build:15:0: ERROR: Compiler gfortran cannot compile programs.

Is there a guide how I install it properly from git?

eli-schwartz · 2023-09-22T22:11:00Z

git clone https://github.com/mesonbuild/meson.git $HOME/meson
[...]
$HOME/meson/meson.py setup [...]

This is a correct implementation of running meson from git. To be sure, you should see that it's reporting a version number of 1.2.99 rather than reporting a version of 1.2.1

My conclusion is that the answer to @jpakkane's question is "no, using the most recent git version still fails, and does not solve the problem -- it is not fixed yet".

mkoeppe · 2023-09-26T17:42:59Z

Has this failure been reproduced with gfortran that is not coming from Homebrew?
This homebrew-specific configuration Homebrew/homebrew-core@57c7726 by @fxcoudert may be relevant.

fxcoudert · 2023-09-26T22:23:20Z

On macOS Ventura, Homebrew's gcc is unaffected by the changed linked above (we still build it on CI with Xcode 14).

But even when called as ld-classic, the linker output should be matched by the code shown above:

$ /Library/Developer/CommandLineTools/usr/bin/ld-classic -v
@(#)PROGRAM:ld-classic  PROJECT:ld64-907
BUILD 19:12:38 Aug 11 2023
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.0.12.3)

meson.build:15:0: ERROR: Compiler gfortran cannot compile programs.

Do you have the output from that test?

dimpase · 2023-09-28T00:08:06Z

macport seems to have the same issue with the new ld, see https://trac.macports.org/ticket/68225

wmutschl · 2023-10-10T10:06:29Z

Thanks for the updates, I did some more digging and it seems like the linker issue is a known bug, already reported to Apple, and also in the release notes for XCode 15 (and for me the issue is still there in 15.1. Beta):

Binaries using symbols with a weak definition crash at runtime on iOS 14/macOS 12 or older. This impacts primarily C++ projects due to their extensive use of weak symbols. (114813650) (FB13097713)
Workaround: Bump the minimum deployment target to iOS 15, macOS 12, watchOS 8 or tvOS 15, or add -Wl,-ld_classic to the OTHER_LDFLAGS build setting.
Weak symbol imports are linked as non-weak imports, when used from LTO object files. (115521975) (FB13171424)
Workaround: Add -Wl,-weak_reference_mismatches,weak or -Wl,-ld_classic options to the OTHER_LDFLAGS build setting.

Other resources also confirm that the current workaround is to add -ld_classic to linker flags, which forces the older linker to be used.

See:

However, how do I do this to my project? Do I set environmental variables or do I change the meson build script? Sorry, I am very new to this and not very experiences in compilations...
I found this Libreoffice snippet, so would it be correct to add:

 export LDFLAGS="$LDFLAGS -Wl,-ld_classic"

before my meson setup command?

eli-schwartz · 2023-10-10T11:40:04Z

Yes, setting LDFLAGS should work.

wmutschl · 2023-10-10T21:34:09Z

Okay an update that it works, but I am not sure what exactly did it for me.

Installed XCode Command Line Tools 15.1 Beta, issue still persisted.
Did a export LDFLAGS="$LDFLAGS -Wl,-ld_classic" prior to the meson command, and issue was gone (still on 15.1 Beta).

Interestingly, from now on (even after reboot) I don't have to use export LDFLAGS="$LDFLAGS -Wl,-ld_classic" anymore, but it keeps using the ld_classic linker; this is strange isn't it?

wmutschl · 2023-10-10T21:46:36Z

export LDFLAGS="$LDFLAGS -Wl,-ld_classic" with XCode Command Line Tools 15.0 also works. And same weird thing: I only need to run the export LDFLAGS="$LDFLAGS -Wl,-ld_classic" command once before the meson setup. Closing the terminal, restarting the computer, re-opening Terminal, cloning the repo, the meson setup command runs fine (without the export command anymore).

wmutschl · 2023-10-10T22:04:12Z

@svillemot proposed to simply use the native file in our case:

[binaries]
cpp = 'g++-13'
c = 'gcc-13'
flex = '/usr/local/opt/flex/bin/flex'
bison = '/usr/local/opt/bison/bin/bison'

[built-in options]
cpp_args = [ '-I/usr/local/include', '-B', '/usr/local/lib' ]
cpp_link_args = [ '-Wl,-ld_classic' ]
c_link_args = [ '-Wl,-ld_classic' ]
fortran_link_args = [ '-Wl,-ld_classic' ]

I replicated the issue in a clean installed macOS. Using the above native file solves the issue.

dimpase · 2023-10-10T22:07:25Z

So in a new shell LDFLAGS is not set, yet meson works?

wmutschl · 2023-10-10T22:27:20Z

Yes, very weird... I have this on two macs... Anyways, the native file is the better solution.

wmutschl · 2023-10-11T07:19:40Z

I have done another install in a virtual machine and both the native file solution as well as the export command work and solve the issue. Also the persistence of the export command is not there anymore, so all is good, seems to be something specific to my settings.

dimpase · 2023-10-11T15:58:14Z

Yes, very weird... I have this on two macs... Anyways, the native file is the better solution.

could it be "restore apps and windows" macOS feature playing here? (or perhaps a side effect of it)

- Fixes sagemath#36342 - using the fix discussed at mesonbuild/meson#12282  ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [ ] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#36523 Reported by: Matthias Köppe Reviewer(s): John H. Palmieri

wmutschl · 2023-12-20T00:49:23Z

~~Xcode CLT 15.1 fixed this issue, the ld_classic workaround in the native file is not necessary anymore, even though the workaround still continues to work.~~
Nope, 15.1 did not fix this.

mkoeppe mentioned this issue Sep 26, 2023

Build failures with Xcode CLT 15.0: scipy (via meson), ecm sagemath/sage#36342

Closed

wmutschl closed this as completed Oct 11, 2023

anntzer mentioned this issue Oct 18, 2023

[Bug]: build failure with meson + python 3.11/3.12 + macos14.0 matplotlib/matplotlib#27107

Closed

mkoeppe mentioned this issue Oct 23, 2023

scipy: Fix build on macOS with Xcode CLT 15.0 sagemath/sage#36523

Closed

5 tasks

wpbonelli mentioned this issue Oct 30, 2023

chore: unpin python<=3.11, mention Xcode 15 workaround in dev docs MODFLOW-USGS/modflow6#1415

Merged

sagudev mentioned this issue Nov 21, 2023

The options to the linker need to be separate elements in the list, n… servo/mozjs#427

Closed

ksjogo mentioned this issue Dec 21, 2023

Scipy doesn't build on Mac OS pyodide/pyodide#4348

Open

ketch mentioned this issue Jan 30, 2024

Issue with installation clawpack/pyclaw#716

Closed

tbenthompson mentioned this issue Apr 24, 2024

Pip Install Okada Wrapper Issues tbenthompson/okada_wrapper#15

Closed

wendyfyx mentioned this issue May 29, 2024

Broken source installation dipy/dipy#3242

Closed

meteorcloudy mentioned this issue Jul 9, 2024

Tracking issue for impact of 2024 MacOS fleet expansion bazelbuild/continuous-integration#1981

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macOS: LD version detection is unable to detect linker for compiler gfortran with Xcode CLT 15.0 #12282

macOS: LD version detection is unable to detect linker for compiler gfortran with Xcode CLT 15.0 #12282

wmutschl commented Sep 22, 2023 •

edited

Loading

eli-schwartz commented Sep 22, 2023

wmutschl commented Sep 22, 2023

wmutschl commented Sep 22, 2023

eli-schwartz commented Sep 22, 2023

wmutschl commented Sep 22, 2023 •

edited

Loading

eli-schwartz commented Sep 22, 2023

wmutschl commented Sep 22, 2023

wmutschl commented Sep 22, 2023

jpakkane commented Sep 22, 2023

wmutschl commented Sep 22, 2023

eli-schwartz commented Sep 22, 2023

mkoeppe commented Sep 26, 2023

fxcoudert commented Sep 26, 2023 •

edited

Loading

dimpase commented Sep 28, 2023

wmutschl commented Oct 10, 2023

eli-schwartz commented Oct 10, 2023

wmutschl commented Oct 10, 2023 •

edited

Loading

wmutschl commented Oct 10, 2023

wmutschl commented Oct 10, 2023 •

edited

Loading

dimpase commented Oct 10, 2023

wmutschl commented Oct 10, 2023

wmutschl commented Oct 11, 2023 •

edited

Loading

dimpase commented Oct 11, 2023

wmutschl commented Dec 20, 2023 •

edited

Loading

macOS: LD version detection is unable to detect linker for compiler gfortran with Xcode CLT 15.0 #12282

macOS: LD version detection is unable to detect linker for compiler gfortran with Xcode CLT 15.0 #12282

Comments

wmutschl commented Sep 22, 2023 • edited Loading

eli-schwartz commented Sep 22, 2023

wmutschl commented Sep 22, 2023

wmutschl commented Sep 22, 2023

eli-schwartz commented Sep 22, 2023

wmutschl commented Sep 22, 2023 • edited Loading

eli-schwartz commented Sep 22, 2023

wmutschl commented Sep 22, 2023

wmutschl commented Sep 22, 2023

jpakkane commented Sep 22, 2023

wmutschl commented Sep 22, 2023

eli-schwartz commented Sep 22, 2023

mkoeppe commented Sep 26, 2023

fxcoudert commented Sep 26, 2023 • edited Loading

dimpase commented Sep 28, 2023

wmutschl commented Oct 10, 2023

eli-schwartz commented Oct 10, 2023

wmutschl commented Oct 10, 2023 • edited Loading

wmutschl commented Oct 10, 2023

wmutschl commented Oct 10, 2023 • edited Loading

dimpase commented Oct 10, 2023

wmutschl commented Oct 10, 2023

wmutschl commented Oct 11, 2023 • edited Loading

dimpase commented Oct 11, 2023

wmutschl commented Dec 20, 2023 • edited Loading

wmutschl commented Sep 22, 2023 •

edited

Loading

wmutschl commented Sep 22, 2023 •

edited

Loading

fxcoudert commented Sep 26, 2023 •

edited

Loading

wmutschl commented Oct 10, 2023 •

edited

Loading

wmutschl commented Oct 10, 2023 •

edited

Loading

wmutschl commented Oct 11, 2023 •

edited

Loading

wmutschl commented Dec 20, 2023 •

edited

Loading