Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Building on macos-14 fails given tool problems #300

Open
matthewfeickert opened this issue Jun 24, 2024 · 3 comments
Open

Building on macos-14 fails given tool problems #300

matthewfeickert opened this issue Jun 24, 2024 · 3 comments

Comments

@matthewfeickert
Copy link
Member

As in conda-forge/fastjet-feedstock#10, trying to build with macos-latest (macos-14) and auto64 fails in the building wheels stage with

Details: 
  Building wheels for collected packages: fastjet
    Created temporary directory: /private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-wheel-b__xkee5
    Destination directory: /private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-wheel-b__xkee5
    Building wheel for fastjet (pyproject.toml): started
    Running command Building wheel for fastjet (pyproject.toml)
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.macosx-11.0-arm64-cpython-312
    creating build/lib.macosx-11.0-arm64-cpython-312/fastjet
    copying src/fastjet/version.py -> build/lib.macosx-11.0-arm64-cpython-312/fastjet
    copying src/fastjet/_singleevent.py -> build/lib.macosx-11.0-arm64-cpython-312/fastjet
    copying src/fastjet/__init__.py -> build/lib.macosx-11.0-arm64-cpython-312/fastjet
    copying src/fastjet/_pyjet.py -> build/lib.macosx-11.0-arm64-cpython-312/fastjet
    copying src/fastjet/_multievent.py -> build/lib.macosx-11.0-arm64-cpython-312/fastjet
    copying src/fastjet/_generalevent.py -> build/lib.macosx-11.0-arm64-cpython-312/fastjet
    copying src/fastjet/_utils.py -> build/lib.macosx-11.0-arm64-cpython-312/fastjet
    running egg_info
    writing src/fastjet.egg-info/PKG-INFO
    writing dependency_links to src/fastjet.egg-info/dependency_links.txt
    writing requirements to src/fastjet.egg-info/requires.txt
    writing top-level names to src/fastjet.egg-info/top_level.txt
    reading manifest template 'MANIFEST.in'
    adding license file 'LICENSE'
    writing manifest file 'src/fastjet.egg-info/SOURCES.txt'
    running build_ext
    patching file 'src/ClusterSequence.cc'
    Detected Mac OSX
  
    On Mac OSX, ./autogen.sh requires the GNU libtool and libtoolize scripts,
    renamed glibtol and glibtoolize respectively by Apple to avoid
    conflict with Apple-provided libtool.
  
    They should be already installed, together with Apple developer tools,
    in /usr/bin. If you see this message, they are not.
  
    The GNU version can be retrieved from http://ftp.gnu.org/gnu/libtool/.
    If they are (re)installed from sources in a different location,
    autogen.sh should then be modified accordingly.
    make: *** No targets specified and no makefile found.  Stop.
    Traceback (most recent call last):
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
        main()
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
        return _build_backend().build_wheel(wheel_directory, config_settings,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 415, in build_wheel
        return self._build_with_temp_dir(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
        self.run_setup()
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
        exec(code, locals())
      File "<string>", line 192, in <module>
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
        return distutils.core.setup(**attrs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
        return run_commands(dist)
               ^^^^^^^^^^^^^^^^^^
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
        dist.run_commands()
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
        self.run_command(cmd)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command
        super().run_command(command)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
        cmd_obj.run()
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/command/bdist_wheel.py", line 373, in run
        self.run_command("build")
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
        self.distribution.run_command(command)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command
        super().run_command(command)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
        cmd_obj.run()
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 132, in run
        self.run_command(cmd_name)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
        self.distribution.run_command(command)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command
        super().run_command(command)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
        cmd_obj.run()
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 93, in run
        _build_ext.run(self)
      File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/pip-build-env-_kwn1o6c/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
        self.build_extensions()
      File "<string>", line 109, in build_extensions
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 571, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['make', '-j']' returned non-zero exit status 2.
    error: subprocess-exited-with-error
    
    × Building wheel for fastjet (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: /private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/bin/python /private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/tmpzncpiu4y
    cwd: /Users/runner/work/fastjet/fastjet
    Building wheel for fastjet (pyproject.toml): finished with status 'error'
    ERROR: Failed building wheel for fastjet
  Failed to build fastjet
  ERROR: Failed to build one or more wheels
  Exception information:
  Traceback (most recent call last):
    File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
      status = run_func(*args)
               ^^^^^^^^^^^^^^^
    File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
      return func(self, options, args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/b3/2xm02wpd21qgrpkck5q1c6k40000gn/T/cibw-run-xlbfqb8i/cp312-macosx_arm64/build/venv/lib/python3.12/site-packages/pip/_internal/commands/wheel.py", line 181, in run
      raise CommandError("Failed to build one or more wheels")
  pip._internal.exceptions.CommandError: Failed to build one or more wheels

with the relevant

    On Mac OSX, ./autogen.sh requires the GNU libtool and libtoolize scripts,
    renamed glibtol and glibtoolize respectively by Apple to avoid
    conflict with Apple-provided libtool.
  
    They should be already installed, together with Apple developer tools,
    in /usr/bin. If you see this message, they are not.
  
    The GNU version can be retrieved from http://ftp.gnu.org/gnu/libtool/.
    If they are (re)installed from sources in a different location,
    autogen.sh should then be modified accordingly.

which comes from the fastjet C++ source code's autogen.sh:

# change executable names when running on Macs
# if test x`uname` = xDarwin; then
if [ `uname` == "Darwin" ]; then
  echo "Detected Mac OSX"
  if [ x$(which glibtool) == "x" ]; then    
    echo ""
    echo "On Mac OSX, $0 requires the GNU libtool and libtoolize scripts,"
    echo "renamed glibtol and glibtoolize respectively by Apple to avoid"
    echo "conflict with Apple-provided libtool."
    echo ""
    echo "They should be already installed, together with Apple developer tools,"
    echo "in /usr/bin. If you see this message, they are not."
    echo ""
    echo "The GNU version can be retrieved from http://ftp.gnu.org/gnu/libtool/."
    echo "If they are (re)installed from sources in a different location,"
    echo "autogen.sh should then be modified accordingly."
    exit
  fi
  LIBTOOL=glibtool
  LIBTOOLIZE=glibtoolize
fi

The best thing would be to fix this up upstream, but this also need to get fixed in the releases we have out now with patching the wheel buidling.
@matthewfeickert
Copy link
Member Author

I think this is to some degree some of the pains of PR #4 again in a different format. Moving the underlying build system off of autogen would help a lot I think.

@matthewfeickert
Copy link
Member Author

matthewfeickert commented Aug 14, 2024 •
edited
Loading

I emailed the FastJet developers on this back on 2024-06-24 and Gavin responded quickly with the following:

Do you have working builds of FastJet behaving as expected for Apple silicon macOS?

Yes, we do, regularly (and I just double checked a fresh clone on my M2). But we haven’t tried builds where libtool comes from some other source — that range of variation goes beyond what we would normally explore.

I can see that the autogen.sh script could fail in that context. Mostly what that script is doing is checking that all tools are available, and clearly that’s not diagnosing things correctly for the situation you have in mind.

An alternative to autogen.sh is to run

cd plugins/SISCone/siscone && autoreconf -i && cd ../../.. && autoreconf -i
./configure [with your usual args]

That would simply assume all tools are correctly available. Might that offer an adequate workaround?

Best,
Gavin

I haven't tried this yet on my Apple silicon machine, but this might be something we should try.

@matthewfeickert matthewfeickert mentioned this issue Sep 15, 2024
Open
@matthewfeickert
Copy link
Member Author

Related: #193 (comment)

@matthewfeickert matthewfeickert mentioned this issue Sep 16, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@matthewfeickert