[BUG] breaks numpy in cross-compiling environment #3549

th0ma7 · 2022-08-20T12:52:19Z

setuptools version

setuptools >= 64.x

Python version

python 3.10.x

OS

Linux

Additional environment information

Environment is using the Synology DSM toolchain for cross-compiling for multiple archs (armv5, v7, v8, ppc, x64, i686).

Description

While preparing python 3.10.6 update I've took care to update all build requirements, including setuptools.
It hapens that it made numpy to fail to build starting with setuptools >= 64.x (also fails on 65.x).
Full build logs can be found here: https://github.com/SynoCommunity/spksrc/runs/7917533394?check_suite_focus=true
Associated SynoCommunity PR: SynoCommunity/spksrc#5395

Expected behavior

numpy to build properly (like all other regular wheel being cross-compiled)

How to Reproduce

Can be reproduced easily using the SynoCommunity (spksrc) cross-compiling environment.

Output

Output is mostly hidden under pip call:

===>  _PYTHON_HOST_PLATFORM=x86_64-pc-linux-gnu /home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/cross/bin/pip wheel --disable-pip-version-check --no-binary :all: --cache-dir /home/spksrc/py310.6/spksrc/spk/bazarr/../../distrib/pip --no-deps --wheel-dir /home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/wheelhouse --no-build-isolation greenlet==1.1.2
Collecting greenlet==1.1.2
  Using cached greenlet-1.1.2.tar.gz (91 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: greenlet
  Building wheel for greenlet (setup.py): started
  Building wheel for greenlet (setup.py): finished with status 'done'
  Created wheel for greenlet: filename=greenlet-1.1.2-cp310-cp310-linux_x86_64.whl size=142558 sha256=d3ac376cd9b322f76ee60208a103be9f52d989d3b3a8a1185fb9857c4a28de25
  Stored in directory: /home/spksrc/py310.6/spksrc/distrib/pip/wheels/51/01/b7/16995feb30f4eed2504aed2260a4058a1164472c9be5ab44b1
Successfully built greenlet
make[3]: Leaving directory '/home/spksrc/py310.6/spksrc/spk/bazarr'
===>  [numpy]           
make[3]: Entering directory '/home/spksrc/py310.6/spksrc/spk/bazarr'
===>  _PYTHON_HOST_PLATFORM=x86_64-pc-linux-gnu /home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/cross/bin/pip wheel --disable-pip-version-check --no-binary :all: --cache-dir /home/spksrc/py310.6/spksrc/spk/bazarr/../../distrib/pip --no-deps --wheel-dir /home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/wheelhouse --no-build-isolation numpy==1.22.4
Collecting numpy==1.22.4
  Using cached numpy-1.22.4.zip (11.5 MB)
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [75 lines of output]
      Running from numpy source directory.
      running dist_info
      running build_src
      INFO: build_src
      creating /tmp/pip-modern-metadata-8a5sb6si/numpy.egg-info
      writing /tmp/pip-modern-metadata-8a5sb6si/numpy.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-modern-metadata-8a5sb6si/numpy.egg-info/dependency_links.txt
      writing entry points to /tmp/pip-modern-metadata-8a5sb6si/numpy.egg-info/entry_points.txt
      writing top-level names to /tmp/pip-modern-metadata-8a5sb6si/numpy.egg-info/top_level.txt
      writing manifest file '/tmp/pip-modern-metadata-8a5sb6si/numpy.egg-info/SOURCES.txt'
      /home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/egg_info.py:643: SetuptoolsDeprecationWarning: Custom 'build_py' does not implement 'get_data_files_without_manifest'.
      Please extend command classes from setuptools instead of distutils.
        warnings.warn(
      INFO: unifing config_cc, config, build_clib, build_ext, build commands --compiler options
      INFO: unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
      Traceback (most recent call last):
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/build_meta.py", line 369, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/build_meta.py", line 474, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/build_meta.py", line 334, in run_setup
          exec(code, locals())
        File "<string>", line 461, in <module>
        File "<string>", line 453, in setup_package
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
          self.run_command(cmd)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/dist_info.py", line 96, in run
          self.egg_info.run()
        File "/tmp/pip-wheel-zx645048/numpy_62d5a1a8355144f781d4c843ede91f36/numpy/distutils/command/egg_info.py", line 25, in run
          _egg_info.run(self)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 308, in run
          self.find_sources()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 316, in find_sources
          mm.run()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 560, in run
          self.add_defaults()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 597, in add_defaults
          sdist.add_defaults(self)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/sdist.py", line 107, in add_defaults
          self._add_defaults_build_sub_commands()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/sdist.py", line 127, in _add_defaults_build_sub_commands
          self.filelist.extend(chain.from_iterable(files))
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 503, in extend
          self.files.extend(filter(self._safe_path, paths))
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/sdist.py", line 126, in <genexpr>
          files = (c.get_source_files() for c in cmds if hasattr(c, "get_source_files"))
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/command/sdist.py", line 125, in <genexpr>
          cmds = (self.get_finalized_command(c) for c in missing_cmds)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 306, in get_finalized_command
          cmd_obj.ensure_finalized()
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 109, in ensure_finalized
          self.finalize_options()
        File "/tmp/pip-wheel-zx645048/numpy_62d5a1a8355144f781d4c843ede91f36/numpy/distutils/command/config_compiler.py", line 69, in finalize_options
          v = getattr(c, a)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 157, in __getattr__
          return _build_ext.build_ext.__getattr__(self, name)
        File "/home/spksrc/py310.6/spksrc/spk/bazarr/work-x64-6.1/crossenv/build/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 105, in __getattr__
          raise AttributeError(attr)
      AttributeError: fcompiler. Did you mean: 'compiler'?
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
make[3]: *** [../../mk/spksrc.wheel.mk:139: cross-compile-wheel-numpy] Error 1
make[3]: Leaving directory '/home/spksrc/py310.6/spksrc/spk/bazarr'
make[2]: *** [../../mk/spksrc.wheel.mk:81: build_wheel_target] Error 1
make[2]: Leaving directory '/home/spksrc/py310.6/spksrc/spk/bazarr'
make[1]: *** [../../mk/spksrc.spk.mk:603: build-arch-x64-6.1] Error 1
make[1]: Leaving directory '/home/spksrc/py310.6/spksrc/spk/bazarr'
make: *** [../../mk/spksrc.spk.mk:597: arch-x64-6.1] Error 2

The text was updated successfully, but these errors were encountered:

boegel · 2022-12-08T13:35:07Z

@th0ma7 I'm hitting this same problem when installing numpy from source (no cross-compiling, but fine).

Did you make any progress on this?

freemedom · 2022-12-11T14:51:24Z

@th0ma7 I'm hitting this same problem when installing numpy from source (no cross-compiling, but fine).

Did you make any progress on this?

Try downgrading the setuptools version?

th0ma7 · 2022-12-11T14:57:16Z

Downgrading setuptools is all i could find for now

mboisson · 2023-02-22T14:08:33Z

Hitting the same issue. Need to downgrade setuptools to <64 on our system.

jeanbez · 2023-04-26T17:11:03Z

Hitting the same issue in MacOS, the solution to downgrade setuptools to <64 worked.

mboisson · 2023-08-30T14:16:40Z

This seems to have disappeared for me with recent versions of setuptools and/or numpy

th0ma7 · 2023-09-03T12:14:53Z

@mboisson Thnx! indeed it did and can confirm SynoCommunity/spksrc#5866

mboisson · 2023-09-05T12:52:37Z

To be more specific, after more testing, setuptools>63 is broken to build numpy<1.24, but numpy>=1.24 builds fine with the latest setuptools.

th0ma7 · 2023-09-07T01:51:12Z

I concur, you are absolutely and had to revert back to older setuptools due to a need for an older numpy. But indeed now working with 1.24+

ReluXingZeng · 2023-12-04T04:03:38Z

I am having this issue with setuptools==65.5.1 as well as numpy==1.26.0, so I am not sure if the newest numpy (as of writing, 1.26.2) has already fixed for setuptools.

update: upgrade to newest setuptools setuptools==69.0.2 and numpy==1.26.2 also does not work for me. Ubuntu 22 is my operating system so unfortunately I can't downgrade to numpy<1.24

th0ma7 added bug Needs Triage Issues that need to be evaluated for severity and status. labels Aug 20, 2022

th0ma7 mentioned this issue Aug 20, 2022

BUG: numpy fails to cross-compiling with setuptools >= 64.x numpy/numpy#22157

Closed

starnight mentioned this issue Sep 26, 2022

The numpy builds failed with org.gnome.Sdk//43 flathub/org.laptop.TurtleArtActivity#8

Open

bbhtt mentioned this issue Sep 29, 2022

Runtime 43 flathub/com.endlessm.photos#9

Merged

th0ma7 mentioned this issue Dec 23, 2022

py310-py311: Update to latest wheel and rework numpy management SynoCommunity/spksrc#5531

Merged

10 tasks

This was referenced Feb 20, 2023

pip install fails (error: implicit declaration of function 'H5close' is invalid in C99) NREL/PyDSS#109

Closed

Support for Apple's arm64 architecture (osx-arm64) PyTables/PyTables#832

Closed

pasdeloup added a commit to pasdeloup/chaquopy that referenced this issue Aug 17, 2023

Downgrade setuptools to 63 (cf pypa/setuptools#3549)

53ca3df

pasdeloup mentioned this issue Aug 17, 2023

Fix package: numpy chaquo/chaquopy#935

Merged

th0ma7 closed this as completed Sep 3, 2023

th0ma7 mentioned this issue Sep 3, 2023

py311-310: Update to latest version SynoCommunity/spksrc#5866

Merged

10 tasks

th0ma7 reopened this Sep 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] breaks numpy in cross-compiling environment #3549

[BUG] breaks numpy in cross-compiling environment #3549

th0ma7 commented Aug 20, 2022

boegel commented Dec 8, 2022

freemedom commented Dec 11, 2022

th0ma7 commented Dec 11, 2022

mboisson commented Feb 22, 2023

jeanbez commented Apr 26, 2023

mboisson commented Aug 30, 2023

th0ma7 commented Sep 3, 2023

mboisson commented Sep 5, 2023

th0ma7 commented Sep 7, 2023

ReluXingZeng commented Dec 4, 2023 •

edited

Loading

[BUG] breaks numpy in cross-compiling environment #3549

[BUG] breaks numpy in cross-compiling environment #3549

Comments

th0ma7 commented Aug 20, 2022

setuptools version

Python version

OS

Additional environment information

Description

Expected behavior

How to Reproduce

Output

boegel commented Dec 8, 2022

freemedom commented Dec 11, 2022

th0ma7 commented Dec 11, 2022

mboisson commented Feb 22, 2023

jeanbez commented Apr 26, 2023

mboisson commented Aug 30, 2023

th0ma7 commented Sep 3, 2023

mboisson commented Sep 5, 2023

th0ma7 commented Sep 7, 2023

ReluXingZeng commented Dec 4, 2023 • edited Loading

ReluXingZeng commented Dec 4, 2023 •

edited

Loading