Could not build wheels for XXX which use PEP 517 #1516

albireox · 2019-10-29T20:36:45Z

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.

OS version and name: macOS 10.15
Poetry version: 1.0.0b4

Issue

#11 provides a workaround for custom build systems (e.g., Cython extensions) that works well with poetry. However, when you try to do pip install . on a project that has a custom build.py the build process fails with

ERROR: Could not build wheels for seiya which use PEP 517 and cannot be installed directly

This means, among other things, that one cannot use ReadTheDocs to document the code. This happens with my code but I have also tested it with pendulum.

Is there a workaround for this?

The text was updated successfully, but these errors were encountered:

albireox · 2019-10-29T22:02:31Z

Doing some more Googling on this I came up with the solution described here. One can create a setup.py file such as

from distutils.core import setup

import numpy

from Cython.Build import cythonize
from setuptools.extension import Extension


global setup_kwargs


extensions = [
    Extension('seiya.cube.cubify',
              ['seiya/cube/cubify.pyx'],
              include_dirs=[numpy.get_include()])
]


setup_kwargs = {}
setup_kwargs.update({'ext_modules': cythonize(extensions, annotate=True)})


setup(**setup_kwargs)

and remove the need of using build.py. As far as I can see this works fine (even allows editable installs!). When you do poetry install it creates an UNKNOWN.egg-info directory with PKG-INFO in which all the fields are UNKNOWN. That does not seem to affect poetry but I'm not sure what will happen when you build a distribution (I have run poetry build and the results look ok, with the correct metadata, but I haven't tested it all the way by uploading it to PyPI and installing from a fresh environment).

IMHO poetry really needs to define a way to deal with extensions. There are many projects that depend on a custom build system and this kind of workarounds are getting old ...

albireox · 2019-12-18T08:36:37Z

@sdispater Any comments on this? I'm seeing problems with this workaround with a recent project that requires compilation of extensions and being able to install using pip install . and I cannot find a good solution.

flokno · 2020-03-31T13:58:23Z

+1 from me and probably related to #2214

I use numpy to compile a small fortran module. Installation works fine with poetry install, but with pip install ., pip complains that it does not find a setup.py. From what I can see this should be easy to fix since poetry install builds a temporary setup.py as far as I understand.

Resolves: python-poetry/poetry#1516

albireox · 2020-05-15T02:43:02Z

Thanks @abn for fixing this. However, I cannot completely make it work. If I add

[build-system]
build-backend = "poetry.core.masonry.api"
requires = ["poetry-core>=1.0.0a6"]

to my pyproject.toml it fails with

  Building wheel for sdss-flicamera (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/bin/python3.8 /Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/tmpzqz9ab7n
       cwd: /Users/albireo/Documents/Code/sdss5/flicamera
  Complete output (24 lines):
  Traceback (most recent call last):
    File "setup.py", line 2, in <module>
      from setuptools import setup
  ModuleNotFoundError: No module named 'setuptools'
  Traceback (most recent call last):
    File "/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 204, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/api.py", line 57, in build_wheel
      return unicode(WheelBuilder.make_in(poetry, Path(wheel_directory)))
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 56, in make_in
      wb.build()
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 82, in build
      self._build(zip_file)
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 101, in _build
      self._run_build_command(setup)
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 129, in _run_build_command
      subprocess.check_call(
    File "/Users/albireo/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 364, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/bin/python3.8', 'setup.py', 'build', '-b', 'build']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for sdss-flicamera
Failed to build sdss-flicamera
ERROR: Could not build wheels for sdss-flicamera which use PEP 517 and cannot be installed directly

However, if I do

[build-system]
build-backend = "poetry.core.masonry.api"
requires = ["poetry-core>=1.0.0a6", "setuptools>=46.0.0"]

Then it works. Is this expected and if so, should the build-system documentation be updated?

I'm running this on a virtual environment using pyenv with Python 3.8.2. Note that the virtual environment has setuptools installed but that doesn't seem to make a difference.

abn · 2020-05-15T05:42:43Z

@albireox what's the version of pip in your environment? Can you try updating it?

albireox · 2020-05-15T15:39:54Z

With the latest pip 20.1 the problem still happens (that's what I had been using originally).

albireox · 2020-05-15T19:59:54Z

Maybe related to #2276 (comment)

virtustate · 2020-05-22T19:40:01Z

Same problem working through https://www.pythoncheatsheet.org/blog/python-projects-with-poetry-and-vscode-part-1/ installing pendulum. Conda installs pendulum fine. I use Conda as a pip fallback for things like scipy on Windows.

abn · 2020-09-30T10:39:13Z

@albireox the issue is the isolated environment created by pip when building th wheel. #2666 might help mitigate this issue and so will #2826 since that will mean we rely on virtualenv everytime to build packages.

If he issue persists, please create a new issue, so we can track it and discuss it correctly. (#3001 maybe?).

albireox · 2020-09-30T17:05:19Z

@abn I tried this again, without the setuptools requires, and now it seems to work fine. If I see the issue pop up again I'll open a dedicated issue.

ag-hcoder · 2020-11-18T06:54:22Z

Any solution yet?

finswimmer · 2020-11-18T07:27:47Z

@ag-hcoder: I believe the problem here is setuptools itself. See #3153 (comment)

syahdeini · 2021-06-02T05:45:18Z

I use python 3.8.0, and this problem still occurs


    error: Command "clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/bzip2/include -I/usr/local/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/bzip2/include -I/usr/local/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/common -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath -Inumpy/core/include -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/include/numpy -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/distutils/include -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/Users/XXX/Library/Caches/pypoetry/virtualenvs/aep-T5QCnjDx-py3.8/include -I/Users/XXX/.pyenv/versions/3.8.0/include/python3.8 -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/common -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/npymath -c build/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath/loops_unary_fp.dispatch.c -o build/temp.macosx-11.1-x86_64-3.8/build/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath/loops_unary_fp.dispatch.o -MMD -MF build/temp.macosx-11.1-x86_64-3.8/build/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath/loops_unary_fp.dispatch.o.d -msse -msse2 -msse3" failed with exit status 1
    ----------------------------------------
    ERROR: Failed building wheel for numpy
  Failed to build numpy
  ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly
  WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
  You should consider upgrading via the '/Users/XXX/Library/Caches/pypoetry/virtualenvs/aep-T5QCnjDx-py3.8/bin/python -m pip install --upgrade pip' command.

jordi-crespo · 2021-08-24T08:48:03Z

Any updates on this error?

github-actions · 2024-03-02T18:06:00Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

albireox added the kind/bug Something isn't working as expected label Oct 29, 2019

albireox mentioned this issue Oct 29, 2019

PEP-517 and pip install --editable with pip 19 #1279

Closed

2 tasks

abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020

Fix PEP-517 issues for projects using build scripts

10d7f39

Resolves: python-poetry/poetry#1516

abn mentioned this issue Mar 31, 2020

Fix PEP-517 issues for projects using build scripts python-poetry/poetry-core#12

Merged

abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020

Fix PEP-517 issues for projects using build scripts

a9edfd7

Resolves: python-poetry/poetry#1516

abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020

Fix PEP-517 issues for projects using build scripts

0328610

Resolves: python-poetry/poetry#1516

abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020

Fix PEP-517 issues for projects using build scripts

808ca5c

Resolves: python-poetry/poetry#1516

abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020

Fix PEP-517 issues for projects using build scripts

b8acbaf

Resolves: python-poetry/poetry#1516

abn added a commit to abn/poetry-core that referenced this issue Apr 7, 2020

Fix PEP-517 issues for projects using build scripts

93da0d3

Resolves: python-poetry/poetry#1516

abn added a commit to abn/poetry-core that referenced this issue Apr 9, 2020

Fix PEP-517 issues for projects using build scripts

f5d92c4

Resolves: python-poetry/poetry#1516

abn mentioned this issue Apr 10, 2020

Add support for non pure python projects #11

Closed

abn closed this as completed in python-poetry/poetry-core#12 Apr 12, 2020

abn added a commit to python-poetry/poetry-core that referenced this issue Apr 12, 2020

Fix PEP-517 issues for projects using build scripts

6aadea9

Resolves: python-poetry/poetry#1516

aronchick mentioned this issue Apr 8, 2021

pip installs numpy but poetry does not #3894

Closed

3 tasks

FedericoStra mentioned this issue Feb 9, 2022

Add example pyproject.toml using poetry instead of setuptools FedericoStra/cython-package-example#1

Open

github-actions bot locked as resolved and limited conversation to collaborators Mar 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not build wheels for XXX which use PEP 517 #1516

Could not build wheels for XXX which use PEP 517 #1516

albireox commented Oct 29, 2019 •

edited

Loading

albireox commented Oct 29, 2019

albireox commented Dec 18, 2019

flokno commented Mar 31, 2020 •

edited

Loading

albireox commented May 15, 2020 •

edited

Loading

abn commented May 15, 2020

albireox commented May 15, 2020

albireox commented May 15, 2020

virtustate commented May 22, 2020

abn commented Sep 30, 2020

albireox commented Sep 30, 2020

ag-hcoder commented Nov 18, 2020

finswimmer commented Nov 18, 2020

syahdeini commented Jun 2, 2021

jordi-crespo commented Aug 24, 2021

github-actions bot commented Mar 2, 2024

Could not build wheels for XXX which use PEP 517 #1516

Could not build wheels for XXX which use PEP 517 #1516

Comments

albireox commented Oct 29, 2019 • edited Loading

Issue

albireox commented Oct 29, 2019

albireox commented Dec 18, 2019

flokno commented Mar 31, 2020 • edited Loading

albireox commented May 15, 2020 • edited Loading

abn commented May 15, 2020

albireox commented May 15, 2020

albireox commented May 15, 2020

virtustate commented May 22, 2020

abn commented Sep 30, 2020

albireox commented Sep 30, 2020

ag-hcoder commented Nov 18, 2020

finswimmer commented Nov 18, 2020

syahdeini commented Jun 2, 2021

jordi-crespo commented Aug 24, 2021

github-actions bot commented Mar 2, 2024

albireox commented Oct 29, 2019 •

edited

Loading

flokno commented Mar 31, 2020 •

edited

Loading

albireox commented May 15, 2020 •

edited

Loading