Poetry not using wheels and trying to build from source on Big Sur

[damienrj]

• [ x] I am on the latest Poetry version.
• [ x] I have searched the issues of this repo and believe that this is not a duplicate.
• [ x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Big Sur 11.0.1
• Poetry version: 1.1.4
• x86

Issue

When I try to install packages poetry trys to install from source, which can lead to various build failuers. But if I use pip 20.3 or greater I can download the wheel.

 pip install numpy==1.18.5
Looking in indexes: https://pypi.org/simple
Collecting numpy==1.18.5
  Downloading numpy-1.18.5-cp37-cp37m-macosx_10_9_x86_64.whl (15.1 MB)
     |████████████████████████████████| 15.1 MB 5.8 MB/s
Installing collected packages: numpy
Successfully installed numpy-1.18.5

will work fine for example. My guess is the version of pip poetry install uses is too old.

[damienrj]

This will be solved by https://github.com/python-poetry/poetry-core/pull/125/files

[damienrj]

A workaround for now

RUN poetry export --without-hashes --dev -f requirements.txt > requirements.txt && \
    pip install -r requirements.txt

And only use poetry lock or poetry add --lock since the lock file generated by poetry right now won't even install in a linux container.

[NathanHowell]

as @jtc42 mentioned in #3196, running poetry config experimental.new-installer false also fixes it for me

[damienrj]

I haven't actually been able to install my whole lock file yet while it works fine after converting to requirements.txt
Get failures as it still tries to build packages like cryptography (3.4.5) or grpcio (1.35.0) that work fine via pip. Only solution so far has been converting to requirements.txt.

I also tried clearing the cache and making a new lock file but no difference.

  Processing /Users/damien/Library/Caches/pypoetry/artifacts/98/8a/2a/6a96fba41c057cbf67cb4d48fd53937c817c2f6e7b3c809da18b28c18d/cryptography-3.4.5.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Building wheels for collected packages: cryptography
    Building wheel for cryptography (PEP 517): started
    Building wheel for cryptography (PEP 517): finished with status 'error'

Both install fine with pip

❯ pip install cryptography==3.4.5 grpcio==1.35.0
Looking in indexes: https://pypi.org/simple, https://nexus3.sqcorp.co/repository/pypi-square-general/simple
Collecting cryptography==3.4.5
  Downloading cryptography-3.4.5-cp36-abi3-macosx_10_10_x86_64.whl (2.0 MB)
     |████████████████████████████████| 2.0 MB 10.1 MB/s
Requirement already satisfied: cffi>=1.12 in /Users/damien/.pyenv/versions/3.7.4/envs/test/lib/python3.7/site-packages (from cryptography==3.4.5) (1.14.5)
Collecting grpcio==1.35.0
  Using cached grpcio-1.35.0-cp37-cp37m-macosx_10_10_x86_64.whl (3.7 MB)
Requirement already satisfied: six>=1.5.2 in /Users/damien/.pyenv/versions/3.7.4/envs/test/lib/python3.7/site-packages (from grpcio==1.35.0) (1.15.0)
Requirement already satisfied: pycparser in /Users/damien/.pyenv/versions/3.7.4/envs/test/lib/python3.7/site-packages (from cffi>=1.12->cryptography==3.4.5) (2.20)
Installing collected packages: grpcio, cryptography
Successfully installed cryptography-3.4.5 grpcio-1.35.0

[ulope]

I had the same issue with cryptography.
Upgrading pip to 21.0.1 (via poetry run pip install -U pip) solved the problem.

This now leads me to the question how the pip version used is determined? Is it based on the version installed in the base Python or is there a specific version pinned / bundled in poetry?

[phillipuniverse]

@ulope thanks for the advice, that fixed my problem with numpy and pandas. Looks like we are still building a wheel for fbprophet but upgrading pip reduced my overall poetry install time by ~15 minutes!

[qoire]

This advice fixed it for me (atleast when installing pandas). Not sure about root cause.

[omBratteng]

Both #3591 (comment) and #3591 (comment) worked for me when trying to install uamqp v1.4.0.

Though #3591 (comment) is a better solution until fixed, as it requires less commands to be run.

[andrewsonin]

poetry config experimental.new-installer false doesn't work for me. Poetry still downloads tar.gz sources instead of installing from wheels (that are available).

[Kavan72]

I can confirm that I'm getting the same error, poetry downloads tar.gz sources instead of installing from wheels which is available in PyPI. here is the log of error https://pastebin.com/mm1EMKF4
I'm using poetry 1.1.13

[alon1samuel]

I'm also having the same problem with poetry 1.1.13 with Kaleido 0.2.1 post -

Managed to solve it with (workaround) -
poetry add kaleido@0.2.1

If it helps anyone

[abn]

@alon1samuel the issue here is that the post1 release only distributes arm7 wheels for some reason.

$ curl -sL https://pypi.org/pypi/kaleido/json | jq -r '.releases["0.2.1.post1"][] | .filename'
kaleido-0.2.1.post1-py2.py3-none-manylinux2014_armv7l.whl
$ curl -sL https://pypi.org/pypi/kaleido/json | jq -r '.releases["0.2.1"][] | .filename'
kaleido-0.2.1-py2.py3-none-macosx_10_11_x86_64.whl
kaleido-0.2.1-py2.py3-none-macosx_11_0_arm64.whl
kaleido-0.2.1-py2.py3-none-manylinux1_x86_64.whl
kaleido-0.2.1-py2.py3-none-manylinux2014_aarch64.whl
kaleido-0.2.1-py2.py3-none-win32.whl
kaleido-0.2.1-py2.py3-none-win_amd64.whl

When poetry does poetry add package it finds the latest version of the package and sets that as the reqirement. And since this is 0.2.1.post1, and the resolver is satisfied, when it comes time to install the package to your environment, we do not find a platform wheel compatible nor an sdist we can build from. Hence the failure.

[choucavalier]

I'm still having this issue. Poetry is building my packages from source and it takes ages

[neersighted]

I am closing this for now as the initial issue is solved, and the people chiming in have a variety of issues with no good description or reproduction.

@tgy, if you are having issues, please open a Discussion or join Discord for advice. If you have done extensive testing/have enough knowledge of Poetry to think there may be a bug, feel free to open an issue -- however, the issue is most likely usage or local configuration, as Poetry has good support for macOS wheels on modern versions of pip.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.