pip 19.1 breaks Poetry

[sfermigier]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: OSX 10.13
• Poetry version: 0.12.2
• Link of a Gist with the contents of your pyproject.toml file: https://github.com/abilian/abilian-core/blob/master/pyproject.toml

Issue

After upgrading pip to version 19.1 (relased 5 hours ago), I can't install my projects using poetry anymore:

abilian-core ❯ poetry install
Installing dependencies from lock file

Nothing to install or update

  - Installing abilian-core (0.11.0-dev)
A setup.py file already exists. Using it.

[EnvCommandError]
Command ['/Users/fermigier/.virtualenvs/abilian-core/bin/pip', 'install', '-e', '/Users/fermigier/projects/abilian-core',
 '--no-deps'] errored with the following output:
Obtaining file:///Users/fermigier/projects/abilian-core
ERROR: Error installing 'file:///Users/fermigier/projects/abilian-core': editable mode is not supported for pyproject.toml-style projects. This project is being processed as pyproject.toml-style because it has a pyproject.toml file with a "build-backend" key in the "build_system" value. See PEP 517 for the relevant specification.

install [--no-dev] [--dry-run] [-E|--extras EXTRAS] [--develop DEVELOP]

(Downgrading pip to 19.0.3 restores the usual behavior).

Not sure if it's a pip or a poetry issue, but I'm posting it here so that poetry users are aware of the issue.

[dbanty]

This is intentional Pip behavior. They implemented it knowing full well it would break Poetry (see pypa/pip#6314).

Basically, if you have a build-system defined in your pyproject.toml, Pip won’t let you install in editable mode, because technically that isn’t supported by PEP-517.

There’s also a Poetry issue #826 that addresses it, though the title should probably be updated to be easier to find.

Commenting out the build system piece of your pyproject.toml or using older versions of Pip are current workarounds. Note that the underlying issue of conflicting backend dependencies will happen even on Pip 19.0.3

[vlcinsky]

In our case automated tests of python package using tox failed as tox is pulling the latest pip.

Workaround is using tox-pip-version plugin allowing to declare pip to be installed into tox created virtualenv.

In our case fixing the pip to version 19.0.3 fixed the problem (so our experience differers from the one of @dbanty ).

[pappasam]

I have an open PR that addresses this issue. Please give it some 🎉 , maybe someone will notice it then

[m-aciek]

Fixed in 0.12.13.

[ape364]

still facing this issue
travis log
how to fix this?

[abn]

@ape364 the issue right now looks like is caused due to the python get-poetry.py --preview -y step installing the latest available preview version 1.0.0a2. Which does not include the recent fix for this issue. Will have to wait for a merge into develop and new pre-release.

[vlcinsky]

@ape364 CI scripts shall have deterministic environment to run (as much as is feasible). For this reason we run our CI jobs for python packages (in GitLab) using docker image with pre-installed tox, poetry and devpi commands.

Anyway, even with this setup we have failed as tox pulled the latest pip which broke the job. This we have fixed adding tox-pip-version tox plugin allowing to fix pip to known version.

[ape364]

but I see correct version in the log:

$ pip --version
pip 19.0.3 from /home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/pip (python 3.7)

As I understand, 19.1 breaks Poetry, but 19.0.3 should work 😞

[vlcinsky]

@ape364 I think, you are right that pip 19.0.3 shall not break poetry (at least this is what we experienced with poetry==12.12).

Anyway, I have noticed someone reporting that even pip 19.0.3 can break it. I thing it is wrong and reacted to that, but I got no feedback. (sorry, I cannot find the issue now, I think it was in poetry but could be also in pip.)

[pappasam]

@vlcinsky pip 19.0.3 broke if you used a private package index in your pyproject.toml. That said, the latest poetry release fixes this issue by temporarily deleting pyproject.toml, avoiding pip's pep 517 quirks. See commit 2ed53be

@sfermigier can you verify that the latest stable version of poetry resolves this issue and close?

[vlcinsky]

@pappasam thanks for pip 19.0.3 clarification. Private index is important use case for us. We had another problem (installing from it, even poetry 12:14 does not installs) and we ended up using version 1.0.0a2.

[rafalkrupinski]

running poetry 12.14, still experiencing the issue

[dsevero]

Worked fine with python==3.7.3, poetry==0.12.14 and pip==19.1

[riverfr0zen]

Hmm, python==3.7.3, poetry==0.12.14 and pip==19.1 didn't work for me. Still had to downgrade to pip==19.0.3. Is there something particular with python 3.7.3 that would make it work, or was that just informational?

Still fails with

...editable mode is not supported for pyproject.toml-style projects. This project is being   
processed as pyproject.toml-style because it has a pyproject.toml file with a "build-backend" key in the  
 "build_system" value. See PEP 517 for the relevant specification.     

[butla]

For me works with setup reported by @dsevero, doesn't work if I have poetry==0.12.15.

[rafalkrupinski]

python 3.7.2
pip 19.1 broken, 19.0.3 works

It's a bit weird that works for some and doesn't for others. Could it be some other tool or library causing the problems, like setuptools?

[riverfr0zen]

I'm not sure of what's going on with the internals here, but this may provide some insight (or further perplexity):

So, I have a repo 'Y' on Github that uses regular virtualenv + pip 19.1.1. This repo in turn requires (through a requirements.txt file) another Github repo 'X' that uses poetry & pyproject.toml for its packaging.

Cloning repo Y and running pip (19.1.1) install within the virtualenv succesfully installs the repo X package.

However, I have another project, Z, which is also using poetry for package management and also requires repo X (an entry such as this in pyproject.toml: graphene-mongo-extras = {git = "https://github.com/riverfr0zen/graphene-mongo-extras.git"}). When running poetry install or poetry update with project Z, I get the error described in this issue, and only downgrading to 19.0.3 as mentioned above fixes it.

Hope that made sense.

[vlcinsky]

@riverfr0zen what version of poetry are you using? Did the problem hapṕen when using poetry version 0.12.15?

Another question: if you modify project Z and remove the dependency on project Y, is the problem still present? I guess, the problem will still be there.

My understanding of your problems is as follows:

• poetry has problems installing project in development mode (e.g. by poetry install or poetry update) when using pip version 19.1 (I am not sure about 19.1.1 which might revert the changes causing the problem)

• your project X is not suffering from this issue as it is installing (possibly in development mode) base packages via pip which does it on it's own way. Dependency on project Y is fine, pip will read declared builder for that project and install it properly.

• your project Z is primarily based on poetry and here you attempt (using poetry install) the installation of it in development mode. This shall fail, if you have pip version 19.1. The latest poetry 0.12.15 shall fix that.

In any case, make sure you report proper version of poetry and pip you are using.

[riverfr0zen]

Ok, somehow I missed upgrading to 0.12.15.

Thanks for your reply. After upgrading, I cleared out the old virtualenv, cloned 'project Z', and tried to install and was successful. I did have to upgrade poetry's pip, but I'm not sure if that is because of something in my setup. Anyway, here were my steps in case someone finds it helpful.

irf@metroplex:~/tmp/hippo-appserver$ git checkout develop
Branch 'develop' set up to track remote branch 'develop' from 'origin'.
Switched to a new branch 'develop'
irf@metroplex:~/tmp/hippo-appserver$ pyenv local 3.6.7
irf@metroplex:~/tmp/hippo-appserver$ which python
/home/irf/.pyenv/shims/python
irf@metroplex:~/tmp/hippo-appserver$ which pip
/home/irf/.pyenv/shims/pip
irf@metroplex:~/tmp/hippo-appserver$ pip --version
pip 19.1.1 from /home/irf/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip (python 3.6)
irf@metroplex:~/tmp/hippo-appserver$ poetry run pip --version
Creating virtualenv hippo-appserver-py3.6 in /home/irf/.cache/pypoetry/virtualenvs
pip 10.0.1 from /home/irf/.cache/pypoetry/virtualenvs/hippo-appserver-py3.6/lib/python3.6/site-packages/pip (python 3.6)
irf@metroplex:~/tmp/hippo-appserver$ poetry run pip install --upgrade pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/5c/e0/be401c003291b56efc55aeba6a80ab790d3d4cece2778288d65323009420/pip-19.1.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-19.1.1
irf@metroplex:~/tmp/hippo-appserver$ poetry install 

The install was successful, including the git repo ('project Y').

[Serkan-devel]

Currently I can't use some packages because they require a higher pip version to be installed, which breaks poetry

[vlcinsky]

@Serkan-devel if you provide more specific details (which packages, what pip version is required, and mainly - what poetry version you are using), it shall be simpler to fix it.

Namely: do you youse poetry==0.12.15?

[Serkan-devel]

I'm using poetry 0.12.15 and tried to install cryptography==1.9, however I got a very long error message with this at the end

You are using pip version 19.0.3, however version 19.1.1 is available.                                                                                                                 
You should consider upgrading via the 'pip install --upgrade pip' command.

[vlcinsky]

@Serkan-devel the message (which is actually a warning and not an error message) you report happens any time, you use pip older than the latest one. This has nothing to do with poetry and this does not affect the result.

[Serkan-devel]

But doesn't poetry still break when using pip 19.1?

[vlcinsky]

@Serkan-devel the latest poetry works well with pip version 19.1.x for most packages. There might be particular problems.

Use this issue to improve the situation by properly reporting problems you experience and provide enough details such as OS, pip and poetry version, packages making problems and details about the problems (such as error messages)

[Serkan-devel]

Will open a newer issue because I have something different

[dsevero]

The only problem I've had so far is with enum34, but this has to do with the fact that numba isn't checking python versions with PEP508 environment markers. Hence, since I'm using python==3.6.7 and enum34 is installed, it breaks pip upon trying to install packages via ssh.

No other issues so far!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.