Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

requirements.txt not installable by pip when local package depends on another #147

Open
3 tasks done
jyggen opened this issue Mar 2, 2020 · 4 comments
Open
3 tasks done

Comments

@jyggen
Copy link

jyggen commented Mar 2, 2020

  • I am on the latest Poetry version.

  • I have searched the issues of this repo and believe that this is not a duplicate.

  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

  • OS version and name: Docker (python:3.7-slim)

  • Poetry version: 1.1.4

  • Link of a Gist with the contents of your pyproject.toml file: https://github.com/jyggen/poetry-example

Issue

pip is unable to install the dependencies using the requirements.txt from poetry export when the main application is depending on the local path package B, and where package B in turn depends on local path package A. I've created a repository (see above) that can reliably reproduce the error by running docker build ..

Dependency Graph:

app 
|-- package-b
    |-- package-a

Example output:

$ docker build .
Sending build context to Docker daemon  81.92kB
Step 1/8 : FROM python:3.7-slim
 ---> 84de2ffd919d
Step 2/8 : ENV POETRY_VERSION=1.0.4
 ---> Using cache
 ---> 7b1b2eaa0c59
Step 3/8 : ENV POETRY_HOME=/usr/local/poetry
 ---> Using cache
 ---> 1b0452673941
Step 4/8 : RUN apt-get update && apt-get install -y wget   && wget https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py   && python get-poetry.py
 ---> Using cache
 ---> 60b085346606
Step 5/8 : ENV PATH="/usr/local/poetry/bin:${PATH}"
 ---> Using cache
 ---> 4799b083ead0
Step 6/8 : COPY . /build
 ---> fdf2a1ac60a3
Step 7/8 : WORKDIR /build/app
 ---> Running in cb8b784b1f88
Removing intermediate container cb8b784b1f88
 ---> 67acda5f4b2c
Step 8/8 : RUN poetry export --without-hashes -f requirements.txt   | sed 's/-e //'   | pip install -r /dev/stdin
 ---> Running in dbafc8616663
Processing /build/package-a
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Processing /build/package-b
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpnhnp2nm6
         cwd: /tmp/pip-req-build-2y1q91vj
    Complete output (16 lines):
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
        main()
      File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 110, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-t7gqct2d/overlay/lib/python3.7/site-packages/poetry/masonry/api.py", line 38, in prepare_metadata_for_build_wheel
        poetry = Factory().create_poetry(Path("."))
      File "/tmp/pip-build-env-t7gqct2d/overlay/lib/python3.7/site-packages/poetry/factory.py", line 99, in create_poetry
        package.add_dependency(name, constraint)
      File "/tmp/pip-build-env-t7gqct2d/overlay/lib/python3.7/site-packages/poetry/packages/package.py", line 333, in add_dependency
        develop=constraint.get("develop", True),
      File "/tmp/pip-build-env-t7gqct2d/overlay/lib/python3.7/site-packages/poetry/packages/directory_dependency.py", line 36, in __init__
        raise ValueError("Directory {} does not exist".format(self._path))
    ValueError: Directory ../package-a does not exist
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpnhnp2nm6 Check the logs for full command output.
The command '/bin/sh -c poetry export --without-hashes -f requirements.txt   | sed 's/-e //'   | pip install -r /dev/stdin' returned a non-zero code: 1

I've also seen a related error (where a local package depends on another local package that's also required by the main app) occur in an actual project when trying to install the export output with pip, but haven't been able to create a reproducible case yet. The error output, however, is the following:

ERROR: Could not find a version that satisfies the requirement package-a (from package-b==0.1.0->-r /dev/stdin (line 9)) (from versions: none)
ERROR: No matching distribution found for package (from package-b==0.1.0->-r /dev/stdin (line 9))
@jyggen jyggen changed the title requirements.txt not installable by pip when local packages depend on each other requirements.txt not installable by pip when local package depends on another Mar 2, 2020
@jyggen
Copy link
Author

jyggen commented Mar 17, 2020

Looking at it more closely, it seems to be plagued by the same issue as python-poetry/poetry#266, namely that package-b is moved to the temporary build directory and thus can't resolve the path to package-a.

Sadly it doesn't seem to be fixed by python-poetry/poetry#2114 and, after scouring through the code, there doesn't seem to be an obvious way to obtain and send an equivalent of original_root to Factory().create_poetry() in prepare_metadata_for_build_wheel() either.

@jyggen
Copy link
Author

jyggen commented Jun 24, 2020

This is still an issue in 1.0.9, despite the recent dependency resolution- and inconsistency fixes.

$ poetry export --without-hashes -f requirements.txt | sed 's/-e //' > requirements.txt && pip install -r requirements.txt
...
ERROR: Could not find a version that satisfies the requirement package-a (from package-b==0.1.0->-r requirements.txt (line 3)) (from versions: none)
ERROR: No matching distribution found for package-a (from package-b==0.1.0->-r requirements.txt (line 3))

@soleares
Copy link

@jyggen I'm not seeing this issue in 1.1.4 anymore. It was still occurring for me in 1.1. Are you still seeing it or can this be closed out?

@jyggen
Copy link
Author

jyggen commented Feb 11, 2021

Seems like it's still occuring. Not the exact same error, but a similar one. I've updated the example code in OP to use latest Poetry.

Step 8/9 : RUN poetry export --without-hashes -f requirements.txt > requirements.txt
 ---> Running in 4aebd374a046
Removing intermediate container 4aebd374a046
 ---> 7ad62c9be344
Step 9/9 : RUN  cat requirements.txt && ls /build && pip install -r requirements.txt
 ---> Running in 43afc9198e48
package-a @ /build/package-a; python_version >= "3.7" and python_version < "4.0"
package-b @ /build/package-b; python_version >= "3.7" and python_version < "4.0"
Dockerfile
app
package-a
package-b
ERROR: Invalid requirement: 'package-a @ /build/package-a' (from line 1 of requirements.txt)
Hint: It looks like a path. File 'package-a @ /build/package-a' does not exist.
The command '/bin/sh -c cat requirements.txt && ls /build && pip install -r requirements.txt' returned a non-zero code: 1

As seen by the ls /build right before pip install, /build/package-a very much exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants