Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New resolver: Resolver quits when a given set of dependencies is uninstallable #9246

Closed
di opened this issue Dec 8, 2020 · 2 comments · Fixed by #9264 or #9289
Closed

New resolver: Resolver quits when a given set of dependencies is uninstallable #9246

di opened this issue Dec 8, 2020 · 2 comments · Fixed by #9264 or #9289
Milestone

Comments

@di
Copy link
Member

di commented Dec 8, 2020

What did you want to do?

Install an unresolvable set of dependencies and get a ResolutionImpossible error.

python3 -m pip install flask==1.1.1 diamond-dependency==1.0.0 --upgrade --upgrade-strategy only-if-needed --no-warn-script-location --no-warn-conflicts --force-reinstall --no-compile --user

Output

$ python3 -m pip install flask==1.1.1 diamond-dependency==1.0.0 --upgrade --upgrade-strategy only-if-needed --no-warn-script-location --no-warn-conflicts --force-reinstall --no-compile --user
Collecting diamond-dependency==1.0.0
  Downloading diamond_dependency-1.0.0-py3-none-any.whl (1.1 kB)
Collecting flask==1.1.1
  Downloading Flask-1.1.1-py2.py3-none-any.whl (94 kB)
     |████████████████████████████████| 94 kB 1.7 MB/s
Collecting click>=5.1
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
     |████████████████████████████████| 82 kB 849 kB/s
Collecting itsdangerous>=0.24
  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2>=2.10.1
  Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
     |████████████████████████████████| 125 kB 8.8 MB/s
Collecting MarkupSafe>=0.23
  Downloading MarkupSafe-1.1.1.tar.gz (19 kB)
Collecting sub-dependency-a
  Downloading sub_dependency_a-1.0.0-py3-none-any.whl (1.1 kB)
Collecting sub-dependency-b
  Downloading sub_dependency_b-1.0.0-py3-none-any.whl (1.1 kB)
INFO: pip is looking at multiple versions of sub-dependency-a to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of markupsafe to determine which version is compatible with other requirements. This could take a while.
Collecting MarkupSafe>=0.23
  Downloading MarkupSafe-1.1.0.tar.gz (18 kB)
  Downloading MarkupSafe-1.0.tar.gz (14 kB)
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1qalpscj/markupsafe_4ed9795d2ce54eb69fc8b87f9e9e4c69/setup.py'"'"'; __file__='"'"'/tmp/pip-install-1qalpscj/markupsafe_4ed9795d2ce54eb69fc8b87f9e9e4c69/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-l25ks531
         cwd: /tmp/pip-install-1qalpscj/markupsafe_4ed9795d2ce54eb69fc8b87f9e9e4c69/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-1qalpscj/markupsafe_4ed9795d2ce54eb69fc8b87f9e9e4c69/setup.py", line 6, in <module>
        from setuptools import setup, Extension, Feature
    ImportError: cannot import name 'Feature' from 'setuptools' (/usr/local/lib/python3.9/site-packages/setuptools/__init__.py)
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

It seems like the given combination of setuptools and markupsafe causes an exception to be raised at install-time for markupsafe. Unfortunately this also causes the resolver to exit as well.

What I expected to happen

I expected the resolver to catch/ignore this error in order to ignore uninstallable combinations of dependencies and continue attempting to solve the dependency set.

@uranusjr
Copy link
Member

uranusjr commented Dec 8, 2020

This is because the wheel-building code is inherited from the legacy resolver. It is fine for the legacy resolver to simply abort because it does not know how to try another version, but the same implementation works poorly in the new resolver. Some refactroing is needed to make this work.

copybara-service bot pushed a commit to GoogleCloudPlatform/buildpacks that referenced this issue Dec 8, 2020
The test verifies that builds with conflicting transitive dependencies fail in Python 3.8 and 3.9. The conflicts used to be found using `pip check`. `pip` was recently updated to include a new resolver that checks for conflicts during installation, removing the need to run `pip check`.

This change:
* Removes `flask==1.1.1` from the requirements.txt files because it causes pypa/pip#9246.
* Updates the test case error message regex to check for the new message as well (for now; after :latest stack images are updated we can remove pip check for 3.8+).

PiperOrigin-RevId: 346346873
Change-Id: Ic8bb0c0679dd9cc52ece4a537a25dc6586349a66
@edmorley
Copy link
Contributor

The fix for this (#9264) was reverted in #9293 - should this issue be reopened?

@uranusjr uranusjr reopened this Dec 15, 2020
@uranusjr uranusjr added this to the 20.3.4 milestone Dec 15, 2020
bors bot referenced this issue in duckinator/emanate Dec 15, 2020
204: Update pip to 20.3.3 r=duckinator a=pyup-bot


This PR updates [pip](https://pypi.org/project/pip) from **20.3.1** to **20.3.3**.



<details>
  <summary>Changelog</summary>
  
  
   ### 20.3.3
   ```
   ===================

Bug Fixes
---------

- Revert &quot;Skip candidate not providing valid metadata&quot;, as that caused pip to be overeager about downloading from the package index. (`9264 &lt;https://github.com/pypa/pip/issues/9264&gt;`_)
   ```
   
  
  
   ### 20.3.2
   ```
   ===================

Features
--------

- New resolver: Resolve direct and pinned (``==`` or ``===``) requirements first
  to improve resolver performance. (`9185 &lt;https://github.com/pypa/pip/issues/9185&gt;`_)
- Add a mechanism to delay resolving certain packages, and use it for setuptools. (`9249 &lt;https://github.com/pypa/pip/issues/9249&gt;`_)

Bug Fixes
---------

- New resolver: The &quot;Requirement already satisfied&quot; log is not printed only once
  for each package during resolution. (`9117 &lt;https://github.com/pypa/pip/issues/9117&gt;`_)
- Fix crash when logic for redacting authentication information from URLs
  in ``--help`` is given a list of strings, instead of a single string. (`9191 &lt;https://github.com/pypa/pip/issues/9191&gt;`_)
- New resolver: Correctly implement PEP 592. Do not return yanked versions from
  an index, unless the version range can only be satisfied by yanked candidates. (`9203 &lt;https://github.com/pypa/pip/issues/9203&gt;`_)
- New resolver: Make constraints also apply to package variants with extras, so
  the resolver correctly avoids backtracking on them. (`9232 &lt;https://github.com/pypa/pip/issues/9232&gt;`_)
- New resolver: Discard a candidate if it fails to provide metadata from source,
  or if the provided metadata is inconsistent, instead of quitting outright. (`9246 &lt;https://github.com/pypa/pip/issues/9246&gt;`_)

Vendored Libraries
------------------

- Update vendoring to 20.8

Improved Documentation
----------------------

- Update documentation to reflect that pip still uses legacy resolver by default in Python 2 environments. (`9269 &lt;https://github.com/pypa/pip/issues/9269&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pip
  - Changelog: https://pyup.io/changelogs/pip/
  - Homepage: https://pip.pypa.io/
</details>



205: Update pytest to 6.2.1 r=duckinator a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **6.1.2** to **6.2.1**.



*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
bors bot referenced this issue in duckinator/emanate Jan 31, 2021
215: Update pip to 21.0.1 r=duckinator a=pyup-bot


This PR updates [pip](https://pypi.org/project/pip) from **20.3.3** to **21.0.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 21.0.1
   ```
   ===================

Bug Fixes
---------

- commands: debug: Use packaging.version.parse to compare between versions. (`9461 &lt;https://github.com/pypa/pip/issues/9461&gt;`_)
- New resolver: Download and prepare a distribution only at the last possible
  moment to avoid unnecessary network access when the same version is already
  installed locally. (`9516 &lt;https://github.com/pypa/pip/issues/9516&gt;`_)

Vendored Libraries
------------------

- Upgrade packaging to 20.9
   ```
   
  
  
   ### 21.0
   ```
   =================

Deprecations and Removals
-------------------------

- Drop support for Python 2. (`6148 &lt;https://github.com/pypa/pip/issues/6148&gt;`_)
- Remove support for legacy wheel cache entries that were created with pip
  versions older than 20.0. (`7502 &lt;https://github.com/pypa/pip/issues/7502&gt;`_)
- Remove support for VCS pseudo URLs editable requirements. It was emitting
  deprecation warning since version 20.0. (`7554 &lt;https://github.com/pypa/pip/issues/7554&gt;`_)
- Modernise the codebase after Python 2. (`8802 &lt;https://github.com/pypa/pip/issues/8802&gt;`_)
- Drop support for Python 3.5. (`9189 &lt;https://github.com/pypa/pip/issues/9189&gt;`_)
- Remove the VCS export feature that was used only with editable VCS
  requirements and had correctness issues. (`9338 &lt;https://github.com/pypa/pip/issues/9338&gt;`_)

Features
--------

- Add ``--ignore-requires-python`` support to pip download. (`1884 &lt;https://github.com/pypa/pip/issues/1884&gt;`_)
- New resolver: Error message shown when a wheel contains inconsistent metadata
  is made more helpful by including both values from the file name and internal
  metadata. (`9186 &lt;https://github.com/pypa/pip/issues/9186&gt;`_)

Bug Fixes
---------

- Fix a regression that made ``pip wheel`` do a VCS export instead of a VCS clone
  for editable requirements. This broke VCS requirements that need the VCS
  information to build correctly. (`9273 &lt;https://github.com/pypa/pip/issues/9273&gt;`_)
- Fix ``pip download`` of editable VCS requirements that need VCS information
  to build correctly. (`9337 &lt;https://github.com/pypa/pip/issues/9337&gt;`_)

Vendored Libraries
------------------

- Upgrade msgpack to 1.0.2.
- Upgrade requests to 2.25.1.

Improved Documentation
----------------------

- Render the unreleased pip version change notes on the news page in docs. (`9172 &lt;https://github.com/pypa/pip/issues/9172&gt;`_)
- Fix broken email link in docs feedback banners. (`9343 &lt;https://github.com/pypa/pip/issues/9343&gt;`_)


.. note

    You should *NOT* be adding new change log entries to this file, this
    file is managed by towncrier. You *may* edit previous change logs to
    fix problems like typo corrections or such.

    To add a new change log entry, please see
        https://pip.pypa.io/en/latest/development/contributing/#news-entries

.. towncrier release notes start
   ```
   
  
  
   ### 20.3.4
   ```
   ===================

Features
--------

- ``pip wheel`` now verifies the built wheel contains valid metadata, and can be
  installed by a subsequent ``pip install``. This can be disabled with
  ``--no-verify``. (`9206 &lt;https://github.com/pypa/pip/issues/9206&gt;`_)
- Improve presentation of XMLRPC errors in pip search. (`9315 &lt;https://github.com/pypa/pip/issues/9315&gt;`_)

Bug Fixes
---------

- Fixed hanging VCS subprocess calls when the VCS outputs a large amount of data
  on stderr. Restored logging of VCS errors that was inadvertently removed in pip
  20.2. (`8876 &lt;https://github.com/pypa/pip/issues/8876&gt;`_)
- Fix error when an existing incompatibility is unable to be applied to a backtracked state. (`9180 &lt;https://github.com/pypa/pip/issues/9180&gt;`_)
- New resolver: Discard a faulty distribution, instead of quitting outright.
  This implementation is taken from 20.2.2, with a fix that always makes the
  resolver iterate through candidates from indexes lazily, to avoid downloading
  candidates we do not need. (`9203 &lt;https://github.com/pypa/pip/issues/9203&gt;`_)
- New resolver: Discard a source distribution if it fails to generate metadata,
  instead of quitting outright. This implementation is taken from 20.2.2, with a
  fix that always makes the resolver iterate through candidates from indexes
  lazily, to avoid downloading candidates we do not need. (`9246 &lt;https://github.com/pypa/pip/issues/9246&gt;`_)

Vendored Libraries
------------------

- Upgrade resolvelib to 0.5.4.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pip
  - Changelog: https://pyup.io/changelogs/pip/
  - Homepage: https://pip.pypa.io/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants