Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adapt virtualenv 20 API #7718

Closed
wants to merge 7 commits into from
Closed

Adapt virtualenv 20 API #7718

wants to merge 7 commits into from

Conversation

uranusjr
Copy link
Member

Close #7698. Draft PR filed for full CI status.

@uranusjr uranusjr force-pushed the virtualenv-20 branch 2 times, most recently from 3764555 to 719d19f Compare February 11, 2020 06:37
@uranusjr
Copy link
Member Author

virtualenv 20.0 induces a bug in utils/virtualenv.py. Environments it creates have both real_prefix and base_prefix, but only respects pyvenv.cfg, not no-global-site-packages.txt. A fix is included in this PR.

@uranusjr
Copy link
Member Author

I’ve fixed most compatibility issues, but can’t wrap my head around why the run_with_build_env tests are failing to clean up the temp dir. They seem unrelated…

@uranusjr uranusjr mentioned this pull request Feb 11, 2020
Merged
@uranusjr
Use virtualenv from Git for SCM compatibility
3decd9d
@uranusjr
Convert virtualenv usages in test to 20+ API
4eae4ac 
virtualenv 20.0+ completely revamps its API:

* There's no path_locations() anymore. Stdlib sysconfig can replace it
  completely.
* The Python API to create an environment is split into several pieces.
  Use the entry point function for now to provide the functionality.
@uranusjr
Remove virtualenv fixup
ac8cec6 
I *think* virtualenv fixed these? We'll see.
@uranusjr
Always use PEP 405 sitecustomize
682bbf9
@pradyunsg
Copy link
Member

Possibly related to #7778?

@uranusjr uranusjr closed this Feb 25, 2020
@uranusjr uranusjr reopened this Feb 25, 2020
@uranusjr
Copy link
Member Author

virtualenv (temporarily) removed the setup_requires entry in the master branch, so if there were any effects, they’re not present anymore.

@uranusjr
Copy link
Member Author

So I still don’t think #7778 is a direct cause, but the solution might fix the failures, since the sitecustomize.py in question is exactly the thing not get cleaned up. I am puzzled why it’s not though…

@pradyunsg
Copy link
Member

Could you look into the egg-info vs dist-info mismatch errors?

@uranusjr
Copy link
Member Author

So I just realised… why are tests on Travis run with --with-venv, even on Python 2? (What does this even mean?)

@uranusjr
Sync system site packages on user_site_packages
bc9c8df 
This is what the old virtualenv implementation did, so we match that.
@uranusjr uranusjr mentioned this pull request Feb 27, 2020
Open
@pradyunsg pradyunsg mentioned this pull request Apr 8, 2020
Closed
@asottile
Copy link
Contributor

this is listed in the changelog for 20.1b1 but is not merged -- I'm a bit confused :S

@uranusjr
Copy link
Member Author

uranusjr commented Apr 30, 2020 •
edited
Loading

I think they are two different things. The changelog you mentioned is probably referring to #7719 (I didn’t actually check), which fixed a conditional to make pip detect virtualenv 20.0 correctly. This one is strictly about fixing tests that rely on the virtualenv API in 19.x.

bors bot referenced this pull request in duckinator/emanate May 13, 2020
@bors @pyup-bot
Merge #118
dc75234 
118: Update pip to 20.1 r=duckinator a=pyup-bot


This PR updates [pip](https://pypi.org/project/pip) from **20.0.2** to **20.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 20.1
   ```
   =================

Process
-------

- Document that pip 21.0 will drop support for Python 2.7.

Features
--------

- Add ``pip cache dir`` to show the cache directory. (`7350 &lt;https://github.com/pypa/pip/issues/7350&gt;`_)

Bug Fixes
---------

- Abort pip cache commands early when cache is disabled. (`8124 &lt;https://github.com/pypa/pip/issues/8124&gt;`_)
- Correctly set permissions on metadata files during wheel installation,
  to permit non-privileged users to read from system site-packages. (`8139 &lt;https://github.com/pypa/pip/issues/8139&gt;`_)
   ```
   
  
  
   ### 20.1b1
   ```
   ===================

Deprecations and Removals
-------------------------

- Remove emails from AUTHORS.txt to prevent usage for spamming, and only populate names in AUTHORS.txt at time of release (`5979 &lt;https://github.com/pypa/pip/issues/5979&gt;`_)
- Remove deprecated ``--skip-requirements-regex`` option. (`7297 &lt;https://github.com/pypa/pip/issues/7297&gt;`_)
- Building of local directories is now done in place, instead of a temporary
  location containing a copy of the directory tree. (`7555 &lt;https://github.com/pypa/pip/issues/7555&gt;`_)
- Remove unused ``tests/scripts/test_all_pip.py`` test script and the ``tests/scripts`` folder. (`7680 &lt;https://github.com/pypa/pip/issues/7680&gt;`_)

Features
--------

- pip now implements PEP 610, so ``pip freeze`` has better fidelity
  in presence of distributions installed from Direct URL requirements. (`609 &lt;https://github.com/pypa/pip/issues/609&gt;`_)
- Add ``pip cache`` command for inspecting/managing pip&#39;s wheel cache. (`6391 &lt;https://github.com/pypa/pip/issues/6391&gt;`_)
- Raise error if ``--user`` and ``--target`` are used together in ``pip install`` (`7249 &lt;https://github.com/pypa/pip/issues/7249&gt;`_)
- Significantly improve performance when ``--find-links`` points to a very large HTML page. (`7729 &lt;https://github.com/pypa/pip/issues/7729&gt;`_)
- Indicate when wheel building is skipped, due to lack of the ``wheel`` package. (`7768 &lt;https://github.com/pypa/pip/issues/7768&gt;`_)
- Change default behaviour to always cache responses from trusted-host source. (`7847 &lt;https://github.com/pypa/pip/issues/7847&gt;`_)
- An alpha version of a new resolver is available via ``--unstable-feature=resolver``. (`988 &lt;https://github.com/pypa/pip/issues/988&gt;`_)

Bug Fixes
---------

- Correctly freeze a VCS editable package when it is nested inside another VCS repository. (`3988 &lt;https://github.com/pypa/pip/issues/3988&gt;`_)
- Correctly handle ``%2F`` in URL parameters to avoid accidentally unescape them
  into ``/``. (`6446 &lt;https://github.com/pypa/pip/issues/6446&gt;`_)
- Reject VCS URLs with an empty revision. (`7402 &lt;https://github.com/pypa/pip/issues/7402&gt;`_)
- Warn when an invalid URL is passed with ``--index-url`` (`7430 &lt;https://github.com/pypa/pip/issues/7430&gt;`_)
- Use better mechanism for handling temporary files, when recording metadata
  about installed files (RECORD) and the installer (INSTALLER). (`7699 &lt;https://github.com/pypa/pip/issues/7699&gt;`_)
- Correctly detect global site-packages availability of virtual environments
  created by PyPA’s virtualenv&gt;=20.0. (`7718 &lt;https://github.com/pypa/pip/issues/7718&gt;`_)
- Remove current directory from ``sys.path`` when invoked as ``python -m pip &lt;command&gt;`` (`7731 &lt;https://github.com/pypa/pip/issues/7731&gt;`_)
- Stop failing uninstallation, when trying to remove non-existent files. (`7856 &lt;https://github.com/pypa/pip/issues/7856&gt;`_)
- Prevent an infinite recursion with ``pip wheel`` when ``$TMPDIR`` is within the source directory. (`7872 &lt;https://github.com/pypa/pip/issues/7872&gt;`_)
- Significantly speedup ``pip list --outdated`` by parallelizing index interaction. (`7962 &lt;https://github.com/pypa/pip/issues/7962&gt;`_)
- Improve Windows compatibility when detecting writability in folder. (`8013 &lt;https://github.com/pypa/pip/issues/8013&gt;`_)

Vendored Libraries
------------------

- Update semi-supported debundling script to reflect that appdirs is vendored.
- Add ResolveLib as a vendored dependency.
- Upgrade certifi to 2020.04.05.1
- Upgrade contextlib2 to 0.6.0.post1
- Upgrade distro to 1.5.0.
- Upgrade idna to 2.9.
- Upgrade msgpack to 1.0.0.
- Upgrade packaging to 20.3.
- Upgrade pep517 to 0.8.2.
- Upgrade pyparsing to 2.4.7.
- Remove pytoml as a vendored dependency.
- Upgrade requests to 2.23.0.
- Add toml as a vendored dependency.
- Upgrade urllib3 to 1.25.8.

Improved Documentation
----------------------

- Emphasize that VCS URLs using git, git+git and git+http are insecure due to
  lack of authentication and encryption (`1983 &lt;https://github.com/pypa/pip/issues/1983&gt;`_)
- Clarify the usage of --no-binary command. (`3191 &lt;https://github.com/pypa/pip/issues/3191&gt;`_)
- Clarify the usage of freeze command in the example of Using pip in your program (`7008 &lt;https://github.com/pypa/pip/issues/7008&gt;`_)
- Add a &quot;Copyright&quot; page. (`7767 &lt;https://github.com/pypa/pip/issues/7767&gt;`_)
- Added example of defining multiple values for options which support them (`7803 &lt;https://github.com/pypa/pip/issues/7803&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pip
  - Changelog: https://pyup.io/changelogs/pip/
  - Homepage: https://pip.pypa.io/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
@pradyunsg pradyunsg mentioned this pull request May 15, 2020
Closed
@deveshks deveshks mentioned this pull request May 20, 2020
Closed
@frenzymadness
Copy link
Contributor

I'd like to help move this forward. I've rebased the PR to the current master branch (no conflicts) and I am waiting for the results from tox now. Is there any way how to restart CI here so I can compare the results I'll have from local tox?

Also, why pip uses the very latest virtualenv from its master branch instead of the latest released version?

@pradyunsg
Copy link
Member

Also, why pip uses the very latest virtualenv from its master branch instead of the latest released version?

Since historically, pip's use of virtualenv in it's test suite was the most comprehensive set of tests for virtualenv. We can probably switch to using released versions now.

@pradyunsg
Copy link
Member

@frenzymadness Feel free to file a new draft PR for trying out the CI. :)

@frenzymadness frenzymadness mentioned this pull request Jun 15, 2020
Closed
@jku
Copy link
Contributor

jku commented Jun 16, 2020

Could you look into the egg-info vs dist-info mismatch errors?

At least in some tests (like test_basic_install_from_local_directory[])

  • on master pip ends up doing legacy setup.py install
  • on this branch wheel seems to be available so it ends up building the package.

How is wheel there when the virtualenv is created with "--no-wheel" ?

@frenzymadness
Copy link
Contributor

How is wheel there when the virtualenv is created with "--no-wheel" ?

Please, take a look here #8441

Maybe we could close the old PRs to not confuse others?

@uranusjr uranusjr closed this Jun 17, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@pradyunsg pradyunsg pradyunsg left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@uranusjr @pradyunsg @asottile @frenzymadness @jku