-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Temporarily restore --build-dir for PyCharm #9193
Comments
@pradyunsg Would it be a good idea to restore the option before PyCharm can release a fix on their end? |
Do we know why PyCharm developers did not see the warning about that option being deprecated? Maybe we should improve on that. |
so this is basically a pycharm issue? Is 6 months warning enough? Probably. Should this critical breaking change be coordinated between the biggest IDE's developers and the developers of the most important executable in the python ecosystem? Probably. Where do we go from here? We have to open a PyCharm issue? |
All major IDE developers have access to the pip changelog. The pip developer team has no way to know what deprecated interfaces these IDEs use (against emitting warnings for 6+ months), especially sometimes when those sources are behind closed doors. Yes, you should be opening a PyCharm issue. The issue lies with them at the end of the day. Should they perhaps have a test suite against the master branch/RC versions of pip? Probably. |
Would it help if the option was still present as a no-op ? |
The PyCharm issue https://youtrack.jetbrains.com/issue/PY-45712. Seems there was a commit to remove the deprecated flag yesterday on the PyCharm code base JetBrains/intellij-community@a618daa |
@sbidoul I was going to say yes, leaving the option as a no-op for a short while might be helpful. But if JetBrains have already committed the change, it sounds like they are in a position to release a fix, so I think we can leave it to them. |
Ok, this annoys me more than I should probably allow it to. pip's maintainers have communicated about this change across multiple public channels, (as noted earlier) this option has been printing warnings on every invocation for the past few releases and we even had a beta release for 20.3 out for nearly 1 month, where this option was removed. I'm genuinely not sure what more we could've done to avoid this situation from our end. From my perspective, no reports/requests came in related to this deprecation from end users, so this was supposed to be "the easy one" -- I'd even dropped it from our 20.3 release announcement publicity push, because I thought: well, this removal seemed to affect no one so why call it out in the emails (LOL). To imply that pip's devs didn't do enough feels exceedingly dismissive of our efforts, and is the kind of thing that results in folks walking away from projects. sigh Sorry, I got worked up there.
Yes. It seems like PyCharm has been able to make the changes on their end to accomodate for the removal, but I'd imagine it might still be necessary to reduce breakage as they roll out the fix. If someone from the PyCharm team could clarify/provide an answer to the question:
I don't want to reintroduce the I'm happy to cut a pip 20.3.1 later today (assuming someone can file a PR adding this, with a comment pointing back to this issue), if that would help with this breakage for PyCharm's users. |
Agreed. Can I also note that the comments I saw from the PyCharm developers were polite, reasonable, and not at all confrontational, so I'd like to publicly thank them for taking a completely reasonable approach here. Drive-by criticism of either project over this from 3rd-party bystanders is neither helpful nor welcome, in my view. |
Fair enough, will do.
That said maybe this should have been a semver major release change on the pip side?
it should be noted that a user has to adopt this pip change to break things and it can be rolled back in the same way.
maybe its me but i dont read all the release notes for the hundreds of projects i use and track but a major release increment usually gets for attention. a breaking change appearing between x.2.4 to x.3.0 seems to violate this seemingly basic heads up but i didnt review this issue close enough to know that my assertion here is correct. if it is i dont know what semver is for anymore.
apologies if this seems strident i dont mean it to be. i had some intensive rollbacks i had to do where i was used to trusting these upgrades without doubt.
anyway, its mostly a pycharm issue i agree but a breaking change is a breaking change. my limited perspective says that the built in warning systems did not work.
…On Dec 1, 2020, 2:14 AM -0800, Bernát Gábor ***@***.***>, wrote:
> Should this critical breaking change be coordinated between the biggest IDE's developers and the developers of the most important executable in the python ecosystem? Probably.
All major IDE developers have access to the pip changelog. The pip developer team has no way to know what deprecated interfaces these IDEs use (against emitting warnings for 6+ months), especially sometimes when those sources are behind closed doors. Yes, you should be opening a PyCharm issue. The issue lies with them at the end of the day. Should they perhaps have a test suite against the master branch of pip? Probably.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
If you take a look at pip version number (2020.4) you'll notice that's a fairly big number. It's because pip does not use semantic versioning but instead https://calver.org/overview.html. As such there's no concept of major release you might be familiar with semver.
The warning message printed on every invocation for the last few months (at least) should suffice as a notification of you're doing something wrong. Please familiarize yourself with it https://calver.org/overview.html#when-to-use-calver 👍 |
If we followed Semver -- sure, it would've been -- but we don't, for a lot of reasons that I won't go into. I'll flag that pip has been one of the most prominent projects using CalVar for over 2 years now. We also have a documented deprecation policy about making breaking changes: https://pip.pypa.io/en/stable/development/release-process/#deprecation-policy
This is definitely true. I do want us to identify why this happened, so that we can avoid this class of issues in the future. I say that we let this thread be "at rest" now, until tomorrow or until one of the PyCharm developers responds here. :) |
Hi everyone! I'm from a PyCharm team. We didn't notice the deprecation because this option was used only in an internal script that we normally don't launch manually. We can definitely improve in this regard by reporting any deprecation messages from the tool to stderr, at least in the internal/EAP mode as an early diagnostic, so that it doesn't happen again. As for why we needed Making |
A modest proposal: if you add a For best results increase the delay with every subsequent release. |
This can be interpreted as one of those passive-aggressive interactions 🤔 What about people who can't upgrade and pin, should they be penalized? |
@mgedmin I presume that your phrase "modest proposal" was meant to mark your comment as satirical, and not to be taken seriously. Am I correct? [Also, @east825, thank you & thanks to PyCharm for sponsoring the Python Software Foundation and helping financially contribute to the infrastructure of Python!] |
@brainwane I've passed unexpected, but heart-warming praises to the team :) |
@pradyunsg Would it be possible to keep the dummy no-op option for another 6 months (maybe even hiding it in the |
@pradyunsg @east825 I'll do the PR within the next hours. |
Yes, I'm 100% on board for doing this. I'm pretty sure that no one in @pypa/pip-committers would be opposed to this, but, if they are, I'd imagine we'll hear from them here. ;) @sbidoul wheee! Thanks! :) |
There you go: #9198. |
FYI folks - I plan to keep this issue open, until the re-removal of the option in 2021. That way we can cross check before 21.1 that PyCharm folks have been able to successfully transition most of their users. |
Can this error break venv setup in any way? |
@SergeiBond It can interfere, if you're automatically installing a package in it, e.g. Django or Flask, through a new project wizard. An empty virtual environment is still created in this case, though. |
It looks like the new release of PyCharm (2020.2.5) got it fixed? PyCharm people, please confirm :) |
@sfelixkim Correct! 2020.2.5 was released specifically because of this fix. It's also included in 2020.1.5 and the upcoming 2020.3. |
I'll cut a release in my lunch break today. :) |
First of all, huge thanks to everyone involved in addressing this problem so promptly! I don't want to be a pain in the neck, but, nonetheless, can someone please still elaborate a bit on how the behavior of |
I'm not a 100% sure on the history, but based on what you described earlier, yes, it's redundant for PyCharm's use case now. :) I've listed what seem to be the changelog entries relevant to this option (from https://pip.pypa.io/en/latest/news/). Folks can Ctrl+F to figure out the timeline and all that. :)
|
Apologies, thanks for the vigorous response. Truly appreciated. |
I took the liberty to create the 21.1 milestone add this to it. |
Thanks @uranusjr! FWIW, 20.3.1 is out with the requested fix. |
Hmm, weird: I got this error when I was actually starting a new Flask project with venv. However, everything seems to be working fine despite the error. It seems that Flask package got installed just fine. |
@pradyunsg Thanks a lot for the excerpt from the changelog. Apparently, #2122 is just what I was looking for. Shame on me, somehow I couldn't find it myself right away. All in all, the option indeed seems long overdue for removal from our scripts. |
194: Update pytest-pylint to 0.18.0 r=duckinator a=pyup-bot This PR updates [pytest-pylint](https://pypi.org/project/pytest-pylint) from **0.17.0** to **0.18.0**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest-pylint - Changelog: https://pyup.io/changelogs/pytest-pylint/ - Repo: https://github.com/carsongee/pytest-pylint </details> 197: Update pip to 20.3.1 r=duckinator a=pyup-bot This PR updates [pip](https://pypi.org/project/pip) from **20.2.4** to **20.3.1**. <details> <summary>Changelog</summary> ### 20.3.1 ``` =================== Deprecations and Removals ------------------------- - The --build-dir option has been restored as a no-op, to soften the transition for tools that still used it. (`9193 <https://github.com/pypa/pip/issues/9193>`_) ``` ### 20.3 ``` - Introduce a new ResolutionImpossible error, raised when pip encounters un-satisfiable dependency conflicts (`8546 <https://github.com/pypa/pip/issues/8546>`_, `8377 <https://github.com/pypa/pip/issues/8377>`_) - Add a subcommand ``debug`` to ``pip config`` to list available configuration sources and the key-value pairs defined in them. (`6741 <https://github.com/pypa/pip/issues/6741>`_) - Warn if index pages have unexpected content-type (`6754 <https://github.com/pypa/pip/issues/6754>`_) - Allow specifying ``--prefer-binary`` option in a requirements file (`7693 <https://github.com/pypa/pip/issues/7693>`_) - Generate PEP 376 REQUESTED metadata for user supplied requirements installed by pip. (`7811 <https://github.com/pypa/pip/issues/7811>`_) - Warn if package url is a vcs or an archive url with invalid scheme (`8128 <https://github.com/pypa/pip/issues/8128>`_) - Parallelize network operations in ``pip list``. (`8504 <https://github.com/pypa/pip/issues/8504>`_) - Allow the new resolver to obtain dependency information through wheels lazily downloaded using HTTP range requests. To enable this feature, invoke ``pip`` with ``--use-feature=fast-deps``. (`8588 <https://github.com/pypa/pip/issues/8588>`_) - Support ``--use-feature`` in requirements files (`8601 <https://github.com/pypa/pip/issues/8601>`_) Bug Fixes --------- - Use canonical package names while looking up already installed packages. (`5021 <https://github.com/pypa/pip/issues/5021>`_) - Fix normalizing path on Windows when installing package on another logical disk. (`7625 <https://github.com/pypa/pip/issues/7625>`_) - The VCS commands run by pip as subprocesses don't merge stdout and stderr anymore, improving the output parsing by subsequent commands. (`7968 <https://github.com/pypa/pip/issues/7968>`_) - Correctly treat non-ASCII entry point declarations in wheels so they can be installed on Windows. (`8342 <https://github.com/pypa/pip/issues/8342>`_) - Update author email in config and tests to reflect decommissioning of pypa-dev list. (`8454 <https://github.com/pypa/pip/issues/8454>`_) - Headers provided by wheels in .data directories are now correctly installed into the user-provided locations, such as ``--prefix``, instead of the virtual environment pip is running in. (`8521 <https://github.com/pypa/pip/issues/8521>`_) Vendored Libraries ------------------ - Vendored htmlib5 no longer imports deprecated xml.etree.cElementTree on Python 3. - Upgrade appdirs to 1.4.4 - Upgrade certifi to 2020.6.20 - Upgrade distlib to 0.3.1 - Upgrade html5lib to 1.1 - Upgrade idna to 2.10 - Upgrade packaging to 20.4 - Upgrade requests to 2.24.0 - Upgrade six to 1.15.0 - Upgrade toml to 0.10.1 - Upgrade urllib3 to 1.25.9 Improved Documentation ---------------------- - Add ``--no-input`` option to pip docs (`7688 <https://github.com/pypa/pip/issues/7688>`_) - List of options supported in requirements file are extracted from source of truth, instead of being maintained manually. (`7908 <https://github.com/pypa/pip/issues/7908>`_) - Fix pip config docstring so that the subcommands render correctly in the docs (`8072 <https://github.com/pypa/pip/issues/8072>`_) - replace links to the old pypa-dev mailing list with https://mail.python.org/mailman3/lists/distutils-sig.python.org/ (`8353 <https://github.com/pypa/pip/issues/8353>`_) - Fix example for defining multiple values for options which support them (`8373 <https://github.com/pypa/pip/issues/8373>`_) - Add documentation for the ResolutionImpossible error that helps the user fix dependency conflicts (`8459 <https://github.com/pypa/pip/issues/8459>`_) - Add feature flags to docs (`8512 <https://github.com/pypa/pip/issues/8512>`_) - Document how to install package extras from git branch and source distributions. (`8576 <https://github.com/pypa/pip/issues/8576>`_) ``` ### 20.3b1 ``` =================== Deprecations and Removals ------------------------- - ``pip freeze`` will stop filtering the ``pip``, ``setuptools``, ``distribute`` and ``wheel`` packages from ``pip freeze`` output in a future version. To keep the previous behavior, users should use the new ``--exclude`` option. (`4256 <https://github.com/pypa/pip/issues/4256>`_) - Deprecate support for Python 3.5 (`8181 <https://github.com/pypa/pip/issues/8181>`_) - Document that certain removals can be fast tracked. (`8417 <https://github.com/pypa/pip/issues/8417>`_) - Document that Python versions are generally supported until PyPI usage falls below 5%. (`8927 <https://github.com/pypa/pip/issues/8927>`_) - Deprecate ``--find-links`` option in ``pip freeze`` (`9069 <https://github.com/pypa/pip/issues/9069>`_) Features -------- - Add ``--exclude`` option to ``pip freeze`` and ``pip list`` commands to explicitly exclude packages from the output. (`4256 <https://github.com/pypa/pip/issues/4256>`_) - Allow multiple values for --abi and --platform. (`6121 <https://github.com/pypa/pip/issues/6121>`_) - Add option ``--format`` to subcommand ``list`` of ``pip cache``, with ``abspath`` choice to output the full path of a wheel file. (`8355 <https://github.com/pypa/pip/issues/8355>`_) - Improve error message friendliness when an environment has packages with corrupted metadata. (`8676 <https://github.com/pypa/pip/issues/8676>`_) - Make the ``setup.py install`` deprecation warning less noisy. We warn only when ``setup.py install`` succeeded and ``setup.py bdist_wheel`` failed, as situations where both fails are most probably irrelevant to this deprecation. (`8752 <https://github.com/pypa/pip/issues/8752>`_) - Check the download directory for existing wheels to possibly avoid fetching metadata when the ``fast-deps`` feature is used with ``pip wheel`` and ``pip download``. (`8804 <https://github.com/pypa/pip/issues/8804>`_) - When installing a git URL that refers to a commit that is not available locally after git clone, attempt to fetch it from the remote. (`8815 <https://github.com/pypa/pip/issues/8815>`_) - Include http subdirectory in ``pip cache info`` and ``pip cache purge`` commands. (`8892 <https://github.com/pypa/pip/issues/8892>`_) - Cache package listings on index packages so they are guarenteed to stay stable during a pip command session. This also improves performance when a index page is accessed multiple times during the command session. (`8905 <https://github.com/pypa/pip/issues/8905>`_) - New resolver: Tweak resolution logic to improve user experience when user-supplied requirements conflict. (`8924 <https://github.com/pypa/pip/issues/8924>`_) - Support Python 3.9. (`8971 <https://github.com/pypa/pip/issues/8971>`_) - Log an informational message when backtracking takes multiple rounds on a specific package. (`8975 <https://github.com/pypa/pip/issues/8975>`_) - Switch to the new dependency resolver by default. (`9019 <https://github.com/pypa/pip/issues/9019>`_) - Remove the ``--build-dir`` option, as per the deprecation. (`9049 <https://github.com/pypa/pip/issues/9049>`_) Bug Fixes --------- - Propagate ``--extra-index-url`` from requirements file properly to session auth, so that keyring auth will work as expected. (`8103 <https://github.com/pypa/pip/issues/8103>`_) - Allow specifying verbosity and quiet level via configuration files and environment variables. Previously these options were treated as boolean values when read from there while through CLI the level can be specified. (`8578 <https://github.com/pypa/pip/issues/8578>`_) - Only converts Windows path to unicode on Python 2 to avoid regressions when a POSIX environment does not configure the file system encoding correctly. (`8658 <https://github.com/pypa/pip/issues/8658>`_) - List downloaded distributions before exiting ``pip download`` when using the new resolver to make the behavior the same as that on the legacy resolver. (`8696 <https://github.com/pypa/pip/issues/8696>`_) - New resolver: Pick up hash declarations in constraints files and use them to filter available distributions. (`8792 <https://github.com/pypa/pip/issues/8792>`_) - Avoid polluting the destination directory by resolution artifacts when the new resolver is used for ``pip download`` or ``pip wheel``. (`8827 <https://github.com/pypa/pip/issues/8827>`_) - New resolver: If a package appears multiple times in user specification with different ``--hash`` options, only hashes that present in all specifications should be allowed. (`8839 <https://github.com/pypa/pip/issues/8839>`_) - Tweak the output during dependency resolution in the new resolver. (`8861 <https://github.com/pypa/pip/issues/8861>`_) - Correctly search for installed distributions in new resolver logic in order to not miss packages (virtualenv packages from system-wide-packages for example) (`8963 <https://github.com/pypa/pip/issues/8963>`_) - Do not fail in pip freeze when encountering a ``direct_url.json`` metadata file with editable=True. Render it as a non-editable ``file://`` URL until modern editable installs are standardized and supported. (`8996 <https://github.com/pypa/pip/issues/8996>`_) Vendored Libraries ------------------ - Fix devendoring instructions to explicitly state that ``vendor.txt`` should not be removed. It is mandatory for ``pip debug`` command. Improved Documentation ---------------------- - Add documentation for '.netrc' support. (`7231 <https://github.com/pypa/pip/issues/7231>`_) - Add OS tabs for OS-specific commands. (`7311 <https://github.com/pypa/pip/issues/7311>`_) - Add note and example on keyring support for index basic-auth (`8636 <https://github.com/pypa/pip/issues/8636>`_) - Added initial UX feedback widgets to docs. (`8783 <https://github.com/pypa/pip/issues/8783>`_, `8848 <https://github.com/pypa/pip/issues/8848>`_) - Add ux documentation (`8807 <https://github.com/pypa/pip/issues/8807>`_) - Update user docs to reflect new resolver as default in 20.3. (`9044 <https://github.com/pypa/pip/issues/9044>`_) - Improve migration guide to reflect changes in new resolver behavior. (`9056 <https://github.com/pypa/pip/issues/9056>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pip - Changelog: https://pyup.io/changelogs/pip/ - Homepage: https://pip.pypa.io/ </details> Co-authored-by: pyup-bot <github-bot@pyup.io> Co-authored-by: Ellen Marie Dash <me@duckie.co>
I… forgot what we should do for 21.2. Does anyone here still remember the context? |
I think we just need to remove the --build-dir option that is a no-op. That said, it has no maintenance costs so I'd be inclined to leave it in for a few more releases. |
The change necessary here is to revert #9198. Happy to defer this to the next release too. :) |
Let’s leave this for the 21.1 RM to decide. |
This has been moved to 21.3 in #9783. Changing the milestone. |
And, there's now a PR for removing this option. @east825 Do you know if there's any reason to not go forward with the migration plan from 6 months ago (i.e. removing the no-op flag now)? |
@pradyunsg I think it's completely fine to drop it now, go ahead. Thanks for keeping it for us for that long. |
Awesome. Thanks for confirming! :) |
Done in #10485. |
Environment
Description
Cannot install any packages when using PyCharm python interpreter.
The virtual environment is created using virtualenv.
No problem when executing
pip install [package]
on either PyCharm terminal or cmd prompt directly.Problem resolved when pip is downgraded to 20.2 (had to be done on terminal by
pip install pip==20.2.4
)Expected behavior
Install packages successfully.
How to Reproduce
Output
executed command:
Error occurred:
proposed solution:
command output:
The text was updated successfully, but these errors were encountered: