bpo-25988: Do not expose abstract collection classes in the collections module. #10596

serhiy-storchaka · 2018-11-19T08:25:54Z

https://bugs.python.org/issue25988

ilevkivskyi · 2018-11-19T11:16:27Z

I think it makes sense to ping pip folks about the test failure. @pfmoore @dstufft not something urgent, since 3.8 is still far away, but it would be great if pip didn't use the deprecated collections imports like collections.Mapping and switched to collections.abc.Mapping instead, thanks!.

dstufft · 2018-11-19T12:11:32Z

This is actually in urllib3, we just bundle urllib3, it looks fixed in urllib3 and in pip and we just need to update CPython's bundled copy of pip.

ilevkivskyi · 2018-11-19T12:13:39Z

we just need to update CPython's bundled copy of pip.

Great, thanks for looking into this!

dstufft · 2018-11-19T12:23:16Z

#10598

dstufft · 2018-11-19T12:43:32Z

The above has been merged, you should be able to rebase this PR.

tirkarthi · 2018-11-21T04:12:04Z

This might cause make doctest to fail since some of the build dependencies might be incompatible https://bugs.python.org/issue35109

cc: @JulienPalard

serhiy-storchaka · 2018-12-20T08:02:40Z

Tests still are failing because of incompatibility in html5lib bundled with pip.

  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpcbwhlouc/pip-18.1-py2.py3-none-any.whl/pip/_vendor/html5lib/_trie/_base.py", line 3, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/home/travis/build/python/cpython/Lib/collections/__init__.py)

Note also syntax warnings produced by requests.

/tmp/tmpocpc284j/pip-18.1-py2.py3-none-any.whl/pip/_vendor/requests/status_codes.py:18: SyntaxWarning: invalid escape sequence \o

tirkarthi · 2018-12-20T08:25:05Z

Warning in html5lib's _trie/_base.py seems to have been fixed upstream : html5lib/html5lib-python#403

Upstream issue I opened while testing : html5lib/html5lib-python#402

tirkarthi · 2018-12-20T08:31:07Z

Requests also seem to be fixed : psf/requests#4728

So I guess pip's vendor folder needs to be updated for both libraries?

rhettinger · 2018-12-23T04:07:49Z

FWIW, it would be reasonable to delay this be one additional point release.

Also, if there are known major projects that still do the old imports, we should submit bug reports to those projects (I've already done some of these).

tirkarthi · 2018-12-23T04:52:16Z

Also, if there are known major projects that still do the old imports,

Our docs build had this problem with a dependency that had this issue and now it's fixed upstream. I think it's better to do it now that gives more time and will help users and libraries who test their code with alpha and beta candidates if they have these imports.

tirkarthi · 2019-01-22T23:37:27Z

@dstufft @pradyunsg https://bugs.python.org/issue35807 updates pip to 19.0 but I still see the vendored copy using html5lib that has this warning. Does pip update vendored version copy only after a release from html5lib? Should I file an issue in Pip's repo for this?

pradyunsg · 2019-02-04T16:37:30Z

(sorry for the delay in responding; swamped in GitHub notifications currently)

Does pip update vendored version copy only after a release from html5lib?

Yep. pip downloads and vendors only versions released on PyPI.

Should I file an issue in Pip's repo for this?

This isn't necessary since we basically do a round of vendoring updates before a release which should cover this, but yeah, it won't hurt either. Please feel free to.

…ns module.

serhiy-storchaka · 2019-03-07T06:55:30Z

pip 19.0.3 still is not compatible with this PR.

pradyunsg · 2019-03-07T17:19:37Z

Yea, there hasn't been an update for html5lib yet. :(

I pinged the author on #pypa-dev on Freenode today about this.

methane · 2019-03-27T11:19:16Z

If we postpone the removal, can we change warning class from DeprecationWarning to ImportWarning to nosy more people?

Maybe, many Python developers doesn't care about DeprecationWarning after Python 3.2.

pradyunsg · 2019-08-04T13:40:17Z

Ah. So that explains the documentation build failing. Thanks for the super quick response to a trivial question!

yan12125 · 2019-08-05T19:35:50Z

I gave it a try - the jinja issue is the last blocker. If I use an unreleased commit of jinja [1], tests are green [2].

[1] https://github.com/yan12125/cpython/commit/e6641707fb87774bf6359d8a584047cc94cc5b68#diff-c8dee9c3c068ad3447bbf459950a35b5
[2] https://travis-ci.org/yan12125/cpython/builds/567688247

vstinner · 2019-08-12T17:11:30Z

The issue was fixed in 2.10.2 more than a year ago, but this version is not released yet.

That's surprising. Why was a tag created in Git with no public release on PyPI?

Jinja2 fix for collections.abc: pallets/jinja@593ee1e

yan12125 · 2019-08-12T17:42:59Z

Seems 2.10.2 is not tagged yet (or it was tagged but later removed): https://github.com/pallets/jinja/tags

sersorrel · 2019-08-14T14:17:01Z

There was an issue filed against Jinja2 in April asking when the fix would be included in a release, but the maintainers aren't saying much: pallets/jinja#973

yan12125 · 2019-10-04T18:44:06Z

A new version of Jinja is out - http://jinja.palletsprojects.com/en/2.10.x/changelog/#version-2-10-2. This pull request can go forward now.

And the git conflict seems trivial to fix :) c4106af

A regression is discovered in Jinja 2.10.2 (pallets/jinja#1069). It might be better to wait for 2.10.3.

pradyunsg · 2019-10-04T19:06:11Z

The new (2.10.3) jinja release has been made! We're unblocked on this! :)

yan12125 · 2019-10-05T04:36:54Z

I gave it another try - with this PR rebased [1] and Jinja 2.10.3 installed, tests are still green [2].

[1] ~~https://github.com/yan12125/cpython/commits/collections-abc~~ (branch removed as this PR has been merged)
[2] https://travis-ci.org/yan12125/cpython/builds/593814893

pradyunsg · 2019-10-07T10:46:07Z

Yay! We managed to do this in under an year! ^>^

…ns module. (pythonGH-10596)

…recation warnings ### What changes were proposed in this pull request? This PR aims to remove deprecation warnings by importing ABCs from `collections.abc` instead of `collections`. - python/cpython#10596 ### Why are the changes needed? This will remove deprecation warnings in Python 3.7 and 3.8. ``` $ python -V Python 3.7.5 $ python python/pyspark/resultiterable.py python/pyspark/resultiterable.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working class ResultIterable(collections.Iterable): ``` ### Does this PR introduce any user-facing change? No, this doesn't introduce user-facing change ### How was this patch tested? Manually because this is about deprecation warning messages. Closes #26835 from tirkarthi/spark-30205-fix-abc-warnings. Authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

…es in the collections module. (GH-10596)" (GH-18545) This reverts commit ef092fe. Update collections __getattr__() and documentation to defer aliases removal to Python 3.10.

serhiy-storchaka requested a review from rhettinger as a code owner November 19, 2018 08:25

the-knights-who-say-ni added the CLA signed label Nov 19, 2018

bedevere-bot added the awaiting merge label Nov 19, 2018

serhiy-storchaka force-pushed the collections-abc branch from c1ffdbc to 6e1d424 Compare November 19, 2018 13:36

rhettinger approved these changes Nov 26, 2018

View reviewed changes

tirkarthi mentioned this pull request Nov 27, 2018

Using ABCs from collections instead of collections.abc is deprecated. python-babel/babel#609

Merged

serhiy-storchaka mentioned this pull request Dec 20, 2018

Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated html5lib/html5lib-python#405

Closed

rhettinger approved these changes Dec 23, 2018

View reviewed changes

This was referenced Dec 23, 2018

Add a six.moves for collections and collections.abc in preparation fo… benjaminp/six#241

Closed

DeprecationWarning in flask.sessions pallets/flask#3053

Closed

tirkarthi mentioned this pull request Feb 4, 2019

Update vendored copy of html5lib for Python 3.8 compatibility pypa/pip#6237

Closed

bpo-25988: Do not expose abstract collection classes in the collectio…

95262b1

…ns module.

serhiy-storchaka force-pushed the collections-abc branch from 6e1d424 to 95262b1 Compare March 5, 2019 07:54

Merge branch 'master' into collections-abc

d4a6798

Merge branch 'master' into collections-abc

0eb46a3

serhiy-storchaka merged commit ef092fe into python:master Oct 7, 2019

serhiy-storchaka deleted the collections-abc branch October 7, 2019 09:10

bedevere-bot removed the awaiting merge label Oct 7, 2019

jacobneiltaylor pushed a commit to jacobneiltaylor/cpython that referenced this pull request Dec 5, 2019

bpo-25988: Do not expose abstract collection classes in the collectio…

0a80b93

…ns module. (pythonGH-10596)

This was referenced Jan 4, 2020

Use ABC imports from collections.abc for Python 3.9 compatibility basho/riak-python-client#538

Open

Transition testing framework from nose/chai to pytest arrow-py/arrow#594

Closed

tirkarthi mentioned this pull request Jan 19, 2020

Importing ABC directly from collections module will be ImportError in Python 3.9 aws/aws-cli#4853

Closed

hugovk mentioned this pull request Jan 21, 2020

Fix collections.abc imports for Python 3.9 aws/aws-cli#4860

Merged

tirkarthi mentioned this pull request Apr 21, 2020

Import ABC from collections.abc for Python 3. rethinkdb/rethinkdb-python#187

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-25988: Do not expose abstract collection classes in the collections module. #10596

bpo-25988: Do not expose abstract collection classes in the collections module. #10596

serhiy-storchaka commented Nov 19, 2018 •

edited by bedevere-bot

Loading

ilevkivskyi commented Nov 19, 2018

dstufft commented Nov 19, 2018

ilevkivskyi commented Nov 19, 2018

dstufft commented Nov 19, 2018

dstufft commented Nov 19, 2018

tirkarthi commented Nov 21, 2018

serhiy-storchaka commented Dec 20, 2018

tirkarthi commented Dec 20, 2018

tirkarthi commented Dec 20, 2018

rhettinger commented Dec 23, 2018

tirkarthi commented Dec 23, 2018 •

edited

Loading

tirkarthi commented Jan 22, 2019

pradyunsg commented Feb 4, 2019

serhiy-storchaka commented Mar 7, 2019

pradyunsg commented Mar 7, 2019

methane commented Mar 27, 2019

pradyunsg commented Aug 4, 2019

yan12125 commented Aug 5, 2019

vstinner commented Aug 12, 2019

yan12125 commented Aug 12, 2019

sersorrel commented Aug 14, 2019

yan12125 commented Oct 4, 2019 •

edited

Loading

pradyunsg commented Oct 4, 2019 •

edited

Loading

yan12125 commented Oct 5, 2019 •

edited

Loading

pradyunsg commented Oct 7, 2019

bpo-25988: Do not expose abstract collection classes in the collections module. #10596

bpo-25988: Do not expose abstract collection classes in the collections module. #10596

Conversation

serhiy-storchaka commented Nov 19, 2018 • edited by bedevere-bot Loading

ilevkivskyi commented Nov 19, 2018

dstufft commented Nov 19, 2018

ilevkivskyi commented Nov 19, 2018

dstufft commented Nov 19, 2018

dstufft commented Nov 19, 2018

tirkarthi commented Nov 21, 2018

serhiy-storchaka commented Dec 20, 2018

tirkarthi commented Dec 20, 2018

tirkarthi commented Dec 20, 2018

rhettinger commented Dec 23, 2018

tirkarthi commented Dec 23, 2018 • edited Loading

tirkarthi commented Jan 22, 2019

pradyunsg commented Feb 4, 2019

serhiy-storchaka commented Mar 7, 2019

pradyunsg commented Mar 7, 2019

methane commented Mar 27, 2019

pradyunsg commented Aug 4, 2019

yan12125 commented Aug 5, 2019

vstinner commented Aug 12, 2019

yan12125 commented Aug 12, 2019

sersorrel commented Aug 14, 2019

yan12125 commented Oct 4, 2019 • edited Loading

pradyunsg commented Oct 4, 2019 • edited Loading

yan12125 commented Oct 5, 2019 • edited Loading

pradyunsg commented Oct 7, 2019

serhiy-storchaka commented Nov 19, 2018 •

edited by bedevere-bot

Loading

tirkarthi commented Dec 23, 2018 •

edited

Loading

yan12125 commented Oct 4, 2019 •

edited

Loading

pradyunsg commented Oct 4, 2019 •

edited

Loading

yan12125 commented Oct 5, 2019 •

edited

Loading