Meta-ticket: Remove Python 2 support for Sage 9.2; upgrades enabled by the removal

[mkoeppe]

From https://groups.google.com/forum/#!topic/sage-devel/vYlbnAwKATM "drop python2 compatibility in 9.1" (thread Jan 5-22, 2020).

Removal of Python 2 support in Sage 9.2 has been announced at https://wiki.sagemath.org/Python3-Switch and https://wiki.sagemath.org/ReleaseTours/sage-9.1

Removal tickets:

• For Sage 9.2: Remove python 2 support from sagelib #28000 For Sage 9.2: Remove python 2 support from sagelib
• For Sage 9.2: Remove Python 2 support from the build system and CI scripts #29669 For Sage 9.2: Remove Python 2 support from the build system and CI scripts
• Remove calls to 'long(...)', etc. #27826 - remove long hack for Python 3 in doctest/forker.py, remove all calls to long()
• Remove packages that handle backporting from Python 3 to Python 2 #29754 - remove Python 2 package and packages which backports various parts of Python

Upgrade tickets:

Please check that upgraded packages are compatible with Python >= 3.6 (and note this on the upgrade ticket) because we may wish to support Python 3.6 venv (#29033).

• upgrade to ipython 7 #28197 - upgrade to ipython 7 / prompt-toolkit 2. See also Sort dictionaries for doctests independent of ipythons pretty print #29042
• Upgrade: Python 3.8.5 #27754 - Upgrade: Python 3.8.x (see #27754 comment:55 in particular)
• numpy >= 1.17 - Upgrade NumPy to 1.18.5, scipy to 1.4.1, add pybind11 package #29425 Upgrade Cython to 0.29.16, NumPy to 1.18.2 to fix numpy build error on local-conda-forge-macos-standard
• Update sympy to 1.6 #29730 Update sympy to 1.6
• matplotlib-3: Update to matplotlib 3 #29547. Related: Messed up labels in crystal plot #28449 and #27866 comment:25
• sphinx-2: Update to sphinx 3 #28856, Fix sphinxify deprecation warnings with sphinx 2.2 #29095
• scipy-1.3+: Fix deprecation warnings with scipy 1.4 #29115
• rpy2 3.x: Support rpy 3.x #28988, upgrade rpy2 package 2.8.2 -> 3.3.5, upgrade R to 3.6.3, add new dependencies #29441
• networkx 2.4: networkx 2.3 compatibility #28450
• Upgrade setuptools, setuptools_scm, pip (2020-06), add package wheel; remove zope_interface #29803 - Upgrade setuptools, setuptools_scm, pip

Other:

• compatibility with python 3.8... needs details
  • "at this point the major issues are python 3.8 and ipython 7, and I don't see how one could support several versions of them without forking hundreds of doctests. Both updates require multi-thousand-lines patches, due to changes in dict sorting and hashes."
• remove src/sage/cpython/_py2_random.py
  • This was added in py3: problems with tests that use random_element #24508 and is only used during doctesting in order to ensure consistent random results between Python 2 and 3.

Reference: Tickets that kept Python 2 compatibility but introduced a performance penalty for Python 2:

• get rid of part of itervalues #29103 (get rid of part of itervalues)

See also:

• Meta-ticket: Review of Python 3 features that sagelib should use systematically #29756 Meta-ticket: Review of Python 3 features that sagelib should use systematically

CC: @fchapoton @embray @jdemeyer @slel @antonio-rojas @kiwifb @timokau @jhpalmieri @tscrim @isuruf @nbruin @orlitzky @vbraun @dimpase @videlec @kliem

Component: python3

Keywords: sd109

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29141

[antonio-rojas]

comment:1

python 3.8 update is tracked in #27754, there is a list of issues that need fixing. rpy3 support is merged already in 9.1.beta2 (#28988), as well as networkx 2.4, which is also py3-only (#28450). IMO #28190 is no longer worth the effort at this point, provided that python2 support is to be dropped at 9.2 in the worst case.

[mkoeppe]

comment:2

Please feel free to update the ticket description.

[mkoeppe]

comment:8

I have just updated https://wiki.sagemath.org/Python3-Switch to say that the upcoming 9.1 will still support Python 2

[kiwifb]

comment:12

I am working on #29042 and I noticed that neither #28000 or #29669 remove # py2 and # py3 for doctesting. Is there a ticket for that which should then be in the summary, or is it something that is planned to be removed on the longer term?

[jhpalmieri]

comment:15

@kiwifb: I don't know of any ticket for removing py2 and py3 tags.

[kiwifb]

comment:16

Another thing that I spotted scanning stuff in #29633. We have a lot python feature backport packages in our tree. We may want to get rid of them as we get rid of python 2.7 and anything older than python 3.7 in fact. Look at enum34 if you don't get what I mean.

[slel]

comment:17

Another thing that I spotted scanning stuff in #29633.
We have a lot of Python feature backport packages in our tree.
We may want to get rid of them as we get rid of Python 2.7
and anything older than Python 3.7 in fact. Look at enum34
if you don't get what I mean.

Consider keeping support Python 3.6, whose official support is till 2021-12-23. See #29033.

Python 3.5 support ends in 2020-09, so fine to drop it. See life cycle of Python versions:

• currently or recently supported versions: https://www.python.org/downloads/
• older versions: https://devguide.python.org/devcycle/#end-of-life-branches

[mkoeppe]

Changed keywords from none to sd109

[jhpalmieri]

comment:19

Replying to @kiwifb:

Another thing that I spotted scanning stuff in #29633. We have a lot python feature backport packages in our tree. We may want to get rid of them as we get rid of python 2.7 and anything older than python 3.7 in fact. Look at enum34 if you don't get what I mean.

I'm working on this. I'll open up a ticket and post a link here some time soon.

[jhpalmieri]

comment:20

Replying to @jhpalmieri:

I'm working on this. I'll open up a ticket and post a link here some time soon.

See #29754.

[mkoeppe]

comment:26

We can close this ticket as completed.

[dimpase]

Reviewer: Dima Pasechnik