Python 3.7 End-of-Life and Upgrading to Python 3.11

[JacksonBurns]

Python 3.7 is reaching end of life this summer and will no longer receive security updates. Ultimately because we distribute on Docker now we can continue to ship the outdated Python version for a while before things actually stop working, but it will atrophy.

We will need to increment our minor version of Python, preferably to 3.11 for the speed improvements and longest support window. I say this rather than just shooting for 3.8, since I believe the additional effort of incrementing 4 minor versions will not be much worse than 1 (see below).

I anticipate that this will be an absolute nightmare, but a (forced) opportunity for some cleanup.

• pydas and pydqed were originally written in Python 2.7, hacked to get into Python 3.7, and will likely go kicking and screaming into Python 3.11
• pyjulia and diffeqpy are both presumably locked to Python 3.7
• Our remarkably outdated build of chemprop (0.0.1) only supports 3.7, though chemprop v2 will support 3.11 and newer.
• many of the packages we built and 'maintain' on the conda rmg channel expect Python 3.7 and will need to be rebuilt (just especially the ones mentioned above)
• Cython will need to be incremented to 0.29 (from 0.27)

This might take long enough that we end up targeting 3.12, which promises some very nice parallelism improvements so maybe that isn't such a bad thing.

Some things we should get done (wrt clearing tech debt) before attempting this:

• release v3.2.0 via RMG-Py release v3.2.0 #2311
• Removing Dead, Deprecated, and Unreachable Code #975 delete what we aren't using so don't have to try and upgrade it for no purpose
• Breaking Change on Aug. 1: Switch from nosetest to pytest and Overhaul Testing Layout #2380 as broken as nose is now, it will be even more broken-er in Python 3.11
• environment updates, including Simplify the RMG anaconda Dependencies  #2375, Create system image of RMS using Julia #2379 (and its PR Make and use a julia system image #2443), and Updating Conda environment requirements #2322 which should hopefully make it easier to upgrade by reducing # of dependencies and their complexity
• Fix None of our regression tests use RMS reactors #2442 (and [Regression Testing]: Need Documentation for Adding Regression Tests #2429 by consequence) so that we can validate results with RMS reactors in 3.11 versions

[JacksonBurns]

#2429 was resolved by @JacksonBurns in the process of assisting @hwpang, who is resolving #2442 via #2446

[rwest]

Regarding chemprop. You wrote above and in the environment.yml file:

• "Our remarkably outdated build of chemprop (0.0.1) only supports 3.7, though chemprop v2 will support 3.11 and newer."
• "Our build of this is version 0.0.1 (!!) and we are using parts of the API that are now gone. Need a serious PR to fix this."

How significant are the changes needed, and who is best to work on it? I'm rather unfamiliar with chemprop and RMG's use of it. Could someone start a draft PR to continue discussion?

[JacksonBurns]

#2456 may end up being related - we have a custom build of rdkit on the rmg conda channel that should also be replaced before attempting to increment to Python 3.11.

[rwest]

Should this be a "project" with multiple issues or and PRs?

[JacksonBurns]

#2456 may end up being related - we have a custom build of rdkit on the rmg conda channel that should also be replaced before attempting to increment to Python 3.11.

It wasn't

[JacksonBurns]

Should this be a "project" with multiple issues or and PRs?

It could be, I just don't love the projects interface - open to trying it though.

[JacksonBurns]

Our build of symmetry should be replaced with this.

[JacksonBurns]

Migrating this to GitHub Project Board.