Python 3.5 support broken

[Germar]

Hi,
I just fired up my (quite old) Test-VM for BiT. It is still running with Ubuntu 16.04 which comes with Python 3.5.2
Commit 8e0397f comes with a new string format I didn't know before which breakes Python 3.5 support.

$ python3
Python 3.5.2 (default, Sep 16 2022, 21:16:04) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print(f"foo")
  File "<stdin>", line 1
    print(f"foo")
               ^
SyntaxError: invalid syntax

As Ubuntu 16.04 is not End of Life yet (April 2026) and we still build the PPA for 16.04 I'd prefere to keep compatibility. As far as I can see f"foo {bar()}" is just a replacement for "foo {}".format(bar())

[aryoda]

I think we have also introduced new package dependencies that require at least Python 3.6 so there'd be waiting more work for us to be backward compatible.

@buhtz Can you remember which ones?

I am not sure but LTE normally meant only 5 years but there seams to be another 5 years "Extended Security Maintenance (ESM)" period now:

https://ubuntu.com/16-04

Shall we really support such an old version with new updates (the old packaged BiT version should still work; we discussed a few weeks ago not to backport fixes to old versions due to the high maintenance efforts).

Any ideas/comments/proposals?

[emtiu]

This was discussed here on the mailing list a couple of weeks ago. We didn't make a fixed policy decision, but the general consensus was that 3.6 (and therefore also 3.5) was too old to support in active development.

[buhtz]

The facts

Python Software Foundation

Official details about Python versions and their support can be found at https://devguide.python.org/versions/.

• Python 3.5: Latest release was Sept. 2020. Reached end of life.
• Python 3.6: Latest release (and only for security reasons) was Sept. 2021 (3.6.15). It's latest regular bugfix release (3.6.8) was Dec. 2018. So official support was droped 4 years ago.
• Python 3.7: It is the oldes supported Python release. It is now only in the security fix phase. The final support end will be next year in June 2023.

Ubuntu

Disclaimer: It is not a secret that I have "my problems" with Ubuntu. But I was using it for many years so I have some experience with it and my opinions about it. But I'll try to be objective here. 😅

What does LTS mean in Ubuntu?

Usually LTS versions are supported 5 years. This phase can be extended for paying users with the Extended Security Maintenance (ESM). Looking into Ubuntus Wiki about it Releaes:

• 16.04 LTS suppored ended in April 2021.
• 18.04 is the oldest and still supported LTS. It supports ends next year.
• 20.04 LTS (used by our TravisCI) ends in April 2025.
• 22.04 LTS is the freshest and latest which support ends April 2027.

In fact Ubuntu does have three alive LTS lines. And more then three when you count the ESM for paying users.

What LTS really mean in Ubuntu!

As I mentioned on bit-dev as an example. Ubuntu 18 is shipped with Python 3.6.7 which has known security issues. Python 3.6.18 is still out there but Ubuntu doesn't fix it. So Ubuntu-LTS is for the trash only and doesn't mean anything. It is a marketing buzzword. They are a company with payed people and not able to even do security fixes of essential system elements.

Debian

Sorry I have to do this. ;) The image of Debian is to be old, slow and conservative. And that image isn't wrong. But comparing the supported software versions between Debian and Ubuntu (the oldest LTS) the latter is much more outdated then Debian.

Core of the universe

IMHO the question we need to answer is what is the core of the universe for us at BackInTime upstream? The market, security or our ressources?

My opinion

Let's imagine that Ubuntu and all other distribution doing perfect LTS releases. An LTS shouldn't be an orientation for upstream developers. LTS is the responsibilty of the GNU/Linux distribution. When they label a release "LTS" then they are responsible to check for security issues, fixing them and even backporting the fixes if upstream doesn't support the used version anymore (e.g. the python interpreter). Upstream is IMHO in depended of the LTS concept.

It should also be considered that the "backintime" package "in Ubuntu" isn't real Ubuntu. It is a community package. It is packed by persons in there spare time and they don't get payed for it. Canonical (as the Ubuntu company) even don't have to do the package and just should use the original Debian package. But they refuse to officialy support Back In Time. The Ubuntu users often don't know or don't understand about the difference between the real Ubuntu and the community packages.

BIT is not part of official Ubuntu. Even if it would be a part of it Canonical itself is responsible in the context of its LTS versions to keep BIT's security and heavy bugs up-2-date with our upstream repo.

The official support of Python by the Python Software Foundation weights much more than the "support" and use of Python in different GNU/Linux distributions. It is not our responsibility if some distros acting unsecure and "supporting" outdated Python interpreters.

Currently we technically support Python 3.6 (which is dead) because some outdated (but "supported") Distros do use it. As you can see I'm not happy about that but it is as it is.

I don't see a value but just much pain when stepping back into the past and also reactivate the Python 3.5 support for BIT.

Considering our ressources and our release cycles I would suggest to leave the Python support (>=3.6) as it is for the next BIT release (1.3.3) and drop Python 3.6 and 3.7 for the release after that.

Python 3.7 support ends next year. Even 3.8 and 3.9 are just only in security phase. Our next release after the next (1.3.4; "das übernächste") will take time and will take much more time until it reach the package repositories of Distros like Debian and Ubuntu. Rolling Release distros are no problem because the use the latest release from us and from Python also.

[aryoda]

I am not quite sure if "old" LTS versions must really use old Python versions or if there is an upgrade path (eg. via Ubuntu point releases).

On the practical side: Should we add a check for the python version when BiT is started
and throw an error for unsupported python versions?

Currently users do get only strange tracebacks instead of a clear error message in case of old python versions (see Germar's example above)...

[emtiu]

I am not quite sure if "old" LTS versions must really use old Python versions or if there is an upgrade path (eg. via Ubuntu point releases).

On the practical side: Should we add a check for the python version when BiT is started and throw an error for unsupported python versions?

Currently users do get only strange tracebacks instead of a clear error message in case of old python versions (see Germar's example above)...

That's a good idea. It could save a lot of time for users and us to have a clear message about this.

[buhtz]

On the practical side: Should we add a check for the python version when BiT is started and throw an error for unsupported python versions?

Kind of but not in code.

Usually you have a setup.py/setup.cfg/project.toml/requirements.txt where you add the minimal python version. We don't have that yet because our repo is in terms of today Python Packaging recommendations not a Python project.

We use make. So I would say make/configure should check for the correct python version.

Of course someone could run BIT out of the repo without using make or any other install-process before. We can't catch that rare and unusual cases yet.

[Germar]

AFAIK Ubuntu has his own Security-Team which backports security fixes from upstream into the released versions (same as Debian does). So even as Python 3.5 is not supported by the Python Foundation anymore, it still get security-fixes. Also you get ESM Support for up to 5 machines for free. I use this because I didn't found time to upgrade some 16.04 instances yet.

Anyways! I'm fine with ditching those old Versions and Releases. I just thought a simple change in formatting shouldn't be the reason for ditching it.

[buhtz]

@aryoda You are more the make person. 😄

Can you implement kind of a Python-version check in the configure/make files please?

After that I would say we can close this issue?

[aryoda]

I have assigned this issue to myself (currently with low prio)

[buhtz]

Just a note for myself. Python 3.6 is not shipped with Ubuntu 22.04 "Jammy".
When migrating TravisCI from Ubuntu 20 to 22 we need a extra machine for Python 3.6.
See PR #1415

[buhtz]

After our latest release I want to warm up my arguments for restricting BIT's Python support to version >= 3.8 and dropping 3.7 and older.

• Python 3.7 support (by Python itself) ends this year.
• Python 3.8 and 3.9 are just only in security phase. Even we support them this versions are kind of "old" and not in active development anymore.

Of course users preferences and environments are also important. But in our situation IMHO the primary argument is our limited resource. IMHO no one gets hurt if we drop that old Python versions.

EDIT: My point is that currently dev is not released and IMHO won't get released before next year. Support for >= 3.8 means what we support in the future for the upcoming release. "Support" for Python 3.7 ends 2023-06-27 (PEP527); next months. And also 3.8 and 3.9 do only get security fixes and not actively developed anymore.