Elpy fails with Python 3.12, distutils has been removed

[robewald]

Summary

Python 3.12 has removed distutils Therefore the rpc initialization script fails.

Steps to reproduce

My configuration

OS

Mac OS Sonoma 14.6.1
homebrew install pipx python@3.12 virtualenv

Result of (elpy-config)

Elpy Configuration

Emacs.............: 29.4
Elpy..............: Not found (Python), 1.35.0 (Emacs Lisp)
Virtualenv........: py_docker (/Users/robert/.virtualenvs/py_docker/)
Interactive Python: python3 3.12.6 (/Users/robert/.virtualenvs/py_docker/bin/python3)
RPC virtualenv....: rpc-venv (/Users/robert/.emacs.d/elpy/rpc-venv)
 Python...........: python3 nil (/Users/robert/.emacs.d/elpy/rpc-venv/bin/python3)
 Jedi.............: Not found
 Autopep8.........: Not found
 Yapf.............: Not found
 Black............: Not found
Syntax checker....: Not found (flake8)

Warnings

The Python interpreter could not find the elpy module. Please report
to: https://github.com/jorgenschaefer/elpy/issues/new.

There was an unexpected problem starting the RPC process. Please check
the following output to see if this makes sense to you. To me, it
doesn't.

Traceback (most recent call last):
  File "<string>", line 9, in <module>
ModuleNotFoundError: No module named 'distutils'

[genovese]

I too have this problem, which causes spurious syntax error highlighting with new syntax.

From a quick look, it appears that distutils is only needed for LooseVersion, and this is only used for a version comparison. There are two packages that seem to have reasonable and likely sufficient replacements for LooseVersion: packaging.version (in packaging) and looseversion. The former seems more established, so you could perhaps replace

from distutils.version import LooseVersion

with

try:
    from distutils.version import LooseVersion
except ModuleNotFoundError:
    from packaging.version import parse as LooseVersion

or similarly with the looseversion package which is intended as a drop-in replacement.

[genovese]

An update: Making this change solved the problem for me.

I made this adjustment to the variable elpy-config--get-config in elpy.el and made a python 3.12 virtual env ~/.emacs.d/elpy/new-rpc-venv and set elpy-rpc-virtualenv-path to that directory. It turns out that packaging is installed as part of the other installations in the venv (black, flake8, ...) and so the modified code just works.

[genovese]

P.S. elpy/blackutil.py and elpy/jedybackend.py use pkg_resources which is also deprecated and removed in Python 3.12. This raises a messsage from the elpy backend but is not fatal. All that is used is pkg_resources.parse_version, so perhaps packaging.parse can be used instead. Alternatively, pkg_resources has been moved from the standard library to setuptools apparently.

I'll experiment with changing this when I have a moment, but I thought I'd mention in as a follow up to the earlier comments.
Thanks

P.P.S. The new version is recognized and seems to function, but there do appear to still be some issues using the new version. I haven't tracked down the problem, so it's not as cleancut as I had hoped.

[genovese]

I've seemingly solved the problem with three changes.

1. The adjustment to elpy.el described above

try:
    from distutils.version import LooseVersion
except ModuleNotFoundError:
    from packaging.version import parse as LooseVersion

2. In elpy/blackutil.py, replace the pkg_resources import attempt with

try:
    from pkg_resources import parse_version
except ImportError:  # pragma: no cover
    try:
        from packaging.version import parse as parse_version
    except ImportError:  # pragma: no cover
        def parse_version(*arg, **kwargs):
            raise Fault("Neither `packaging` nor`pkg_resources` could be imported, "
                        "please reinstall Elpy RPC virtualenv with"
                        " `M-x elpy-rpc-reinstall-virtualenv`", code=400)

The packaging library seems to be in the rpc venv, so just need to make sure it is in the venv used, which it seems to be without effort in my experience when syntax checkers and such are added.
3. In elpy/jedibackend.py, do the same thing as the previous item; exactly the same code and need for packaging.

I've been using this for a bit without difficulty and can finally upgrade to the newer versions for the RPC venv.
The last two additions are important; otherwise one gets repeated process sentinel errors.