import setuptools fails on importing distutils on an installed Python 3.12 (without distutils)

[vstinner]

Install a fresh Python 3.12:

./configure --with-pydebug --prefix /opt/py312
make
make install

import setuptools fails on importing distutils on this fresh Python 3.12 (without distutils):

$ /opt/py312/bin/python3 
Python 3.12.0a1+ (heads/main:f09da28768, Nov  4 2022, 17:22:12) [GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/py312/lib/python3.12/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/__init__.py", line 77, in do_override
    ensure_local_distutils()
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/__init__.py", line 60, in ensure_local_distutils
    importlib.import_module('distutils')
  File "/opt/py312/lib/python3.12/importlib/__init__.py", line 124, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils'

This setuptools installation miss distutils. I guess that make install used ensurepip to install setuptools. I have these versions:

$ /opt/py312/bin/python3 -m pip list
Package    Version
---------- -------
pip        22.3
setuptools 65.5.0

ensurepip has:

$ grep _VERSION Lib/ensurepip/__init__.py 
_SETUPTOOLS_VERSION = "65.5.0"
_PIP_VERSION = "22.3"

[vstinner]

I just removed distutils packages: commit.

[vstinner]

If I install Python without ensurepip:

./configure --with-pydebug --prefix /opt/py312 --with-ensurepip=no
make
make install

Well, pip and setuptools are missing, as expected:

$ /opt/py312/bin/python3
>>> import pip
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pip'

>>> import setuptools
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools'

If I install pip manually:

cd /opt/py312
wget https://bootstrap.pypa.io/get-pip.py 
/opt/py312/bin/python3 get-pip.py 

In this case, pip and setuptools work as expected:

$ /opt/py312/bin/python3 
Python 3.12.0a1+ (heads/main:f09da28768, Nov  4 2022, 17:33:42) [GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools
<module 'setuptools' from '/opt/py312/lib/python3.12/site-packages/setuptools/__init__.py'>

>>> import distutils
>>> distutils
<module 'distutils' (/opt/py312/lib/python3.12/site-packages/setuptools/_distutils/__init__.py)>

distutils is available directly thanks to /opt/py312/lib/python3.12/site-packages/distutils-precedence.pth:

$ /opt/py312/bin/python3 
Python 3.12.0a1+ (heads/main:f09da28768, Nov  4 2022, 17:33:42) [GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import distutils
>>> distutils
<module 'distutils' (/opt/py312/lib/python3.12/site-packages/setuptools/_distutils/__init__.py)>

$ cat /opt/py312/lib/python3.12/site-packages/distutils-precedence.pth
import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'local') == 'local'; enabled and __import__('_distutils_hack').add_shim(); 

[vstinner]

Oh wait, I can reproduce the issue if pip is imported before setuptools:

$ /opt/py312/bin/python3 
Python 3.12.0a1+ (heads/main:f09da28768, Nov  4 2022, 17:33:42) [GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
>>> import setuptools
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/py312/lib/python3.12/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/__init__.py", line 77, in do_override
    ensure_local_distutils()
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/__init__.py", line 60, in ensure_local_distutils
    importlib.import_module('distutils')
  File "/opt/py312/lib/python3.12/importlib/__init__.py", line 124, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils'

[vstinner]

Hum, I misunderstood the problem. I opened pypa/setuptools#3661 instead.