Python 3.12 Support

[snej]

🐞Describing the bug

• I'm following the directions at "Installing Core ML Tools".
• I went through "Set Up Conda": created the virtual environment, ran conda install pip.
• Then when I ran pip install -U coremltools the installation failed with a Python exception in the installer:

(coremltools-env) $  pip install -U coremltools
Collecting coremltools
  Downloading coremltools-7.1.tar.gz (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 6.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/7d/hl3k4nxj5_72y9dcxt0brwwm0000gp/T/pip-install-te23914i/coremltools_7669cbec5c1f436d949cfe9db26f0157/setup.py", line 8, in <module>
          import imp
      ModuleNotFoundError: No module named 'imp'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Stack Trace

      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/7d/hl3k4nxj5_72y9dcxt0brwwm0000gp/T/pip-install-te23914i/coremltools_7669cbec5c1f436d949cfe9db26f0157/setup.py", line 8, in <module>
          import imp
      ModuleNotFoundError: No module named 'imp'

System environment (please complete the following information):

• Conda 23.11.0
• Python 3.12.1 (installed by Conda)
• macOS Sonoma 14.2.1
• MacBook Pro 16", 2021 (M1 Pro, 32GB RAM)

[snej]

I'm not a Python expert, but I looked up the module imp and it's been deprecated since Python 3.4, and no longer exists at all in the Python 3.12.1 that Conda installed.

(coremltools-env) $  python --version
Python 3.12.1
(coremltools-env) $  python
Python 3.12.1 | packaged by Anaconda, Inc. | (main, Jan 19 2024, 09:45:58) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'imp'

[TobyRoseman]

We don't support Python 3.12 yet. That is the root cause here.

Related to #1846

[snej]

I was able to work around the problem by running conda install python=3.11 -- after that, installing coremltools ran successfully.

[snej]

If this is considered an enhancement, then the bug is on the Installing CoreML Tools web page -- it should tell people to run conda install python=3.11 before installing coremltools. Otherwise anyone following the directions on that web page is going to run into the same problem I did. Is there a separate repo for the docs where this bug can be filed?

[TobyRoseman]

@snej - Right at the top of the document you reference, it says:

The current version of coremltools  includes wheels for Python 3.7, 3.8, 3.9, 3.10, and 3.11.

[johnnynunez]

tensorflow 2.16.1 is out supporting 3.12

[teelrabbit]

This was addressed in #2170 @YifanShenSZ

[TobyRoseman]

This was addressed in #2170 @YifanShenSZ

There are several other things that need to be addressed before we can support Python 3.12. At a minimum, we still need to update all of our test dependencies to versions that support Python 3.12

[ambitious-octopus]

What's the status of python 3.12 support?

[johnnynunez]

What's the status of python 3.12 support?

3.13 is out in October

[YifanShenSZ]

Our 8.1 release has added python 3.12 support 🎊 Please try it out

[alexmyczko]

@YifanShenSZ it builds fine, however on Debian sid i get the following:

$ python
Python 3.12.7 (main, Nov  8 2024, 17:55:36) [GCC 14.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import coremltools
Fail to import BlobReader from libmilstoragepython. No module named 'coremltools.libmilstoragepython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelAssetProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLCPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLGPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLNeuralEngineComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
Fail to import BlobWriter from libmilstoragepython. No module named 'coremltools.libmilstoragepython'
Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
Failed to load _MLComputePlanProxy: No module named 'coremltools.libcoremlpython'
>>> 

[YifanShenSZ]

@alexmyczko would ubuntu work for you? We build and test linux wheels on ubuntu

If you have to stick to Debian sid 🙏 could you please file an issue and share reproduce details