Can't generate code with python 3.12

[chaen]

When running with python 3.12, the code generation fails, while it works fine with python 3.11

$ autorest --python --input-file=/tmp/pytest-of-chaen/pytest-4/test_regenerate_client0/openapi.json --models-mode=msrest --namespace=client --output-folder=/home/chaen/dirac/diracx-project/diracx/diracx-client/src/diracx 
AutoRest code generation utility [cli version: 3.7.1; node: v20.9.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    | AutoRest core version selected from configuration: ^3.7.0.
installing core... [========================================] 100% | 2/2
info    |    Loading AutoRest core      '/home/chaen/.autorest/@autorestcore@3.10.2/nodemodules/@autorest/core/dist' (3.10.2)
info    |    Installing AutoRest extension '@autorest/python' (latest -> 6.13.6)
installing... [========================================] 100% | 16/16
info    |    Installed AutoRest extension '@autorest/python' (latest->6.13.6)
info    |    Installing AutoRest extension '@autorest/modelerfour' (~4.23.5 -> 4.23.7)

info    |    Installed AutoRest extension '@autorest/modelerfour' (~4.23.5->4.23.7)
warning | PreCheck/SchemaMissingType | The schema 'Bodyauth_token-granttype' with an undefined type and 'allOf'/'anyOf'/'oneOf' is a bit ambiguous. This has been auto-corrected to 'type:object'

[...]

warning | Modeler/MissingType | The schema 'paths·11v7oic·api-jobs-job_id-remove·post·responses·200·content·application-json·schema' has no type or format information whatsoever. Location:
   file:///tmp/pytest-of-chaen/pytest-4/testregenerate_client0/openapi.json#/components/schemas/paths·11v7oic·api-jobs-jobid-remove·post·responses·200·content·application-json·schema
API Exception: {'type': 'AttributeError', 'args': ("module 'pkgutil' has no attribute 'ImpImporter'",), 'message': "module 'pkgutil' has no attribute 'ImpImporter'"}
Traceback (most recent call last):
  File "/home/chaen/.autorest/@autorest_python@6.13.6/node_modules/@autorest/python/venv/lib/python3.12/site-packages/jsonrpc/manager.py", line 120, in _get_responses
    result = method(*request.args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chaen/.autorest/@autorest_python@6.13.6/node_modules/@autorest/python/autorest/jsonrpc/server.py", line 48, in Process
    from ..m2r import M2RAutorest as PluginToLoad
  File "/home/chaen/.autorest/@autorest_python@6.13.6/node_modules/@autorest/python/autorest/m2r/__init__.py", line 11, in <module>
    import m2r2
  File "/home/chaen/.autorest/@autorest_python@6.13.6/node_modules/@autorest/python/venv/lib/python3.12/site-packages/m2r2.py", line 16, in <module>
    from pkg_resources import get_distribution
  File "/home/chaen/.autorest/@autorest_python@6.13.6/node_modules/@autorest/python/venv/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2191, in <module>
    register_finder(pkgutil.ImpImporter, find_on_path)
                    ^^^^^^^^^^^^^^^^^^^
AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
fatal   | Process() cancelled due to failure 
error   |   Error: Server error
error   | Autorest completed with an error. If you think the error message is unclear, or is a bug, please declare an issues at https://github.com/Azure/autorest/issues with the error message you are seeing.

The issue seems to come from the setuptools version pinned for the virtualenv.
The pinning was done in #2296 (but I could not find a reason to it) and then updated in 8a1fcfe ?

If I update the setuptools version in the venv, it works

$ /home/chaen/.autorest/@autorest_python@6.13.6/node_modules/@autorest/python/venv/bin/python3 -m pip install --upgrade setuptools
Requirement already satisfied: setuptools in /home/chaen/.autorest/@autorest_python@6.13.6/node_modules/@autorest/python/venv/lib/python3.12/site-packages (65.5.1)
Collecting setuptools
  Using cached setuptools-69.2.0-py3-none-any.whl.metadata (6.3 kB)
Using cached setuptools-69.2.0-py3-none-any.whl (821 kB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 65.5.1
    Uninstalling setuptools-65.5.1:
      Successfully uninstalled setuptools-65.5.1
Successfully installed setuptools-69.2.0

$ autorest --python --input-file=/tmp/pytest-of-chaen/pytest-4/test_regenerate_client0/openapi.json --models-mode=msrest --namespace=client --output-folder=/home/chaen/dirac/diracx-project/diracx/diracx-client/src/diracx 
AutoRest code generation utility [cli version: 3.7.1; node: v20.9.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    | AutoRest core version selected from configuration: ^3.7.0.
info    |    Loading AutoRest core      '/home/chaen/.autorest/@autorestcore@3.10.2/nodemodules/@autorest/core/dist' (3.10.2)
info    |    Loading AutoRest extension '@autorest/python' (latest->6.13.6)
info    |    Loading AutoRest extension '@autorest/modelerfour' (~4.23.5->4.23.7)
warning | PreCheck/SchemaMissingType | The schema 'Bodyauth_token-granttype' with an undefined type and 'allOf'/'anyOf'/'oneOf' is a bit ambiguous. This has been auto-corrected to 'type:object'

[...]

warning | Modeler/MissingType | The schema 'paths·11v7oic·api-jobs-job_id-remove·post·responses·200·content·application-json·schema' has no type or format information whatsoever. Location:
   file:///tmp/pytest-of-chaen/pytest-4/testregenerate_client0/openapi.json#/components/schemas/paths·11v7oic·api-jobs-jobid-remove·post·responses·200·content·application-json·schema
warning | [autorest.codegen.models.basebuilder.__init__:81] Not going to generate operation "build_auth_token_request" because we are unable to generate this type of operation right now. Please write your own custom operation in the "patch.py" file following https://aka.ms/azsdk/python/dpcodegen/python/customize
warning | [autorest.codegen.models.basebuilder.__init__:81] Not going to generate operation "token" because we are unable to generate this type of operation right now. Please write your own custom operation in the "patch.py" file following https://aka.ms/azsdk/python/dpcodegen/python/customize
info    | Autorest completed in 9.96s. 43 files generated.

Could the version be relaxed again, or updated ?

Thanks a lot

Detailed versions:

$ python --version
Python 3.12.2

$ autorest --version
AutoRest code generation utility [cli version: 3.7.1; node: v20.9.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest


Showing All Installed Extensions

 Type       Extension Name                           Version      Location 
 extension  @autorest/python                         6.13.6       /home/chaen/.autorest/@autorest_python@6.13.6
 extension  @autorest/modelerfour                    4.23.7       /home/chaen/.autorest/@autorest_modelerfour@4.23.7
 core       @autorest/core                           3.10.2       /home/chaen/.autorest/@autorest_core@3.10.2

[iscai-msft]

@chaen this is a known issue with python 3.12. You can follow the steps here to attempt to have pip work with python 3.12. If those don't work, I would downgrade to python 3.11.

I'm going to close this issue because it's 3.12 related, not related to the python emitter. Thanks!

[chaen]

@iscai-msft I am afraid the problem is not pip.
This was reported to pip to which they reply the problem is in setuptools (pypa/pip#12179 (comment))
and setuptools fixed it in v66.1.0

[chaen]

I've tried a PR to see if the tests are still passing with a more recent version #2455