install.sh does not work on Ubuntu 22.04 LTS (Jammy Jellyfish) due to python 3.10 (IDFGH-7994)

[cfriedt]

Environment

• Development Kit: N/A
• Kit version: N/A
• Module or chip used: N/A
• IDF version 7bfb89e
• Build System: N/A
• Compiler version: N/A
• Operating System: Linux
• (Windows only) environment type: N/A
• Using an IDE?: No
• Power Supply: N/A

Problem Description

The esp-idf fails to install under Ubuntu Jammy due to python 3.10. Namely the python3.10-venv package no longer exists (fwir it has been integrated into python?).

It's non-obvious to me how to solve this issue in the install.sh script.

Expected Behavior

esp-idf should be installed

Actual Behavior

esp-id install fails

Steps to reproduce

1. install / upgrade to Ubuntu 22.04
2. try to install esp-idf

Code to reproduce this issue

./install.sh
Skipping openocd-esp32@v0.11.0-esp32-20220706 (already installed)
Installing Python environment and packages
Python 3.10.4
/home/cfriedt/.espressif/python_env/idf5.0_py3.10_env/bin/python: No module named pip
WARNING: pip is not available in the existing virtual environment, new virtual environment will be created.
WARNING: Removing the existing Python environment in /home/cfriedt/.espressif/python_env/idf5.0_py3.10_env
Creating a new Python environment in /home/cfriedt/.espressif/python_env/idf5.0_py3.10_env
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/home/cfriedt/.espressif/python_env/idf5.0_py3.10_env/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

Traceback (most recent call last):
  File "/home/cfriedt/workspace/esp-idf/tools/idf_tools.py", line 2517, in <module>
    main(sys.argv[1:])
  File "/home/cfriedt/workspace/esp-idf/tools/idf_tools.py", line 2509, in main
    action_func(args)
  File "/home/cfriedt/workspace/esp-idf/tools/idf_tools.py", line 1991, in action_install_python_env
    subprocess.check_call([sys.executable, '-m', 'venv',
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'venv', '--clear', '--upgrade-deps', '/home/cfriedt/.espressif/python_env/idf5.0_py3.10_env']' returned non-zero exit status 1.

sudo apt install python3.10-venv
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package python3.10-venv is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'python3.10-venv' has no installation candidate

Debug Logs

See above

Other items if possible

N/A

[cfriedt]

From microsoft/WSL#8327

The change below fixed it on my machine, which will unblock me for the time being. Note: this problem should not be considered "solved" as upstream is still broken. Please modify the fix as you see fit.

diff --git a/tools/idf_tools.py b/tools/idf_tools.py
index 698b4a6d12..e9063066ac 100755
--- a/tools/idf_tools.py
+++ b/tools/idf_tools.py
@@ -1551,6 +1551,7 @@ def action_install_python_env(args):  # type: ignore
         if with_seeder_option:
             virtualenv_options += ['--seeder', 'pip']
 
+        os.environ['SETUPTOOLS_USE_DISTUTILS']='stdlib'
         subprocess.check_call([sys.executable, '-m', 'virtualenv',
                                *virtualenv_options,
                                idf_python_env_path],

[dobairoland]

Thanks @cfriedt for your report. We will try to reproduce it and get back to you.

[dobairoland]

The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

This message is not printed by ESP-IDF.

The package does exist for Ubuntu 22.04 LTS: https://packages.ubuntu.com/jammy/python3.10-venv

My guess is you don't have neither security nor universe Ubuntu repositories enabled. Here is a guide how to do it: https://itsfoss.com/ubuntu-repositories/

[cfriedt]

This message is not printed by ESP-IDF.

I copied the wrong error message after updating, sorry, but the message was printed from the command that install.sh ran (see stack trace above).

My guess is you don't have neither security nor universe Ubuntu repositories enabled. Here is a guide how to do it: https://itsfoss.com/ubuntu-repositories/

Both repositories were enabled.

sudo apt install -y python3.10-venv
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help resolve the situation:

The following packages have unmet dependencies:
 python3.10-venv : Depends: python3.10 (= 3.10.4-3) but 3.10.4-3ubuntu0.1 is to be installed
E: Unable to correct problems, you have held broken packages.

In any case, the provided fix worked for me.

[dobairoland]

Thank you for the additional information. I'm sorry but it still looks like an Ubuntu issue. It is well explained by the message.

The solution should be to run apt update to update the databases and have compatible python3.10 and python3.10-venv packages. It could help to reinstall them if they are really broken.

[cfriedt]

@dobairoland - yes, I think it was an Ubuntu issue. It seems that their upgrade mechanism rarely ever works smoothly. I just did a fresh install and cannot reproduce this issue.