install-poetry.py fails with "no module named 'virtualenv'" when run with Python from the Microsoft Store

[yrro]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10 20H2 (10.0.19042.1052)

• Poetry version: N/A

• Link of a Gist with the contents of your pyproject.toml file: N/A

Issue

I'm trying the new installer with Pyton 3.9 from the Microsoft Store.

PS C:\> get-command python

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     python.exe                                         0.0.0.0    C:\Users\sam\AppData\Local\Microsoft\WindowsApps\python.exe

PS C:\> (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py -UseBasicParsing).Content | python
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Scripts

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

You are installing 1.1.6. When using the current installer, this version does not support updating using the 'self update' command. Please use 1.2.0a1 or later.
Installing Poetry (1.1.6)
Installing Poetry (1.1.6): Creating environment
Traceback (most recent call last):
  File "<stdin>", line 831, in <module>
  File "<stdin>", line 827, in main
  File "<stdin>", line 440, in run
  File "<stdin>", line 462, in install
  File "<stdin>", line 526, in make_env
ModuleNotFoundError: No module named 'virtualenv'

When install-poetry.py runs pip to install the virtualenv module, it doesn't check the return code. If I change the installer to use subprocess.check_call then I get:

Installing Poetry (1.1.6): Creating environment
An error has occured: Command '['C:\\Users\\sam\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\python.exe', '-m', 'pip', 'install', 'virtualenv', '-t', 'C:\\Users\\sam\\AppData\\Local\\Temp\\tmpsno44236']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "c:\users\sam\appdata\local\temp\install-poetry.py", line 440, in run
    self.install(version)
  File "c:\users\sam\appdata\local\temp\install-poetry.py", line 462, in install
    env_path = self.make_env(version)
  File "c:\users\sam\appdata\local\temp\install-poetry.py", line 518, in make_env
    subprocess.check_call(
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:\\Users\\sam\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\python.exe', '-m', 'pip', 'install', 'virtualenv', '-t', 'C:\\Users\\sam\\AppData\\Local\\Temp\\tmpsno44236']' returned non-zero exit status 1.

... but I've no idea where the output of pip is disappearing to.

If I try to run pip manually, I get...

PS C:\Users\sam\AppData\Local\Temp> python -m pip install virtualenv -t vvv
ERROR: Can not combine '--user' and '--target'
WARNING: You are using pip version 21.1.1; however, version 21.1.2 is available.
You should consider upgrading via the 'C:\Users\sam\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe -m pip install --upgrade pip' command.

No idea what's going on there. But I think the installer should have caught the error & displayed it to me.

[yrro]

Root cause: C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\pip.ini contains:

[install]
user=yes
[uninstall]
user=yes
[freeze]
user=yes
[list]
user=yes

[yrro]

FWIW, with my fix, the installed poetry.exe doesn't actaully work:

PS C:\Users\sam\AppData\Local\Temp> python install-poetry.py
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Scripts

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

You are installing 1.1.6. When using the current installer, this version does not support updating using the 'self update' command. Please use 1.2.0a1 or later.
Do you want to continue? ([y]/n) y
Installing Poetry (1.1.6)
Installing Poetry (1.1.6): Creating environment
Installing Poetry (1.1.6): Installing Poetry
Installing Poetry (1.1.6): Creating script
Installing Poetry (1.1.6): Done

Poetry (1.1.6) is installed now. Great!

To get started you need Poetry's bin directory (C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Scripts) in your `PATH`
environment variable.

Alternatively, you can call Poetry explicitly with `C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Scripts\poetry`.

You can test that everything is set up by executing:

`poetry --version`

PS C:\Users\sam\AppData\Local\Temp> poetry
poetry: The term 'poetry' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
PS C:\Users\sam\AppData\Local\Temp> C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Scripts\poetry
ResourceUnavailable: Program 'poetry.exe' failed to run: No application is associated with the specified file for this operation.At line:1 char:1
+ C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3 …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.

It looks like poetry.exe is a symlink to a non-existent path:

PS C:\Users\sam\AppData\Local\Temp> ls C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Scripts\poetry.exe

    Directory: C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Scripts

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
la---          22/06/2021    10:40              0 poetry.exe -> C:\Users\sam\AppData\Roaming\pypoetry\venv\Scripts\poetry.exe

PS C:\Users\sam\AppData\Local\Temp> ls  C:\Users\sam\AppData\Roaming\pypoetry\venv\Scripts\poetry.exe
Get-ChildItem: Cannot find path 'C:\Users\sam\AppData\Roaming\pypoetry\venv\Scripts\poetry.exe' because it does not exist.

And indeed C:\Users\sam\AppData\Roaming\pypoetry does not exist. Maybe it should point to C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\Roaming\pypoetry, which does exit -- but that also wouldn't work, because C:\Users\sam\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\Roaming\pypoetry\venv\Scripts\poetry.exe does not exist (although pip.exe, python.exe etc do exist in the same Scripts directory).

... if you think it's worth it trying to get poetry working with the Microsoft Store python, and think my pull request is a sensible way to get past the first hurdle, I'm happy to open a separate issue to figure the next part out...

[sboomi]

I have more or less the same issue with Ubuntu 20.08, when I try to execute the following:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3.8 -

I believe it's because of that part of the installation:

        with temporary_directory() as tmp_dir:
            subprocess.call(
                [sys.executable, "-m", "pip", "install", "virtualenv", "-t", tmp_dir],
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
            )

            sys.path.insert(0, tmp_dir)

            import virtualenv

            virtualenv.cli_run([str(env_path), "--clear"])

        return env_path

The virtualenv installed lacks a special module necessary to proceed with, which is the virtualenv.seed.via_app_data one. I tried uninstalling virtualenv multiple times without any success.

[klDen]

I have the same issue running with the latest script.
Using the previous installer works for me even though it's marked as deprecated :

# latest and fails https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py
docker run -it python:3.7 bash -c "apt -yqq update && apt -y install --no-install-recommends curl &&  curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | /usr/bin/python3 -"

# deprecated and works https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py
docker run -it python:3.7 bash -c "apt -yqq update && apt -y install --no-install-recommends curl &&  curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | /usr/bin/python3 -"

[yrro]

... if you think it's worth it trying to get poetry working with the Microsoft Store python, and think my pull request is a sensible way to get past the first hurdle, I'm happy to open a separate issue to figure the next part out...

Alternatively, if it is not a goal to support Python from the Microsoft Store, I think the installer should bug out early and loudly!

[shniubobo]

Having the same problem on Ubuntu 18.04 WSL.

At first it was because of pip not being installed by default for the system python, but after sudo apt install python3-pip the problem persisted. Then I tried installing virtualenv manually to see what was going on:

$ mktemp -d
/tmp/tmp.YuayGzcNwi
$ python3 -m pip install virtualenv -t /tmp/tmp.YuayGzcNwi
[unrelated logs omitted]
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 360, in run
    prefix=options.prefix_path,
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "/usr/lib/python3/dist-packages/pip/locations.py", line 153, in distutils_scheme
    i.finalize_options()
  File "/usr/lib/python3.6/distutils/command/install.py", line 274, in finalize_options
    raise DistutilsOptionError("can't combine user with prefix, "
distutils.errors.DistutilsOptionError: can't combine user with prefix, exec_prefix/home, or install_(plat)base

But if I use a newer version of pip, virtualenv can be successfully installed:

$ mktemp -d
/tmp/tmp.4j1OAffTGx
$ python3.9 -m pip install virtualenv -t /tmp/tmp.4j1OAffTGx
[unrelated logs omitted]
Installing collected packages: six, platformdirs, filelock, distlib, backports.entry-points-selectable, virtualenv
Successfully installed backports.entry-points-selectable-1.1.0 distlib-0.3.2 filelock-3.0.12 platformdirs-2.2.0 six-1.16.0 virtualenv-20.7.0

$ python3 -m pip -V  # system python with system pip
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
$ python3.9 -m pip -V
pip 21.2.3 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

And with this version of pip, poetry can be successfully installed:

$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3.9 -
[unrelated logs omitted]
Poetry (1.1.7) is installed now. Great!

You can test that everything is set up by executing:

`poetry --version`
$ poetry --version
Poetry version 1.1.7

---

So there are three problems with install-poetry.py:

1. It assumes that pip is installed.
2. It assumes that an up-to-date pip is installed.
3. The pip install error message is suppressed.

One possible solution to the first and second problems: provide a version of virtualenv with all of its dependency vendorized, and let the script download this version of virtualenv directly instead of using pip.

Edit:
Please note that the solution in yrro@dea343d will not work for old version of pip, including the python3-pip package provided by apt. The option --no-user does not exist as of that version.

[yrro]

FYI python -m pip install poetry works fine with Python from the Microsoft Store, so maybe that can be suggested as a way to install with that distribution of Python rather than the installer script?

Downside - you have to put %USERPROFILE%\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts into your user's PATH by hand. IMO this should be done automatically when Python is installed...

[abn]

The root cause of this issue was resolved by #4099. See the bottom part of #4463 (comment) for more information.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.