Poetry does not use pyenv virtualenvs

[simonmeggle]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Mac OS arm64 (M1)

• Poetry version: 1.1.13

Issue

I have installed poetry with curl -sSL https://install.python-poetry.org | python3 - and it is located in /Users/simon/.poetry/bin/poetry.

I want poetry to install the dependencies into the current virtual environment.

My poetry config:

❯ poetry config --list
cache-dir = "/Users/simon/Library/Caches/pypoetry"
experimental.new-installer = true
installer.parallel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/simon/Library/Caches/pypoetry/virtualenvs

My pyproject.toml:

❯ cat pyproject.toml
[tool.poetry]
name = "testproject"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.10"
robotframework = "5"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

# Homebrew default python 
❯ python -V
Python 2.7.18
❯ mkdir testproject && cd $?
# create a venv with python 3.10.5
❯ pyenv virtualenv 3.10.5 venv-testproject
# stick this environment to this project
❯ pyenv local venv-testproject
❯ python -V
Python 3.10.5
❯ which poetry
/Users/simon/.poetry/bin/poetry
❯ poetry --version
Poetry version 1.1.13
# now try to install the dependencies into the active venv
❯ poetry install -vvv
Creating virtualenv testproject-zM3EHoG7-py3.10 in /Users/simon/Library/Caches/pypoetry/virtualenvs
Using virtualenv: /Users/simon/Library/Caches/pypoetry/virtualenvs/testproject-zM3EHoG7-py3.10
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 1 install, 0 updates, 0 removals

  • Installing robotframework (5.0): Downloading... 100%
  • Installing robotframework (5.0): Installing...
  • Installing robotframework (5.0)
...
...

And here we are. All the blogs and tutorials I have read so far tell me that poetry should use the currently active venv.
But instead, it creates another virtualenv in /Users/simon/Library/Caches/pypoetry/virtualenvs/.

I also discovered the fact that poetry installed by Homebrew does not work because it has a hardcoded Python3.9 interpreter.
Poetry is now installed as recommended - but it does not want to play with pyenv's virtualenvs.

Any clue what is wrong here?
Thanks & regards,
Simon

[simonmeggle]

I also installed the current 1.2.0b2 version and set the new experimental option:

poetry config virtualenvs.prefer-active-python true

But even with that, it sticks on creating own environments.

[finswimmer]

Hello @simonmeggle,

I've never used pyenv virtualenv until now, but I guess it will neither create a .venv folder within the project nor set the env variable VIRTUAL_ENV. If that's the case, there is no (generic) way for Poetry to know about the venv provided through pyenv.

fin swimmer

[simonmeggle]

If that's the case, there is no (generic) way for Poetry to know about the venv provided through pyenv.

I can't quite agree with that. Even if I explicitly tell poetry to use the environment...

poetry env use $(pyenv which python)

...it doesn't work.

According to the release notes of 1.2.0b1 ("Added experimental option virtualenvs.prefer-active-python to detect current activated python") it should work exactly like this.

...or have I misunderstood something?

[dimbleby]

This looks as though it is duplicating #5947: that one claims that if you set virtualenvs.prefer-active-python to true then some things work with pyenv and some don't.

(If they are indeed duplicates, this one is clearer so probably better to close the other)

[simonmeggle]

OK, seems to be related to pyenv-virtualenv. See this issue:
pyenv/pyenv-virtualenv#348

[johnziebro]

Consider using the the Poetry Preview Release 1.2.0b1 with config prefer-active-python as described here.

[simonmeggle]

Consider using the the Poetry Preview Release 1.2.0b1 with config prefer-active-python as described here.

Please see this issue at pyenv-virtualenv where I am already using version 1.2.0b2 of poetry.

The setting virtualenvs.prefer-active-python is set to false in both examples (the working and non-working one).
If I set $VIRTUAL_ENV by hand (or: in the script), Poetry uses the active venv as expected - regardless of prefer-active-python.

[neersighted]

This is not a Poetry issue / the root cause has been identified -- when you poetry env use <python> the actual Python executable is resolved, so pyenv-virtualenv's method of 'entering' a virtual env cannot be detected.

Whether or not this is desirable can be revisited at a later date, but for now this is the expected and documented behavior and pyenv-virtualenv really should use a supported method of setting the virtual environment (as overriding the python executable will cause many tools to 'miss' the virtualenv).