Poetry not using specified virtual environment

[TNonet]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name:

>>>cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

• Poetry version: 1.1.14
• Link of a Gist with the contents of your pyproject.toml file:

Issue

Poetry seems to use a different python version when installing packages than specified in the install output:

poetry install -vvv
The currently activated Python version 3.10.6 is not supported by the project (>=3.9,<3.10).
Trying to find and use a compatible version.
Trying python3
Using python3 (3.9.13)
Creating virtualenv [PACKAGE]-rMAC0O7B-py3.9 in /home/circleci/.cache/pypoetry/virtualenvs
Using virtualenv: /home/circleci/.cache/pypoetry/virtualenvs/[PACKAGE]-rMAC0O7B-py3.9
Installing dependencies from lock file

Finding the necessary packages for the current system

  Stack trace:

  8  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  7  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  6  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│
      173│     def __repr__(self):  # type: () -> str

  5  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│
    →  92│         return self.handle()
       93│
       94│     def handle(self):  # type: () -> Optional[int]

  4  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/install.py:71 in handle
       69│         self._installer.verbose(self._io.is_verbose())
       70│
    →  71│         return_code = self._installer.run()
       72│
       73│         if return_code != 0:

  3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:103 in run
      101│         local_repo = Repository()
      102│
    → 103│         return self._do_install(local_repo)
      104│
      105│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

  2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:305 in _do_install
      303│
      304│         with solver.use_environment(self._env):
    → 305│             ops = solver.solve(use_latest=self._whitelist)
      306│
      307│         # We need to filter operations so that packages

  1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:65 in solve
       63│         with self._provider.progress():
       64│             start = time.time()
    →  65│             packages, depths = self._solve(use_latest=use_latest)
       66│             end = time.time()
       67│

  SolverProblemError

  The current project's Python requirement (3.10.6) is not compatible with some of the required packages Python requirement:
    - tfcausalimpact requires Python >=3, <3.10, so it will not be satisfied for Python 3.10.6

  Because tfcausalimpact (0.0.9) requires Python >=3, <3.10
   and no versions of tfcausalimpact match >0.0.9,<0.0.10, tfcausalimpact is forbidden.
  So, because [PACKAGE] depends on tfcausalimpact (^0.0.9), version solving failed.

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

  • Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties

    For tfcausalimpact, a possible solution would be to set the `python` property to "<empty>"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

Poetry claims to use Using python3 (3.9.13), but when installing, it uses the 3.10.6 python.

  system
* 3.9.13 (set by /home/circleci/.pyenv/version)
  3.10.6

[alealv]

I have also the same problem!

[finswimmer]

Hey @TNonet,

please try with the current preview release 1.2.0rc1. I guess the reason for the issue is similar to what's described in this fix #5086.

fin swimmer

[john-sandall]

Linking to my comment here in case it's the same issue: #6565 (comment)

I found Poetry could not find the virtualenv it was being run from, but if I change the name key of [tool.poetry] of pyproject.toml to something completely different, then it all works fine. Be interesting to know if anyone else has this issue and can also fix it by changing this same value in pyproject.toml

[neersighted]

@john-sandall you have a different issue -- please open a new issue if you can reproduce it in a container.

[joshorr]

@finswimmer @neersighted This may help, I have a way to repo this situation:

You first need pyenv installed, and then python 3.10 + 3.9 installed via pyenv.

You then activate both versions of python:

:~> pyenv install 3.9.14
:~> pyenv install 3.10.7
:~> pyenv versions
  system
  3.9.14 (set by /Users/myuser/.pyenv/version)
  3.10.7 (set by /Users/myuser/.pyenv/version)

Then activate both versions, so you can use either version depending on the particular project's requirements. I work on multiple projects and some need 3.10 and others need 3.9. Need to activate both so pyenv knows what minor version of python to use for 3.9 and 3.10:

:~> pyenv global 3.9.14 3.10.7
  system
* 3.9.14 (set by /Users/myuser/.pyenv/version)
* 3.10.7 (set by /Users/myuser/.pyenv/version)

Next, you create a basic project via poetry, but tell it the python version to use is ~3.9:

:~> cd src/o
:~/src/o> mkdir example-proj
:~/src/o> cd example-proj/
:~/src/o/example-proj> mkdir example_proj
:~/src/o/example-proj> touch example_proj/__init__.py
:~/src/o/example-proj> touch README.md
:~/src/o/example-proj> poetry init

This command will guide you through creating your pyproject.toml config.

Package name [example-proj]:  
Version [0.1.0]:  
Description []:  
Author [..., n to skip]:  
License []:  
Compatible Python versions [^3.10]:  ~3.9

Would you like to define your main dependencies interactively? (yes/no) [yes] no
Would you like to define your development dependencies interactively? (yes/no) [yes] no
Generated file

[tool.poetry]
name = "example-proj"
version = "0.1.0"
description = ""
authors = [""]
readme = "README.md"
packages = [{include = "example_proj"}]

[tool.poetry.dependencies]
python = "~3.9"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"


Do you confirm generation? (yes/no) [yes] 

Then you do a poetry install, it will say that 3.10.7 is not supported, it then find 3.9.14 and tries to use that:

:~/src/o/example-proj> poetry install
The currently activated Python version 3.10.7 is not supported by the project (~3.9).
Trying to find and use a compatible version. 
Using python3 (3.9.14)
Creating virtualenv example-proj-ayXlD45f-py3.9 in /Users/myuser/Library/Caches/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Installing the current project: example-proj (0.1.0)

After dependencies are installed, poetry thinks it's using 3.9, but the actual version of python in the virtual-env is 3.10:

:~/src/o/example-proj> poetry run python --version
The currently activated Python version 3.10.7 is not supported by the project (~3.9).
Trying to find and use a compatible version. 
Using python3 (3.9.14)
Python 3.10.7

I am using poetry 1.2.2:

:~/src/o/example-proj> poetry --version
Poetry (version 1.2.2)

The workaround I've found is if you do poetry env use 3.9 BEFORE you do anything that makes poetry create the virtual environment for the first time it will properly create the virtual environment:

:~/src/o/example-proj> rm -r /Users/myuser/Library/Caches/pypoetry/virtualenvs/example-proj-ayXlD45f-py3.9
:~/src/o/example-proj> poetry env use 3.9
Creating virtualenv example-proj-ayXlD45f-py3.9 in /Users/myuser/Library/Caches/pypoetry/virtualenvs
Using virtualenv: /Users/myuser/Library/Caches/pypoetry/virtualenvs/example-proj-ayXlD45f-py3.9
:~/src/o/example-proj> poetry install
Installing dependencies from lock file

Installing the current project: example-proj (0.1.0)
:~/src/o/example-proj> poetry run python --version
Python 3.9.14
:~/src/o/example-proj> 

[finswimmer]

@joshorr Thanks a lot for providing this step by step guide 👍

However, I'm not able to reproduce this with Poetry 1.2.2. Which version are you using?

fin swimmer

[joshorr]

Using 1.2.2; i mentioned it in the step by step guide.

I suppose I could try this out in a blank machine, see if it’s something with my environment.

It very reliably repo’s for me with my current setup.

[finswimmer]

Using 1.2.2; i mentioned it in the step by step guide.

I swear it wasn't there when I read it 🤞 😄 🙈

[joshorr]

It seems like I was able to reproduce this in a docker container build I did the other day. I'll try to clean and simplify it up to the bare minimum and post something early next week.

[finswimmer]

Hello everyone,

you haven't come back to this issue for a long while. So I assume this is solved in the meantime and I can close this ticket.

Feel free to leave a comment if you disagree.

fin swimmer

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.