Re-use of virtual environments across envs in tox4

[fdallmeier]

Issue

In tox <4 it was possible to re-use the same python virtual environment across envs by specifying a shared envdir. Tox 4 however re-creates the virtual environment if the env name is different to the env name that was used to create the virtual environment.
It seems that tox 4 remembers the env name used to create the virtual environment in .tox-info.json and then decides to re-create the virtual environment if the env name changed.

I was not able to spot this change documented under https://tox.wiki/en/latest/faq.html#breaking-changes-in-tox-4. Also, I was not able to find an alternative way to re-use virtual environments across envs in tox 4.

Use case

A common use case for me is to run pytest and pylint with the same set of dependencies. Re-using the same virtual environment avoids the expensive creation of the multiple copies of the same virtual environment. Since dependencies change frequently this cost is significant. In practice, I also run more tools than just pytest and pylint via tox.
Also, I often run the entire test suite locally. For this I run tox which runs all envs in the envlist. If the virtual environment is re-created between every command I spend a lot of time waiting for the exact same packages to be installed.

Environment

• OS: Ubuntu 20.04
• Python 3.11.1
• tox 3.28.0 and tox 4.0.19

Example

Here an example that demonsrates the behavior of tox 3 and tox 4 on the same tox.ini.

tox.ini:

[tox]
envlist = 
    pytest
    pylint
skipsdist = true

[testenv]
# reuse virtual environment across commands
envdir = {toxworkdir}/.venv

deps = 
    pytest
    pylint

commands =
    pytest: pytest main.py
    pylint: pylint main.py

The main.py under test:

# pylint: disable=missing-module-docstring,missing-function-docstring
def one():
    return 1

def test_one():
    assert one() == 1

Bash script demonstrating above described behavior that tox 4 does not re-use venvs where tox 3 does.

#!/bin/bash

# echo commands as they are executed
set -x 

# remove old virtual environments
rm -rf .tox tox3-venv tox4-venv

# create virtual environment for tox3
python -m venv tox3-venv 
tox3-venv/bin/pip --quiet install "tox==3.28.0"

# create virtual environment for tox4
python -m venv tox4-venv 
tox4-venv/bin/pip --quiet install "tox==4.0.19"

# run tox3
tox3-venv/bin/tox -e pytest
tox3-venv/bin/tox -e pylint

# remove old virtual environments
rm -rf .tox

# run tox4
# run each command twice to demonstrate that the env is reused with same env name
tox4-venv/bin/tox -e pytest
tox4-venv/bin/tox -e pytest
tox4-venv/bin/tox -e pylint
tox4-venv/bin/tox -e pylint

Output of the script:

+ rm -rf .tox tox3-venv tox4-venv
+ python -m venv tox3-venv
+ tox3-venv/bin/pip --quiet install tox==3.28.0
+ python -m venv tox4-venv
+ tox4-venv/bin/pip --quiet install tox==4.0.19
+ tox3-venv/bin/tox -e pytest
pytest create: /home/fd/.tox/.venv
pytest installdeps: pytest, pylint
pytest installed: astroid==2.12.13,attrs==22.2.0,dill==0.3.6,iniconfig==1.1.1,isort==5.11.4,lazy-object-proxy==1.8.0,mccabe==0.7.0,packaging==22.0,platformdirs==2.6.2,pluggy==1.0.0,pylint==2.15.9,pytest==7.2.0,tomlkit==0.11.6,wrapt==1.14.1
pytest run-test-pre: PYTHONHASHSEED='2731687734'
pytest run-test: commands[0] | pytest main.py
======================================================================= test session starts =======================================================================
platform linux -- Python 3.11.1, pytest-7.2.0, pluggy-1.0.0
cachedir: .tox/.venv/.pytest_cache
rootdir: /home/fd
collected 1 item                                                                                                                                                  

main.py .                                                                                                                                                   [100%]

======================================================================== 1 passed in 0.01s ========================================================================
_____________________________________________________________________________ summary _____________________________________________________________________________
  pytest: commands succeeded
  congratulations :)
+ tox3-venv/bin/tox -e pylint
pylint installed: astroid==2.12.13,attrs==22.2.0,dill==0.3.6,iniconfig==1.1.1,isort==5.11.4,lazy-object-proxy==1.8.0,mccabe==0.7.0,packaging==22.0,platformdirs==2.6.2,pluggy==1.0.0,pylint==2.15.9,pytest==7.2.0,tomlkit==0.11.6,wrapt==1.14.1
pylint run-test-pre: PYTHONHASHSEED='3576645533'
pylint run-test: commands[0] | pylint main.py

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

_____________________________________________________________________________ summary _____________________________________________________________________________
  pylint: commands succeeded
  congratulations :)
+ rm -rf .tox
+ tox4-venv/bin/tox -e pytest
pytest: install_deps> python -I -m pip install pylint pytest
pytest: commands[0]> pytest main.py
======================================================================= test session starts =======================================================================
platform linux -- Python 3.11.1, pytest-7.2.0, pluggy-1.0.0
cachedir: .tox/.venv/.pytest_cache
rootdir: /home/fd
collected 1 item                                                                                                                                                  

main.py .                                                                                                                                                   [100%]

======================================================================== 1 passed in 0.01s ========================================================================
  pytest: OK (6.58=setup[6.27]+cmd[0.31] seconds)
  congratulations :) (6.64 seconds)
+ tox4-venv/bin/tox -e pytest
pytest: commands[0]> pytest main.py
======================================================================= test session starts =======================================================================
platform linux -- Python 3.11.1, pytest-7.2.0, pluggy-1.0.0
cachedir: .tox/.venv/.pytest_cache
rootdir: /home/fd
collected 1 item                                                                                                                                                  

main.py .                                                                                                                                                   [100%]

======================================================================== 1 passed in 0.01s ========================================================================
  pytest: OK (0.36=setup[0.04]+cmd[0.31] seconds)
  congratulations :) (0.42 seconds)
+ tox4-venv/bin/tox -e pylint
pylint: recreate env because env type changed from {'name': 'pytest', 'type': 'VirtualEnvRunner'} to {'name': 'pylint', 'type': 'VirtualEnvRunner'}
pylint: remove tox env folder /home/fd/.tox/.venv
pylint: install_deps> python -I -m pip install pylint pytest
pylint: commands[0]> pylint main.py

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

  pylint: OK (6.88=setup[6.21]+cmd[0.67] seconds)
  congratulations :) (6.95 seconds)
+ tox4-venv/bin/tox -e pylint
pylint: commands[0]> pylint main.py

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

  pylint: OK (0.72=setup[0.04]+cmd[0.68] seconds)
  congratulations :) (0.79 seconds)

We see that in tox 3, execution of tox -e pytest after tox -e pytest leads to the virtual environment being re-used.

With tox 4, the virtual environment is only re-used if the same env is used, e.g. executing tox -e pytest after tox -e pytest. Executing tox -e pylint after tox -e pytest leads to tox 4 recreating the virtual environment which tox 4 tells us via

pylint: recreate env because env type changed from {'name': 'pytest', 'type': 'VirtualEnvRunner'} to {'name': 'pylint', 'type': 'VirtualEnvRunner'}

[jugmac00]

Thanks for the extensive bug report.

I am pretty sure we had this discussion quite recently, but I cannot find it again.

From what I recall is that re-use of environments worked for tox v3 by chance, so was never officially supported.

Even as this is not a planned change of behavior, this should be still listed in the mentioned faq.

Waiting for @gaborbernat to confirm, and maybe he recalls the issue where this was discussed.

FWIW there is also a very old issue where it was mentioned that re-use of environments is not supported by tox core, but could be supported by a plugin, see #425

Keeping this issue open until at least the faq is updated.

[jugmac00]

And FWIW.. the usual way to define what you want is...

[tox]
envlist = 
    py311
    pylint
skipsdist = true

[testenv]
deps = pytest
commands = pytest main.py

[testenv:pylint]
deps = pylint
commands = pylint main.py

This all said, you could also have a look at https://pre-commit.com/ to run your linters, which is very common in the Python community.

[tox]
envlist = py311, lint
skipsdist = true

[testenv]
deps = pytest
commands = pytest main.py

[testenv:lint]
deps = pre-commit
commands = pre-commit run --all-files

[gaborbernat]

In tox \u003C4 it was possible to re-use the same python virtual environment across envs by specifying a shared envdir.

This was never actually supported, worked only by chance because our env detection logic wasn't good enough. We don't plan to support this behavior.

[jugmac00]

I'll add an entry to the faq/changes list later today.

[masenf]

I've published a plugin to support this workflow as tox-ignore-env-name-mismatch

[broken-wheel]

Thank you @masenf!!!
I was pouring through documentations and other plugins (and even ChatGPT :-P) to see how to write a tox plugin for just this purpose. You just saved me a ton of time.

[gaborbernat]

I'll just say that the core team does not agree with the approach taken, and we don't plan to offer support for the interface used to create this, but we'll not disallow it either 👍 in practice it means if you submit a bug to tox and you're using this plugin you're on your own 🙂 we'll close the issue without addressing the issue you're running into.

[masenf]

we don't plan to offer support for the interface used to create this

Is this referring to Info._content being accessed and modified by the plug-in? Or extending VirtualEnvRunner.cache? Or a different interface?

[gaborbernat]

The former, it's private for a reason. Also note that we don't really support sidestepping a core check of the tool, even if the python language allows it. 😅

[gaborbernat]

It is a smart workaround of the problem though, just not an elegant solution to the underlying problem 😎