Extras not installed correctly when using isolated_build=true

[janfreyberg]

I am using flit to manage my package builds. My TOML file looks like:

[build-system]
requires = ["flit"]
build-backend = "flit.buildapi"

[tool.flit.metadata]
module = "pytorch_revgrad"
author = "Jan Freyberg"
author-email = "jan.freyberg@gmail.com"
home-page = "https://github.com/janfreyberg/pytorch-revgrad"
classifiers = ["License :: OSI Approved :: MIT License"]
requires-python = ">=3.5"
description-file="README.md"
requires = [
    "numpy",
    "torch>=1.0.0"
]

[tool.flit.metadata.requires-extra]
test = [
    "tox",
    "pytest",
    "pytest-cov",
    "flake8",
    "pytest-flake8",
    "coveralls"
]

I'd like to use the test requires-extra section to manage my dependencies. Accordingly, my tox.ini file looks like:

[tox]
envlist=py35,py36,py37
isolated_build = true
extras =
    test

[testenv]
commands =
    python -m pytest

However, while I don't fully understand the reason, it seems as if the extra dependencies are not installed:

$ tox -e py37 -rvv
using tox.ini: /Users/jan/pytorch-revgrad/tox.ini (pid 29128)
using tox-3.14.3 from /Users/jan/pytorch-revgrad/.venv/lib/python3.8/site-packages/tox/__init__.py (pid 29128)
.package start: getenv /Users/jan/pytorch-revgrad/.tox/.package
.package cannot reuse: -r flag
.package create: /Users/jan/pytorch-revgrad/.tox/.package
/Users/jan/pytorch-revgrad/.venv/bin/python (/Users/jan/pytorch-revgrad/.venv/bin/python) is {'executable': '/Users/jan/pytorch-revgrad/.venv/bin/python', 'name': 'python', 'version_info': [3, 8, 0, 'final', 0], 'version': '3.8.0 (default, Nov 27 2019, 15:14:08) \n[Clang 11.0.0 (clang-1100.0.20.17)]', 'is_64': True, 'sysplatform': 'darwin'}
.package uses /Users/jan/pytorch-revgrad/.venv/bin/python
setting PATH=/Users/jan/pytorch-revgrad/.tox/.package/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29131] /Users/jan/pytorch-revgrad/.tox$ /Users/jan/pytorch-revgrad/.venv/bin/python -m virtualenv --no-download --python /Users/jan/pytorch-revgrad/.venv/bin/python .package
Already using interpreter /Users/jan/pytorch-revgrad/.venv/bin/python
Using base prefix '/Users/jan/.pyenv/versions/3.8.0'
New python executable in /Users/jan/pytorch-revgrad/.tox/.package/bin/python
Installing setuptools, pip, wheel...
done.
.package installdeps: flit
setting PATH=/Users/jan/pytorch-revgrad/.tox/.package/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29136] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/.package/bin/python -m pip install flit
Looking in indexes: https://pypi.org/simple
Collecting flit
  Using cached https://files.pythonhosted.org/packages/80/f6/b10b8541b07caa4b680bdd84744056a401f142e639c9c316adaa88b9fa78/flit-2.2.0-py3-none-any.whl
Collecting requests
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting docutils
  Using cached https://files.pythonhosted.org/packages/81/44/8a15e45ffa96e6cf82956dd8d7af9e666357e16b0d93b253903475ee947f/docutils-0.16-py2.py3-none-any.whl
Collecting pytoml
  Using cached https://files.pythonhosted.org/packages/a5/47/c7f8a0f210ad18576840922e0b504f0b7f5f73aea4a52ab14c5b58517edf/pytoml-0.1.21-py2.py3-none-any.whl
Collecting flit_core>=2.2.0
  Using cached https://files.pythonhosted.org/packages/25/4c/0b1ed660937d96ed192c376d3983dd7b052b887c8041ae020c950c0d06f0/flit_core-2.2.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Using cached https://files.pythonhosted.org/packages/e8/74/6e4f91745020f967d09332bb2b8b9b10090957334692eb88ea4afe91b77f/urllib3-1.25.8-py2.py3-none-any.whl
Collecting certifi>=2017.4.17
  Using cached https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Installing collected packages: urllib3, certifi, idna, chardet, requests, docutils, pytoml, flit-core, flit
Successfully installed certifi-2019.11.28 chardet-3.0.4 docutils-0.16 flit-2.2.0 flit-core-2.2.0 idna-2.8 pytoml-0.1.21 requests-2.22.0 urllib3-1.25.8
WARNING: You are using pip version 19.3.1; however, version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
.package finish: getenv /Users/jan/pytorch-revgrad/.tox/.package after 6.96 seconds
.package start: finishvenv
write config to /Users/jan/pytorch-revgrad/.tox/.package/.tox-config1 as 'ecf9d3c64af8da48f51ade97d1900cd7454f27b5829c29f13cb2cc293f175282 /Users/jan/pytorch-revgrad/.venv/bin/python\n3.14.3 0 0 0\n00000000000000000000000000000000 flit'
.package finish: finishvenv  after 0.01 seconds
.package start: get-build-requires /Users/jan/pytorch-revgrad/.tox/.package
setting PATH=/Users/jan/pytorch-revgrad/.tox/.package/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29142] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/.package/bin/python .venv/lib/python3.8/site-packages/tox/helper/build_requires.py flit.buildapi '' >.tox/.package/log/.package-0.log
.package finish: get-build-requires /Users/jan/pytorch-revgrad/.tox/.package after 0.20 seconds
.package start: build_requires /Users/jan/pytorch-revgrad/.tox/.package
setting PATH=/Users/jan/pytorch-revgrad/.tox/.package/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29143] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/.package/bin/python -m pip install numpy 'torch>=1.0.0' 'ipython ; extra == "dev"' 'tox ; extra == "test"' 'pytest ; extra == "test"' 'pytest-cov ; extra == "test"' 'flake8 ; extra == "test"' 'pytest-flake8 ; extra == "test"' 'coveralls ; extra == "test"'
Ignoring ipython: markers 'extra == "dev"' don't match your environment
Ignoring tox: markers 'extra == "test"' don't match your environment
Ignoring pytest: markers 'extra == "test"' don't match your environment
Ignoring pytest-cov: markers 'extra == "test"' don't match your environment
Ignoring flake8: markers 'extra == "test"' don't match your environment
Ignoring pytest-flake8: markers 'extra == "test"' don't match your environment
Ignoring coveralls: markers 'extra == "test"' don't match your environment
Looking in indexes: https://pypi.org/simple
Collecting numpy
  Using cached https://files.pythonhosted.org/packages/a7/06/6d616fb5fb423db595b1502cbd873f3f2025f2fd8509046c771a20c4302a/numpy-1.18.1-cp38-cp38-macosx_10_9_x86_64.whl
Collecting torch>=1.0.0
  Using cached https://files.pythonhosted.org/packages/53/e1/f2ba65890bb3ed98eda66ec9195e97b9b1b5824117ff4c8ad3b3f298e464/torch-1.4.0-cp38-none-macosx_10_9_x86_64.whl
Installing collected packages: numpy, torch
Successfully installed numpy-1.18.1 torch-1.4.0
WARNING: You are using pip version 19.3.1; however, version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
.package finish: build_requires /Users/jan/pytorch-revgrad/.tox/.package after 9.25 seconds
.package start: finishvenv
.package finish: finishvenv  after 0.01 seconds
.package start: perform-isolated-build /Users/jan/pytorch-revgrad/.tox/.package
setting PATH=/Users/jan/pytorch-revgrad/.tox/.package/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29145] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/.package/bin/python .venv/lib/python3.8/site-packages/tox/helper/build_isolated.py .tox/dist flit.buildapi '' >.tox/.package/log/.package-1.log
pytorch_revgrad-0.1.tar.gz

.package finish: perform-isolated-build /Users/jan/pytorch-revgrad/.tox/.package after 1.01 seconds
copying new sdistfile to '/Users/jan/.tox/distshare/pytorch_revgrad-0.1.tar.gz'
package .tmp/package/1/pytorch_revgrad-0.1.tar.gz links to dist/pytorch_revgrad-0.1.tar.gz (/Users/jan/pytorch-revgrad/.tox)
py37 start: getenv /Users/jan/pytorch-revgrad/.tox/py37
py37 cannot reuse: -r flag
py37 create: /Users/jan/pytorch-revgrad/.tox/py37
python3.7 (/Users/jan/.pyenv/shims/python3.7) is {'executable': '/Users/jan/.pyenv/versions/3.7.5rc1/bin/python3.7', 'name': 'python', 'version_info': [3, 7, 5, 'candidate', 1], 'version': '3.7.5rc1 (default, Nov 27 2019, 15:23:39) \n[Clang 11.0.0 (clang-1100.0.20.17)]', 'is_64': True, 'sysplatform': 'darwin'}
py37 uses /Users/jan/.pyenv/versions/3.7.5rc1/bin/python3.7
setting PATH=/Users/jan/pytorch-revgrad/.tox/py37/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29181] /Users/jan/pytorch-revgrad/.tox$ /Users/jan/pytorch-revgrad/.venv/bin/python -m virtualenv --no-download --python /Users/jan/.pyenv/versions/3.7.5rc1/bin/python3.7 py37
Running virtualenv with interpreter /Users/jan/.pyenv/versions/3.7.5rc1/bin/python3.7
Already using interpreter /Users/jan/.pyenv/versions/3.7.5rc1/bin/python3.7
Using base prefix '/Users/jan/.pyenv/versions/3.7.5rc1'
New python executable in /Users/jan/pytorch-revgrad/.tox/py37/bin/python3.7
Also creating executable in /Users/jan/pytorch-revgrad/.tox/py37/bin/python
Installing setuptools, pip, wheel...
done.
py37 finish: getenv /Users/jan/pytorch-revgrad/.tox/py37 after 3.90 seconds
py37 start: installpkg /Users/jan/pytorch-revgrad/.tox/.tmp/package/1/pytorch_revgrad-0.1.tar.gz
py37 inst: /Users/jan/pytorch-revgrad/.tox/.tmp/package/1/pytorch_revgrad-0.1.tar.gz
write config to /Users/jan/pytorch-revgrad/.tox/py37/.tox-config1 as '13208ff0ed71c6500eb63d0f6f0939d93dc1b46269505dc461860d09ac9bc0c1 /Users/jan/.pyenv/versions/3.7.5rc1/bin/python3.7\n3.14.3 0 0 0'
setting PATH=/Users/jan/pytorch-revgrad/.tox/py37/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29189] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/py37/bin/python -m pip install --exists-action w .tox/.tmp/package/1/pytorch_revgrad-0.1.tar.gz
Looking in indexes: https://pypi.org/simple
Processing ./.tox/.tmp/package/1/pytorch_revgrad-0.1.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Collecting torch>=1.0.0
  Using cached https://files.pythonhosted.org/packages/4a/72/0282449efe6e8a7ab6354ac990b8275bd8c881dcbf95b3ef0a041da3897b/torch-1.4.0-cp37-none-macosx_10_9_x86_64.whl
Collecting numpy
  Using cached https://files.pythonhosted.org/packages/2f/5b/2cc2b9285e8b2ca8d2c1e4a2cbf1b12d70a2488ea78170de1909bca725f2/numpy-1.18.1-cp37-cp37m-macosx_10_9_x86_64.whl
Building wheels for collected packages: pytorch-revgrad
  Building wheel for pytorch-revgrad (PEP 517) ... done
  Created wheel for pytorch-revgrad: filename=pytorch_revgrad-0.1-cp37-none-any.whl size=2764 sha256=6af2264fcfb20aa53ece821009f342d9882248dc13f2b428c81fa803f1df5f08
  Stored in directory: /Users/jan/Library/Caches/pip/wheels/d7/a0/61/edb972be6fa8f71ccd47b95aaa37625a51e11572bf735d2d26
Successfully built pytorch-revgrad
Installing collected packages: torch, numpy, pytorch-revgrad
Successfully installed numpy-1.18.1 pytorch-revgrad-0.1 torch-1.4.0
WARNING: You are using pip version 19.3.1; however, version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
py37 finish: installpkg /Users/jan/pytorch-revgrad/.tox/.tmp/package/1/pytorch_revgrad-0.1.tar.gz after 25.06 seconds
py37 start: envreport
setting PATH=/Users/jan/pytorch-revgrad/.tox/py37/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29209] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/py37/bin/python -m pip freeze >.tox/py37/log/py37-0.log
py37 finish: envreport  after 0.27 seconds
py37 installed: numpy==1.18.1,pytorch-revgrad==0.1,torch==1.4.0
py37 start: run-test-pre
py37 run-test-pre: PYTHONHASHSEED='2124118408'
py37 finish: run-test-pre  after 0.00 seconds
py37 start: run-test
py37 run-test: commands[0] | python -m pytest
setting PATH=/Users/jan/pytorch-revgrad/.tox/py37/bin:/Users/jan/pytorch-revgrad/.venv/bin:/Users/jan/.pyenv/bin:/Users/jan/.pyenv/shims:/Users/jan/scipts:/Users/jan/.local/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/2.6.0/bin:/Users/jan/.cargo/bin:/Users/jan/scripts:/Users/jan/.cabal/bin:/Users/jan/.ghcup/bin:/Users/jan/scripts/google-cloud-sdk/bin:/Users/jan/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[29213] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/py37/bin/python -m pytest
/Users/jan/pytorch-revgrad/.tox/py37/bin/python: No module named pytest
ERROR: InvocationError for command /Users/jan/pytorch-revgrad/.tox/py37/bin/python -m pytest (exited with code 1)
py37 finish: run-test  after 0.02 seconds
py37 start: run-test-post
py37 finish: run-test-post  after 0.00 seconds
__________________________________________________________ summary ___________________________________________________________
ERROR:   py37: commands failed
cleanup /Users/jan/pytorch-revgrad/.tox/.tmp/package/1/pytorch_revgrad-0.1.tar.gz

In particular, I am unclear why these lines occur:

[29143] /Users/jan/pytorch-revgrad$ /Users/jan/pytorch-revgrad/.tox/.package/bin/python -m pip install numpy 'torch>=1.0.0' 'ipython ; extra == "dev"' 'tox ; extra == "test"' 'pytest ; extra == "test"' 'pytest-cov ; extra == "test"' 'flake8 ; extra == "test"' 'pytest-flake8 ; extra == "test"' 'coveralls ; extra == "test"'
Ignoring ipython: markers 'extra == "dev"' don't match your environment
Ignoring tox: markers 'extra == "test"' don't match your environment
Ignoring pytest: markers 'extra == "test"' don't match your environment
Ignoring pytest-cov: markers 'extra == "test"' don't match your environment
Ignoring flake8: markers 'extra == "test"' don't match your environment
Ignoring pytest-flake8: markers 'extra == "test"' don't match your environment
Ignoring coveralls: markers 'extra == "test"' don't match your environment

I tried to look through issues mentioning "extras" or "isolated_build" but couldn't find anything that matches this problem.

Any help would be appreciated!

Jan

[astrojuanlu]

I confirm I'm observing the same

[gaborbernat]

Feel free to open a PR against master and fix it. My available efforts at the moment are aimed at fixing this as part of #1394, but that probably will take a while (ETA September).

[gaborbernat]

I think this has been fixed now in tox version 4. Please check it out! If it still happens can you submit a reproducible to it?