Ensure tox_extend_envs list can be read twice
#3598
Merged
+28
−4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously (PR tox-dev#3591) a new hook point was added. The tests checked that it works well with the `tox config` and `tox list` commands. However, `tox run` turned out to have a problem — it would complain that there's no such env, when invoked: ```console ROOT: 170 E HandledError| provided environments not found in configuration file: pip-compile-tox-env-lock [tox/run.py:23] ``` Turned out, this was because the config object is being interated twice in some subcommands. This in turn iterates over the discovered additional ephemeral environments list object. The implementation passes an iterator into it and so when it's first accessed, it's exhausted and the second attempt does not give the same envs, causing inconsistency. The patch solves this by using `itertools.tee()`, making sure that the underlying iterable is always cached and it's possible to repeat iteration as many times as possible without loosing the data in the process.
Contributor
Author
|
Urgh.. it looks like the fix doesn't work on older Pythons (3.11 and lower). I'll take a look. |
Before Python 3.11, the first value is the the underlying iterator. And we still exhaust it. So for backwards compatibility, we request two iterators and disregard the first one.
Contributor
Author
|
@gaborbernat this is now ready for inclusion in v4.29.1 |
gaborbernat
approved these changes
Sep 2, 2025
Contributor
Author
|
@gaborbernat could you release when you have time? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously (PR #3591) a new hook point was added. The tests checked that it works well with the
tox configandtox listcommands. However,tox runturned out to have a problem — it would complain that there's no such env, when invoked:Turned out, this was because the config object is being interated twice in some subcommands. This in turn iterates over the discovered additional ephemeral environments list object. The implementation passes an iterator into it and so when it's first accessed, it's exhausted and the second attempt does not give the same envs, causing inconsistency.
The patch solves this by using
itertools.tee(), making sure that the underlying iterable is always cached and it's possible to repeat iteration as many times as possible without loosing the data in the process.tox -e fix)docs/changelogfolder