GH-106747: Prepare pathlib globbing for dir_fd support. #116319
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The present implementation of `pathlib.Path.glob()` creates a series of 'selectors' that each handle a part of the pattern. The selectors are connected together in `glob()`, without the use of recursion. One very subtle property of this scheme is that each selector is exhaused *before* its successor selector - for example when globbing `*/*.py`, the selector for `*` is exhausted prior to the selector for `*.py`. This doesn't make any difference when globbing strings, but it does prevent us from adding `dir_fd` support, because there's no good moment to call `os.close(fd)` after opening a directory for scanning. This patch refactors globbing to work much as it did in 3.12, where each selector is responsible for creating and feeding its own successor. This inverts the order of selector exhaustion, and so will make it much easier to add `dir_fd` support. There's one behaviour change here: I've removes deduplication of results, and so in some very specific circumstances (multiple non-consecutive `**` segments in pattern, and either `follow_symlinks=None` or `..` segments separating them), `glob()` can yield the same path more than once. Note that `glob.glob()` can also yield duplicate results - see pythonGH-104269.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The present implementation of
pathlib.Path.glob()creates a series of 'selectors' that each handle a part of the pattern. The selectors are connected together inglob(), without the use of recursion. One very subtle property of this scheme is that each selector is exhausted before its successor selector - for example when globbing*/*.py, the selector for*is exhausted prior to the selector for*.py. This doesn't make any difference when globbing strings, but it does prevent us from addingdir_fdsupport, because there's no good moment to callos.close(fd)after opening a directory for scanning.This patch refactors globbing to work much as it did in 3.12, where each selector is responsible for creating and feeding its own successor. This inverts the order of selector exhaustion, and so will make it much easier to add
dir_fdsupport.There's one behaviour change here: I've removed deduplication of results, and so in some quite specific circumstances (multiple non-consecutive
**segments in pattern, and eitherfollow_symlinks=Noneor..segments separating them),glob()can yield the same path more than once. Note thatglob.glob()can also yield duplicate results - see GH-104269.