option to not follow symlinks when globbing

[o11c]

BPO	29475
Nosy	@ethanfurman, @o11c, @wimglenn
Files	python-glob-symlink.diff: Patch to add symlinks= argument to glob.glob and glob.iglob

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-02-07.21:26:15.742>
labels = ['type-feature', 'library']
title = 'option to not follow symlinks when globbing'
updated_at = <Date 2017-09-12.23:05:45.646>
user = 'https://github.com/o11c'

bugs.python.org fields:

activity = <Date 2017-09-12.23:05:45.646>
actor = 'wim.glenn'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-02-07.21:26:15.742>
creator = 'o11c'
dependencies = []
files = ['46566']
hgrepos = []
issue_num = 29475
keywords = ['patch']
message_count = 3.0
messages = ['287256', '287279', '302002']
nosy_count = 3.0
nosy_names = ['ethan.furman', 'o11c', 'wim.glenn']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue29475'
versions = ['Python 3.6']

[o11c]

Background:
I have a data hierarchy with a lot of "sibling" symlinked directories/files. I want to glob only the non-symlink files, because it's a *huge* performance increase.

Before os.scandir, I was using a local copy of glob.py and calling os.path.islink every time, which was faster for my use case, but unacceptable for upstreaming. With os.scandir, my new patch should be acceptable.

The patch includes tests.

Current discussion points:

• Am I making the right decision to still accept symlinks for fully-literal components (in glob0)? It doesn't apply to my use-case, and I imagine some people might want to handle that case separately.
• Are my tests sufficient? I just copied and modified the existing symlink tests.
• Should my flags TODO be implemented before this patch? IMO it would be clearer after, even if it makes the diffs longer.

Future discussion points (don't derail):

• Should my flags TODO be implemented internally (this would significantly shrink any future patches)? (I can work on this)
• Should flags also be exposed externally?
• What additional flags might be useful? (my list: GLOB_ERR, GLOB_MARK, ~GLOB_NOSORT, ~GLOB_NOESCAPE, GLOB_PERIOD, GLOB_BRACE, GLOB_TILDE_CHECK, GLOB_ONLYDIR (+ equivalent for files - also, why doesn't os.scandir have accessors for the other types without doing an unnecessary stat?))
• Is there a bitwise enum (or equivalently, enum set) in the standard library so flags can get sane reprs? (I've implemented this before, but imagine it would be overwhelmed with bikeshedding if it doesn't exist yet)
• Should pathlib really be implementing globbing on its own? That makes it hard to ensure feature parity. Perhaps the glob module needs some additional APIs? (I don't want to work on pathlib itself)

[ethanfurman]

Before talking about the patch, have you signed the Contributer License Agreement yet? The issue tracker isn't showing that you have. Check out https://www.python.org/psf/contrib/contrib-form/ to do so.

---

The symlinks flag: can you give some glob examples showing when, and when not, symlinks will be matched when the symlinks param is False?

A bit flags enum: There is now an IntFlag Enum type which implements bit flags.

[wimglenn]

+1, would like to use this feature too, and I would like it also in pathlib.PosixPath.glob

[gpshead]

The upcoming 3.13 release gains a follow_symlinks= argument to the pathlib.Path.glob API: https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.glob via #77609.

[barneygale]

I plan to work on this once #77609 and #116380 are done.