Skip to content

Commit fbe6a09

Browse files
authored
GH-101357: Suppress OSError from pathlib.Path.exists() and is_*() (#118243)
Suppress all `OSError` exceptions from `pathlib.Path.exists()` and `is_*()` rather than a selection of more common errors as we do presently. Also adjust the implementations to call `os.path.exists()` etc, which are much faster on Windows thanks to GH-101196.
1 parent d8e0e00 commit fbe6a09

File tree

5 files changed

+92
-125
lines changed

5 files changed

+92
-125
lines changed

Doc/library/pathlib.rst

+40-35
Original file line numberDiff line numberDiff line change
@@ -873,7 +873,7 @@ Methods
873873
^^^^^^^
874874

875875
Concrete paths provide the following methods in addition to pure paths
876-
methods. Many of these methods can raise an :exc:`OSError` if a system
876+
methods. Some of these methods can raise an :exc:`OSError` if a system
877877
call fails (for example because the path doesn't exist).
878878

879879
.. versionchanged:: 3.8
@@ -885,6 +885,15 @@ call fails (for example because the path doesn't exist).
885885
instead of raising an exception for paths that contain characters
886886
unrepresentable at the OS level.
887887

888+
.. versionchanged:: 3.14
889+
890+
The methods given above now return ``False`` instead of raising any
891+
:exc:`OSError` exception from the operating system. In previous versions,
892+
some kinds of :exc:`OSError` exception are raised, and others suppressed.
893+
The new behaviour is consistent with :func:`os.path.exists`,
894+
:func:`os.path.isdir`, etc. Use :meth:`~Path.stat` to retrieve the file
895+
status without suppressing exceptions.
896+
888897

889898
.. classmethod:: Path.cwd()
890899

@@ -951,6 +960,8 @@ call fails (for example because the path doesn't exist).
951960
.. method:: Path.exists(*, follow_symlinks=True)
952961

953962
Return ``True`` if the path points to an existing file or directory.
963+
``False`` will be returned if the path is invalid, inaccessible or missing.
964+
Use :meth:`Path.stat` to distinguish between these cases.
954965

955966
This method normally follows symlinks; to check if a symlink exists, add
956967
the argument ``follow_symlinks=False``.
@@ -1067,11 +1078,10 @@ call fails (for example because the path doesn't exist).
10671078

10681079
.. method:: Path.is_dir(*, follow_symlinks=True)
10691080

1070-
Return ``True`` if the path points to a directory, ``False`` if it points
1071-
to another kind of file.
1072-
1073-
``False`` is also returned if the path doesn't exist or is a broken symlink;
1074-
other errors (such as permission errors) are propagated.
1081+
Return ``True`` if the path points to a directory. ``False`` will be
1082+
returned if the path is invalid, inaccessible or missing, or if it points
1083+
to something other than a directory. Use :meth:`Path.stat` to distinguish
1084+
between these cases.
10751085

10761086
This method normally follows symlinks; to exclude symlinks to directories,
10771087
add the argument ``follow_symlinks=False``.
@@ -1082,11 +1092,10 @@ call fails (for example because the path doesn't exist).
10821092

10831093
.. method:: Path.is_file(*, follow_symlinks=True)
10841094

1085-
Return ``True`` if the path points to a regular file, ``False`` if it
1086-
points to another kind of file.
1087-
1088-
``False`` is also returned if the path doesn't exist or is a broken symlink;
1089-
other errors (such as permission errors) are propagated.
1095+
Return ``True`` if the path points to a regular file. ``False`` will be
1096+
returned if the path is invalid, inaccessible or missing, or if it points
1097+
to something other than a regular file. Use :meth:`Path.stat` to
1098+
distinguish between these cases.
10901099

10911100
This method normally follows symlinks; to exclude symlinks, add the
10921101
argument ``follow_symlinks=False``.
@@ -1122,46 +1131,42 @@ call fails (for example because the path doesn't exist).
11221131

11231132
.. method:: Path.is_symlink()
11241133

1125-
Return ``True`` if the path points to a symbolic link, ``False`` otherwise.
1126-
1127-
``False`` is also returned if the path doesn't exist; other errors (such
1128-
as permission errors) are propagated.
1134+
Return ``True`` if the path points to a symbolic link, even if that symlink
1135+
is broken. ``False`` will be returned if the path is invalid, inaccessible
1136+
or missing, or if it points to something other than a symbolic link. Use
1137+
:meth:`Path.stat` to distinguish between these cases.
11291138

11301139

11311140
.. method:: Path.is_socket()
11321141

1133-
Return ``True`` if the path points to a Unix socket (or a symbolic link
1134-
pointing to a Unix socket), ``False`` if it points to another kind of file.
1135-
1136-
``False`` is also returned if the path doesn't exist or is a broken symlink;
1137-
other errors (such as permission errors) are propagated.
1142+
Return ``True`` if the path points to a Unix socket. ``False`` will be
1143+
returned if the path is invalid, inaccessible or missing, or if it points
1144+
to something other than a Unix socket. Use :meth:`Path.stat` to
1145+
distinguish between these cases.
11381146

11391147

11401148
.. method:: Path.is_fifo()
11411149

1142-
Return ``True`` if the path points to a FIFO (or a symbolic link
1143-
pointing to a FIFO), ``False`` if it points to another kind of file.
1144-
1145-
``False`` is also returned if the path doesn't exist or is a broken symlink;
1146-
other errors (such as permission errors) are propagated.
1150+
Return ``True`` if the path points to a FIFO. ``False`` will be returned if
1151+
the path is invalid, inaccessible or missing, or if it points to something
1152+
other than a FIFO. Use :meth:`Path.stat` to distinguish between these
1153+
cases.
11471154

11481155

11491156
.. method:: Path.is_block_device()
11501157

1151-
Return ``True`` if the path points to a block device (or a symbolic link
1152-
pointing to a block device), ``False`` if it points to another kind of file.
1153-
1154-
``False`` is also returned if the path doesn't exist or is a broken symlink;
1155-
other errors (such as permission errors) are propagated.
1158+
Return ``True`` if the path points to a block device. ``False`` will be
1159+
returned if the path is invalid, inaccessible or missing, or if it points
1160+
to something other than a block device. Use :meth:`Path.stat` to
1161+
distinguish between these cases.
11561162

11571163

11581164
.. method:: Path.is_char_device()
11591165

1160-
Return ``True`` if the path points to a character device (or a symbolic link
1161-
pointing to a character device), ``False`` if it points to another kind of file.
1162-
1163-
``False`` is also returned if the path doesn't exist or is a broken symlink;
1164-
other errors (such as permission errors) are propagated.
1166+
Return ``True`` if the path points to a character device. ``False`` will be
1167+
returned if the path is invalid, inaccessible or missing, or if it points
1168+
to something other than a character device. Use :meth:`Path.stat` to
1169+
distinguish between these cases.
11651170

11661171

11671172
.. method:: Path.iterdir()

Lib/glob.py

+4-8
Original file line numberDiff line numberDiff line change
@@ -340,7 +340,7 @@ def __init__(self, sep, case_sensitive, case_pedantic=False, recursive=False):
340340

341341
# Low-level methods
342342

343-
lstat = operator.methodcaller('lstat')
343+
lexists = operator.methodcaller('exists', follow_symlinks=False)
344344
add_slash = operator.methodcaller('joinpath', '')
345345

346346
@staticmethod
@@ -516,12 +516,8 @@ def select_exists(self, path, exists=False):
516516
# Optimization: this path is already known to exist, e.g. because
517517
# it was returned from os.scandir(), so we skip calling lstat().
518518
yield path
519-
else:
520-
try:
521-
self.lstat(path)
522-
yield path
523-
except OSError:
524-
pass
519+
elif self.lexists(path):
520+
yield path
525521

526522
@classmethod
527523
def walk(cls, root, top_down, on_error, follow_symlinks):
@@ -562,7 +558,7 @@ def walk(cls, root, top_down, on_error, follow_symlinks):
562558

563559

564560
class _StringGlobber(_Globber):
565-
lstat = staticmethod(os.lstat)
561+
lexists = staticmethod(os.path.lexists)
566562
scandir = staticmethod(os.scandir)
567563
parse_entry = operator.attrgetter('path')
568564
concat_path = operator.add

Lib/pathlib/_abc.py

+9-82
Original file line numberDiff line numberDiff line change
@@ -13,32 +13,12 @@
1313

1414
import functools
1515
from glob import _Globber, _no_recurse_symlinks
16-
from errno import ENOENT, ENOTDIR, EBADF, ELOOP, EINVAL
16+
from errno import ENOTDIR, ELOOP
1717
from stat import S_ISDIR, S_ISLNK, S_ISREG, S_ISSOCK, S_ISBLK, S_ISCHR, S_ISFIFO
1818

1919

2020
__all__ = ["UnsupportedOperation"]
2121

22-
#
23-
# Internals
24-
#
25-
26-
_WINERROR_NOT_READY = 21 # drive exists but is not accessible
27-
_WINERROR_INVALID_NAME = 123 # fix for bpo-35306
28-
_WINERROR_CANT_RESOLVE_FILENAME = 1921 # broken symlink pointing to itself
29-
30-
# EBADF - guard against macOS `stat` throwing EBADF
31-
_IGNORED_ERRNOS = (ENOENT, ENOTDIR, EBADF, ELOOP)
32-
33-
_IGNORED_WINERRORS = (
34-
_WINERROR_NOT_READY,
35-
_WINERROR_INVALID_NAME,
36-
_WINERROR_CANT_RESOLVE_FILENAME)
37-
38-
def _ignore_error(exception):
39-
return (getattr(exception, 'errno', None) in _IGNORED_ERRNOS or
40-
getattr(exception, 'winerror', None) in _IGNORED_WINERRORS)
41-
4222

4323
@functools.cache
4424
def _is_case_sensitive(parser):
@@ -450,12 +430,7 @@ def exists(self, *, follow_symlinks=True):
450430
"""
451431
try:
452432
self.stat(follow_symlinks=follow_symlinks)
453-
except OSError as e:
454-
if not _ignore_error(e):
455-
raise
456-
return False
457-
except ValueError:
458-
# Non-encodable path
433+
except (OSError, ValueError):
459434
return False
460435
return True
461436

@@ -465,14 +440,7 @@ def is_dir(self, *, follow_symlinks=True):
465440
"""
466441
try:
467442
return S_ISDIR(self.stat(follow_symlinks=follow_symlinks).st_mode)
468-
except OSError as e:
469-
if not _ignore_error(e):
470-
raise
471-
# Path doesn't exist or is a broken symlink
472-
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
473-
return False
474-
except ValueError:
475-
# Non-encodable path
443+
except (OSError, ValueError):
476444
return False
477445

478446
def is_file(self, *, follow_symlinks=True):
@@ -482,14 +450,7 @@ def is_file(self, *, follow_symlinks=True):
482450
"""
483451
try:
484452
return S_ISREG(self.stat(follow_symlinks=follow_symlinks).st_mode)
485-
except OSError as e:
486-
if not _ignore_error(e):
487-
raise
488-
# Path doesn't exist or is a broken symlink
489-
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
490-
return False
491-
except ValueError:
492-
# Non-encodable path
453+
except (OSError, ValueError):
493454
return False
494455

495456
def is_mount(self):
@@ -518,13 +479,7 @@ def is_symlink(self):
518479
"""
519480
try:
520481
return S_ISLNK(self.lstat().st_mode)
521-
except OSError as e:
522-
if not _ignore_error(e):
523-
raise
524-
# Path doesn't exist
525-
return False
526-
except ValueError:
527-
# Non-encodable path
482+
except (OSError, ValueError):
528483
return False
529484

530485
def is_junction(self):
@@ -542,14 +497,7 @@ def is_block_device(self):
542497
"""
543498
try:
544499
return S_ISBLK(self.stat().st_mode)
545-
except OSError as e:
546-
if not _ignore_error(e):
547-
raise
548-
# Path doesn't exist or is a broken symlink
549-
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
550-
return False
551-
except ValueError:
552-
# Non-encodable path
500+
except (OSError, ValueError):
553501
return False
554502

555503
def is_char_device(self):
@@ -558,14 +506,7 @@ def is_char_device(self):
558506
"""
559507
try:
560508
return S_ISCHR(self.stat().st_mode)
561-
except OSError as e:
562-
if not _ignore_error(e):
563-
raise
564-
# Path doesn't exist or is a broken symlink
565-
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
566-
return False
567-
except ValueError:
568-
# Non-encodable path
509+
except (OSError, ValueError):
569510
return False
570511

571512
def is_fifo(self):
@@ -574,14 +515,7 @@ def is_fifo(self):
574515
"""
575516
try:
576517
return S_ISFIFO(self.stat().st_mode)
577-
except OSError as e:
578-
if not _ignore_error(e):
579-
raise
580-
# Path doesn't exist or is a broken symlink
581-
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
582-
return False
583-
except ValueError:
584-
# Non-encodable path
518+
except (OSError, ValueError):
585519
return False
586520

587521
def is_socket(self):
@@ -590,14 +524,7 @@ def is_socket(self):
590524
"""
591525
try:
592526
return S_ISSOCK(self.stat().st_mode)
593-
except OSError as e:
594-
if not _ignore_error(e):
595-
raise
596-
# Path doesn't exist or is a broken symlink
597-
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
598-
return False
599-
except ValueError:
600-
# Non-encodable path
527+
except (OSError, ValueError):
601528
return False
602529

603530
def samefile(self, other_path):

Lib/pathlib/_local.py

+34
Original file line numberDiff line numberDiff line change
@@ -502,12 +502,46 @@ def stat(self, *, follow_symlinks=True):
502502
"""
503503
return os.stat(self, follow_symlinks=follow_symlinks)
504504

505+
def exists(self, *, follow_symlinks=True):
506+
"""
507+
Whether this path exists.
508+
509+
This method normally follows symlinks; to check whether a symlink exists,
510+
add the argument follow_symlinks=False.
511+
"""
512+
if follow_symlinks:
513+
return os.path.exists(self)
514+
return os.path.lexists(self)
515+
516+
def is_dir(self, *, follow_symlinks=True):
517+
"""
518+
Whether this path is a directory.
519+
"""
520+
if follow_symlinks:
521+
return os.path.isdir(self)
522+
return PathBase.is_dir(self, follow_symlinks=follow_symlinks)
523+
524+
def is_file(self, *, follow_symlinks=True):
525+
"""
526+
Whether this path is a regular file (also True for symlinks pointing
527+
to regular files).
528+
"""
529+
if follow_symlinks:
530+
return os.path.isfile(self)
531+
return PathBase.is_file(self, follow_symlinks=follow_symlinks)
532+
505533
def is_mount(self):
506534
"""
507535
Check if this path is a mount point
508536
"""
509537
return os.path.ismount(self)
510538

539+
def is_symlink(self):
540+
"""
541+
Whether this path is a symbolic link.
542+
"""
543+
return os.path.islink(self)
544+
511545
def is_junction(self):
512546
"""
513547
Whether this path is a junction.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Suppress all :exc:`OSError` exceptions from :meth:`pathlib.Path.exists` and
2+
``is_*()`` methods, rather than a selection of more common errors. The new
3+
behaviour is consistent with :func:`os.path.exists`, :func:`os.path.isdir`,
4+
etc. Use :meth:`Path.stat` to retrieve the file status without suppressing
5+
exceptions.

0 commit comments

Comments
 (0)