Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-101357: Suppress OSError from pathlib.Path.exists() and is_*() #118243

Merged
merged 7 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 40 additions & 35 deletions Doc/library/pathlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -874,7 +874,7 @@ Methods
^^^^^^^

Concrete paths provide the following methods in addition to pure paths
methods. Many of these methods can raise an :exc:`OSError` if a system
methods. Some of these methods can raise an :exc:`OSError` if a system
call fails (for example because the path doesn't exist).

.. versionchanged:: 3.8
Expand All @@ -886,6 +886,15 @@ call fails (for example because the path doesn't exist).
instead of raising an exception for paths that contain characters
unrepresentable at the OS level.

.. versionchanged:: 3.14

The methods given above now return ``False`` instead of raising any
:exc:`OSError` exception from the operating system. In previous versions,
some kinds of :exc:`OSError` exception are raised, and others suppressed.
The new behaviour is consistent with :func:`os.path.exists`,
:func:`os.path.isdir`, etc. Use :meth:`~Path.stat` to retrieve the file
status without suppressing exceptions.


.. classmethod:: Path.cwd()

Expand Down Expand Up @@ -952,6 +961,8 @@ call fails (for example because the path doesn't exist).
.. method:: Path.exists(*, follow_symlinks=True)

Return ``True`` if the path points to an existing file or directory.
``False`` will be returned if the path is invalid, inaccessible or missing.
Use :meth:`Path.stat` to distinguish between these cases.

This method normally follows symlinks; to check if a symlink exists, add
the argument ``follow_symlinks=False``.
Expand Down Expand Up @@ -1068,11 +1079,10 @@ call fails (for example because the path doesn't exist).

.. method:: Path.is_dir(*, follow_symlinks=True)

Return ``True`` if the path points to a directory, ``False`` if it points
to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a directory. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a directory. Use :meth:`Path.stat` to distinguish
between these cases.

This method normally follows symlinks; to exclude symlinks to directories,
add the argument ``follow_symlinks=False``.
Expand All @@ -1083,11 +1093,10 @@ call fails (for example because the path doesn't exist).

.. method:: Path.is_file(*, follow_symlinks=True)

Return ``True`` if the path points to a regular file, ``False`` if it
points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a regular file. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a regular file. Use :meth:`Path.stat` to
distinguish between these cases.

This method normally follows symlinks; to exclude symlinks, add the
argument ``follow_symlinks=False``.
Expand Down Expand Up @@ -1123,46 +1132,42 @@ call fails (for example because the path doesn't exist).

.. method:: Path.is_symlink()

Return ``True`` if the path points to a symbolic link, ``False`` otherwise.

``False`` is also returned if the path doesn't exist; other errors (such
as permission errors) are propagated.
Return ``True`` if the path points to a symbolic link, even if that symlink
is broken. ``False`` will be returned if the path is invalid, inaccessible
or missing, or if it points to something other than a symbolic link. Use
:meth:`Path.stat` to distinguish between these cases.


.. method:: Path.is_socket()

Return ``True`` if the path points to a Unix socket (or a symbolic link
pointing to a Unix socket), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a Unix socket. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a Unix socket. Use :meth:`Path.stat` to
distinguish between these cases.


.. method:: Path.is_fifo()

Return ``True`` if the path points to a FIFO (or a symbolic link
pointing to a FIFO), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a FIFO. ``False`` will be returned if
the path is invalid, inaccessible or missing, or if it points to something
other than a FIFO. Use :meth:`Path.stat` to distinguish between these
cases.


.. method:: Path.is_block_device()

Return ``True`` if the path points to a block device (or a symbolic link
pointing to a block device), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a block device. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a block device. Use :meth:`Path.stat` to
distinguish between these cases.


.. method:: Path.is_char_device()

Return ``True`` if the path points to a character device (or a symbolic link
pointing to a character device), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a character device. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a character device. Use :meth:`Path.stat` to
distinguish between these cases.


.. method:: Path.iterdir()
Expand Down
12 changes: 4 additions & 8 deletions Lib/glob.py
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ def __init__(self, sep, case_sensitive, case_pedantic=False, recursive=False):

# Low-level methods

lstat = operator.methodcaller('lstat')
lexists = operator.methodcaller('exists', follow_symlinks=False)
add_slash = operator.methodcaller('joinpath', '')

@staticmethod
Expand Down Expand Up @@ -516,12 +516,8 @@ def select_exists(self, path, exists=False):
# Optimization: this path is already known to exist, e.g. because
# it was returned from os.scandir(), so we skip calling lstat().
yield path
else:
try:
self.lstat(path)
yield path
except OSError:
pass
elif self.lexists(path):
yield path

@classmethod
def walk(cls, root, top_down, on_error, follow_symlinks):
Expand Down Expand Up @@ -562,7 +558,7 @@ def walk(cls, root, top_down, on_error, follow_symlinks):


class _StringGlobber(_Globber):
lstat = staticmethod(os.lstat)
lexists = staticmethod(os.path.lexists)
scandir = staticmethod(os.scandir)
parse_entry = operator.attrgetter('path')
concat_path = operator.add
Expand Down
91 changes: 9 additions & 82 deletions Lib/pathlib/_abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,32 +13,12 @@

import functools
from glob import _Globber, _no_recurse_symlinks
from errno import ENOENT, ENOTDIR, EBADF, ELOOP, EINVAL
from errno import ENOTDIR, ELOOP
from stat import S_ISDIR, S_ISLNK, S_ISREG, S_ISSOCK, S_ISBLK, S_ISCHR, S_ISFIFO


__all__ = ["UnsupportedOperation"]

#
# Internals
#

_WINERROR_NOT_READY = 21 # drive exists but is not accessible
_WINERROR_INVALID_NAME = 123 # fix for bpo-35306
_WINERROR_CANT_RESOLVE_FILENAME = 1921 # broken symlink pointing to itself

# EBADF - guard against macOS `stat` throwing EBADF
_IGNORED_ERRNOS = (ENOENT, ENOTDIR, EBADF, ELOOP)

_IGNORED_WINERRORS = (
_WINERROR_NOT_READY,
_WINERROR_INVALID_NAME,
_WINERROR_CANT_RESOLVE_FILENAME)

def _ignore_error(exception):
return (getattr(exception, 'errno', None) in _IGNORED_ERRNOS or
getattr(exception, 'winerror', None) in _IGNORED_WINERRORS)


@functools.cache
def _is_case_sensitive(parser):
Expand Down Expand Up @@ -450,12 +430,7 @@ def exists(self, *, follow_symlinks=True):
"""
try:
self.stat(follow_symlinks=follow_symlinks)
except OSError as e:
if not _ignore_error(e):
raise
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False
return True

Expand All @@ -465,14 +440,7 @@ def is_dir(self, *, follow_symlinks=True):
"""
try:
return S_ISDIR(self.stat(follow_symlinks=follow_symlinks).st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_file(self, *, follow_symlinks=True):
Expand All @@ -482,14 +450,7 @@ def is_file(self, *, follow_symlinks=True):
"""
try:
return S_ISREG(self.stat(follow_symlinks=follow_symlinks).st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_mount(self):
Expand Down Expand Up @@ -518,13 +479,7 @@ def is_symlink(self):
"""
try:
return S_ISLNK(self.lstat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_junction(self):
Expand All @@ -542,14 +497,7 @@ def is_block_device(self):
"""
try:
return S_ISBLK(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_char_device(self):
Expand All @@ -558,14 +506,7 @@ def is_char_device(self):
"""
try:
return S_ISCHR(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_fifo(self):
Expand All @@ -574,14 +515,7 @@ def is_fifo(self):
"""
try:
return S_ISFIFO(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_socket(self):
Expand All @@ -590,14 +524,7 @@ def is_socket(self):
"""
try:
return S_ISSOCK(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def samefile(self, other_path):
Expand Down
34 changes: 34 additions & 0 deletions Lib/pathlib/_local.py
Original file line number Diff line number Diff line change
Expand Up @@ -514,12 +514,46 @@ def stat(self, *, follow_symlinks=True):
"""
return os.stat(self, follow_symlinks=follow_symlinks)

def exists(self, *, follow_symlinks=True):
"""
Whether this path exists.

This method normally follows symlinks; to check whether a symlink exists,
add the argument follow_symlinks=False.
"""
if follow_symlinks:
return os.path.exists(self)
return os.path.lexists(self)

def is_dir(self, *, follow_symlinks=True):
"""
Whether this path is a directory.
"""
if follow_symlinks:
return os.path.isdir(self)
return PathBase.is_dir(self, follow_symlinks=follow_symlinks)

def is_file(self, *, follow_symlinks=True):
"""
Whether this path is a regular file (also True for symlinks pointing
to regular files).
"""
if follow_symlinks:
return os.path.isfile(self)
return PathBase.is_file(self, follow_symlinks=follow_symlinks)

def is_mount(self):
"""
Check if this path is a mount point
"""
return os.path.ismount(self)

def is_symlink(self):
"""
Whether this path is a symbolic link.
"""
return os.path.islink(self)

def is_junction(self):
"""
Whether this path is a junction.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Suppress all :exc:`OSError` exceptions from :meth:`pathlib.Path.exists` and
``is_*()`` methods, rather than a selection of more common errors. The new
behaviour is consistent with :func:`os.path.exists`, :func:`os.path.isdir`,
etc. Use :meth:`Path.stat` to retrieve the file status without suppressing
exceptions.
Loading