Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-99726: Adds os.statx function and associated constants #99755

Closed
wants to merge 30 commits into from
Closed
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
6f7e9fa
gh-99726: Adds os.statx function and associated constants
zooba Nov 24, 2022
21b5f50
Update configure
zooba Nov 24, 2022
8fd982e
Fix POSIX build issues
zooba Nov 24, 2022
2b85372
Revert unrelated pyconfig.h.in changes
zooba Nov 24, 2022
a813f84
Revert generated files
zooba Nov 24, 2022
5a6ce8e
Update pyconfig.h.in
zooba Nov 24, 2022
6f54522
Fixup configure
zooba Nov 24, 2022
cda6886
Fix docs
zooba Nov 24, 2022
0db9341
Check for STATX_MNT_ID
zooba Nov 24, 2022
e278102
Properly exclude statx
zooba Nov 24, 2022
7844111
Add missing fields
zooba Nov 24, 2022
a787ae7
More field initialization
zooba Nov 24, 2022
b1182c3
More uninitialised fields
zooba Nov 24, 2022
cc5a159
endif
zooba Nov 24, 2022
ce24e79
Switch to less-confusing conditional attributes. Other fixes
zooba Nov 25, 2022
4a6e5ee
Few fixes
zooba Nov 25, 2022
7835bde
Make init config tests more reliable on Windows builds
zooba Nov 25, 2022
64111f4
Merge remote-tracking branch 'cpython/main' into gh-99726
zooba Nov 25, 2022
d53c013
Merge remote-tracking branch 'cpython/main' into gh-99726
zooba Nov 28, 2022
991876d
Update DeviceType check and rest of stdlib
zooba Nov 28, 2022
2d90f4d
Fix refleak and tests
zooba Nov 28, 2022
a6d3386
More test fixes
zooba Nov 28, 2022
ed5f370
Fix os.statx assumption
zooba Nov 28, 2022
4e95220
Documentation improvements
zooba Nov 29, 2022
3cc69aa
Missed one doc change
zooba Nov 29, 2022
025a4e6
Nope, it was applied fine
zooba Nov 29, 2022
8ee94aa
Handle block devices better
zooba Nov 29, 2022
bbcc6ee
Put statx in correct os sets, and include in pythoninfo
zooba Nov 29, 2022
768ba42
Merge remote-tracking branch 'cpython/main' into gh-99726
zooba Dec 5, 2022
cadeecb
Merge branch 'main' into gh-99726
carljm Dec 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 189 additions & 23 deletions Doc/library/os.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1841,6 +1841,16 @@ features:
function on your platform using :data:`os.supports_follow_symlinks`.
If it's unavailable, using it will raise a :exc:`NotImplementedError`.

On Windows, ``follow_symlinks`` applies to name-surrogate reparse points,
including symlinks and junctions. This type of reparse point is a symbolic
link to another path on the system. All other types of reparse point (e.g.
tiered storage) are always traversed, except that reparse points that are
not supported by the system are operated on directly.

If a path is either a symlink or a junction that cannot be traversed to the
final path, :func:`os.path.realpath` can be used to resolve as much of the
target path as possible.



.. function:: access(path, mode, *, dir_fd=None, effective_ids=False, follow_symlinks=True)
Expand Down Expand Up @@ -2795,17 +2805,6 @@ features:
This function can support :ref:`specifying a file descriptor <path_fd>` and
:ref:`not following symlinks <follow_symlinks>`.

On Windows, passing ``follow_symlinks=False`` will disable following all
name-surrogate reparse points, which includes symlinks and directory
junctions. Other types of reparse points that do not resemble links or that
the operating system is unable to follow will be opened directly. When
following a chain of multiple links, this may result in the original link
being returned instead of the non-link that prevented full traversal. To
obtain stat results for the final path in this case, use the
:func:`os.path.realpath` function to resolve the path name as far as
possible and call :func:`lstat` on the result. This does not apply to
dangling symlinks or junction points, which will raise the usual exceptions.

.. index:: module: stat

Example::
Expand Down Expand Up @@ -2839,18 +2838,75 @@ features:
``follow_symlinks=False`` had been specified instead of raising an error.


.. function:: statx(path, mask, *, dir_fd=None, follow_symlinks=True, flags=0)

Get selected fields of the status of a file or a file descriptor. Perform the
equivalent of a :c:func:`statx` system call on the given path. *path* may be
specified as either a string or bytes -- directly or indirectly through the
:class:`PathLike` interface -- or as an open file descriptor. Return a
:class:`stat_result` object.

*mask* is a bitwise combination of the ``STATX_*`` attributes in the
:mod:`stat` module, indicating which fields the caller intends to use. Note
that the set of fields returned may differ from what's requested, if the
operating system or file system does not support the metadata, or if it can
provide additional fields with no extra effort.

This function normally follows symlinks; to stat a symlink add the argument
``follow_symlinks=False``.

This function can support :ref:`specifying a file descriptor <path_fd>` and
:ref:`not following symlinks <follow_symlinks>`.

.. index:: module: stat

Example::

>>> import os, stat
>>> statinfo = os.statx('somefile.txt', stat.STATX_SIZE)
>>> statinfo
os.stat_result(st_mode=33188, st_ino=0, st_dev=0,
st_nlink=1, st_uid=0, st_gid=0, st_size=264, st_atime=0,
st_mtime=0, st_ctime=0)
>>> statinfo.stx_mask & stat.STATX_SIZE
512
>>> statinfo.st_size
264

.. seealso::

:func:`stat`, :func:`fstat` and :func:`lstat` functions.

.. versionadded:: 3.12
Added ``statx`` function.


.. class:: stat_result

Object whose attributes correspond roughly to the members of the
:c:type:`stat` structure. It is used for the result of :func:`os.stat`,
:func:`os.fstat` and :func:`os.lstat`.
:func:`os.fstat`, :func:`os.lstat` and :func:`os.statx`.

Attributes that are optional under :func:`os.stax` calls are marked
with the mask value that will be present in :attr:`stx_mask` when the
value is present. This can be tested even for regular ``stat`` calls,
as ``stx_mask`` will be initialized with a suitable value even if the
underlying system call uses regular ``stat``.

Attributes may be missing from this struct if they are not supported by
the underlying operating system. When :func:`os.statx` is supported, all
``stx_*`` attributes are present, but their ``st_*`` equivalents may not
be. :attr:`stx_mask` is always present, even if ``statx`` is not supported.

Attributes:

.. attribute:: st_mode

File mode: file type and file mode bits (permissions).

This field is set when ``stx_mask`` contains
:data:`stat.STATX_TYPE` and/or :data:`stat.STATX_MODE`.

.. attribute:: st_ino

Platform dependent, but if non-zero, uniquely identifies the
Expand All @@ -2861,70 +2917,128 @@ features:
<https://msdn.microsoft.com/en-us/library/aa363788>`_ on
Windows

This field is set when ``stx_mask`` contains :data:`stat.STATX_INO`.

.. attribute:: st_dev

Identifier of the device on which this file resides.

On Windows, this field is set with :data:`stat.STATX_INO`.
Other platforms always set this value.

.. attribute:: st_nlink

Number of hard links.

This field is set when ``stx_mask`` contains :data:`stat.STATX_NLINK`.

.. attribute:: st_uid

User identifier of the file owner.

This field is set when ``stx_mask`` contains :data:`stat.STATX_UID`.

.. attribute:: st_gid

Group identifier of the file owner.

This field is set when ``stx_mask`` contains :data:`stat.STATX_GID`.

.. attribute:: st_size

Size of the file in bytes, if it is a regular file or a symbolic link.
The size of a symbolic link is the length of the pathname it contains,
without a terminating null byte.

This field is set when ``stx_mask`` contains :data:`stat.STATX_SIZE`.

.. attribute:: stx_mask

Flags indicating which values were set. :func:`os.statx` allows specifying
a mask, though the result may include more or less than requested. Other
``stat`` functions set a default value representing the information they
return.

Timestamps:

.. attribute:: st_atime

Time of most recent access expressed in seconds.

This field is set when ``stx_mask`` contains :data:`stat.STATX_ATIME`.

.. attribute:: st_mtime

Time of most recent content modification expressed in seconds.

This field is set when ``stx_mask`` contains :data:`stat.STATX_MTIME`.

.. attribute:: st_ctime

Platform dependent:

* the time of most recent metadata change on Unix,
* the time of creation on Windows, expressed in seconds.
* the time of creation on Windows, expressed in seconds, except
when :data:`stat.STATX_CTIME` is in :attr:`stx_mask`, in which
case this is the time of the most recent metadata change

This field is set when ``stx_mask`` contains :data:`stat.STATX_CTIME`.

.. attribute:: st_atime_ns

Time of most recent access expressed in nanoseconds as an integer.

This field is set when ``stx_mask`` contains :data:`stat.STATX_ATIME`.

.. attribute:: st_mtime_ns

Time of most recent content modification expressed in nanoseconds as an
integer.

This field is set when ``stx_mask`` contains :data:`stat.STATX_MTIME`.

.. attribute:: st_ctime_ns

Platform dependent:

* the time of most recent metadata change on Unix,
* the time of creation on Windows, expressed in nanoseconds as an
integer.
integer, except when :data:`stat.STATX_CTIME` is in :attr:`stx_mask`,
in which case this is the time of the most recent metadata change

This field is set when ``stx_mask`` contains :data:`stat.STATX_CTIME`.

.. attribute:: st_birthtime

Time of file creation, if available. The attribute may not be present if
your operating system does not support the field.

This field is set with :data:`stat.STATX_BTIME`, but is only present when
supported by regular ``stat`` calls. See also: :attr:`stx_btime`.

.. attribute:: stx_btime

Alias of :attr:`st_birthtime` that is always present when :func:`statx`
is supported. If ``st_birthtime`` is also present, its value will be
identical.

This field is set when ``stx_mask`` contains :data:`stat.STATX_BTIME`.

.. attribute:: stx_btime_ns

Time of file creation expressed in nanoseconds as an integer.

This field is set when ``stx_mask`` contains :data:`stat.STATX_BTIME`.

.. note::

The exact meaning and resolution of the :attr:`st_atime`,
:attr:`st_mtime`, and :attr:`st_ctime` attributes depend on the operating
system and the file system. For example, on Windows systems using the FAT
or FAT32 file systems, :attr:`st_mtime` has 2-second resolution, and
:attr:`st_atime` has only 1-day resolution. See your operating system
documentation for details.
:attr:`st_mtime`, :attr:`st_ctime`, :attr:`st_birthtime` and
:attr:`stx_btime` attributes depend on the operating system and the file
system. For example, on Windows systems using the FAT or FAT32 file
systems, :attr:`st_mtime` has 2-second resolution, and :attr:`st_atime`
has only 1-day resolution.
See your operating system documentation for details.

Similarly, although :attr:`st_atime_ns`, :attr:`st_mtime_ns`,
and :attr:`st_ctime_ns` are always expressed in nanoseconds, many
Expand All @@ -2943,30 +3057,71 @@ features:
Number of 512-byte blocks allocated for file.
This may be smaller than :attr:`st_size`/512 when the file has holes.

This field is set with :data:`stat.STATX_BLOCKS`, but is only present
when supported by regular ``stat`` calls. See also: :attr:`stx_blocks`.

.. attribute:: st_blksize

"Preferred" blocksize for efficient file system I/O. Writing to a file in
smaller chunks may cause an inefficient read-modify-rewrite.

This field is set with :data:`stat.STATX_BLOCKSIZE`, but is only present
when supported by regular ``stat`` calls. See also: :attr:`stx_blksize`.

.. attribute:: st_rdev

Type of device if an inode device.

This field is always set when appropriate, but is only present when
supported by regular ``stat`` calls. See also: :attr:`stx_rdev`.

.. attribute:: st_flags

User defined flags for file.

.. attribute:: stx_attributes

Additional attribute flags (``STATX_ATTR_*`` values).

This field is only set for calls using :func:`os.statx`.

.. attribute:: stx_attributes_mask

Attribute flags (``STATX_ATTR_*``) that were supported on the file system
containing the file. Flags not set in this mask are meaningless in
:attr:`stx_attributes`.

This field is only set for calls using :func:`os.statx`.

.. attribute:: stx_blocks

Alias of :attr:`st_blocks` that is always present when :func:`statx` is
supported. If ``st_blocks`` is also present, its value will be identical.

This field is set with :data:`stat.STATX_BLOCKS`.

.. attribute:: stx_blksize

Alias of :attr:`st_blksize` that is always present when :func:`statx` is
supported. If ``st_blksize`` is also present, its value will be identical.

This field is set with :data:`stat.STATX_BLKSIZE`.

.. attribute:: stx_rdev

Alias of :attr:`st_rdev` that is always present when :func:`statx` is
supported. If ``st_rdev`` is also present, its value will be identical.

This field is always set, when appropriate, by all ``stat`` calls.


On other Unix systems (such as FreeBSD), the following attributes may be
available (but may be only filled out if root tries to use them):

.. attribute:: st_gen

File generation number.

.. attribute:: st_birthtime

Time of file creation.

On Solaris and derivatives, the following attributes may also be
available:

Expand Down Expand Up @@ -2998,12 +3153,16 @@ features:
:c:func:`GetFileInformationByHandle`. See the ``FILE_ATTRIBUTE_*``
constants in the :mod:`stat` module.

This field is requested with :data:`stat.STATX_TYPE`.

.. attribute:: st_reparse_tag

When :attr:`st_file_attributes` has the ``FILE_ATTRIBUTE_REPARSE_POINT``
set, this field contains the tag identifying the type of reparse point.
See the ``IO_REPARSE_TAG_*`` constants in the :mod:`stat` module.

This field is requested with :data:`stat.STATX_TYPE`.

The standard module :mod:`stat` defines functions and constants that are
useful for extracting information from a :c:type:`stat` structure. (On
Windows, some items are filled with dummy values.)
Expand Down Expand Up @@ -3039,6 +3198,13 @@ features:
files as :const:`S_IFCHR`, :const:`S_IFIFO` or :const:`S_IFBLK`
as appropriate.

.. versionchanged:: 3.12
Added the :attr:`stx_mask` member along with :func:`statx`.

.. versionchanged:: 3.12
Added the :attr:`st_birthtime` member on Windows.


.. function:: statvfs(path)

Perform a :c:func:`statvfs` system call on the given path. The return value is
Expand Down
2 changes: 2 additions & 0 deletions Include/internal/pycore_fileutils.h
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ struct _Py_stat_struct {
int st_ctime_nsec;
unsigned long st_file_attributes;
unsigned long st_reparse_tag;
time_t st_btime;
int st_btime_nsec;
};
#else
# define _Py_stat_struct stat
Expand Down
Loading