pythongh-117151: IO performance improvement, increase io.DEFAULT_BUFF…

…ER_SIZE to 128k, adjust open() to use max(st_blksize, io.DEFAULT_BUFFER_SIZE)

Doc/library/functions.rst

@@ -1381,10 +1381,11 @@ are always available. They are listed here in alphabetical order.
  :func:`io.TextIOWrapper.reconfigure`. When no *buffering* argument is
  given, the default buffering policy works as follows:
 
- * Binary files are buffered in fixed-size chunks; the size of the buffer is
- chosen using a heuristic trying to determine the underlying device's "block
- size" and falling back on :const:`io.DEFAULT_BUFFER_SIZE`. On many systems,
- the buffer will typically be 4096 or 8192 bytes long.
+ * Binary files are buffered in fixed-size chunks; the size of the buffer
+ is set to ``max(io.DEFAULT_BUFFER_SIZE, st_blksize)`` using a heuristic
+ trying to determine the underlying device's "block size" when available
+ and falling back on :const:`io.DEFAULT_BUFFER_SIZE`.
+ On most systems, the buffer will typically be 131072 bytes long.
 
  * "Interactive" text files (files for which :meth:`~io.IOBase.isatty`
  returns ``True``) use line buffering. Other text files use the policy

Lib/_pyio.py

@@ -23,8 +23,8 @@
  valid_seek_flags.add(os.SEEK_HOLE)
  valid_seek_flags.add(os.SEEK_DATA)
 
-# open() uses st_blksize whenever we can
-DEFAULT_BUFFER_SIZE = 8 * 1024 # bytes
+# open() uses max(st_blksize, io.DEFAULT_BUFFER_SIZE) when st_blksize is available
+DEFAULT_BUFFER_SIZE = 128 * 1024 # bytes
 
 # NOTE: Base classes defined here are registered with the "official" ABCs
 # defined in io.py. We don't use real inheritance though, because we don't want
@@ -248,8 +248,7 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
  except (OSError, AttributeError):
  pass
  else:
- if bs > 1:
- buffering = bs
+ buffering = max(bs, DEFAULT_BUFFER_SIZE)
  if buffering < 0:
  raise ValueError("invalid buffering size")
  if buffering == 0:

Misc/NEWS.d/next/Core and Builtins/2024-04-30-14-03-09.gh-issue-117151.yt2H8c.rst

@@ -0,0 +1,4 @@
+Increase ``io.DEFAULT_BUFFER_SIZE`` from 8k to 128k. Adjust :func:`open` on
+Linux to use ``max(io.DEFAULT_BUFFER_SIZE, device block size)`` rather than
+the device block size. Improve I/O write performance by up to 5 times.
+Patch by Romain Morotti.

Modules/_io/_iomodule.h

@@ -78,7 +78,7 @@ extern Py_ssize_t _PyIO_find_line_ending(
 */
 extern int _PyIO_trap_eintr(void);
 
-#define DEFAULT_BUFFER_SIZE (8 * 1024) /* bytes */
+#define DEFAULT_BUFFER_SIZE (128 * 1024) /* bytes */
 
 /*
  * Offset type for positioning.