gh-124370: Add "howto" for free-threaded Python #124371

colesbury · 2024-09-23T19:23:01Z

This is a guide aimed at people writing Python code, as oppposed to the existing guide for C API extension authors.

Issue: Python "HOWTO" guide for free-threading #124370

📚 Documentation preview 📚: https://cpython-previews--124371.org.readthedocs.build/en/124371/howto/free-threading-python.html

This is a guide aimed at people writing Python code, as oppposed to the existing guide for C API extension authors.

Doc/howto/free-threading-python.rst

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

ZeroIntensity · 2024-09-23T20:42:36Z

Doc/howto/free-threading-python.rst

+* :ref:`function <user-defined-funcs>` objects declared at the module level
+* :ref:`method <instance-methods>` descriptors
+* :ref:`code <code-objects>` objects
+* :term:`module` objects and their dictionaries
+* :ref:`classes <classes>` (type objects)


It's probably worth noting somewhere in here that these objects (and immortalization itself) are implementation details, and very much subject to change.

Doc/howto/free-threading-python.rst

JelleZijlstra · 2024-09-23T20:47:53Z

Doc/howto/free-threading-python.rst

+mechanism for decisions related to the build configuration.
+
+
+Thread Safety


I feel like this section is probably among the most important for most users. As I understand it, the general rule is that you write thread-safe code under free-threading the same way that you'd write thread-safe code under previous versions of Python; the rules have not really changed. That could be made more explicit.

I think that's our goal for the implementation, but I'm a bit hesitant to promise too much here, especially because it's not always clear what was thread-safe under the GIL.

Doc/howto/free-threading-python.rst

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

gpshead

High level comments:

Overall, great! I'm not reading it with an eye for what has been left out. But its a doc, I consider those "living" and future updates would deal with that.
Perhaps include the word "experimental" in the top level title to help temper expectations?
Consider adding a reference to PEP 703 to the doc for context for those interested in how this came to be and what the initial long term plan and expectations are.
In the final Single-threaded performance section, I predict someone not aware of larger long term plans and PEP 703 and the earlier proof of concept work you built on top of a couple releases to freak out at the thought of a 40% slower interpreter. I'd go beyond the "This overhead is expected to be reduced in the Python 3.14." statement and indicate what level of free-threaded build performance delta we're ultimately aiming for in the future (without promising which version it happens in).

mpage · 2024-09-24T03:46:04Z

LGTM. +1 for @gpshead's comment about including some detail about what single-threaded performance delta we're ultimately shooting for.

Doc/howto/free-threading-python.rst

eendebakpt · 2024-09-24T05:36:06Z

Doc/howto/free-threading-python.rst

+---------
+
+Sharing the same iterator object between multiple threads is generally not
+safe and threads may see duplicate or missing elements when iterating or crash


Is the possible crashing of the interpreter expected to be addressed in 3.14?

As a consequence of iterators not being thread safe, some modules are not safe either (e g. json, itertools). Should this be mentioned somewhere?

vstinner

LGTM. This HOWTO can be adjusted/elaborated later if needed.

nedbat

Thanks for doing this! I love that we are putting out information aimed at the users.

nedbat · 2024-09-24T11:46:37Z

Doc/howto/free-threading-python.rst

+*********************************
+
+Starting with the 3.13 release, CPython has experimental support for running
+with the :term:`global interpreter lock` (GIL) disabled in a configuration


"configuration" makes me think of a configuration file I might need to create. Can we say "build of Python"? Or at least "build configuration".

nedbat · 2024-09-24T11:47:19Z

Doc/howto/free-threading-python.rst

+with the :term:`global interpreter lock` (GIL) disabled in a configuration
+called :term:`free threading`.  This document describes the implications of
+free threading for Python code.  See :ref:`freethreading-extensions-howto` for
+information on how to write C extensions that support the free-threaded build.


We seem to switch between "free threading" and "free-threaded" kind of randomly. I can't decide if this is OK or if we should choose one and stick with it.

I think it's fine. We also say both "multithreaded" and "multithreading" depending on the context.

nedbat · 2024-09-24T11:48:35Z

Doc/howto/free-threading-python.rst

+
+The free-threaded build of CPython can optionally run with the global
+interpreter lock enabled, such as when :envvar:`PYTHON_GIL` is set to ``1``,
+or when importing an extension module that requires the GIL.


This could use more explanation. When might I want to set this environment variable? "an extension module that requires the GIL": is this something I have to figure out, or does Python know this itself somehow?

I don't want to get into the details of the environment variable here. The important point here is that the free-threaded build can run with the GIL enabled -- that's not obvious to most people. The environment variable is mentioned here just as an example of that.

nedbat · 2024-09-24T11:53:01Z

Doc/howto/free-threading-python.rst

+The free-threaded build of the 3.13 release makes some objects :term:`immortal`
+in order to avoid reference count contention that would prevent efficient
+multi-threaded scaling.  This means that these objects are never deallocated.
+This is expected to be addressed in Python 3.14 with


"addressed" how? That makes it sound like immortal objects are a problem. Are they?

nedbat · 2024-09-24T11:55:21Z

Doc/howto/free-threading-python.rst

+* :ref:`classes <classes>` (type objects)
+
+The immortalization of these objects happens the first time a thread is started
+after the main thread.


After reading about immortalization, I don't understand the implications for me as a Python programmer. Why do I care that they are now immortal? These all sound like things that would have never been deallocated anyway. Are there unusual circumstances that I should be considering?

Hopefully most people won't care, but some programs create these sorts of things in a loop.

Doc/howto/free-threading-python.rst

nedbat · 2024-09-24T11:58:58Z

Doc/howto/free-threading-python.rst

+called :term:`free threading`.  This document describes the implications of
+free threading for Python code.  See :ref:`freethreading-extensions-howto` for
+information on how to write C extensions that support the free-threaded build.
+


I wonder if we could add a quick snapshot of the overall plan: if everything works out, eventually free threading will be the only build, etc. Also, maybe a statement about how most programmer won't need to be concerned with this, we're doing a lot to keep everyday Python programs behaving the same, etc.

I would perhaps add a note in the seealso block that refers to the PEP and adds 1 or 2 highlights:

performance conscious users may wish to try the free threaded version

most users can safely continue to use the default CPython version

when free threading is no longer experimental, the versions will converge into one version for all.

nedbat · 2024-09-24T12:00:12Z

Doc/howto/free-threading-python.rst

+
+The :func:`sys._is_gil_enabled` function will return ``False`` if the global
+interpreter lock is currently disabled.  This is the recommended mechanism for
+decisions like whether to use multithreading or multiprocessing.


Is it likely that people will write code that examines this variable and chooses between multithreading and multiprocessing? It seems unlikely to me, but maybe library authors will? When should it be used?

"How do I distinguish between the free-threaded and GIL-enabled build?" is a fairly common question, although I don't think people make much use of it other than for quick debugging and sanity checks.

I would break this section into two sections:

Identifying free threading builds (with the last paragraph)

Reenabling the GIL with a free threading build (first two paragraphs)

bedevere-app · 2024-09-24T12:02:08Z

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

Co-authored-by: mpage <mpage@cs.stanford.edu>

willingc

Looking good @colesbury. I made some wording suggestions that should add clarity for Ned and others.

Doc/howto/free-threading-python.rst

willingc · 2024-09-24T20:32:06Z

Doc/howto/free-threading-python.rst

+called :term:`free threading`.  This document describes the implications of
+free threading for Python code.  See :ref:`freethreading-extensions-howto` for
+information on how to write C extensions that support the free-threaded build.
+


I would perhaps add a note in the seealso block that refers to the PEP and adds 1 or 2 highlights:

performance conscious users may wish to try the free threaded version

most users can safely continue to use the default CPython version

when free threading is no longer experimental, the versions will converge into one version for all.

Doc/howto/free-threading-python.rst

willingc · 2024-09-24T20:37:41Z

Doc/howto/free-threading-python.rst

+
+The :func:`sys._is_gil_enabled` function will return ``False`` if the global
+interpreter lock is currently disabled.  This is the recommended mechanism for
+decisions like whether to use multithreading or multiprocessing.


I would break this section into two sections:

Identifying free threading builds (with the last paragraph)

Reenabling the GIL with a free threading build (first two paragraphs)

Doc/howto/free-threading-python.rst

willingc · 2024-09-24T20:51:53Z

Doc/howto/free-threading-python.rst

+---------------
+
+The free-threaded build of the 3.13 release makes some objects :term:`immortal`
+in order to avoid reference count contention that would prevent efficient


Suggested change

in order to avoid reference count contention that would prevent efficient

which have reference counts that are never modified. Since the immortal object is guaranteed not to be

deallocated, efficient multi-threading scaling is possible by avoiding reference count contention between

threads.

I've reworked this a bit. Would you please take another look at it? As you wrote, immortalization means reference counts are not modified and objects are not guaranteed not to be deallocated. I wanted to get across that:

Reference counts not being modified enables efficient multi-threaded scaling.

Objects not being deallocated is a potential problem because some applications may leak memory. That's being addressed in 3.14. We don't expect this to be a problem for most applications, but it's something to watch out for if you run your application with refleak checks.

Doc/howto/free-threading-python.rst

willingc · 2024-09-24T21:03:06Z

Doc/howto/free-threading-python.rst

+less of an impact.  This overhead is expected to be reduced in Python
+3.14.   We are aiming for an overhead of 10% or less on the pyperformance


Suggested change

less of an impact. This overhead is expected to be reduced in Python

3.14. We are aiming for an overhead of 10% or less on the pyperformance

less of an impact. This overhead is expected to be reduced in Python

3.14 by implementing more efficient reference counting and garbage collection. We are aiming for an overhead of 10% or less on the pyperformance

We expect that the biggest source of improvement will be reenabling the specializing adaptive interpreter in 3.14. It's currently disabled in the 3.13 free-threaded build. Is that worth mentioning here?

I don't expect any big performance impact for reference counting and garbage collection changes in 3.14

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

willingc

Love the updates @colesbury. This how to is a nice addition 😄

nedbat · 2024-09-25T15:28:27Z

Doc/howto/free-threading-python.rst

+
+The GIL may also automatically be enabled when importing a C-API extension
+module that is not explicitly marked as supporting free threading.  See
+:c:macro:`Py_MOD_GIL_NOT_USED` for more details.


Since this document is for Python programmers, I think we should avoid sending them off to look at C macros. As a Python programmer, my question now would be, "how can I tell if my C extensions support free threading or not?" Is there a way to know without examining the C source code?

I can remove the link and mention of the C macro. There isn't a great answer for "how can I tell if my C extensions support free threading?"

Read the docs for the package in question. I think this is often the best answer for "does the extensions support 3.13?" or whatever new Python version comes out

pip install it.

Check out https://py-free-threading.github.io/tracking/ or https://hugovk.github.io/free-threaded-wheels/

If the extension is not marked as supporting free-threading, then a warning will be printed when it is imported (and the GIL will be automatically enable)

Those are good answers!

willingc · 2024-09-25T18:51:17Z

@nedbat If you are good with today's change, please approve and merge. Nice work by @colesbury and everyone who gave feedback. Let's get this out in the wild and we can fine tune if/when needed. 🎉

* main: (69 commits) Add "annotate" SET_FUNCTION_ATTRIBUTE bit to dis. (python#124566) pythongh-124412: Add helpers for converting annotations to source format (python#124551) pythongh-119180: Disallow instantiation of ConstEvaluator objects (python#124561) For-else deserves its own section in the tutorial (python#123946) Add 3.13 as a version option to the crash issue template (python#124560) pythongh-123242: Note that type.__annotations__ may not exist (python#124557) pythongh-119180: Make FORWARDREF format look at __annotations__ first (python#124479) pythonGH-58058: Add quick reference for `ArgumentParser` to argparse docs (pythongh-124227) pythongh-41431: Add `datetime.time.strptime()` and `datetime.date.strptime()` (python#120752) pythongh-102450: Add ISO-8601 alternative for midnight to `fromisoformat()` calls. (python#105856) pythongh-124370: Add "howto" for free-threaded Python (python#124371) pythongh-121277: Allow `.. versionadded:: next` in docs (pythonGH-121278) pythongh-119400: make_ssl_certs: update reference test data automatically, pass in expiration dates as parameters python#119400 (pythonGH-119401) pythongh-119180: Avoid going through AST and eval() when possible in annotationlib (python#124337) pythongh-124448: Update Windows builds to use Tcl/Tk 8.6.15 (pythonGH-124449) pythongh-123884 Tee of tee was not producing n independent iterators (pythongh-124490) pythongh-124378: Update test_ttk for Tcl/Tk 8.6.15 (pythonGH-124542) pythongh-124513: Check args in framelocalsproxy_new() (python#124515) pythongh-101100: Add a table of class attributes to the "Custom classes" section of the data model docs (python#124480) Doc: Use ``major.minor`` for documentation distribution archive filenames (python#124489) ...

vstinner · 2024-09-26T09:01:53Z

This is a guide aimed at people writing Python code, as oppposed to the existing guide for C API extension authors.

Thanks for writing down this guide @colesbury! It will be very valuable :-)

miss-islington-app · 2024-10-01T17:12:02Z

Thanks @colesbury for the PR, and @nedbat for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

* pythongh-124370: Add "howto" for free-threaded Python This is a guide aimed at people writing Python code, as oppposed to the existing guide for C API extension authors. * Add missing new line * Update Doc/howto/free-threading-python.rst Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * interned -> immortalized * Apply suggestions from code review Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> * Update Doc/howto/free-threading-python.rst Co-authored-by: mpage <mpage@cs.stanford.edu> * Update docs * Apply suggestions from code review Co-authored-by: Carol Willing <carolcode@willingconsulting.com> * A few more updates * Additional comment on immortal objects * Mention specializing adaptive interpreter * Remove trailing whitespace * Remove mention of C macro --------- (cherry picked from commit 68e384c) Co-authored-by: Sam Gross <colesbury@gmail.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: mpage <mpage@cs.stanford.edu> Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

bedevere-app · 2024-10-01T17:12:13Z

GH-124860 is a backport of this pull request to the 3.13 branch.

…124860) gh-124370: Add "howto" for free-threaded Python (GH-124371) (cherry picked from commit 68e384c) Co-authored-by: Sam Gross <colesbury@gmail.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: mpage <mpage@cs.stanford.edu> Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

pythongh-124370: Add "howto" for free-threaded Python

5b4a8aa

This is a guide aimed at people writing Python code, as oppposed to the existing guide for C API extension authors.

colesbury added the topic-free-threading label Sep 23, 2024

bedevere-app bot mentioned this pull request Sep 23, 2024

Python "HOWTO" guide for free-threading #124370

Closed

bedevere-app bot added docs Documentation in the Doc dir skip news labels Sep 23, 2024

Add missing new line

0252cf5

colesbury marked this pull request as ready for review September 23, 2024 20:33

colesbury requested a review from willingc as a code owner September 23, 2024 20:33

bedevere-app bot added the awaiting core review label Sep 23, 2024

ZeroIntensity reviewed Sep 23, 2024

View reviewed changes

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved

JelleZijlstra reviewed Sep 23, 2024

View reviewed changes

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved

colesbury and others added 2 commits September 23, 2024 13:42

Update Doc/howto/free-threading-python.rst

df1489e

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

interned -> immortalized

5658fa8

ZeroIntensity reviewed Sep 23, 2024

View reviewed changes

JelleZijlstra reviewed Sep 23, 2024

View reviewed changes

hugovk reviewed Sep 23, 2024

View reviewed changes

Apply suggestions from code review

ae0f637

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

colesbury requested review from DinoV and mpage September 23, 2024 23:23

gpshead reviewed Sep 23, 2024

View reviewed changes

mpage reviewed Sep 24, 2024

View reviewed changes

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved

eendebakpt reviewed Sep 24, 2024

View reviewed changes

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved

eendebakpt reviewed Sep 24, 2024

View reviewed changes

vstinner approved these changes Sep 24, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Sep 24, 2024

nedbat requested changes Sep 24, 2024

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting merge labels Sep 24, 2024

colesbury and others added 2 commits September 24, 2024 09:31

Update Doc/howto/free-threading-python.rst

1554fb9

Co-authored-by: mpage <mpage@cs.stanford.edu>

Update docs

3b173e0

willingc reviewed Sep 24, 2024

View reviewed changes

colesbury and others added 5 commits September 24, 2024 14:13

Apply suggestions from code review

91ec1ca

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

A few more updates

6078cfd

Additional comment on immortal objects

e02ed00

Mention specializing adaptive interpreter

526737e

Remove trailing whitespace

f24bdb7

willingc approved these changes Sep 25, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting changes labels Sep 25, 2024

nedbat reviewed Sep 25, 2024

View reviewed changes

Remove mention of C macro

a3ad6b3

willingc approved these changes Sep 25, 2024

View reviewed changes

nedbat approved these changes Sep 25, 2024

View reviewed changes

nedbat merged commit 68e384c into python:main Sep 25, 2024
23 checks passed

bedevere-app bot removed the awaiting merge label Sep 25, 2024

colesbury deleted the gh-124370-python-free-threaded-howto branch October 1, 2024 17:10

colesbury added the needs backport to 3.13 bugs and security fixes label Oct 1, 2024

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Oct 1, 2024

		mechanism for decisions related to the build configuration.


		Thread Safety

-in order to avoid reference count contention that would prevent efficient
+which have reference counts that are never modified. Since the immortal object is guaranteed not to be
+deallocated, efficient multi-threading scaling is possible by avoiding reference count contention between
+threads.

		less of an impact. This overhead is expected to be reduced in Python
		3.14. We are aiming for an overhead of 10% or less on the pyperformance

gh-124370: Add "howto" for free-threaded Python #124371

gh-124370: Add "howto" for free-threaded Python #124371

Conversation

colesbury commented Sep 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gpshead left a comment

Choose a reason for hiding this comment

mpage commented Sep 24, 2024

Choose a reason for hiding this comment

vstinner left a comment

Choose a reason for hiding this comment

nedbat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bedevere-app bot commented Sep 24, 2024

willingc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willingc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willingc commented Sep 25, 2024

vstinner commented Sep 26, 2024

miss-islington-app bot commented Oct 1, 2024

bedevere-app bot commented Oct 1, 2024

colesbury commented Sep 23, 2024 •

edited

Loading