Is there a 3.13 nogil container?

[impredicative]

I want a 3.13 nogil container, also with all optimizations, etc. I assume that the current 3.13 containers use the GIL.

[tianon]

We don't have a "nogil" image currently. For 3.13 we're using the default compilation settings, which AFAIK still results in the GIL. Unfortunately, compiling Python is pretty heavy, so expanding our current support matrix to include new "nogil" variants is also going to be a tough sell. 🙈

[tianon]

I'm not sure how you got there from what I said -- perhaps I can clarify: we do compile Python (currently ~42 times per architecture, in fact, and we support ~10 architectures where compilation is necessary), and it's very heavy to do so, and adding more variants would require us to do so even more times, so is not something we're willing to consider unless/until it is an officially recommended configuration (ideally the default configuration, but we'd be willing to consider an official statement along the lines of "everyone should try this out").

[impredicative]

nogil is the future of Python. It doesn't make sense to not support it. If this request seems strange, that may be so only because I am the first person to ask for it. Rest assured, there will be many more who will progressively ask for it. The pressure will build. This request is easy to reject now, but only now.

[tianon]

I agree that nogil is the future of Python, and there's nothing strange about the request. If I were completely opposed to the idea in general, I probably would've closed the issue already. It's more a matter of resourcing/prioritization, not desire/understanding. If the demand were actively higher, it would also be a lot easier to justify the added resources/maintenance spend -- the best way for folks to signal that is to react with 👍 on the top post in this thread.

To put that another way, my answer is more accurately "not yet", not "never".

If you'd like to help with the work, figuring out what changes are necessary to our current Dockerfiles would be a really useful first step.

[ldeluigi]

An experimental JIT-enabled build would also be great to have

[sekrause]

Instead of prodiving a whole second set of "nogil" Docker images the free-threaded executable could also be part of the normal images just like it is done with the official Windows and macOS installers:

CPython now has experimental support for running in a free-threaded mode, with the global interpreter lock (GIL) disabled. This is an experimental feature and therefore is not enabled by default. The free-threaded mode requires a different executable, usually called python3.13t or python3.13t.exe. Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers, or CPython can be built from source with the --disable-gil option.

https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython

So we would have /usr/local/bin/python3.13t in every image and call this to use the free-threaded mode.

Unfortunately that still means that all of Python has to be compiled twice.

[impredicative]

@sekrause But that would break a lot of tooling that expects Python to be at the prior file path.

[sekrause]

But that would break a lot of tooling that expects Python to be at the prior file path.

The normal paths would still be there, but point to the normal GIL version. So if you don't adjust your tooling everything will be as before.

[abebus]

If anyone seeking for some basic image to play with, I've made one: https://github.com/abebus/free-threaded-python-docker-image

[SDAravind]

Python images without GIL and JIT enabled should be part of docker images. I have been using docker as a dev container and to experiment new versions of python. Thanks to all the work been done so far.

[pitrou]

I'm a bit surprised that you're going out of your way to provide images for alpha versions of Python (if I'm reading https://hub.docker.com/_/python correctly, there are images for Python 3.14.0a2), but not for free-threaded builds which are an official feature of released Pythons. While these builds are still marked experimental, many Python packages have started using them to ensure they are free-threading compatible.

For example in Apache Arrow we have our own image for CI until the official Python Docker images support free-threaded builds:
https://github.com/apache/arrow/blob/main/ci/docker/python-free-threaded-wheel-manylinux-test-imports.dockerfile

Many Python open source projects are probably using similar home-grown recipes due to lack of official Docker image availability.

[impredicative]

I think four builds are needed:

1. GIL and noJIT (current build)
2. noGIL and noJIT (this issue)
3. GIL and JIT
4. noGIL and JIT

Is this correct?

[abebus]

@impredicative

4. noGIL and JIT

is not possible to build (yet)

[pitrou]

Those are unrelated, perhaps open a separate issue for JIT builds?

(but I'd argue that JIT builds are unimportant for now, as they don't provide tangible benefits)

[impredicative]

Those are unrelated, perhaps open a separate issue for JIT builds?

It matters because the naming convention has to account for all possible variations. Also, it might help in understanding that bundling multiple versions in the same image is not a great idea.

(but I'd argue that JIT builds are unimportant for now, as they don't provide tangible benefits)

JIT may not matter too much now, but it's the sauce that could get Python closer to Julia-like speeds, so it may matter in the long term.