Add OpenSSL 3.5 #782
Add OpenSSL 3.5 #782
Conversation
This means all supported Unix platforms (CPython 3.9+), and Windows from CPython 3.11+. This initial attempt copies what we did for OpenSSL 3.0.
|
I'm pleasantly surprised that my naïve attempt worked for so many platforms. Tested one of the macOS builds locally and all seems to be in order: >>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 3.5.2 5 Aug 2025'
>>> hex(ssl.OPENSSL_VERSION_NUMBER)
'0x30500020'
>>> ssl.OPENSSL_VERSION_INFO
(3, 5, 0, 2, 0)
>>> import socket
... import ssl
...
... hostname = 'www.python.org'
... context = ssl.create_default_context()
...
... with socket.create_connection((hostname, 443)) as sock:
... with context.wrap_socket(sock, server_hostname=hostname) as ssock:
... print(ssock.version())
...
TLSv1.3Failing jobs are:
(also one AArch64 Windows job timed out, but I'm ignoring that for now) Looking at some logs from the failed jobs: Linux riscv64Linux s390xWindows x86-64 |
Linux riscv64
Some options:
Example patch: @@ -94,8 +94,8 @@
.globl riscv_vlen_asm
.type riscv_vlen_asm,\@function
riscv_vlen_asm:
- csrr $ret, vlenb
- slli $ret, $ret, 3
+ csrr a0, 0xc22
+ slli a0, a0, 3
ret
.size riscv_vlen_asm,.-riscv_vlen_asm |
Linux s390x
Options:
|
Windows x86-64
Options:
Looks like we source a nasm binary from python/cpython-bin-deps and we're using the latest (only) version available: https://github.com/python/cpython-bin-deps/tree/nasm-2.11.06 |
This has been done to support OpenSSL 3.5 that uses the `cijne` opcode. This is still a very conservative arch-level. Gentoo also made this change when updating to OpenSSL 3.5, and to quote the news item from Andreas K. Hüttel: > The z10 Enterprise Class (2097 series) was introduced in February 2008, > which essentially means everyone except hardware archaeologists should be fine.
This is to support building OpenSSL 3.5. Version 2.16.03 is the latest stable version at the time of writing.
|
Looks like I messed up my patching copypasta: Ah, when building cpython we explicitly copy patches in: for f in sorted(os.listdir(SUPPORT)):
if f.endswith(".patch"):
build_env.copy_file(SUPPORT / f) |
|
s390x builds are failing still, but with building bdb of all things: I don't know if it's an issue with GitHub, but I can't download the log archive, so I'm going to try |
Edward-Knight
force-pushed
the
openssl_5.3
branch
from
f9acf84 to
3f7c4cb
Compare
Our GCC version doesn't know it, so replace the `vlenb` variable with `0xc22`. See openssl/openssl#23011.
Edward-Knight
force-pushed
the
openssl_5.3
branch
from
3f7c4cb to
ec2491e
Compare
s390x bdb configure issue with
|
|
It seems to try and run the CXX and fails: |
|
There is a Current options I'm thinking of:
|
|
Nice, thank you for the PR and the detailed investigations! The workarounds you have so far seem generally fine. Let me try to find some context on why we currently get nasm from cpython-bin-deps. Also FYI I'm hoping to figure out how to use a newer OS baseline for compiling (without affecting our target OS for runtime) which may help with the problem of older toolchains in general, so patches to work around older toolchains seem perfectly fine in the meantime. We may also have the option of using our recent Clang version instead of the OS GCC version. @namrata-ibm and @jonathan-albrecht-ibm, since you've both raised issues about s390x support in the past - do you have any thoughts on whether it's okay to increase the arch baseline to |
|
It seems like we've been here before! The issue I'm having seems to be the same as #512 (comment)
The fix for #512 was always setting the CXX environment variable (d0ed97f). However, since we don't actually install the C++ compiler in the cross image, autoconf is still unhappy in the same way. I don't know which of these is true (and why it's otherwise been working...):
I'm going to attempt to fix this by installing the C++ compilers into the Docker images. This should prevent autoconf from falling back to |
Edward-Knight
force-pushed
the
openssl_5.3
branch
from
ec2491e to
5a9a739
Compare
This is a follow-on from astral-sh#512 / d0ed97f where we started always setting the CXX environment variable to ensure we don't accidentally try and use the host (non-cross-compiling) C++ compiler. In attempting to upgrade to a more modern s509x minimum arch-level (z10), we found that the configure script for bdb fails. It tries to check the C++ compiler is working (respecting the CXX environment variable that is set), but fails to run the pre-processor with `$CXX -E` (as the C++ compiler isn't actually installed). It then falls back to `/lib/cpp`, which up until now has been working fine. Now that we're specifying `-march=z10` this fails as `/lib/cpp` doesn't know about `z10`. I don't know if we actually need to use the C++ compiler, but I'm adding it in to at least satisfy the configure script, and do the correct thing should we need it (rather than the incorrect thing of falling back to the host compiler).
Edward-Knight
force-pushed
the
openssl_5.3
branch
from
5a9a739 to
6284af1
Compare
|
I either don't have permission to or just don't know how to retry the failing jobs in the latest pipeline but I believe they will pass if given another go 👍 The changes are otherwise ready for review @geofft. Some notes:
|
|
Triggered a re-run.
If it's unused, yes (and it will make your diff slightly nicer :) )
We might be able to test this in qemu or something? In theory, this sort of question should be particularly easy to answer for RISC-V specifically without finding the right hardware.... |
Now we're building with OpenSSL 3.5 instead of 3.0, we no longer need the build configuration around.
|
I've removed OpenSSL 3.0 (and CI is failing due to issues grabbing cygwin again). It looks like OpenSSL is going to accept this PR which is very similar to the patch we're applying, so I think we're good on that front: openssl/openssl#28422 @geofft are there any other changes you'd like? |
@geofft Sorry for the delay. We discussed this with some colleagues and even though |
Fixes #775