Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Non-deterministic bytecode in recent versions of Python #1419

Closed
bastimeyer opened this issue Nov 21, 2022 · 1 comment
Closed

Non-deterministic bytecode in recent versions of Python #1419

bastimeyer opened this issue Nov 21, 2022 · 1 comment

Comments

@bastimeyer
Copy link

Apologies if this is not the right place, but I don't know if it's an issue in cpython itself, or if it's coming from the build scripts/configs/dependencies of the manylinux images, so I'm opening this issue here first.

I'm using the manylinux images in order to build AppImages for my application, which bundle one of the pre-built python environments (after modifying the RPATH of each binary via patchelf and making them relative to the AppImage's root directory). While building the AppImages and installing my application in the container via pip, bytecode gets compiled, and not just for the python application and its dependencies, but also for the stdlib during the execution of pip, as no stdlib bytecode is included in any of the manylinux images. Since my intention is to build reproducible AppImages, I'm expecting that everything is deterministic when building and copying the AppImage contents, including the compiled bytecode. The compiled bytecode however is not deterministic/reproducible, despite having set the PYTHONHASHSEED=0 and SOURCE_DATE_EPOCH=... environment variables.

The issue apparently seems to have only been introduced recently, and only in some versions of CPython. I've already spent a couple of hours figuring out the reason for this and testing stuff with the compileall module, but to no success.

That's why I've created this simple BASH script which compiles the missing bytecode of the stdlib in each included CPython environment, in multiple different manylinux images, and it prints a sha256 checksum of all the compiled bytecode, so it can be compared against different runs.

#!/usr/bin/env bash

IMAGES=(
  quay.io/pypa/manylinux2014_x86_64@sha256:012a15a28de1377c716577f109cbc40ffa3477498e2fdb0409e3b69b852b705e
  quay.io/pypa/manylinux2014_x86_64@sha256:17de698632b5255a4e3c26bf2cbd4a1c2d46662130503bf9660531517621651e
  quay.io/pypa/manylinux2014_x86_64@sha256:9b463efac479efbcab6dec77eca28c5cfa0c5ef64f13ac184eb7117dc1f8edda
  quay.io/pypa/manylinux2014_x86_64:latest
)
PYTHON=(
  cp37-cp37m
  cp38-cp38
  cp39-cp39
  cp310-cp310
  cp311-cp311
)
NUM=5


for image in "${IMAGES[@]}"; do
  docker image pull -q "$image"
done

for image in "${IMAGES[@]}"; do
  echo "Image: $image"
  for py in "${PYTHON[@]}"; do
    for i in $(seq 1 $NUM); do
      dir="/opt/python/$py"
      bin="$dir/bin/python"
      lib="$dir/lib"
      docker run -i --rm "$image" <<EOF
set -eo pipefail
[[ -f "$bin" ]] || exit

version=\$("$bin" --version)
echo -n "\$version ($i): "

# compile stdlib bytecode
SOURCE_DATE_EPOCH=0 PYTHONHASHSEED=0 "$bin" -B -m compileall -q -j0 -f -x 'lib2to3|test' -r9999 "$lib"

# get a list of all compiled files and hash bytecode contents
data=\$(find "$lib" -type f -name '*.pyc' -print0 | LC_ALL=C sort -uz | xargs -0 -n1 -P1 sha256sum)

# get number of compiled files and overall checksum
files=\$(wc -l <<< "\$data")
checksum=\$(sha256sum <<< "\$data" | sed 's/\W//g')

echo "\$files - \$checksum"
EOF
    done
  done
done

As you can see in the following log output, the compiled bytecode was deterministic at some point in some versions, but then it started to compile non-deterministic bytecode. I haven't found the exact point yet where this started.

$ ./deterministic-python-bytecode.sh
quay.io/pypa/manylinux2014_x86_64@sha256:012a15a28de1377c716577f109cbc40ffa3477498e2fdb0409e3b69b852b705e
quay.io/pypa/manylinux2014_x86_64@sha256:17de698632b5255a4e3c26bf2cbd4a1c2d46662130503bf9660531517621651e
quay.io/pypa/manylinux2014_x86_64@sha256:9b463efac479efbcab6dec77eca28c5cfa0c5ef64f13ac184eb7117dc1f8edda
quay.io/pypa/manylinux2014_x86_64:latest
Image: quay.io/pypa/manylinux2014_x86_64@sha256:012a15a28de1377c716577f109cbc40ffa3477498e2fdb0409e3b69b852b705e
Python 3.7.12 (1): 1204 - fd1051dfde7ca9dc4875a31d7aee332bccfc7ec1187d055501c0cc442b1428e5
Python 3.7.12 (2): 1204 - fd1051dfde7ca9dc4875a31d7aee332bccfc7ec1187d055501c0cc442b1428e5
Python 3.7.12 (3): 1204 - fd1051dfde7ca9dc4875a31d7aee332bccfc7ec1187d055501c0cc442b1428e5
Python 3.7.12 (4): 1204 - fd1051dfde7ca9dc4875a31d7aee332bccfc7ec1187d055501c0cc442b1428e5
Python 3.7.12 (5): 1204 - fd1051dfde7ca9dc4875a31d7aee332bccfc7ec1187d055501c0cc442b1428e5
Python 3.8.12 (1): 1198 - 687bdd2d7f6540dd09756813f0ba5d992976705b51cbc16ba1650d6557e124b8
Python 3.8.12 (2): 1198 - 687bdd2d7f6540dd09756813f0ba5d992976705b51cbc16ba1650d6557e124b8
Python 3.8.12 (3): 1198 - 687bdd2d7f6540dd09756813f0ba5d992976705b51cbc16ba1650d6557e124b8
Python 3.8.12 (4): 1198 - 687bdd2d7f6540dd09756813f0ba5d992976705b51cbc16ba1650d6557e124b8
Python 3.8.12 (5): 1198 - 687bdd2d7f6540dd09756813f0ba5d992976705b51cbc16ba1650d6557e124b8
Python 3.9.7 (1): 1205 - 6b69c20f8c166db358a2ca5123d146db679c8263e91c196dc9fc3e041160a80e
Python 3.9.7 (2): 1205 - 6b69c20f8c166db358a2ca5123d146db679c8263e91c196dc9fc3e041160a80e
Python 3.9.7 (3): 1205 - 6b69c20f8c166db358a2ca5123d146db679c8263e91c196dc9fc3e041160a80e
Python 3.9.7 (4): 1205 - 6b69c20f8c166db358a2ca5123d146db679c8263e91c196dc9fc3e041160a80e
Python 3.9.7 (5): 1205 - 6b69c20f8c166db358a2ca5123d146db679c8263e91c196dc9fc3e041160a80e
Python 3.10.0 (1): 1211 - 8396079b44daadc0ea1e4a0a3d682a74162e54c965491cf07a74b9a909d74b50
Python 3.10.0 (2): 1211 - a6a0d55dbfd82fdf1294895d05fe81e36645ea65a160d0f25d0ce707f1ce7002
Python 3.10.0 (3): 1211 - 8396079b44daadc0ea1e4a0a3d682a74162e54c965491cf07a74b9a909d74b50
Python 3.10.0 (4): 1211 - 8396079b44daadc0ea1e4a0a3d682a74162e54c965491cf07a74b9a909d74b50
Python 3.10.0 (5): 1211 - 8396079b44daadc0ea1e4a0a3d682a74162e54c965491cf07a74b9a909d74b50
Image: quay.io/pypa/manylinux2014_x86_64@sha256:17de698632b5255a4e3c26bf2cbd4a1c2d46662130503bf9660531517621651e
Python 3.7.12 (1): 1214 - 6a8f545918b3fdf52462d2732c5a6ebc6057ce132c46c538d6a42eec4a076e3b
Python 3.7.12 (2): 1214 - 6a8f545918b3fdf52462d2732c5a6ebc6057ce132c46c538d6a42eec4a076e3b
Python 3.7.12 (3): 1214 - 6a8f545918b3fdf52462d2732c5a6ebc6057ce132c46c538d6a42eec4a076e3b
Python 3.7.12 (4): 1214 - 6a8f545918b3fdf52462d2732c5a6ebc6057ce132c46c538d6a42eec4a076e3b
Python 3.7.12 (5): 1214 - 6a8f545918b3fdf52462d2732c5a6ebc6057ce132c46c538d6a42eec4a076e3b
Python 3.8.12 (1): 1208 - e86af5979b6690d9777eae5310e2b737b0ed6dcbed5686c22c35540ab2489fc0
Python 3.8.12 (2): 1208 - e86af5979b6690d9777eae5310e2b737b0ed6dcbed5686c22c35540ab2489fc0
Python 3.8.12 (3): 1208 - e86af5979b6690d9777eae5310e2b737b0ed6dcbed5686c22c35540ab2489fc0
Python 3.8.12 (4): 1208 - e86af5979b6690d9777eae5310e2b737b0ed6dcbed5686c22c35540ab2489fc0
Python 3.8.12 (5): 1208 - ee0431980af558f9e05873cc8182c2bcbb7998882d69d55fede4bab16bce975e
Python 3.9.9 (1): 1215 - 608da0e717ace76210dd5d2f7096f60cddb4a788e5efe74cceeba40657de156e
Python 3.9.9 (2): 1215 - 608da0e717ace76210dd5d2f7096f60cddb4a788e5efe74cceeba40657de156e
Python 3.9.9 (3): 1215 - 143b0548a657b89659af236466fcd73a8aad52cacf5842d49cf85c6097969832
Python 3.9.9 (4): 1215 - 608da0e717ace76210dd5d2f7096f60cddb4a788e5efe74cceeba40657de156e
Python 3.9.9 (5): 1215 - 608da0e717ace76210dd5d2f7096f60cddb4a788e5efe74cceeba40657de156e
Python 3.10.1 (1): 1221 - 2ba5262efd1bf277022f1dc8de6c39fff6aab7115006c80f2943eca85869e61b
Python 3.10.1 (2): 1221 - 182ecda38353e5cb22b05f436dd61a1a53d21540c6e9c6bf5983bb239b27f451
Python 3.10.1 (3): 1221 - 2ba5262efd1bf277022f1dc8de6c39fff6aab7115006c80f2943eca85869e61b
Python 3.10.1 (4): 1221 - 05eea8bc5326ec9bc7e110658b5cbdbe0048928cf35445764fdbd082303d26ec
Python 3.10.1 (5): 1221 - 2ba5262efd1bf277022f1dc8de6c39fff6aab7115006c80f2943eca85869e61b
Image: quay.io/pypa/manylinux2014_x86_64@sha256:9b463efac479efbcab6dec77eca28c5cfa0c5ef64f13ac184eb7117dc1f8edda
Python 3.7.13 (1): 1395 - 688e489111d6f873c930da4c75a861a28b1699b34128c80a14262a1bfca2dbc9
Python 3.7.13 (2): 1395 - 688e489111d6f873c930da4c75a861a28b1699b34128c80a14262a1bfca2dbc9
Python 3.7.13 (3): 1395 - 688e489111d6f873c930da4c75a861a28b1699b34128c80a14262a1bfca2dbc9
Python 3.7.13 (4): 1395 - 688e489111d6f873c930da4c75a861a28b1699b34128c80a14262a1bfca2dbc9
Python 3.7.13 (5): 1395 - 688e489111d6f873c930da4c75a861a28b1699b34128c80a14262a1bfca2dbc9
Python 3.8.13 (1): 1389 - c94c2529209a896471aaf055300059601083221dd1a969b5240bdaded46313bc
Python 3.8.13 (2): 1389 - 51606c5bb924338fed7735a7a7ba75cdd8f78d2fbe4d7a7c6273f3a7102c22f0
Python 3.8.13 (3): 1389 - a951d76377b024322ad10aed399b653eccac7c0dbcabc5a403e1fc9889cef761
Python 3.8.13 (4): 1389 - 97a90a47669a55ecb6862a4e70447781fd9914ca549fe7b5fb7f1bfc68a41142
Python 3.8.13 (5): 1389 - 994945968eaa3b8e6914d6fa9796808b582499a4a86b18822a3dc4cb7c363c68
Python 3.9.12 (1): 1397 - 06cc544c2595e433fc792aa314d52668942b747335ded39dfc34a6e808e03b7c
Python 3.9.12 (2): 1397 - 903c3c00a5d4e633d94453a65ad6e230ab610be25671e34d2c68c15280504076
Python 3.9.12 (3): 1397 - 9406c3f51f0740eb17c198f091914b3f3f7ebdf9a79edab6f37ac3125437c77a
Python 3.9.12 (4): 1397 - b3fa2cbd8e84eebe38005a174d65160de03d6cca680eee81992af64d297eb6be
Python 3.9.12 (5): 1397 - ee7bab748e62a0d35747724560247d8adb9a7ed3713f92f0a6b60220d8e14079
Python 3.10.4 (1): 1403 - f7871c1eb722f9a2a1afbb37ab760fc8139339eae6082f687510f452dd0154a4
Python 3.10.4 (2): 1403 - 3302fb470732fde894828194d1508bbe8b4a24ebecc3a1f16f8cf29d5e2da485
Python 3.10.4 (3): 1403 - 2b2b05e88b689fd19a504adc50a84f15fd2e94ccfcd154229d40180258c1940f
Python 3.10.4 (4): 1403 - b9b43810fb9604b64760180e0ecf7a63d182ece475faaf4d46d69df38fa34e5a
Python 3.10.4 (5): 1403 - 49cca15c1c20204151a09760947d05a8e279da3a26ed7e09091c4779e1b7712f
Image: quay.io/pypa/manylinux2014_x86_64:latest
Python 3.7.15 (1): 1387 - 6fbce68391f2d456e1c9f46ecfabb7da575402d329fd716662fa87fcdc192543
Python 3.7.15 (2): 1387 - 6fbce68391f2d456e1c9f46ecfabb7da575402d329fd716662fa87fcdc192543
Python 3.7.15 (3): 1387 - 6fbce68391f2d456e1c9f46ecfabb7da575402d329fd716662fa87fcdc192543
Python 3.7.15 (4): 1387 - 6fbce68391f2d456e1c9f46ecfabb7da575402d329fd716662fa87fcdc192543
Python 3.7.15 (5): 1387 - 6fbce68391f2d456e1c9f46ecfabb7da575402d329fd716662fa87fcdc192543
Python 3.8.15 (1): 1379 - f915fdbb13e51b9fa9b4c1754b6a8c04d323c8f7143db3a0122721a5c194e62f
Python 3.8.15 (2): 1379 - 3f5a7e956e45b137b7357a561f707ced4eea00bffa5ab0ef67873a665de16961
Python 3.8.15 (3): 1379 - c326cd54f21f6161c0cc86d5f53f2adf8137f8bb13621de70411a4fe99f62a2e
Python 3.8.15 (4): 1379 - 891a7b78bf40058e519846df36a0db6e81468374a23378346941b0717d0d8919
Python 3.8.15 (5): 1379 - 0c745f4a9fac53e8662ba7e1151bb295b2915de420b1571df9dd64b93433b663
Python 3.9.15 (1): 1387 - fbcec65be2ea586e13eefc5b3e21ffcbfacfb21b8d4af814dfb4c359f5f695c1
Python 3.9.15 (2): 1387 - 4dbcabc5f1e99694c39d540daa3702a003a7ec1a1e592f1d5c840e32208782f7
Python 3.9.15 (3): 1387 - 991b97942e007761e9382e801d939cf7cc7bb65b1a43031f792951da2c629be1
Python 3.9.15 (4): 1387 - 3e36b9baade16fc2d1f801acefb6a5a607834caedb817e2f000099a9465c4f9f
Python 3.9.15 (5): 1387 - c853e45f7f97f8cce63675b38298ff326af65f3f2f8c8b6076e0c5e0ed92b3ed
Python 3.10.8 (1): 1393 - 3943721732a9b6b9b9d34f949426f50078fc2788c99b8c33479b5034d51f2036
Python 3.10.8 (2): 1393 - 3b35dece855fa88d40a84317ebb9c8474c1bb7e59570be4047473218fa1e182b
Python 3.10.8 (3): 1393 - b3872a539ad5aa1f934861baeea039f20ebedf1a5a32280bcaa98217d42c4c1a
Python 3.10.8 (4): 1393 - af334359e666bf56762f29908a686f6c9dfa52812483117ebf1bc6372cf4014c
Python 3.10.8 (5): 1393 - 4c13c7c8fd522d84857782cd3687a4a7b5c7de941fce42f415e6298f1db8be46
Python 3.11.0 (1): 1405 - 0cbafb7f7783e68f0354066f9c2a77f1c60882c0ca4c5800ad6ab96f71849692
Python 3.11.0 (2): 1405 - 0cbafb7f7783e68f0354066f9c2a77f1c60882c0ca4c5800ad6ab96f71849692
Python 3.11.0 (3): 1405 - 0cbafb7f7783e68f0354066f9c2a77f1c60882c0ca4c5800ad6ab96f71849692
Python 3.11.0 (4): 1405 - 0cbafb7f7783e68f0354066f9c2a77f1c60882c0ca4c5800ad6ab96f71849692
Python 3.11.0 (5): 1405 - 0cbafb7f7783e68f0354066f9c2a77f1c60882c0ca4c5800ad6ab96f71849692

3.10.x has been broken the whole time, while 3.7.x and 3.11.x appear stable. The first issues (according to the limited number of images tested) occurred in 3.8.12 and 3.9.9, and after that the results a completely unstable.

Once again, I'm aware that this might be an issue in cpython itself and doesn't belong here, but apparently the number of compiled stdlib modules of the same Python version differs between different manylinux images (see the stable checksums of 3.7.x), so I don't know what this is about, and this suggests that other modifications might have an influence here.
@mayeut
Copy link
Member

mayeut commented Nov 26, 2022

but apparently the number of compiled stdlib modules of the same Python version differs between different manylinux images

You're also pre-compiling site-packages which does contain updates of various tools (among which pip / setuptools) which change across images as updates are available. You'll find the number much more stable if ignoring site-packages.

Once again, I'm aware that this might be an issue in cpython itself and doesn't belong here

Based on python/cpython#73894 (comment), I disabled multiprocessing and achieved something that gives the same hash every time (at least on manylinux2014_aarch64:latest).

Please note that your use-case is not supported by manylinux which only aims at providing images for building wheels. The interpreter is not meant to be portable (it might work or it might not, this is an implementation detail, not a feature).
For this use case, you might want to check:

#!/usr/bin/env bash

IMAGES=(
  quay.io/pypa/manylinux2014_aarch64:latest
)
PYTHON=(
  cp37-cp37m
  cp38-cp38
  cp39-cp39
  cp310-cp310
  cp311-cp311
)
NUM=30


for image in "${IMAGES[@]}"; do
  docker image pull -q "$image"
done

for image in "${IMAGES[@]}"; do
  echo "Image: $image"
  for py in "${PYTHON[@]}"; do
    for i in $(seq 1 $NUM); do
      dir="/opt/python/$py"
      bin="$dir/bin/python"
      lib="$dir/lib"
      docker run -i --rm "$image" <<EOF
set -eo pipefail
[[ -f "$bin" ]] || exit

version=\$("$bin" --version)
echo -n "\$version ($i): "

# compile stdlib bytecode
SOURCE_DATE_EPOCH=0 PYTHONHASHSEED=0 "$bin" -s -S -B -m compileall -q -j 1 -f -x 'lib2to3|test' -r9999 "$lib"

# get a list of all compiled files and hash bytecode contents
data=\$(find "$lib" -type f -name '*.pyc' -print0 | LC_ALL=C sort -uz | xargs -0 -n1 -P1 sha256sum)

# get number of compiled files and overall checksum
files=\$(wc -l <<< "\$data")
checksum=\$(sha256sum <<< "\$data" | sed 's/\W//g')

echo "\$files - \$checksum"
EOF
    done
  done
done

Python 3.7.15 (1): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (2): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (3): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (4): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (5): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (6): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (7): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (8): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (9): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (10): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (11): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (12): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (13): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (14): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (15): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (16): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (17): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (18): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (19): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (20): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (21): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (22): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (23): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (24): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (25): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (26): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (27): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (28): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (29): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.7.15 (30): 1387 - 941461ebad9f31930d9119aa67967da1a5269df551e460a727f4dc8df10fe407
Python 3.8.15 (1): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (2): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (3): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (4): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (5): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (6): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (7): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (8): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (9): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (10): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (11): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (12): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (13): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (14): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (15): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (16): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (17): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (18): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (19): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (20): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (21): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (22): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (23): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (24): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (25): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (26): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (27): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (28): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (29): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.8.15 (30): 1379 - b2076b3608b57036c89bf1d09f2611907056a6ae657741dc1e8162ca31b216b8
Python 3.9.15 (1): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (2): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (3): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (4): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (5): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (6): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (7): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (8): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (9): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (10): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (11): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (12): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (13): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (14): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (15): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (16): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (17): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (18): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (19): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (20): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (21): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (22): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (23): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (24): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (25): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (26): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (27): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (28): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (29): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.9.15 (30): 1387 - b3fde3d1375bc921cf46aa206082c31b125484f57d75a28cfb2ea58d1e6d2192
Python 3.10.8 (1): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (2): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (3): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (4): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (5): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (6): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (7): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (8): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (9): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (10): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (11): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (12): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (13): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (14): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (15): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (16): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (17): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (18): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (19): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (20): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (21): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (22): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (23): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (24): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (25): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (26): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (27): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (28): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (29): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.10.8 (30): 1393 - 825a045c2d55f7331dc6406cdf1ee391f6e73bf3c3910adbe67bba1068bc325f
Python 3.11.0 (1): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (2): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (3): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (4): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (5): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (6): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (7): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (8): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (9): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (10): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (11): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (12): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (13): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (14): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (15): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (16): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (17): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (18): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (19): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (20): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (21): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (22): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (23): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (24): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (25): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (26): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (27): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (28): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (29): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3
Python 3.11.0 (30): 1405 - a87f4fc019fb67a152e3b531ba236c9979ff3f56eb13306bc761ef82f37c1de3

@mayeut mayeut closed this as not planned Won't fix, can't repro, duplicate, stale Nov 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bastimeyer @mayeut