Hash function is FNV on Python 3.10 and 3.11

[gmarkall]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

The builds of Python 3.10.15 and 3.11.10 (maybe others, I have not completed an exhaustive search) use FNV as the hash algorithm, which should not be used (as per PEP 456). This can be observed by running:

$ python -c "import sys; print(sys.hash_info.algorithm)"
fnv

whereas it should be e.g.:

$ python -c "import sys; print(sys.hash_info.algorithm)"
siphash24

In particular this causes Numba to emit warnings that can then cause failures in some test suites (e.g. pytest runs):

UserWarning: FNV hashing is not implemented in Numba. See PEP 456 https://www.python.org/dev/peps/pep-0456/ for rationale over not using FNV. Numba will continue to work, but hashes for built in types will be computed using siphash24. This will permit e.g. dictionaries to continue to behave as expected, however anything relying on the value of the hash opposed to hash as a derived property is likely to not work as expected.

This happens because when cross compiling, HAVE_ALIGNED_REQUIRED is always defined as 1 in CPython's configure script:

https://github.com/python/cpython/blob/0c5fc27217525c4e40b4064e6979f467540c2fc8/configure#L10444-L10445

  if test "$cross_compiling" = yes; then :
  ac_cv_aligned_required=yes

# ...

if test "$ac_cv_aligned_required" = yes ; then
$as_echo "#define HAVE_ALIGNED_REQUIRED 1" >>confdefs.h
fi

then in pyhash.h the default hash algorithm becomes FNV:

https://github.com/python/cpython/blob/0c5fc27217525c4e40b4064e6979f467540c2fc8/Include/pyhash.h#L132-L138

#ifndef Py_HASH_ALGORITHM
#  ifndef HAVE_ALIGNED_REQUIRED
#    define Py_HASH_ALGORITHM Py_HASH_SIPHASH24
#  else
#    define Py_HASH_ALGORITHM Py_HASH_FNV
#  endif /* uint64_t && uint32_t && aligned */
#endif /* Py_HASH_ALGORITHM */

I think this can be worked around by supplying --with-hash-algorithm=siphash24 to the configure script when cross-compiling. I'm trying this out locally, and will provide more information (or a PR) once I make more progress.

cc @bdice

Installed packages

# packages in environment at /home/gmarkall/miniforge3/envs/latest-python-310:
#
# Name                    Version                   Build  Channel
_openmp_mutex             4.5                       2_gnu    conda-forge
bzip2                     1.0.8                h68df207_7    conda-forge
ca-certificates           2024.8.30            hcefe29a_0    conda-forge
ld_impl_linux-aarch64     2.43                 h80caac9_0    conda-forge
libffi                    3.4.2                h3557bc0_5    conda-forge
libgcc                    14.1.0               he277a41_1    conda-forge
libgcc-ng                 14.1.0               he9431aa_1    conda-forge
libgomp                   14.1.0               he277a41_1    conda-forge
libnsl                    2.0.1                h31becfc_0    conda-forge
libsqlite                 3.46.1               hc4a20ef_0    conda-forge
libuuid                   2.38.1               hb4cce97_0    conda-forge
libxcrypt                 4.4.36               h31becfc_1    conda-forge
libzlib                   1.3.1                h68df207_1    conda-forge
ncurses                   6.5                  hcccb83c_1    conda-forge
openssl                   3.3.2                h86ecc28_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
python                    3.10.15         hbf90c55_0_cpython    conda-forge
readline                  8.2                  h8fc344f_1    conda-forge
setuptools                74.1.2             pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h194ca79_0    conda-forge
tzdata                    2024a                h8827d51_1    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h9cdd2b7_0    conda-forge

Environment info

active environment : latest-python-310
    active env location : /home/gmarkall/miniforge3/envs/latest-python-310
            shell level : 5
       user config file : /home/gmarkall/.condarc
 populated config files : /home/gmarkall/miniforge3/.condarc
          conda version : 24.7.1
    conda-build version : not installed
         python version : 3.12.5.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=cortex_a72
                          __conda=24.7.1=0
                          __cuda=12.2=0
                          __glibc=2.35=0
                          __linux=5.15.122=0
                          __unix=0=0
       base environment : /home/gmarkall/miniforge3  (writable)
      conda av data dir : /home/gmarkall/miniforge3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-aarch64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/gmarkall/miniforge3/pkgs
                          /home/gmarkall/.conda/pkgs
       envs directories : /home/gmarkall/miniforge3/envs
                          /home/gmarkall/.conda/envs
               platform : linux-aarch64
             user-agent : conda/24.7.1 requests/2.32.3 CPython/3.12.5 Linux/5.15.122-tegra ubuntu/22.04.3 glibc/2.35 solver/libmamba conda-libmamba-solver/24.7.0 libmambapy/1.5.9
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

[gmarkall]

An attempt at a fix (just Python 3.10 for now whilst discussing the solution) in #719.