Spacy>=3.8.0 fails to install in ARM64 Docker images with Python base

[cgeudens]

When attempting to install spacy>=3.8.0 in a linux/arm64 Docker environment with a 64-bit Python base image, the installation fails during the build process of the blis dependency. I tested for python:3.8, python:3.9, python:3.10, python:3.11, and python:3.12. To reproduce:

docker container run --rm -it --platform=linux/arm64 python:3.10 bash  # but the python version doesn't matter, nor does the base image: bookworm, bullseye, ...
pip install 'spacy>=3.8.0' # fails with the error message below
pip install 'spacy<3.8.0'  # works fine

Error message:

Collecting spacy>=3.8.0
  Downloading spacy-3.8.3.tar.gz (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 17.7 MB/s eta 0:00:00
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [129 lines of output]
      Ignoring numpy: markers 'python_version < "3.9"' don't match your environment
      Collecting setuptools
        Downloading setuptools-75.8.0-py3-none-any.whl (1.2 MB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 15.8 MB/s eta 0:00:00
      Collecting cython<3.0,>=0.25
        Downloading Cython-0.29.37-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (1.8 MB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 28.5 MB/s eta 0:00:00
      Collecting cymem<2.1.0,>=2.0.2
        Downloading cymem-2.0.10.tar.gz (10 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting preshed<3.1.0,>=3.0.2
        Downloading preshed-3.0.9-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (150 kB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 150.3/150.3 kB 16.5 MB/s eta 0:00:00
      Collecting murmurhash<1.1.0,>=0.28.0
        Downloading murmurhash-1.0.11.tar.gz (13 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting thinc<8.4.0,>=8.3.0
        Downloading thinc-8.3.4.tar.gz (193 kB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 193.9/193.9 kB 21.4 MB/s eta 0:00:00
        Installing build dependencies: started
        Installing build dependencies: finished with status 'error'
        error: subprocess-exited-with-error
      
        × pip subprocess to install build dependencies did not run successfully.
        │ exit code: 1
        ╰─> [80 lines of output]
            Collecting setuptools
              Using cached setuptools-75.8.0-py3-none-any.whl (1.2 MB)
            Collecting cython<3.0,>=0.29.0
              Using cached Cython-0.29.37-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (1.8 MB)
            Collecting murmurhash<1.1.0,>=1.0.2
              Using cached murmurhash-1.0.11.tar.gz (13 kB)
              Installing build dependencies: started
              Installing build dependencies: finished with status 'done'
              Getting requirements to build wheel: started
              Getting requirements to build wheel: finished with status 'done'
              Preparing metadata (pyproject.toml): started
              Preparing metadata (pyproject.toml): finished with status 'done'
            Collecting cymem<2.1.0,>=2.0.2
              Using cached cymem-2.0.10.tar.gz (10 kB)
              Installing build dependencies: started
              Installing build dependencies: finished with status 'done'
              Getting requirements to build wheel: started
              Getting requirements to build wheel: finished with status 'done'
              Preparing metadata (pyproject.toml): started
              Preparing metadata (pyproject.toml): finished with status 'done'
            Collecting preshed<3.1.0,>=3.0.2
              Using cached preshed-3.0.9-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (150 kB)
            Collecting blis<1.3.0,>=1.2.0
              Downloading blis-1.2.0.tar.gz (2.4 MB)
                 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 23.7 MB/s eta 0:00:00
              Installing build dependencies: started
              Installing build dependencies: finished with status 'done'
              Getting requirements to build wheel: started
              Getting requirements to build wheel: finished with status 'done'
              Preparing metadata (pyproject.toml): started
              Preparing metadata (pyproject.toml): finished with status 'done'
            Collecting numpy<3.0.0,>=2.0.0
              Using cached numpy-2.2.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.3 MB)
            Building wheels for collected packages: murmurhash, cymem, blis
              Building wheel for murmurhash (pyproject.toml): started
              Building wheel for murmurhash (pyproject.toml): finished with status 'done'
              Created wheel for murmurhash: filename=murmurhash-1.0.11-cp310-cp310-linux_aarch64.whl size=119841 sha256=d204a88a759d0125caf98fd8418d3ae0ef2fa246d55ba45e72ad2ac60292dd9f
              Stored in directory: /root/.cache/pip/wheels/b6/dd/80/8ecbeb72c0e87a74c609da7956e6016554fc1f1169ac87434f
              Building wheel for cymem (pyproject.toml): started
              Building wheel for cymem (pyproject.toml): finished with status 'done'
              Created wheel for cymem: filename=cymem-2.0.10-cp310-cp310-linux_aarch64.whl size=192529 sha256=be26b13d64e3e6f3f379f0a193ac17fb2479ad90000a818ce84bcb478b43687f
              Stored in directory: /root/.cache/pip/wheels/74/f8/fb/2f8843b493972099de993c81ce380922c56ed2effb941e3a5a
              Building wheel for blis (pyproject.toml): started
              Building wheel for blis (pyproject.toml): finished with status 'error'
              error: subprocess-exited-with-error
      
              × Building wheel for blis (pyproject.toml) did not run successfully.
              │ exit code: 1
              ╰─> [21 lines of output]
                  BLIS_COMPILER? None
                  running bdist_wheel
                  running build
                  running build_py
                  creating build/lib.linux-aarch64-cpython-310/blis
                  copying blis/__init__.py -> build/lib.linux-aarch64-cpython-310/blis
                  copying blis/benchmark.py -> build/lib.linux-aarch64-cpython-310/blis
                  copying blis/about.py -> build/lib.linux-aarch64-cpython-310/blis
                  creating build/lib.linux-aarch64-cpython-310/blis/tests
                  copying blis/tests/__init__.py -> build/lib.linux-aarch64-cpython-310/blis/tests
                  copying blis/tests/test_gemm.py -> build/lib.linux-aarch64-cpython-310/blis/tests
                  copying blis/tests/common.py -> build/lib.linux-aarch64-cpython-310/blis/tests
                  copying blis/tests/test_dotv.py -> build/lib.linux-aarch64-cpython-310/blis/tests
                  copying blis/cy.pyx -> build/lib.linux-aarch64-cpython-310/blis
                  copying blis/py.pyx -> build/lib.linux-aarch64-cpython-310/blis
                  copying blis/cy.pxd -> build/lib.linux-aarch64-cpython-310/blis
                  copying blis/__init__.pxd -> build/lib.linux-aarch64-cpython-310/blis
                  running build_ext
                  Build options linux unix
                  BUILD ARCH: cortexa57
                  error: [Errno 2] No such file or directory: '/tmp/pip-install-q3i2el5w/blis_8ba77e0f27a045be890d093379c1a9e2/blis/_src/make/linux-cortexa57.jsonl'
                  [end of output]
      
              note: This error originates from a subprocess, and is likely not a problem with pip.
              ERROR: Failed building wheel for blis
            Successfully built murmurhash cymem
            Failed to build blis
            ERROR: Could not build wheels for blis, which is required to install pyproject.toml-based projects
      
            [notice] A new release of pip is available: 23.0.1 -> 24.3.1
            [notice] To update, run: pip install --upgrade pip
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: subprocess-exited-with-error
      
      × pip subprocess to install build dependencies did not run successfully.
      │ exit code: 1
      ╰─> See above for output.
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      
      [notice] A new release of pip is available: 23.0.1 -> 24.3.1
      [notice] To update, run: pip install --upgrade pip
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[honnibal]

We previously had our own agents set up that built a wide variety of architectures, but I've had to simplify the setup and now we're only using Github-hosted runners to build the wheels.

Github said support for ARM Linux agents would be available for public repos by the end of 2024, but they've moved this back to Q1 2025. Cross-compilation of the wheels isn't viable; it simply takes too long.

We'll have wheels up for ARM Linux as soon as the agents are available. In the meantime you'll have to use spaCy versions prior to v3.8.

[cgeudens]

thank you for clarifying!

[honnibal]

An additional tip: if you use the --only-binary flag to pip install (or set the PIP_ONLY_BINARY environment variable) pip will only solve for requirements with wheels available. It seems you don't have a compilation environment set up in your container (logical), so this will help you get versions you can use.