403 Server Error: Downloading en_core_web_sm fails due to "Compatibility table not found for Spacy v3.7.5"

[mkh1991]

How to reproduce the behaviour

Not sure if it's easily reproducible (it isn't even for me consistently), but during docker build, downloading the en_core_web_sm package fails with a 403 Server error, on occasion. With the --no-cache option during docker build, it still fails occasionally, and retrying the build often leads to a success, but not always. This has only started happening since sometime early this week or late last week, there's been no code change at all for the build or for requirements, and I've never faced this issue prior to now. spaCy claims it can't find the compatibility table, but clearly it can when it doesn't fail.

Failing command

python -m spacy download en_core_web_sm

Error message

81.32 ✘ Server error (403)
81.32 Couldn't fetch compatibility table. Please find a package for your spaCy
81.32 installation (v3.7.5), and download it manually. For more details, see the
81.32 documentation: https://spacy.io/usage/models

Your Environment

• Operating System: Ubuntu 24.04 LTS
• Python Version Used: 3.11
• spaCy Version Used: 3.7.5
• Environment Information: Conda environment within Docker, spaCy is downloaded with pip install -r requirements.txt

[Lixiyao-meow]

Same issue here

[lgnashold]

Same issue here

[timleslie]

Ditto, but with en_core_web_trf.

[Shayfiz]

I am having the same issue but with SpaCy v3.7.4. Did anyone have any luck with downgrading the version ?

[mkh1991]

Fix

Figured out a fix. Use the following requirements in requirements.txt:

spacy==3.7.5
en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl

Use line 2 above instead of downloading it via python -m spacy download en_core_web_sm. Why 3.7.1 for the model? Find out below!

Root Cause

The direct download link throws a 404 for all SpaCy versions from 3.7.2 to 3.7.5: e.g. https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.2/en_core_web_sm-3.7.2-py3-none-any.whl

I tried downgrading to 3.7.1, and installing directly with en_core_web_sm@ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl in requirements.txt and got this rather interesting error message:

23.53 INFO: pip is looking at multiple versions of en-core-web-sm to determine which version is compatible with other requirements. This could take a while.
23.53
23.53 The conflict is caused by:
23.53     The user requested spacy==3.7.1
23.53     en-core-web-sm 3.7.1 depends on spacy<3.8.0 and >=3.7.2

Following this odd recommendation (using spacy==3.7.5 but en_core_web_sm==3.7.1) fixed it though.