sphinx.errors.VersionRequirementError: The sphinxcontrib.applehelp extension used by this project needs at least Sphinx v5.0

[m-albert]

Details

• Read the Docs project URL: https://readthedocs.org/projects/dask-image/
• Build URL (if applicable): https://beta.readthedocs.org/projects/dask-image/builds/23503914/

Expected Result

readthedocs continues building successfully after no changes in dask-image's RTD configuration.

Actual Result

Upstream report of dask/dask-image#351.

Quick summary: RTD fails saying the build requires sphinx>=5. Although the correct sphinx version is pinned in the environment yml file, the build seems to use sphinx=4.5. Interestingly, the print of the environment file shows differences to the dask-image repo env file.

Thanks for any hints on this 🙏

[jakirkham]

Would add this worked in December: https://readthedocs.org/projects/dask-image/builds/22749569/

Then started failing in January: https://readthedocs.org/projects/dask-image/builds/23277934/

Somewhere in there something changed. Hopefully this helps us narrow it down

[agjohnson]

We encountered this bug as well. This error is coming from Sphinx, not Read the Docs though. The cause of the problem is some unpinned dependencies of Sphinx were upgraded in a backwards incompatible way. The fix is to upgrade to version 5, unfortunately there is no way to patch this bug otherwise.

Here's the bug report and discussion at Sphinx:

• sphinx.errors.VersionRequirementError: The sphinxcontrib.applehelp extension used by this project needs at least Sphinx v5.0 sphinx-doc/sphinx#11890

I'll close this discussion here, but hopefully that helps solve your problem.

[jakirkham]

Thanks Anthony! 🙏

We do request Sphinx 5.3.0 on this line and it does appear to get installed by Conda

However there is still Sphinx 4.5.0 version that is getting used, but we didn't install that one

Do we know where that is coming from? How can we make sure RTD uses the Sphinx we installed with Conda?

[agjohnson]

Hah, that's ... unique.

The build commands do look to be running from the conda installed Sphinx, I'm a bit at a loss why that is 4.5.0 though. I'm not the strongest with Conda tooling though.

cc @humitos maybe you have input here?

[agjohnson]

Just a guess, but an unbound sphinx is specified as well: https://beta.readthedocs.org/projects/dask-image/builds/23538687/#220805958--19 -- I believe these are added by our application to ensure default packages are included

Conda reports 5.3.0 is installed though, so not sure if that is important or not.

[jakirkham]

Yeah think this is added (possibly by RTD) in the build somewhere. The environment file only has the pip lines at the end

Do see Sphinx 5.3.0 is installed once

So think that is ok

Do we know what python is used to run python -m sphinx ... later? Maybe that will give us some more clues

[m-albert]

I just recreated the conda environment locally and found that it installed sphinx=4.5.
It seems that sphinx<5 is required by sphinx-book-theme==0.2.0, which is required by dask-sphinx-theme.

[m-albert]

Okay we're not the only dask project that encountered this error 😅 and people have found a workaround: dask/dask-sphinx-theme#68 (comment).

[agjohnson]

I just recreated the conda environment locally and found that it installed sphinx=4.5.
It seems that sphinx<5 is required by sphinx-book-theme==0.2.0, which is required by dask-sphinx-theme.

This was going to be my guess. The same dependency resolution issues happen with pip, but at least pip would report 4.5.0 was installed at the end. Conda reports Sphinx 5.3.0 was installed, which is apparently just lies?

Anyways, glad you have a fix for now. I've also been recommending folks keep up to date on Sphinx releases, as Sphinx is not as backwards compatible these days. I'd aim for Sphinx 7 when you upgrade.

[m-albert]

Thanks @agjohnson !

The same dependency resolution issues happen with pip, but at least pip would report 4.5.0 was installed at the end. Conda reports Sphinx 5.3.0 was installed, which is apparently just lies?

I think it's actually pip that installed sphinx 4.5. Is maybe the --quiet argument in the mamba call preventing this from being printet?

Anyways, glad you have a fix for now. I've also been recommending folks keep up to date on Sphinx releases, as Sphinx is not as backwards compatible these days. I'd aim for Sphinx 7 when you upgrade.

Thanks for sharing this, that's good to know!

[humitos]

That was also my guess: conda and pip doing/reporting different things.

I think it's actually pip that installed sphinx 4.5. Is maybe the --quiet argument in the mamba call preventing this from being printet?

As long as I know, the --quiet argument only removes the progress bar and that kind of verbose output. Mamba is reporting that Sphinx 5.3.0 was installed which is correct:

   + sphinx                             5.3.0  pyhd8ed1ab_0         conda-forge/noarch         2 MB 

However, pip is executed after the conda environment is created and Sphinx is downgraded when installing dask-sphinx-theme via pip as you already realized.

I think you could install the theme from conda-forge as well to avoid this issue using https://github.com/conda-forge/dask-sphinx-theme-feedstock

[m-albert]

If I create the environment with mamba locally without using --quiet I get the mamba output + the pip report:

...
  + sphinx                             5.3.0  pyhd8ed1ab_0          conda-forge/noarch        Cached
...
Installing pip dependencies: / Ran pip subprocess with arguments:
...
  Attempting uninstall: sphinx
    Found existing installation: Sphinx 5.3.0
    Uninstalling Sphinx-5.3.0:
      Successfully uninstalled Sphinx-5.3.0
Successfully installed attrs-23.2.0 beautifulsoup4-4.12.3 dask-sphinx-theme-3.0.5 docutils-0.16 jsonschema-4.21.1 jsonschema-specifications-2023.12.1 pydata-sphinx-theme-0.7.2 referencing-0.33.0 rpds-py-0.18.0 soupsieve-2.5 sphinx-4.5.0 sphinx-book-theme-0.2.0

Maybe it'd be useful to include this output during the RTD build (e.g. by taking out --quiet)?

I think you could install the theme from conda-forge as well to avoid this issue using https://github.com/conda-forge/dask-sphinx-theme-feedstock

This is a great idea, it probably makes sense to stay within conda when possible.

Thanks @humitos for your help!

[humitos]

Maybe it'd be useful to include this output during the RTD build (e.g. by taking out --quiet)?

I opened #11151 to track this.