Is it possible to disable/modify --upgrade-strategy eager when pip installing the source package?

[alejoe91]

Hi RTD Team,

First of all thanks for the amazing service that you provide to the community!

Is it possible to disable/modify --upgrade-strategy eager when pip installing the source package?

Details

• Read the Docs project URL: https://readthedocs.org/projects/spikeinterface/
• Build URL (if applicable): https://readthedocs.org/projects/spikeinterface/builds/15984200/
• Read the Docs username (if applicable): https://readthedocs.org/profiles/alejoe91/

Expected Result

We have a series of dependencies that need to be fixed in order for our documentation to build properly.
We use an environment file that specifies numpy==1.20 because of some external packages that we use for our docs.

Actual Result

Since numpy is one of our requirements and we would like not to force any version in our requirements.txt.
When the dependencies are installed during the build, the --upgrade-stategy eager forces the numpy version to be upgraded to 1.22, which breaks several of our examples.

[humitos]

Hi! We don't currently support changing the --update-strategy. This may be possible once we implement the new builders --but they are not in the roadmap yet and I'm not sure when exactly that will be.

Since you are using conda already for your environment, would it be possible to install your local package with conda instead? Or even using the pip key from inside the conda environment file?

If that is possible, you can probably delegate that to conda instead of Read the Docs installing your package via pip and using -update-strategy eager. Does this make sense to you?

[alejoe91]

@humitos thanks for your reply. I'll try what you suggest and let you know!

[alejoe91]

@humitos seems to work! thank you again for the suggestion :)

[st-bender]

Hi there @humitos
Sorry to comment on a closed issue, but I think you should seriously reconsider this. There is no point in setting all up via conda, only to have the pip install re-installing and upgrading packages that are not supposed to be updated for the documentation. In my opinion this should really be opt-in, at least there should be an opt-out option.
It will also save some traffic because it will stop useless downloads and people trying to get their docs (re)built properly.
Cheers.

[humitos]

@st-bender this use case will be supported once we implement #9062

[st-bender]

Hi @humitos
Thanks for the update, it is very much appreciated!
Meanwhile, I found at least two feasible solutions, which might be worthwhile to add to the documentation/faq.
One is installing the package via conda's pip, which also works in non-editable mode and with extra requirements, by having lines like the following in the conda environment.yml file:

# environment.yml
dependencies:
  - pip
  - pip:
    - .[extras]  # or ..[extras] if the environment file is in a subdirectory

The other option is using jobs, one can put pip install into the configuration's post_create_environment job (for example) and remove the python install step:

# readthedocs.yml
build:
  jobs:
    post_create_environment:
      - python -m pip install --no-cache-dir .[extras]

# no python install section

This latter option got me thinking a bit more, since all is already set up, it would probably be good enough, and maybe easy, to just introduce an install job (instead of using post_create_environment like here). That would be very similar to what people might be used to on other CI/CD services, at least I am. And the rest can even stay at the defaults as they are.
Cheers.

[benbovy]

I'm struggling to get around inconsistent package upgrade done in this build: https://readthedocs.org/projects/ipytone/builds/17209864/.

Context: an indirect dependency (jupyter-cache) has a pinned dependency (nbclient<0.6) which is satisfied during the installation of the conda environment but which is then upgraded to the last release (>6) by RTD, utlimately causing the build to fail.

I've tried several of workarounds like explicitly add and pin these two packages in my environment.yml file, with no luck so far.

I could also install the package via the conda environment file but I guess this wouldn't work well with RTD pull-request builds (a feature that I like a lot)? For some reason, when I tried it the install job (pip upgrade eager) still run, causing the build to fail.

Then I've tried setting up a post_install job to downgrade nbclient<0.6 with pip, but I couldn't see the job in the build logs (maybe I'm doing something wrong?)

I could try using build.commands, but it is experimental and again would it work with pull-request builds?

Is there any reasonable workaround that I've missed?

In general I agree with @st-bender #8890 (comment), it would be great and easy if we could just opt-in or opt-out the --upgrade-strategy eager option.

[humitos]

Context: an indirect dependency (jupyter-cache) has a pinned dependency (nbclient<0.6) which is satisfied during the installation of the conda environment but which is then upgraded to the last release (>6) by RTD, utlimately causing the build to fail.

I suppose that you may be hitting #3829 . I enabled CONDA_APPEND_CORE_REQUIREMENTS to your project as a potential workaround. I triggered a build with this setting but it didn't work either: https://readthedocs.org/projects/ipytone/builds/17224940/

I noticed that you have 2 Read the Docs Config YAML files. It seems that these lines https://github.com/benbovy/ipytone/blob/master/readthedocs.yml#L5-L7 are producing the problem. Would you mind deleting that file and using https://github.com/benbovy/ipytone/blob/master/.readthedocs.yaml instead, which looks more updated?

[benbovy]

I noticed that you have 2 Read the Docs Config YAML files.

Oops yes I totally missed that. It works now without the old config file. Many thanks for your help @humitos!

[valeriupredoi]

hey guys, just wanted to drop a line and thank the Readthedocs devs for implementing build jobs, and to those that are coming here looking for solutions, using said jobs eg https://github.com/ESMValGroup/ESMValTool/pull/2913/files works a treat, and avoids fully the unnecessary pip eager installation 👍 🍺