Bump Sphinx to 1.4.6, or 1.4.1 in conda.

[ajdavis]

Sphinx is 1.4.6 in PyPI now, and 1.4.1 in Anaconda:

https://docs.continuum.io/anaconda/pkg-docs

[agjohnson]

Looks like our docs broke, sphinx_rtd_theme needs separate installation as it's not included in Sphinx anymore.

We'll need to thoroughly test this, we haven't had time for this though. Anyone with an instance feel like running through some QA testing here?

[willingc]

@agjohnson If you point me to the directions for creating an instance, I would be willing to test. conda-forge is also on Sphinx 1.4.6 https://anaconda.org/conda-forge/sphinx

[ericholscher]

http://docs.readthedocs.io/en/latest/install.html should have all the install info needed to set it up locally. Mainly we just need to run through the full builds and make sure it doesn't break out extensions or other bits.

[ericholscher]

Would be cool to use conda-forge to keep Sphinx versions the same, if it's easy, and we can depend on those packages. Who maintains conda-forge?

[willingc]

@ericholscher Thanks for the link. Here's info on conda-forge https://conda-forge.github.io/index.html It's community maintained. More info in this scipy video by Phillip Elson http://pyvideo.org/scipy-2016/community-powered-packaging-with-conda-forge-bof-session-scipy-2016-phillip-elson.html

[willingc]

Quick update: I've got the test site up on OS X 10.11 yet the lxml settings are a bit of a pain with tests since OS X needs additional settings for commands like pip:

CFLAGS=-I/usr/local/opt/libxml2/include/libxml2 LDFLAGS=-L/usr/local/opt/libxml2/lib

Going to find a Linux box to test.

[mpacer]

@willingc Just to follow up on this, on 10.11.16 I don't have CFLAGS or LDFLAGS set, and lxml posed no problems installing.

Hmmm running into some problems when actually running the server, that may be related. Unfortunate that there's no error initially if that is the case.

[SylvainCorlay]

This is related to #2566

[SylvainCorlay]

Besides, there is a package for sphinx_rtd_theme on conda-forge.

[ericholscher]

I've upgraded Sphinx in the pypi version to 1.5, as this was done along with an upgrade of our theme in #2676 -- I didn't touch the theme in conda, since it doesn't update directly from pypi, and as such didn't upgrade sphinx.

I know enough about conda to be dangerous, but the way it packages things over time makes it hard to support along w/ the pypi stuff. I know at one point we were installing from pip & pypi inside conda, but that caused other issues. I'd love it if we had someone willing to be a contributor to RTD, especially to support the conda stuff. It would be great to have someone with a local install that knows conda better than I do who can put some of these things through the paces and be able to provide feedback about how we should support it well.

[astrojuanlu]

I you're interested in pulling the conda version from the conda-forge channel, here you can ping the maintainers to update to 1.5.3:

https://github.com/conda-forge/sphinx-feedstock

cc @humitos

[humitos]

I know at one point we were installing from pip & pypi inside conda, but that caused other issues.

What are those issues? Are they RTD specific?

As far as I know, there is no big issues by installing packages from pypi using pip inside conda.

Anyway, the correct way of bumping up to the latest version of Sphinx in conda seems to be by upgrading the Sphinx version on conda-forge channel but, what are the channels being used in RTD? In this failing build that tries to use python=3.6 (http://readthedocs.org/projects/poliastro/builds/5107503/). I don't see what are the channel being used. Python 3.6 seems to be packaged in conda-forge: https://anaconda.org/conda-forge/python

Should RTD add support for conda-forge channel to install their own base packages?

Besides that, that failing build was built with this environment.yaml (https://github.com/poliastro/poliastro/blob/ad8ced301d805b453a09d252cf0e3ea476423fbc/environment.yml) which adds defaults and conda-forge channels. So, isn't the conda-forge channel being used? Is just a mismatch version between 3.6 and 3.6.0?

On the other hand, we will need to upgrade the sphinx_rtd_theme also to the same version used for pip environments.

[humitos]

The problem here is that sphinx is pinned in 1.3.5 in the conda environment at RTD, and it's incompatible with python=3.6. I was able to reproduce the failing building that I mentioned in my previous comment but also this simple command (outside RTD) fails:

conda install python=3.6 sphinx=1.3.5

[willingc]

FYI...One can build docs using conda with Sphinx if you pin the dependency to Python 3.5 as seen in the environment.yml here.

[willingc]

@ericholscher Let me talk with some folks about being a contributor for conda builds. Thanks for updating re: PyPi builds.

[humitos]

@willingc there is already a PR to build the sphinx=1.5.3 in conda at conda-forge/sphinx-feedstock#10 and conda-forge/sphinx-feedstock#12 but it's failing on windows and we are not sure why. Maybe you can help us to figure this out.

[ericholscher]

@humitos:

What are those issues? Are they RTD specific?

We install some deps from conda, and then some from pip. We were running into issues when users then had a custom environment that also updated the pip installed dependencies. It is likely something that conda could have fixed, but there was a whole host of issues.

I'd love to have you or someone else on board @willingc -- I don't understand the best practices of conda well enough to really properly support it. I imagine we're doing some stuff that makes it harder for people to integrate. Perhaps we could just require a conda environment, and not do any of our own package installation? That would be a pretty sizable backwards incompat change, and vary from how we do normal builds, but perhaps it makes more sense for conda?

I'm weary of depending on conda-forge. Is it more similar to something like pypi for conda, or is it closer to a PPA for Ubuntu? It seems like it's community maintained, but I don't know enough about it to trust it as a default for our builds. I'd love to defer to someone else on that :)

[ericholscher]

Also of note, we'd need our theme upgraded in conda as well. This was a joint upgrade of both packages.

[willingc]

Hi @ericholscher, I would feel completely comfortable with using conda-forge for rtd. I typically use conda-forge when building stuff for Jupyter. One of the nice things about conda-forge is that it is very rigorous with testing before any package is available in the forge. I would say it's more similar to pypi than PPA. FYI @jakirkham for conda-forge, @minrk for Jupyter.

[humitos]

@ericholscher if you want to share the same idea for normal building than conda building, we should use the same pinned packages on both (venv and conda).

Besides, conda-forge it's community maintained but I would say that it's even better than pypi since for a package to be published has to pass all the tests in windows, macosx and linux. In fact, at this moment we are not able to publish Sphinx 1.5.3 because the windows tests are failing (and we are not sure why yet). That's why the latest version of Sphinx in conda is 1.5.2 right now.

Also of note, we'd need our theme upgraded in conda as well. This was a joint upgrade of both packages.

It's done in conda-forge.

[ericholscher]

@humitos @willingc 👍

[ericholscher]

I'm going to close this, and take the conversation to #2713 -- once we address that, we can upgrade conda to Sphinx 1.5 & sphinx_rtd_theme 0.2.4, like our normal virtualenv installs.