Add version dropdown (for other versions of the docs)

[jorisvandenbossche]

We would like to have a version dropdown to switch to a different version of the docs.

Some examples:

Python docs (https://docs.python.org/3/) (also sphinx based, could look how they do it):

Docker docs (https://docs.docker.com/install/overview/):

Readthedocs theme also supports this (so this could also be worth a look how they do it):

[jorisvandenbossche]

@bryevdv implemented this for bokeh in bokeh/bokeh#9330 (see https://docs.bokeh.org/en/dev/index.html), so we can port that back to the main theme.

[bryevdv]

Just some notes about the Bokeh version:

• it releases on a version.json file that presumably would get updated automatically or manually during a release. It lists all the versions to go in the menu explicitly

• this JSON file also lists all previous releases, and this is used to display alert banners for "old release" or "pre-release" at the top

Also, the code for this is currently in a scripts.html template that gets included in every page. I kind of think it might be better to move the version handling to a completely separate JS file that gets loaded in the scripts template. That way if there is ever a time when maintainers want to update the version handling somehow for all versions of the docs, there is just one JS file per docs version to swap out.

[mattip]

+1 for this. FWIW, spyder just released its docs with a modified version of this theme and it has a version template, javascript,css. I don't see how it determines the document versions to list, I think it is connected to the multidocs sphinx extension. That builds the docs for each version, which is not really portable.

[choldgraf]

note that we're discussing this issue in #234 as well...not sure if that's a dupe or not

[bryevdv]

Not sure if it is relevant but just FYI for Bokeh we update/publish a versions.json file as part of our release, that lists the versions that should appear in a menu, as well as specifies the current latest version (so banners get raised when viewing older or dev/rc versions)

[dopplershift]

At a bare minimum, it would be nice to have an easier place to hook in to add/set-up the version chooser. Right now I need to copy/override the entire navbar to inject the <li> and other elements. Happy to submit a PR if there's some agreement on what including even that would look like.

[CAM-Gerlach]

@dopplershift There's also #244 that discusses the sidebar injection portion of this; we did something like this in our downstream fork, but in a more limited fashion.

I don't see how it determines the document versions to list, I think it is connected to the multidocs sphinx extension.

Its based on the tags and branches in the repo (and can use the version and release in your conf.py for the URLs and/or UI names, among other things), with a customizable whitelist for each, so you can customize which get shown. In addition, we have automated build scripting we developed that generates redirects, a tag for the current branch, and automatic stable/latest-type names; this could either be reused, packaged or turned into a Sphinx extension, perhaps.

That builds the docs for each version, which is not really portable.

I'm not sure I follow, but would love to hear more about this. Thanks!

[mattip]

As far as I can tell, and correct me if I am wrong, the multidocs sphinx extension builds the docs for every version at every invocation. The workflow for NumPy is to build the release version of the docs off the release tag, and copy that into a github repo numpy/doc (that already contains prebuilt documentation for previous versions).

[greglucas]

What about adding a new theme option version_switcher that would accept a list of url's and display names? This would make it pretty easy for downstream packages to simply implement in their conf.py theme options by just passing a list of url's and version display string.

In conf.py:

base_url = 'https://example.com/example/doc/'
version_choices = [
                ('latest', '2.1'),
                ('2.0.2', '2.0'),
                ('1.3.3', '1.3')]
version_choices = [base_url + v[0], v[1] for v in version_choices]

html_theme_options = {'theme_version_switcher': version_choices}

Then this theme could add another nav-item to the nav menu to create the dropdown.

        {% if theme_version_switcher | length > 2 %}
          <li class="nav-item version_switcher dropdown">
            <button id="dLabel" class="btn btn-secondary btn-sm dropdown-toggle" type="button" data-toggle="dropdown"
              aria-haspopup="true" aria-expanded="false">
              {{ theme_version_switch[0][1] }}
            </button>
            <div class="dropdown-menu" style="min-width: 6rem;">
              {% for version in theme_version_switcher %}
              <a class="dropdown-item" href="{{ version[0] }}">{{ version[1] }}</a>
              {% endfor %}
            </div>
          </li>
        {% endif %}

This renders as:

Some things to consider from the examples above too:

• everyone is putting the version switcher in different places. Should it be in the nav header, a sidebar, left/right, etc... Should the theme be somewhat opinionated here and just choose one?

• Should a version switch be directly linked to the new version's index page? If I'm on page example.com/v2.3/feature/method1, Should the url be switched to: example.com/v1.2/feature/method1 instead of example.com/v1.2? (Spyder does the former, Bokeh does the latter)

[dopplershift]

@greglucas The problem with that approach is that old versions of docs have to be rebuilt in order to show newer versions in their chooser. That's why the common approach is to put a json file at the top level, where JavaScript within the built docs can read the information on all the available versions and dynamically populate the selector.

[greglucas]

Good point! A similar method could easily be wrapped up in a <script> tag to inject the js.
'theme_version_switcher': '/path/to/versions.json'.

Thinking on the fly here though, I think you'd still need to rebuild all of the old docs once initially to inject that script? But, at least it would be a one-time event rather than for each new release as you bring up.

[choldgraf]

Is there a sphinx package that does this? And if not, I think it could be pretty easy to add a little lightweight one that would:

• Let you use html_sidebars to identify a placeholder for the version switcher
• Use some javascript to read in JSON as described above and then replace the placeholder with a little dropdown

This theme could then depend on that library to re-use the dropdown if it wanted something in the top-bar

[choldgraf]

Now that #248 is merged we have access to html_sidebars. This means that it should be pretty straightforward for people to add the dropdown that @greglucas suggests above, probably with the modification that @dopplershift suggests to have a single source of truth for the versions.

As a start, perhaps @greglucas could make a PR that adds a template for a dropdown selector here:

https://github.com/pandas-dev/pydata-sphinx-theme/tree/master/pydata_sphinx_theme/_templates

and then add a configuration option to add it to the list of html_sidebars that is activated by default in this theme? (as well as documentation for how others could add it manually).

Then a next step could be to find a way to add it to the navbar as well, but it seems this would be a good first-step.

[tupui]

Is this still wanted? I can see two PR but they look stalled. I started to add a rough badge on SciPy and I am not sure if I should just wait for it to happen here or not. xref scipy/scipy#14132

[ThuWangzw]

I'm working on #360 to add a version switcher and a locale switcher. Here are some things I'm considering:

1. I think it's better to rename version.json to config.json as both version switcher and locale switcher need to be configured.
2. Some locale switcher examples:
   Vue
   
   Python
   

[tupui]

Ok thanks I will wait for this to happen then.

[drammock]

@ThuWangzw in case it's helpful, here is how we implemented the version switcher in this theme, in MNE-Python:

https://github.com/mne-tools/mne-python/blob/a22a3ca42687fb63975070cc46ac7754af409732/doc/conf.py#L537

https://github.com/mne-tools/mne-python/blob/ed64dc7bd0750bab2e99a1cfcbce2928eb37e202/doc/_templates/version-switcher.html#L1-L11

https://github.com/mne-tools/mne-python/blob/35e867ad6506d50a4fa1bef71cc2610bf1aad6f3/doc/conf.py#L592-L607

[tupui]

@ThuWangzw in case it's helpful, here is how we implemented the version switcher in this theme, in MNE-Python:

https://github.com/mne-tools/mne-python/blob/a22a3ca42687fb63975070cc46ac7754af409732/doc/conf.py#L537

https://github.com/mne-tools/mne-python/blob/ed64dc7bd0750bab2e99a1cfcbce2928eb37e202/doc/_templates/version-switcher.html#L1-L11

https://github.com/mne-tools/mne-python/blob/35e867ad6506d50a4fa1bef71cc2610bf1aad6f3/doc/conf.py#L592-L607

Woo your doc is really nice @drammock 👍 If you have any feedback for SciPy I would be interested 😅

[choldgraf]

@drammock even better would be a PR or comments on an existing PR for this theme 🙃

[tupui]

@drammock even better would be a PR or comments on an existing PR for this theme 🙃

Agreed! What I meant was more in terms of config with respect to this theme 😃

[drammock]

@drammock even better would be a PR or comments on an existing PR for this theme upside_down_face

😆 it's on my TODO list, but I keep hoping someone else will beat me to it

[drammock]

OK, I just re-read this whole thread, and can make an almost-concrete proposal, based on how we did this in MNE-Python:

1. The version switcher should be implemented as a Jinja template, as we did here. This should allow people some flexibility as to where to put it in their site (sidebar, topbar, etc)
2. It should draw the list of versions and version descriptors from either a JSON file (as @bryevdv did for Bokeh) or from a conf.py variable (as we did in MNE-Python). Need input as to which way to go here. Personally I prefer conf.py (esp. because of # 4 below) but could be convinced otherwise.
3. In our implementation, button color is determined by a custom environment variable BUILD_DEV_HTML that our CIs write, which is then parsed in conf.py here and used in the Jinja template to determine color here. I don't think this is suitable for incorporation into the theme, because it relies on some CI magic that is too specific to MNE-Python's deployment setup. Also, some folks might want more than 2 colors, e.g., @tupui's scipy PR mentions using different colors for old versions, current stable, and dev, so a single boolean env variable can't capture that. If I had to decide myself, I would say to either hard-code the class primary into the theme's version switcher or omit a color-determining class entirely, and let people override the jinja template (or add classes at runtime) if they want to do something more complicated with color. Open to other suggestions here.
4. Needs a way to specify the URL template here. Presumably a conf.py variable should suffice?
5. Do not rely on any multi-version-building sphinx extensions. The theme should be agnostic about that; if people want a multi-version build it ought to come with its own dropdown switcher. I can't easily test how well this dropdown would integrate with such an extension if that extension didn't provide its own dropdown, so if people really want/need that integration, someone else will need to drive the PR (or do a follow-up PR to add it).
6. The warning bar that says "you're on an old version, switch to stable" should not be part of this PR. Our implementation of that is here and it's not a great solution because (1) the current stable version needs updating at each release, and (2) it lives in a separate repo that we use for our docs (similar to what @mattip says they do for NumPy) so it would take some work to figure out how to make that part of the theme.

[choldgraf]

That sounds quite nice to me 🙂 I guess the main question is whether one of the other PR implementations wishes to make a push as well so that toes don't get stepped on. That said, we have lots of not quite done implementations, so my vote is for whatever works and is in ready-to-merge condition 🙂

[ThuWangzw]

@drammock Thanks for your advice! 👍

1. Adding classes at runtime to let people customize btn's color is a nice idea!
2. I also think a URL template is necessary in conf.py. 😄
3. I prefer using config.json(version.json) to specify which versions should be included in version switcher. Other options such as colors of btns can be configured in conf.py.

[drammock]

@ThuWangzw I have a couple questions:

1. can you explain why you prefer a json file over specifying the versions in conf.py? To me conf.py makes more sense because (1) it doesn't require an extra file, (2) if you have an extra file, you'll need to specify in conf.py where to look for it anyway, and (3) any user setting up a site must necessarily be comfortable to edit conf.py but might not be familiar with .json or editing .json by hand, so it could add extra user burden in some cases.
2. you say above that you're working on ✨ ENH: Support version and locale switcher #360 but I don't see any recent commits there... is your work in a fork, or a different PR? I'm trying to figure out whether I should just open a new PR that implements my proposal, or if you or someone else is going to adapt an existing PR (and get it done soon-ish).

[ChaiByte]

@drammock There is a situation that should be considered:

• Assume that we had finished a conf.py to enable version switching in an early version (v1.3) before
• It looks fine. In v1.3 doc, we are able to switch to any version earlier than v1.3 such as v1.2, v1.1 ...
• BUT -- If you are building v1.4 doc now and you edited your conf.py (adding v1.4 info), that will only work for v1.4 doc to access v1.4 doc pages. Because in v1.3 doc's conf.py we can not find any information for v1.4 doc path/name/ or others.
• Now the issue occurs: How can we make sure any earlier version doc can use the latest content to generate a dropdown menu? We could not go back to edit every early doc's conf.py file because it's too troublesome!

For example, if we click Python 3.9 doc, jump to Python 3.7 but can not find 3.9 in the dropdown menu. It's so strange. (Seems Python use switchers.js and python/docsbuild-scripts#91 to solve it. They have to maintain https://github.com/python/docsbuild-scripts/blob/main/build_docs.py and it needs several hours to build each version docs.)

Although you can put a link to the latest/stable document in the menu. But in this way, we need to go to the latest version of the document every time to get the complete drop-down options.

So if we can put a config.json file in someplace and specify the right path in our conf.py, then we can dynamically get the latest information and generate a drop-down menu, which will have the complete options.

@ThuWangzw is working on this branch now. @drammock We need more discussion and suggestions, anyone is welcome~

[drammock]

If you are building v1.4 doc now and you edited your conf.py (adding v1.4 info), that will only work for v1.4 doc to access v1.4 doc pages.

ah, good point @MegChai. that's a good reason to prefer .json over conf.py.

@ThuWangzw is working on this branch now.

OK, I looked at that branch. Comments:

1. opinions may differ on this, but I would recommend that development of this feature should happen within an active PR to the upstream repo (because then the maintainers can see your progress, give advice early, and potentially save you from some effort that will end up not being used). If version-commiter is where this feature is being developed, can you open a new PR from that branch into this repo (and close ✨ ENH: Support version and locale switcher #360)?

2. it was requested by @jorisvandenbossche in ✨ ENH: Support version and locale switcher #360 to implement the version switcher first, and do locale switching in a second, separate PR. The work in the version-commiter branch does both at once.

3. The work in the version-commiter branch adds quite a lot of Javascript, but I don't think that's actually necessary to get a nice version dropdown. Have you looked at the jinja-based implementation that I linked to above? Is there some reason that approach won't work for your use case?

[ChaiByte]

@drammock You are right. #360 will be closed.

We planned to develop on dev branch to make people who follow the PR can get the latest notifications. But version switcher and locale switcher in the dev branch are already used in MegEngine official documentation and it's hard to drop one of them. So now we are using a new branch. When the version-commiter branch is prepared and passes our local test, @ThuWangzw will open a new pull request.

Using config.json will be more flexible to extend more kinds of switchers, or remove. So although the two switchers are implemented in the same PR, it could be easy to maintain in the future. That means the locale switcher could be re-designed if we find a better way later.

[drammock]

@drammock You are right. But version switcher and locale switcher are already used in MegEngine official documentation and it's hard to drop one of them.

That's a good reason why you should not open PRs from a main, master, or dev branch: it constrains your ability to actually incorporate the feedback you get from other developers. See above my advice to close #360 and open a new PR from the other branch.

And using config.json will be more flexible to design and extend to support more kinds of switcher. (Now the two switchers are both dropdown menus.)

I've already acknowledged that using .json instead of conf.py makes sense, for the reason you mentioned above: it allows updating the items in the version switcher even for old versions of the docs. You don't need to convince me again :)

When the version-commiter branch is prepared and passes our local test. @ThuWangzw will open a new pull request. the dev branch is used by MegEngine doc now so we should better just keep it until we get a unanimous solution.

I may be misunderstanding the situation, but it sounds like you might be taking the approach of implementing it the way that works best for you, and then opening a "take it or leave it" pull request for an implementation that you don't intend to change. There's nothing necessarily wrong with that --- sometimes your local needs are urgent and you can't wait for the upstream maintainers to all agree and for a new release to be made --- and it's still nice of you to offer your implementation back to upstream. But if that is the approach you are taking, it seems a little disingenuous to say things like "We need more discussion and suggestions, anyone is welcome". Those discussions and suggestions typically happen in open pull requests, which you've said you will open only after you've finished your implementation. This leaves an impression that you're not actually open to collaborative decision-making about how the implementation should look. In light of that, can you clarify your intentions?

[ChaiByte]

@drammock

Sorry, I may not have explained clearly, we originally planned to update directly on the dev branch but find it will affect the deployment of the current website, and then made a decision yesterday to use a new branch (the version-commiter branch was pushed within 24 hours). Before a new PR is proposed, we will keep #360 open for reference.

the dev branch is used by MegEngine doc now and it's hard to drop one of them.

This sentence seems to be me talking to myself, please ignore it...

[drammock]

ok, thanks for clarifying. I'll assume then that your new PR will be coming soon and I'll save any further feedback for that PR.

[damianavila]

Folks, I am looking at this right now... trying to collect and digest all the information (and different implementations) so I am able to provide good and useful feedback in the current open PRs.

You should expect some comments on those PRs soon.
I might also add some other general thoughts here as well as I am understanding things more deeply.

Thanks for your patience and for the contributions!!

[damianavila]

All,

I have closed the first implementation (#276 because I believe the other two PRs (#433 and #436) are more mature and able to be merged soon.
Particularly, after looking at all the implementations, I have requested some changes in #436 so we can merge that one ASAP and iterate on top of it.

If you @drammock or @ThuWangzw disagree on the proposed path forward, let us know!

[Sachin-Suresh]

@ThuWangzw in case it's helpful, here is how we implemented the version switcher in this theme, in MNE-Python:
https://github.com/mne-tools/mne-python/blob/a22a3ca42687fb63975070cc46ac7754af409732/doc/conf.py#L537
https://github.com/mne-tools/mne-python/blob/ed64dc7bd0750bab2e99a1cfcbce2928eb37e202/doc/_templates/version-switcher.html#L1-L11
https://github.com/mne-tools/mne-python/blob/35e867ad6506d50a4fa1bef71cc2610bf1aad6f3/doc/conf.py#L592-L607

Woo your doc is really nice @drammock 👍 If you have any feedback for SciPy I would be interested 😅

Where can I get the working copy of both versions and languages drop-down, please? Are the above the 3-4 files that are responsible for this functionality ?

[tupui]

@Sachin-Suresh please have a look at the documentation https://pydata-sphinx-theme.readthedocs.io/en/latest/user_guide/version-dropdown.html

[drammock]

Where can I get the working copy of both versions and languages drop-down, please?

the expansion to cover both languages and versions was never implemented (yet).

[Sachin-Suresh]

Thanks @tupui. And sorry to continue this discussion on a closed PR. I have been struggling to implement versions and language in Sphinx Readthedocs theme with limited documentation. It would be great if I get some resources on this as I am new to Sphinx. I would still try the Version switcher documentation Tupui had shared.