WEB/DOC: integration of the main website and documentation

[jorisvandenbossche]

With the new markdown based website (#28014) and the new sphinx documentation theme (#15556, https://dev.pandas.io/pandas-sphinx-theme/), we need to see how to integrate both.

Roughly speaking, I can currently think of three options:

• Have them separate.
  • This is like we have it now, but of course styling-wise they can look unified. But the button for the docs on the main website goes to the doc website without actual integration of menus of both.
• Also show the website navbar on the documentation site:
  • This would mean that when landing on the home page of the docs, you see two navbars: one of the main website, one of the docs. Once you start to scroll down, the navbar of the main website can eg disappear
  • An example of this is the docker docs (https://docs.docker.com/), where there are links from the main website fully at the top, which disappear when scrolling down (then only the navbar menu of the docs is kept).
    I did a quick experiment for this: https://dev.pandas.io/pandas-sphinx-theme/pr-integration-website-double-navbar/. Right now this looks ugly of course due to the non-matching styles and alignment, but to give the idea and I think this can be made look nice with matching styles.
• Have a single shared navbar with menu items of both main website and the sphinx documentation site:
  • Just combining both as they are right now will probably give a too crowded header navbar, so we might need to make some choices what to put there. Currently we would have: Home, Install, Getting Started, User Guide, API Reference, Developer, Release Notes, Community, Donate. Probably Developer and Community could be combined, Release Notes is maybe also not needed on this level (the question is then where to put this though).
    (TODO: will also do an experiment for this option)
  • Having the top-level navigation of the docs in a dropdown (so "docs" itself is only 1 item in the top navbar), as it is now at https://dev.pandas.io/, is also an option to reduce the number of items on the navbar. But personally, I think those should not be hidden in a dropdown, and one of the goals of the new sphinx theme was actually to make this top-level navigation more accessible.

Short term we can use the first option (eg if we want to start using the new theme on the dev docs this week), but I think ideally we want to go to a more integrated option?

---

Besides the choice on how we want it to look, there is also a technical issue to solve: how to put this common navbar in the docs?
We can hardcode this in the html (when building it can eg be included in the layout from a file shared with the website), but this means that this navbar is fixed for older versions of the documentation, and if we do updates to the website this might not match with the header navbar of the released documentation.

[datapythonista]

Worth noting for the items in the navigation bar, that we've got some discussion with NumPy to standardize all that makes sense (numpy/numpy.org#43). I'm personally very happy with what we discussed, I'm generating a version of the website with what I think it makes sense for pandas based on the discussion.

I think 3 is what makes more sense. I'd start by something as simple as creating a theme directory in the repo, with the layout of the web, and use it in the docs too. I guess it'll require few changes, but I created the layout of the website with Sphinx blocks, so shouldn't require so many changes.

The navigation bar is populated from a yaml, I think we should parse the yaml in the Sphinx settings and add it to its context. But I think that would be a good first step, and we'll see if this looks like the right direction or not.

[jorisvandenbossche]

Worth noting for the items in the navigation bar, that we've got some discussion with NumPy to standardize all that makes sense (numpy/numpy.org#43).

OK, that seems an interesting discussion, will need to read up on that. One note though, that from a seeing their current sitemap idea (and also current docs), is that numpy is actually doing option 1: link from the website, but the docs itself are a separate entity (which also eg Julia is doing)

[jorisvandenbossche]

If you say that you would prefer option 3, can you be a bit more detailed? Do you mean with the current "Documentation" dropdown? (as I said above, and before in person I think, I am not a fan of such a dropdown for the main items of docs)

The navigation bar is populated from a yaml, I think we should parse the yaml in the Sphinx settings and add it to its context. But I think that would be a good first step, and we'll see if this looks like the right direction or not.

That's sounds as a good option to get it in sphinx (at least in the static html of the latest version, question is still if we are OK with that for stable + older versions) . As a first step, I wouldn't care too much about automation like that, but first experiment and decide with a certain design.

[datapythonista]

I think having a unifier navigation in all pages including the docs has a lot of value. Many people will land in the docs directly (e.g. read_csv), and not having a way to go from there to the home page, or to see all what we "offer" in the website, creates a poor UX, and limits the visibility of the pages in the website. This is what we have now, and I guess/hope more or less we'll agree on this.

Changing the items of the navigation depending on where you are seems confusing to me (like showing "About us" in the home, and "API reference" in the docs).

Then, the two options I'm happy with are having just the top navigation bar (and then the sidebars you're building), or having the second bar in your prototype.

My preference is the first, not having the second bar, mainly because:

• I think the items will be the same as in the dropdown, so not much value added
• It's simpler to not have it from our side
• Having 4 navigations (top, second top for docs, and two sidebars) is too much for the user
• I think one of the sidebars will also contain links to user guide, API reference... so having it 3 times redundant seems unnecessary

Removing the dropdown in "Documentation" and having the second bar instead sounds reasonable. But I don't see why a dropdown should be a bad thing, and I don't think we want to add even more complexity than the two sidebars to our website. css really sucks, and the last thing we want to spend time with is in having to deal with issues of "your css is broken in X browser". I'd take what bootstrap gives us for free, and not overcomplicate things, if it's just for a personal preference on dropdowns, and there is nothing else that I'm missing.

In any case, worth moving forward one step at a time, and create drafts of the different options once we're in that stage. Will be easier to discuss in practice than in theory I think.

[jorisvandenbossche]

not having a way to go from there to the home page, or to see all what we "offer" in the website, creates a poor UX,

That might well be, but that's how it is in almost all projects that we were comparing with in one of the other issues (numpy, jupyter, spark, julia, ..). They all have a separate website and separate docs, with just a link from the one to the other.
Of course, that doesn't mean we can't do better! Just want to say it's certainly not uncommon (and I only mentioned this option 1 as a short term thing, to already start using the theme).

Changing the items of the navigation depending on where you are seems confusing to me (like showing "About us" in the home, and "API reference" in the docs).

Agree that that would be confusing (that's not what I wanted to suggest, sorry if that was not clear).

I think the items will be the same as in the dropdown, so not much value added

To me, the fact that they are not in a dropdown is a significant value.

I think one of the sidebars will also contain links to user guide, API reference... so having it 3 times redundant seems unnecessary

That's not my current idea for the theme (see the prototype at http://dev.pandas.io/pandas-sphinx-theme/user_guide/index.html), but feedback on it is very welcome.

But I don't see why a dropdown should be a bad thing ... if it's just for a personal preference on dropdowns, and there is nothing else that I'm missing.

Personally, I find dropdowns often a poor UX (often, not always). In this case, having the main navigation level of the docs in a dropdown means an extra click to move there + makes it somewhat hidden.

---

Some examples for possible inspiration:

Django is an example where they have a common header, also on the doc pages: https://docs.djangoproject.com/en/2.2/ref/applications/ (but they don't really have a second nav bar, although they have a bar with "Documentation" title and the search box)
It would be interesting to understand how they do it, but I don't directly see anything related to the common top bar in their doc/ sphinx layouts.

Mapbox docs is an example of two navbars, where the top one disappears when scrolling down (in our case the top one could be the main website one): eg https://docs.mapbox.com/mapbox-gl-js/api/ (personally, on their site and their styling, I don't find this "too much")

Docker also has the main website navigation on their doc landing page: https://docs.docker.com/ (at the full top is the main website navigation, below the "Docker Documentation" banner is the main doc navigation, and when scrolling down, this one sticks at the top).
The difference here is that this main website navigation is only shown on the doc home page, not on all doc pages.

[datapythonista]

Don't have time to work on this now, but to summarize what I'd do:

• Have the theme implemented in Sphinx, including the main layout now used for the web
• Make the web use the Sphinx theme
• Possibly move the web to doc/web/ so the access to theme stuff is more "natural"

And as I mentioned, keep the top bar common. I'd just have that bar, but happy to have a second bar for the docs, since you seem to have a strong preference on that.

[kylekeppler]

FYI, https://dev.pandas.io/about/index.html is not loading CSS.

[kylekeppler]

The "Whats new in 0.25.3" link on the homepage takes you to the 0.25.0 whats new page instead of the 0.25.3 page.

[WillAyd]

Currently releasing 0.25.3 and not announced yet, so might be some broken links during that process

[datapythonista]

FYI, https://dev.pandas.io/about/index.html is not loading CSS.

Thanks for reporting @kylekeppler, I opened #29325 for this.