Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: Build documentation manually instead of using sphinx #23763

Closed
datapythonista opened this issue Nov 18, 2018 · 7 comments
Closed

DOC: Build documentation manually instead of using sphinx #23763

datapythonista opened this issue Nov 18, 2018 · 7 comments
Labels
CI Continuous Integration Dependencies Required and optional dependencies Docs Needs Discussion Requires discussion from core team before further action

Comments

@datapythonista
Copy link
Member

I wasted several days of my life in the last months trying to understand sphinx code, and seeing how make it perform simple things like changing the depth of the sidebar table of contents, add breadcrumbs, and similar things.

Not sure what went wrong with sphinx development, but its code lacks any reasonable architecture, and the complexity for what we make it do is huge compared to what it should be.

So, given that the only two options I see are give up and have a suboptimal website for pandas, or stop using sphinx, I decided to start working on the latter.

There will surely be some challenges, but the core of what we need is simply:

  • Extract information from .rst files (docutil and numpydoc will do all or most of the job in this)
  • Render the extracted content to html templates (jinja2 will do that)
  • Manage the cross-references among documents (that will need to be implemented, but doesn't seem so much work or so challenging)

I'll work in this as a separate reusable project, that can be imported as a dependency.

CC: @jorisvandenbossche @TomAugspurger

@datapythonista datapythonista added Docs CI Continuous Integration Needs Discussion Requires discussion from core team before further action Dependencies Required and optional dependencies labels Nov 20, 2018
@jorisvandenbossche
Copy link
Member

I would be very hesitant to start on such an endeavour.

But I have to admit, I don't really know how many of the features we use are actually already supported by docutils, and how much are add-ons by sphinx.
Eg what about autodoc/autosummary, the ipython directive, math support, special directives (warning, note, versionadded, deprecated, ..), ... ?

Manage the cross-references among documents (that will need to be implemented, but doesn't seem so much work or so challenging)

There is not only the cross-referencing among documents, but also referencing functions, referencing docs of other projects, ..

If it is mainly the final "putting the things together" in a decent structure: I don't know to what extent it is possible to still use sphinx as a kind of preprocessor on the files (to deal with the sphinx-specific constructs), with the idea that it outputs something more general that can be injected in custom templates (just general thought, didn't think this through ..)

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Nov 21, 2018 via email

@datapythonista
Copy link
Member Author

I also agree with @jorisvandenbossche, and that's why I spent SO many hours trying to make sphinx do what we need. But after around one week wasted only in trying to understand how the sidebar toc tree is built, and still not much idea what is going on... I'm quite convinced that even if there will be challenges in building a sphinx replacement, I'll be saving a lot of time. There are so many things I want to customize, and even something very small is incredible complex with sphinx.

In any case, I'll build this as an independent project, I don't expect anyone else from the pandas team to be involved, and won't propose migrating until our website can be fully generated with it. So, no big deal on trying. :)

In any case, I appreciate the feedback, and will update you if I make progress with this.

@jorisvandenbossche
Copy link
Member

Can you give a bit more details on the other things you want to customize that might be difficult with sphinx? (the navigation structure is indeed an important and complicated one)

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Nov 21, 2018 via email

@datapythonista
Copy link
Member Author

@jorisvandenbossche I think there will be plenty of small things we'd like to change in the docs. For example I'd like to add breadcrumbs, or avoid that numpydoc parameters are rendered inside a <blockquote> tag, breaking the formatting.

Will take a look at the dask sidebar, but I think for pandas will render too much information, as it happens now.

@datapythonista
Copy link
Member Author

Closing this, at least for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Dependencies Required and optional dependencies Docs Needs Discussion Requires discussion from core team before further action
Projects
None yet
Development

No branches or pull requests

3 participants