Automatic docs

[williamFalcon]

I know it was proposed before and I tabled it but I think it's time to reconsider so docs can scale easier.

Let's use whatever PyTorch uses and do documentation in the code (i assume that's how it's done?).

Anyone want to take a stab at this? @Borda

[Borda]

I will try, but it may be huge change in code so maybe split it to some smaller tasks...
I would link this action with doctests - #463 (comment)
Btw, could you make be here as assigned... :)

[williamFalcon]

agreed. can we go maybe one page at a time? otherwise there'll be merge conflicts everywhere.
In the meantime we can deploy this version of the docs to a separate URL. Once ready we swap the gh-pages

[Borda]

Just checking the PyTorch and they are using Sphinx which I am also quite familiar... So would you mind to switch to Sphinx or you want to stay with mkdocs? In such case I have to do more research on how to generate API docs etc... or someone else has experience with MKdocs?
^^ @williamFalcon @neggert @Ir1d

[Ir1d]

@Borda I strongly suggest switching to sphinx.
Maintaining API docs with markdown(mkdocs) is harder and not necessary. Generating directly from docstrings is enough for API docs, which can be achieved perfectly with sphinx. Also we could adopt pytorch_sphinx_theme, which has similar UI with pytorch docs and thus more friendly to the community.
Meanwhile, as I've mentioned several times before, the current doc is more like a Cookbook, and maintaining both mkdocs and docstrings is a heavy burden.

[Ir1d]

About implementations, we can setup sphinx in some folder, while leaving the current mkdocs as it is. In this case there won't be much conflict.

It would be nice if we maintain some guide for newcomers in mkdocs, while we maintain detailed api docs in sphinx. What do you think? Or perhaps we just move everything inside sphinx after the migration.

[Borda]

@Ir1d strongly suggest switching to sphinx.

I would like, just wait for @williamFalcon approval

Maintaining API docs with markdown(mkdocs) is harder and not necessary.

I have just found this old issue mkdocs/mkdocs#20 which did not pleased me...

Generating directly from docstrings is enough for API docs, which can be achieved perfectly with sphinx.

I know, I have created a few projects with Sphinx and also you can generate pdf documentation attached with each release...

Also we could adopt pytorch_sphinx_theme, which has similar UI with PyTorch docs and thus more friendly to the community.

Sounds good to me...

Meanwhile, as I've mentioned several times before, the current doc is more like a Cookbook, and maintaining both mkdocs and docstrings is a heavy burden.

We do not remove the cookbook completely, it can stay there until all need sections move to docstrings...

About implementations, we can setup sphinx in some folder, while leaving the current mkdocs as it is. In this case there won't be much conflict.

the MkDocs is mainly just a config :]

It would be nice if we maintain some guide for newcomers in mkdocs, while we maintain detailed api docs in sphinx. What do you think?

I think that we would develop in a PR

[neggert]

I don't see any reason to maintain both Sphinx and mkdocs in the long-term. Sphinx can read markdown (supposedly, I haven't tried it). We might even be able to load the current docs into Sphinx as an intermediate step.

I'm in favor of transitioning to Sphinx, mainly for its ability to auto-generate API docs, but going through and fixing all of our docstrings is going to be a significant undertaking. To be clear, I think fixing docstrings is absolutely something we should do anyway, but let's not pretend this will be an easy switch.

[Borda]

I don't see any reason to maintain both Sphinx and mkdocs in the long-term.

For sure we would stay just with one... I meant that the MD files in MkDOcs can be simply used in Sphinx

Sphinx can read markdown (supposedly, I haven't tried it).

yes, it can handle even ipython notebooks (I have tested)

We might even be able to load the current docs into Sphinx as an intermediate step.

yes

I think fixing docstrings is absolutely something we should do anyway...

I do agree

[Borda]

• WIP - https://github.com/Borda/pytorch-lightning/tree/docs
• generated https://torch-lightning.readthedocs.io/en/docs [private to not confuse users]

[neggert]

As we're getting into this, we'll need to make a decision about docstring styles. It looks like what docstrings we have make some attempt at following the Sphinx native format, while PyTorch uses Google-style docstrings. I'm partial to the Google-style docstrings, as they're more human-readable.

[Borda]

I was using similar to Sphinx, but a bit more compressed

"""[Summary]

:param [ParamType] [ParamName]: [ParamDescription], defaults to [DefaultParamVal]
...
:raises [ErrorType]: [ErrorDescription]
...
:return [ReturnType]: [ReturnDescription]
"""