api: automatic generation

[daavoo]

Extracting new issue from #2770 , as the Python API Reference (kind of) is a separate topic from CLI.

... for Python API, we could use the docstrings content and it could be considered a way of "enforcing" detailed and up-to-date docstrings on the core repo, which are very helpful when using the API inside code editors.

Originally posted by @daavoo in #2770 (comment)

[daavoo]

We might begin to expand DVC Python API so debating this becomes more relevant.

As a potential example of a complete docstring see iterative/dvc@6180d28 .

So, @iterative/docs :

• what are the main points for duplicating content (or worse, the current asymmetric state), when limiting the scope to Python API)?

• How could this be automated (from code docstrings to docs website)

• If the problem of automation is the cost of infrastructure changes, could the Python API Reference live on a separate site (i.e. hosted in github pages associated with dvc repo) ?

IMO, long docstrings are not an issue in Python world so points regarding CLI in (#2770 (comment)) don't really outweigh the cost of manual sync efforts.

Complete docstrings make a more pleasant experience for API users (i.e. IDE integrations) and match the status of the ecosystem/

[shcheklein]

I think there was initial discussion and the conclusion that we didn't want to have a very comprehensive API descriptions (the way you made in that commit). If now consider including everything docs need, it's a different story.

Then we will have the first time situation where we'll have some content rendered on the website that comes from the doc strings. Totally fine, but workflow will become a bit more complicated. At the very least I'm not sure if it will be possible to create preview environment for this, will it require first updating the main repo + running some scripts to do MD in the second one.