docs: add command line full reference using sphinx-click

[lorenzo-cavazzi]

I adopted a solution similar to the one suggested in #3033 . The main difference is the click-generated docs being included in the ui.cli files to better position the content.

I (loosely) took inspiration from the Git documentation where every command has the following sections:

• description
• options
• command-specific section(s) [optional]
• examples [optional]

In our case, after the description, I added a "Commands and options" section, moving relevant explanations and examples at the end in a similar fashion.
If we like this approach, I would separate each command into a different page because the "Renku Command Line" page has grown way too big -- Git does the same.

P.S.
The repetition of the command as a title (just after "Commands and options") is quite annoying, but luckily it will soon be possible to remove it click-contrib/sphinx-click#100 -- we could speed up the process by forking the repo and finalizing that PR, or just waiting (I actually reviewed it since there are a couple of minor inconsistencies).

Here is an example of what we could simply remove with that PR (left is the current output with the additional renku project title). We could also remove the Project commands description.

P.P.S we discussed other approaches, but the content we wrote seems useful: it would be a pity to throw it away, and IMHO integrating sphinx-click led to a decent result.
Of course, I'm open to explore other solutions if you think the result is not as good as it should be.

fix #3033

[Panaetius]

I think separating into different pages makes a lot of sense. And this whole change makes the docs much more usable, thank you!

I wonder if we should make sub sub commands italic or differentiate them further? At least to me, renku config and remove look almost the same in regards to size:

Same with Options which is just 1.6px smaller than the name of the subcommand.
Looking at e.g. https://git-scm.com/docs/git-remote the sub-commands are in their own Commands section and italic. Not that we need to do it this way but it looks cleaner to me. And the different colors help readability a lot as well. Probably also goes well with splitting it to individual files.

Not really related, we should create an issue to investigate https://github.com/click-contrib/click-man to make docs even better. No need to open a browser.

[Panaetius]

Did you intend to remove the cheatsheet entry?

[lorenzo-cavazzi]

Ouch! Nope, that was not intended. I'll restore it

[Panaetius]

Wasn't there before either, but maybe we should write something along the lines of "Merge tool configuration cannot be shared through remote repositories and has to be set up on each clone separately" ? renku clone covers it but something to let the user know that if they do a normal clone, this will be missing.

[lorenzo-cavazzi]

Yes, that makes sense. Otherwise, users might believe that renku clone keeps everything aligned.
I'll add a sentence, feel free to edit it if you prefer different wording.

[Panaetius]

This command is for removing any file (not just ones belonging to a dataset). but if they are in a dataset, it takes care of not having inconsistent metadata.

[olevski]

@lorenzo-cavazzi @Panaetius I know this is more work but how about adding a specific section for each subcommand. Like this:

I did the following to achieve this:

Description
~~~~~~~~~~~

Create, edit and manage the datasets in your Renku project.

This is a core feature of Renku. You might want to go through the examples
listed below to get an idea of how you can create, import, and edit datasets.


``add``
~~~~~~~

.. click:: renku.ui.cli.dataset:add
   :prog: renku dataset add
   :nested: full

``create``
~~~~~~~~~~

.. click:: renku.ui.cli.dataset:create
   :prog: renku dataset create
   :nested: full


Examples
~~~~~~~~

This requires more work but it makes the docs even nicer and much easier to navigate IMO. If we have one section for all subcommands then it is very hard to see where one command ends and another begins. Also it is very hard to understand what subcommands are available.

I think this is def worth doing. Either here or in a followup PR. What do you think?

[Panaetius]

@lorenzo-cavazzi @Panaetius I know this is more work but how about adding a specific section for each subcommand. Like this:

I did the following to achieve this:

Description
~~~~~~~~~~~

Create, edit and manage the datasets in your Renku project.

This is a core feature of Renku. You might want to go through the examples
listed below to get an idea of how you can create, import, and edit datasets.


``add``
~~~~~~~

.. click:: renku.ui.cli.dataset:add
   :prog: renku dataset add
   :nested: full

``create``
~~~~~~~~~~

.. click:: renku.ui.cli.dataset:create
   :prog: renku dataset create
   :nested: full


Examples
~~~~~~~~

This requires more work but it makes the docs even nicer and much easier to navigate IMO. If we have one section for all subcommands then it is very hard to see where one command ends and another begins. Also it is very hard to understand what subcommands are available.

I think this is def worth doing. Either here or in a followup PR. What do you think?

That does look nicer, yes. Though I still think using colors/style could improve it even further. Not sure how easy that'd be to do, might need updating in https://github.com/SwissDataScienceCenter/renku-sphinx-theme

[lorenzo-cavazzi]

Thanks for the suggestions!

I agree with @Panaetius that following more closely how git differentiates sub-commands would be nice. And using different colors/styles might work as well, but that requires changing the theme that is shared with the main renku docs, which doesn't sounds like a good idea. Considering we don't have CSS classes to differentiate between the sub-commands elements and all the other <h5> elements in the docs, keeping everything on a single page would simplify applying changes only there without the risk of messing up later.

I also like the suggestion from @olevski to improve navigability.

I would try the following:

• Split the commands into separate pages
• Apply the suggestion from @olevski on sub-commands

Then we can decide on different formatting if we feel it's still needed.

We can quickly chat about this during tomorrow's standup so we all agree before changing the code.

[lorenzo-cavazzi]

P.S: sorry for force pushing. I messed up badly when merging develop back 🤦

[lorenzo-cavazzi]

Each command now has its own separate page.
I tried a few combinations based on your suggestions.

In the end, I think using @olevski approach to separate the single commands might be too much when there are many sub-commands. You can check it on the renku dataset command

Otoh the final output when using a different styling (closer to the Git documentation) as suggested by @Panaetius looks good to me. You can check it on the renku template command.
P.S. further personalizing some elements is really easy using the override-them.css.

[Panaetius]

I like the second version with italics 👍

[lorenzo-cavazzi]

Ok great, I changed all the commands to adopt the italics+tabbed style and dropped the links to the subcommands.

The following is an example of how the dataset page changed compared to the previous screenshot.

[lokijuhy]

This looks awesome! I agree having each command on its own page makes it easier to navigate.

One potential suggestion: I could see it being useful to users to have a list of all the options for a command, without having to scroll the length of the page and find each heading to see what's available. I see that it's a bit much in the sidebar, but what about a table-of-contents-esque entry at the top of the "Commands and options" section, where the reader can see all commands available, and click to scroll down to the one they want? (add, create, edit, ...)

[lorenzo-cavazzi]

Do you mean something like this? (ignore the style, I just made it on the fly, image it styled as a proper "table of contents")

[lokijuhy]

Yes, precisely!

[lorenzo-cavazzi]

Ok great, I agree it would be useful.

The anchors get automatically generated by the library, which is a good start. Since the commands and options documentation is automatically created, I feel we should automatize the table-of-contents creation as well -- in a similar fashion to what we do with the cheat sheet. Doing it manually seems more error-prone.

That might require a bit of time though, and I have no capacity for this (and the following) sprint on renku-python. Would it make sense to create a follow-up issue so we don't keep this pending for too long?

[lokijuhy]

Yes, that sounds like a good plan!

[lokijuhy]

📚 ✨

[lorenzo-cavazzi]

Here is the follow-up issue: #3086

[rokroskar]

This PR broke the renku docs build, btw. The only reason we didn't notice was because the current submodule still points to a release from June...