feature: Autogenerate man pages

[Alizter]

We add a script to autogenerate man page rules which can be called using:

./dune.exe build @gen-man-rules

When these rules are present, @check, @default and @doc will all update the man pages if there are any changes. Currently we do not delete redundant files.

The motivation here is twofold:

1. We can have command line documentation present in the repo without the need for users to compile the executables.
2. We can observe better what the user comes across.

When all the files are out in the open, it is easy to see we have some nonsensical documentation. Some Dune commands are talking about options that don't apply to them etc.

cc @snowleopard who suggested something like this.

The patch for cmdliner: ocaml-dune/cmdliner#2

Notes:

1. I had to modify cmdliner in order to crawl over the subcommands. cc @emillon does this patch look OK/maintainable? It will probably never make it upstream without good reason.
2. In the future if we have generated includes we can skip the manual rule generation. I've used directory targets instead now.
3. I had to move things around so that the top Dune cmdliner command became visible in a library. Is this OK to do or should something else be done? I have used the executables stanza and not exposed any new library now.

[rgrinberg]

I had to modify cmdliner in order to crawl over the subcommands. cc @emillon does this patch look OK/maintainable? It will probably never make it upstream without good reason.

you need to make a corresponding PR to our cmdliner fork

In the future if we have generated includes we can skip the manual rule generation.

Is this needed? Can't you use directory targets for now?

I had to move things around so that the top Dune cmdliner command became visible in a library. Is this OK to do or should something else be done?

It's not necessary. You can define multiple executables with the executables stanza.

[rgrinberg]

why do we need this change?

[Alizter]

I think bootstrapping did this automatically.

[christinerose]

There’s more for me to go through here, but before I do, I’d like clarification on my questions (so I don’t end up suggesting a ton of things you have to reject because I misunderstood!)

Also, how do you write monospace in .1 files?

[Alizter]

@christinerose Thanks for taking a look at this, however I don't think it will be too useful to review the documentation here. The documentation here is just added to the repository and is what the user will see when they do dune some command --help. We should definitely improve the documentation but that should be in a future PR. The goal of this PR is to just expose it.

[christinerose]

@christinerose Thanks for taking a look at this, however I don't think it will be too useful to review the documentation here. The documentation here is just added to the repository and is what the user will see when they do dune some command --help. We should definitely improve the documentation but that should be in a future PR. The goal of this PR is to just expose it.

Ah! Thanks for letting me know. I was pinged to review as code owner automatically, I guess. It was in my notifications, so I thought it was necessary! Thanks for clarifying.

(I’m glad I didn’t go through all 55 pages then!)

[emillon]

I'll make a more complete review today, but before I dig into the implementation: we're already doing something like that to install man pages, see doc/dune.inc. I don't think you're reusing that mechanism. You should be able to turn these rules into (mode promote) and get them in the source tree "for free".

[Alizter]

@emillon I see that there is a similar script for the installed versions. Two problems:

1. These are not the plain man pages but rather the groff ones.
2. The script is not as extensive as what I am doing currently. (In fact there is even a CR there talking about what I have done).

(mode promote) and get them in the source tree "for free".

I know about this and I had it originally however it doesn't seem right for us. We don't want the runtest alias to actually promote anything and that should really be left up to the user. For this reason, I am investigating adding diff and promotion for directory targets to allow us to promote them separately.

[snowleopard]

Is it necessary to move code around? We have some internal patches (like adding new subcommands) that will require some extra work. If we can avoid moving the code that would be helpful.

[Alizter]

Unfortunately this is the only way I could think of. The entry point for Dune is the same as the cmdliner definition which means the cmdliner cmd cannot be shared with other executables without placing it in its own file.

[snowleopard]

I see, thanks. This is certainly not a blocker, just an extra cost of the current implementation to keep in mind (perhaps, when comparing with alternatives).

[Alizter]

Needs diffing of directories. Had a go at implementing but ran out of time/motivation. Closing for now.