doxygen + markdown to document C/C++ code

[ScottPJones]

This has already been done in JuliaLang/utf8proc, and can be of great help in understanding the different C/C++ modules used to build the core of Julia.
This has already been discussed in #11107, with the suggestion to open a new issue.

[tkelman]

I think there's a pretty strong need to get better about documenting the internals.

Now, to argue the technical merits of the specific approach we use, if we want to do this - it would be quite helpful (but maybe wishful thinking?) if the same system could be used for the C/C++ parts of the runtime as for the Julia parts of the standard library (@doc has some modularity issues for documenting base as you've noted elsewhere).

[StefanKarpinski]

@ScottPJones, it's good practice to keep titles short since they are clipped in various interfaces, including GitHub's issue listings and email subjects. Additional explanation can go into the issue body.

[ScottPJones]

@StefanKarpinski Thanks again for the helpful advice!

[ScottPJones]

@tkelman I've used doxygen for C/C++ code for about 10 years, and found it very useful. Where I'm currently consulting, we've also standardized on doxygen, and are using it for C, C++, Java, JS, Python, and Julia code (currently, in Julia it's just for human eyes, I think there's work to do to on a filter for doxygen to understand Julia, just like Python used to need...). So I think it definitely could be used for all of the code in Julia, and packages written in Julia... (including Base if the issue of making @doc just save the information for later Markdown processing is solved... for now I just wrapped #= =# around the documentation... There is also even a filter for MatLab ;-)

[StefanKarpinski]

Seems like once the doc string format settles down we could teach doxygen to parse it and then use doxygen for everything.

[garrison]

+1. Having better-documented internals will have countless benefits. In particular, I expect that documenting the ownership of various pointers/handles will improve (or at least help us better assess) memory leaks going forward.

[tkelman]

This came up again in #11573.
I'm very meh on doxygen specifically. If we go that way, it looks like there is a choice between annotating with @ markers and annotating with \ markers (or something else?), and I think the latter makes much more sense for Julia code since otherwise there's too much potential for confusion with Julia macros.

[ScottPJones]

As I said in #11573, we'd probably want to pick another character besides @, that could be used inside doc strings... that could simply be replaced by @ before passing on to doxygen processing.
@tkelman Have you actually used doxygen? It helps a lot when dealing with very large projects, with lots of dependencies, and it handles a lot of different languages...

[tkelman]

As a user of libraries that themselves use doxygen, yes. http://www.coin-or.org/Ipopt/doxygen/namespaceIpopt.html

It's an old-fashioned ugly giant disorganized dump of API listings and class diagrams that aren't particularly useful, from what I've seen. Maybe there are more modern better-looking themes for it, or better ways of structuring its output, but I've never really seen it done in a way that I found useful for learning how to use or how to hack on a library.

[ScottPJones]

I use it for C really, so I don't deal with the class diagrams.
To me, it is very useful because you can very quickly find the information on things like the enumerations or defines used with a library, see how a struct is defined, find internal documentation quickly, see what calls what...
It may not be perfect, but I think it is way better than not having any documentation of the internals...