Some way to link comments to Julia constructs

[kk49]

Maybe a ##! comment before or on same line as a type/method/etc... get linked to type/method/etc...

These comments could then be printed from the command line. If something like the doxygen convention is followed, Julia could eventually be processed by doxygen.

[pao]

See also #720.

[JeffBezanson]

Nothing wrong with processing julia source with some external tool, but we're pretty sure we're not going to build anything into the language around this.

[kk49]

I agree, I not asking for a change to the language, but I would like a standard to follow, so that my code (and others) follows what the language developers are doing.

[StefanKarpinski]

For now, comments are just plain old comments — intended only for readers of the code. Library documentation lives in sphinx and the manual lives on the website. Maybe we'll get more integration going forward, but for now it's pretty fast and loose.

[JeffBezanson]

Check out https://github.com/lcw/jocco.

[stevengj]

Still need this. I agree that this probably shouldn't be part of the Julia language per se, but it would be good to:

• "Bless" some standard tool by using it in JuliaLang and in particular in the Julia standard library.
• Rather than roll our own, adopt a mature and popular tool (e.g. Doxygen) and submit patches to make it work better with Julia (both syntactically and also for unusual semantic features like multiple dispatch). (Note that we need a tool that supports LaTeX equations, in order to properly document the mathematical functionality that is Julia's initial focus.)
• Integrate that with the help documentation. (Currently, we have to document every function twice, which is unacceptable.)

The longer we continue with ad hoc documentation systems, the messier our situation becomes and the harder it will be to retrofit a more usable tool.

[StefanKarpinski]

I think there are two issues:

1. A mechanism for making the contents of comments associated with code available to the running program.
2. Interpretation of that data as some help system.

We should focus first on the mechanism and make that happen and then play around with various approaches to interpretation and presentation. The former is also something that arguably makes sense as part of the language, while the second is more of an ecosystem question.

[johnmyleswhite]

I'd be strongly in favor of adopting something based on Doxygen. I've been thinking about this issue a lot recently and always find myself coming back to the basic ideas in Doxygen.

For me, the most important thing (and something I've already started working on) is having a tool that will automatically generate skeletal documentation for existing code.

[stevengj]

@johnmyleswhite, can you hack together a prototype with Doxygen just so that we can get a sense of what it looks like? My main concern about Doxygen is that it seems oriented toward single dispatch, but perhaps we can massage it into something that works for us.

[johnmyleswhite]

Yes, I'll do that this week.

[stevengj]

The main competitor to Doxygen seems to be Sphinx. We are already using its RST formats, but we would need to roll our own way to extract it from the source.

[johnmyleswhite]

I think rolling our own way to extract formatted comments from source is not going to be that difficult: R did it for its roxygen2 package.

For me, the problem with our current use of Sphinx is that we tend to use it more for formatting than for structure. What I like about Doxygen is that it encourages you to write out a proper database about each function instead of hoping that you'll follow the right notational convention to indicate things like the return type signature.

[stevengj]

This quote about Doxygen is the kind of thing I would hope to avoid: The fourth school is the "DoxyGen" school, which lets a program collect a mindless list of hyperlinked variable, procedure, class and filenames, and call that "documentation".

At a minimum, we need to be able to include links to/from our narrative-style manual and the automatically generated function/type reference that we are hoping to generate from source comments, as well as grouping the latter into a shallow hierarchy of logical categories.

[lindahua]

I used Doxygen in some of my previous C++ projects. My feeling towards Doxygen is kind of mixed. It is convenient and works well for not-too-complicated projects/libraries. However, for complex libraries, what it generates is a convoluted cross reference network that is difficult to follow.

I feel that Sphinx tends to encourage people to write more organized documents with cleaner structure.

If we use a variant of Doxygen here, we may also have to consider how to handle codes that are generated by macros. It is not uncommon that we use macros to generate a large collection of functions (e.g. Distributions.jl).

[johnmyleswhite]

The existence of macros makes the use of in-code documentation a little more problematic. What are you thinking we should do Dahua?

[lindahua]

In such cases, we have to document those functions/types separately (Doxygen allows separated documentation). I don't see other solution at this point. (Let Doxygen to expand those macros seems to be asking too much).

[stevengj]

Of course, we could use both Doxygen and Sphinx at the same time...

[fperez]

Would some of you guys be willing to join in the IPython dev meeting on August 29 (or later if that date is bad)? We host public g+ hangouts at 10am PST and would like to go over the design requirements for this with you guys, so that we can better think what to do in IPython to cover Julia's needs.

[stevengj]

August 29 is bad for me.

[sglyon]

Just another thought...

For my Python projects (yes, I'm a Python dev browsing the Julia issue tracker) I use Sphinx and the numpydoc sphinx extension. It is a simple rst based set of paradigms that helps enforce the "database" type inline documentation mentioned by @johnmyleswhite while still allowing sphinx to handle the formatting/cross referencing ect.

[info: I was interested in this post because one of the things I find myself wanting most in julia is a way to replicate how docstrings can be queried for arbitrary built-in and user-written code from the interpreter. Specifically, I often use IPython's X? to browse docstrings and X?? to peek at a function's source]

[ivarne]

In the readline REPL you can you can use ?X to get help on standard library functions, edit(x) to see source, and X(\t to show the possible method signatures.

Currently the docstrings has to be written in a special file. Base auto generate helpstrings.jl from the documentation.

[mschauer]

A canonical and modular way extending the help() system to cover the code in provided packages would be great.

[ViralBShah]

#8588 does this.