Support brief and extended docs (closes #25930)

[timholy]

Demo:

julia> """
           f()
           
       Do something cool with `f`.

       # Extended help

       The mathematics behind `f` is, like, totally intense: `∅`.
       """
       f() = nothing
f

help?> f
search: f fd for fma fld fld1 fill fdio frexp foldr foldl flush floor float

  f()

  Do something cool with f.

  ────────────────────────────────────────────────────────────────────────────

Extended help is available with `??`

help?> ?f
search: f fd for fma fld fld1 fill fdio frexp foldr foldl flush floor float

  f()

  Do something cool with f.

  Extended help
  ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡

  The mathematics behind f is, like, totally intense: ∅.

julia> 

I've checked that this doesn't seem to break Juno. I've not tested jupyter.

[timholy]

I've added the docs and NEWS, so this is ready for any feedback.

[johnnychen94]

A limitation of this feature is that REPL users still can't read those lengthy extended docstrings. I imagine some paging utils (e.g., more) in doc mode would be more useful.

[timholy]

A limitation of this feature is that REPL users still can't read those lengthy extended docstrings. I imagine some paging utils (e.g., more) in doc mode would be more useful.

I think that's mostly an orthogonal PR. Currently help is displayed via the show method for markdown objects. One might have to modify the REPL-loop itself for help mode to be able to switch to using a pager.

[johnnychen94]

Oh, what I intended to say is once there was a pager, the feature proposed in this PR would be more or less useless. IPython uses ?? to get the source codes, I suggest we reserve this "keyword" for that usage.

[timholy]

That's an idea worth exploring. Would we use the pager to handle multiple methods? I.e., if you do ?findall, rather than seeing the horizontal rule divider you just get separate pages, perhaps with an "index" at the beginning to choose the method you want? I've noticed a bit of a readability problem when there are many methods, even when their help is short, so a pager might fix this.

A downside of the pager is that it might be slightly irritating to have to exit the pager each time you do ?function_with_long_help and all you're looking for is a quick API summary.

IPython uses ?? to get the source codes, I suggest we reserve this "keyword" for that usage.

We already have @less and @edit for this. To me it seems that IPython's usage of this is just a way of giving up, "I can't imagine providing any better help than just showing you the source." Personally, I don't think that's a good direction to go in, and that it would be better to say, "ok, you want more help, let me direct you to this more detailed document."

[johnnychen94]

Would we use the pager to handle multiple methods?

I think that makes reading docstrings easier.

A downside of the pager is that it might be slightly irritating to have to exit the pager each time you do ?function_with_long_help and all you're looking for is a quick API summary.

Is something like ?function_with_long_help |> pager achievable?

[timholy]

Is something like ?function_with_long_help |> pager achievable?

Currently that retrieves the documentation for |>. I see the point, but how do you know you "want" the pager until you know whether the documentation will be long? It seems that's something the documentation author needs to figure out, and that the user doesn't (yet) have enough information.

[timholy]

If someone comes along later and adds pager support, will we regret anything here? I think the real issue is whether we're happy with # Extended help being given special meaning.

Would be nice to get a code review, maybe @mortenpi? But if no one has time I'll merge this in the next few days.

[timholy]

I've moved all the functionality to the REPL. I agree, I think this makes more sense.

[fredrikekre]

This is great! Some thoughts:

1. For the case with two methods, but just one of them have extended docs:

       foo(x)
   
   Short help for `foo(x)`.
   
   # Extended help
   
   Extended help for `foo(x)`.
   """
   foo(x) = x
   
   """
       foo(x, y)
   
   Short help for `foo(x, y)`.
   """
   foo(x, y) = x + y
   

   the Extended help is available with ?? is shown below all docstrings:

   help?> foo
   
     foo(x)
   
     Short help for foo(x).
   
     ───────────────────────────────
   
     foo(x, y)
   
     Short help for foo(x, y).
   
     ───────────────────────────────
   
   Extended help is available with `??`
   

   Should it be attached just to the methods that actually has extended docs? I.e. something like:

   help?> foo
   
     foo(x)
   
     Short help for foo(x).
   
   
     Extended help is available with `??`
     ───────────────────────────────
   
     foo(x, y)
   
     Short help for foo(x, y).
   

2. Can we make ?? (literally) work? That way you dont have to type in ?functionname again (or up arrow, home, ?, enter). Should be pretty easy I think by just saving if the last show docstring had extended docs, and if so also save the method. It also matches what is printed, that extended docs are available with ??. (Alternatively we could print Extended docs is available with ??foo`.)

3. Should it be called Extended documentation (Extended docs?) instead?

4. Since this is now included in the REPL code, it would be nice to change the prompt to help??>.

Doesn't have to be adressed in this PR, just some thoughts I had while trying it out.

[timholy]

Extended help is available with ?? is shown below all docstrings

That was deliberate, in case there was a function with n methods and all of them have extended help I was worried it would be annoying to show it each time. It's actually easier to do the thing you suggested, unless we also follow your suggestion about "...available with ??foo" in which case we'd have to come up with a suggestion for invoking each one specifically. That could become awkward if it turns out the type you'd need to specify for foo(::T) spanned many lines (e.g., #14946). Since I think being more explicit about how to get longer help is the more important of these suggestions, I decided to leave it so that the message is printed at the end of however many methods are shown.

Can we make ?? (literally) work?

I like that idea, but a cautionary note:

help?> ?
search:

  Welcome to Julia 1.3.2-pre.0. The full manual is available at

  https://docs.julialang.org/

  as well as many great tutorials and learning resources:

  https://julialang.org/learning/

  For help on a specific function or macro, type ? followed by its name, e.g.
  ?cos, or ?@time, and press enter. Type ; to enter shell mode, ] to enter
  package mode.

I'm a little uneasy giving this special meaning just because it happened to follow a case where there was extended help available.

Alternatively we could print Extended docs is available with ??foo

I went with that suggestion, good idea.

Should it be called Extended documentation (Extended docs?) instead?

Good idea, I've allowed all three.

it would be nice to change the prompt to help??>.

I worry this is more subtle than it sounds. What happens when you hit Ctrl-C at the beginning of the line---does it drop back to non-extended help or the julia prompt? (If we just changed the prompt it would drop back to Julia mode.) What happens when you're paging back through your history, does it remember to access it in extended vs non-extended mode? Really I think the only way to do it right is to add a whole new REPL mode. By requiring the ? at the beginning of the line we get almost all the benefits for one extra character.

[fredrikekre]

Extended help is available with ?? is shown below all docstrings

That was deliberate ....

Yea, and I agree the current approach is less spammy.

Can we make ?? (literally) work?

I like that idea, but a cautionary note: ...

But that is just a default fallback, you get the same from just "Enter" at the empty help prompt.

[timholy]

Thanks for posting that fantastic article. I'm all in favor of formalizing this a bit more. In the context of the blog post, it seems that my vision of # Extended help (which was prompted by docstrings like the one linked in #25930) was mostly about the "explanation" category. The typical Julia convention regarding docstrings seems to start with the "reference" material. In simple cases this seems appropriate, but for methods that have a lot of arguments I could see merit in having a brief summary appear at the top and the full story in # Extended help under a ## Reference section.

In the context of docstrings for individual methods (I'm specifically not talking about whole package ecosystems), the line between "tutorial" and "how to" often seems blurry. I think this is specifically because of the ethic that functions like those in Base should generally do one specific small thing, and so one or two examples pretty much shows you what you need to know. When functions start taking a dozen arguments, then "tutorial" and "how to" take different forms.

[c42f]

Yes, docstrings tend to start with a terse reference section, and that's often all I want to see. Sometimes I look up the help for, say, CartesianIndices (just to pick on one example) and there's all this great detail and examples, but the usage summary which really matters to me has already scrolled off the top of the terminal!

For sure there's unlikely to be something recognizable as a full tutorial or how-to in a docstring. Those tend to need a bigger story which will thread together use of many different functions. Though I wonder whether there's some value in being able to extract the example sections from docstrings to insert into a tutorial in the main docs. It might be tricky because it increases the coupling between the separate docs and the docstrings, but it would make for a super consistent set of examples between related functions.

Anyway, so you're imagining that we could have subsections under extended help? Something like the following perhaps?

<terse API reference>

<short examples>

# Extended help
## Reference

<Reference material for large APIs like CSV.FIle>

## Explanation

<Algorithm description, design rational, academic references>

## Examples

<More extended tutorial-like examples>

Say we had all this structure; what could the system do with it by default to really brings a lot of value for users?

[timholy]

The ?? is clearly a bit too limited to extract those subsections. What about ?r? (reference), ?d? (design=explanation), and ?x? (examples)? So for example ?r?CSV.File would give the full details of the API. ?? would continue to give all of the subsections.

[StefanKarpinski]

I really feel like "short" and "long" modes are all that anyone will remember. Need a quick refresher? Use ?. Need to see all the gory details? Use ?? and get the full docs in a pager.

[tkf]

For sure there's unlikely to be something recognizable as a full tutorial or how-to in a docstring. Those tend to need a bigger story which will thread together use of many different functions.

I think it'd be better to keep the main focus of docstrings to API references and link materials of the other three quadrants just before # Extended help. As you said, tutorials and how-to guides are hard to do with a single function or type. Also, the inclusion of examples does not necessarily mean it's a tutorial or how-to guide. IIRC he was mentioning in the talk that examples for clarifying API is a good fit for API reference too (and I agree). I think a better approach would be to make REPL help?> somehow aware of docs/ so that you can read the tutorials, how-to guides, and discussions in it. (It's a bit tangential but this way you can write tutorials and how-to guides in markdown. I think it's a better file format for writing non-trivial texts.)

[c42f]

I really feel like "short" and "long" modes are all that anyone will remember.

Agreed; so far been unable to think of a compelling use for extra structure in the docstrings, at least from a REPL point of view.

I'm starting to think that the simple answer is that docstrings should be reference documentation only. This is natural for their location interspersed with the source code, where developers only want to see a terse but accurate and complete description.

Aha! And on this point, I finally know why the content of #33936 bothers me. It's tutorial content, but we don't have a place for tutorials.

Also, the inclusion of examples does not necessarily mean it's a tutorial or how-to guide.

Very true.

I think a better approach would be to make REPL help?> somehow aware of docs/ so that you can read the tutorials, how-to guides, and discussions in it.

Great idea! If we could allow easy cross references from the docstrings to the tutorial and explanation sections of the manual, I think we would solve our problem. One thing I've worried about in the past was that the source code is not linked to the explanation sections of the Julia manual. So anyone who either

• reads the source only
• looks at docstrings only with help> mode

is missing out on a great deal of important content.

[tkf]

So anyone who either

• reads the source only
• looks at docstrings only with help> mode

is missing out on a great deal of important content.

I think linking tutorials etc. from the docstring somewhat helps the second kind of readers. Ideally we can use TerminalMenus or something to quickly jump to the relevant section. (Though at this point it feels like we are re-inventing web browser.) But not sure how to help the first kind of readers. Maybe put URL to the relevant section in the documentation?

[c42f]

But not sure how to help the first kind of readers. Maybe put URL to the relevant section in the documentation?

I feel like just the reference in the docstring is enough as that sits next to the source code. It doesn't need to be super convenient to access, but merely a reminder that there's more content to look for. (Personally I often read the source before looking at the docs, but I assume we don't need to optimize for this case :-) )

Baking in URLs isn't ideal as they refer to a particular hosting location. On that note: it would be cool if @refs in docstrings could resolve to https://pkg.julialang.org/docs/... from the REPL and especially if there was a way for @refs of dev'd packages to resolve to localhost.

[tkf]

Baking in URLs isn't ideal as they refer to a particular hosting location.

For docstrings, I totally agree. I was thinking more about lengthy inline comments that explain something about surrounding code blocks. It may be useful to link to a discussion/explanation section of the documentation. It would be certainly nice if this link is also re-locatable. But I think it's OK in this case because the audience of the inline comments is very narrow. (Alternatively it makes sense to use relative file paths.)

[c42f]

It may be useful to link to a discussion/explanation section of the documentation.

Agreed. Actually I made an issue about this very thing a while ago, but it didn't generate much discussion and I kind of forgot about it #29527.

[StefanKarpinski]

Maybe we could have some way of indicating in a docstring which sections of the docstring should be included in the short versus long docs?

[c42f]

which sections of the docstring should be included in the short versus long docs

Do you mean something like # Extended docs, but instead of being just two parts we'd allow long vs short to be interspersed?

One option would be to extend the title convention: for example any heading starting with Extended could go into the long docs. This would be super simple and it's appealing to have a pure markdown approach.

Another option would be to consider extended a kind of metadata which you don't want to render directly. We already have this kind of thing in the Documenter @id tag. Adding a @class annotation like # (Some Heading)[@class Extended] would seem to be a natural extension?

[StefanKarpinski]

Yes, that's what I meant. It does seem like having them interspersed in the long version might be better and the convention of the name starting with "Extended" is pretty clear.

[nalimilan]

I had completely missed this discussion. Very nice! Looking at changes that are needed to use this in DataFrames (JuliaData/DataFrames.jl#2250), I wonder whether we shouldn't treat content appearing after any heading as extended help.

In DataFrames, 90% of the time the PR just adds # Extended help right above # Arguments or # Examples. Doing this automatically would ensure some consistency and enable the feature automatically for most packages

[timholy]

A lot of docstrings look like this, I worry that without the example it will become much less clear.

[rfourquet]

I worry that without the example it will become much less clear.

I agree, and even sometimes examples are part of the specification (but that might be a documentation bug...), e.g. in which order pushfirst!(v, 1, 2, 3) inserts 1, 2, 3 is explicited only in the example.

[nalimilan]

I disagree. The short description should be improved if it's not clear/detailed enough to explain the general behavior of the function without looking at examples (even if examples are great to help understanding). In the findfirst example, we should arguably mention that nothing is returned when no occurrence is found, rather than merely saying it in a comment in the examples section.

EDIT: More generally, I think we need to add precise guidelines to the manual regarding what should be in the short vs. extended help. People may have quite varied expectations and inconsistency risk making this feature counter-productive for users.

[mgkuhn]

English may be the common language for open-source packages, but I'd expect lots of in-house packages to have docstrings in other languages. So baking in a special meaning for the English phrase “Extended help” seems less than ideal. (Also, I prefer not to refer to documentation as “help”: to me “help” is what you offer to people who haven't read the documentation.) It would be nice to be able to start the extended docstring with arbitrary headings, such as “Details”, “Examples”, “Application hints”, “Further information”, “Gory details”, “Full specification”, “Background”, “Compatibility concerns”, “Performance tips”, etc.

@nalimilan's suggestion to simply treat the first heading as the start of extended help sounds simple and practical. That convention works well e.g. for Wikipedia/Mediawiki, where the convention is that there should be a self-contained short lead section of the article before the first heading. If that cuts off examples, perhaps that's a good thing if it encourages authors to make the first section so clear that “Examples” sections really become what they should be: optional additional information.

[timholy]

Your point is a good one but I disagree that we can suppress the current # Examples section of many docstrings without significant loss of clarity, or even that it's possible to write such a clear description that you don't need it.

We might need some other syntactic idea here, but at the moment nothing comes to mind.

[mgkuhn]

Many Markdown applications allow the use of HTML tags in Markdown-formatted text, so HTML could be a source of suitable syntax for an invisible division marker.

One good example is the “more tag” that WordPress uses to separate the introductory part of a blog post (which goes in the front page) from the rest of the full blog post:

<!--more-->

or with an added continuation text:

<!--more Click here for full details-->

That's a stylized SGML/HTML comment section.

Other ideas from HTML/SGML syntax would be adding an element such as <more/> or <more>Click here for full details</more>, or an entity reference such as &more;.

Incidentally, Github's Markdown renders these as

• <!--more--> =
• <more/> =
• <more>Click here for full details</more> = Click here for full details
• &more; = &more;

Also:
In any generated HTML, you want the separation point to be addressable, such that a “Click here for full details” link points the reader at exactly the line where the text continues (like WordPress does). That means the formatter will have to add an id attribute to the next HTML element, as in

<h2 id="more123">Extended help<h2>

However, we can't have the docstring author insert an id like that manually: it needs to be unique on a HTML page that may collect several docstrings, so Documenter.jl needs to put that in.