Wish: Custom sections

[nritsche]

I would like to be able to define custom sections so they don't create warnings when I use them.

This would be very helpful in our project: https://github.com/chime-experiment/dias

for example here

[jnothman]

Would you want to configure how these were treated/displayed?

[nritsche]

Just displaying them somehow would be a good start, I think.

[fkiraly]

Any update on this?

I think numpydoc is great, but it also has a serious drawback when documenting class/object methods or impure functions (with side effects), since there seems to be no clean way to document assumptions on the context and effects on it.

As a very common example, suppose a class/object method that assumes presence of an attribute self.a and writes to an attribute self.b. This needs to be documented by doc standards 101, but there's no clear place where it goes.
Notes? That's for explaining content, algorithm, math. Parameters? self.a is not really a parameter. Description? That should be high-level, not detail documentation regarding context, state, side effects. And so on...

[j9ac9k]

Just going to add that this is a wish list item for pyqtgraph.

In addition to fields like Parameters, we want to add another section, Signals (see example of Qt signals).

Presently this can be done with the napoleon extension with the following bit in conf.py:

napoleon_custom_sections = [("Signals", "params_style")]

Here is the bit in the docs about the functionality:

https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html#confval-napoleon_custom_sections

[stefanv]

That sounds like a reasonable feature to add.

[willingc]

Hi @stefanv. I have found that you can embed a custom section under one of the top level sections by dropping the header level down by one. For example, in a napari docstring, we were getting errors for having a top level section for Events. during Sphinx build. By reducing the level of Events, we were able to have a custom section under Parameters:

class LayerList(SelectableEventedList[Layer]):
    """List-like layer collection with built-in reordering and callback hooks.

    Parameters
    ----------
    data : iterable
        Iterable of napari.layer.Layer

    Events
    ~~~~~~
    inserting : (index: int)
        emitted before an item is inserted at ``index``
    inserted : (index: int, value: T)
        emitted after ``value`` is inserted at ``index``
    removing : (index: int)
        emitted before an item is removed at ``index``
    removed : (index: int, value: T)
        emitted after ``value`` is removed at ``index``
    moving : (index: int, new_index: int)
        emitted before an item is moved from ``index`` to ``new_index``
    moved : (index: int, new_index: int, value: T)
        emitted after ``value`` is moved from ``index`` to ``new_index``
    changed : (index: int, old_value: T, value: T)
        emitted when item at ``index`` is changed from ``old_value`` to
        ``value``
    changed <OVERLOAD> : (index: slice, old_value: List[_T], value: List[_T])
        emitted when items at ``index``es are changed from ``old_value`` to
        ``value``
    reordered : (value: self)
        emitted when the list is reordered (eg. moved/reversed).
    selection.events.changed : (added: Set[_T], removed: Set[_T])
        emitted when the set changes, includes item(s) that have been added
        and/or removed from the set.
    selection.events.active : (value: _T)
        emitted when the current item has changed.
    selection.events._current : (value: _T)
        emitted when the current item has changed. (Private event)

    Notes
    -----

    Note that ``changed`` events are only emitted when an element of the
    list changes, *not* when the list itself changes (for example when items
    are added or removed). For example, ``layerlist.append(layer)`` will emit
    an ``inserted`` event. ``layerlist[idx] = layer`` *will* emit a ``changed``
    event.

    However, the layerlist does not have a way of detecting when an object in
    the list is modified in-place. Therefore, although
    ``layerlist[idx].scale = [2, 1, 1]`` changes the *value* of the layer at
    position ``idx``, a ``changed`` event will not be emitted.

    Examples
    --------

Not a full solution to this issue but reducing the heading level solved our issue and hopefully may help someone else.

h/t to @melissawm for encouraging me to post this here.

[stefanv]

Thanks for that, @willingc! I'm a bit nervous to recommend that as default practice, since it's not in the standard, so may be inadvertently removed in the future. Although, that's probably less likely now that you've mentioned it as a workaround ;)

We should implement an "official" way to support custom headings. I think a start could be: raise errors/warnings on custom sections, which can be switched off. If switched off, custom headings render as usual. That way we have a "strict" section format a "loose" section format for those who need it.

[j9ac9k]

I started working on this, but I'm a bit stuck on how to implement this from an overview perspective without making significant changes.

The main issue I've come across, is that the natural place for users to set the custom sections would be in the config file (pyproject.toml or setup.cfg). The wrinkle becomes that the config only goes as far as DocstringVisitor._get_numpydoc_issues, where all the issues are identified by the call to validate.validate(). Inside the validate function, the specific issues are identified, but the config option is not available there; so I can't do something like ensure a specific section does not report as an error or something like that..

One thing I can do pre-emptively though is modify the list in validate.ALLOWED_SECTIONS since it's mutable, but what I can't do in that way is indicate how a certain section should be treated (should it be "return style" or "parameters style").

Happy to move this discussion elsewhere if this is not the appropriate place for that.

[willingc]

@j9ac9k I'm not sure how strong a need there is for levels or additional sections. My hack solved my problem and I hadn't run into it before. Maybe wait until folks make it a must-have vs nice-to-have

[j9ac9k]

@j9ac9k I'm not sure how strong a need there is for levels or additional sections. My hack solved my problem and I hadn't run into it before. Maybe wait until folks make it a must-have vs nice-to-have

This and PR 558 (intentionally not linking) are the two things stopping pyqtgraph from adopting numpydoc in place of sphinx's napoleon extension, which myself and some of the other maintainers would like to do. I have a hard time characterizing it as a "need", we're able to build docs just fine, but we do like the numpydoc rendered output more and would like to have that ourselves.

When working w/ existing codebases, especially with those I'm unfamiliar with, my strategy is to have as light of a touch as possible (to minimize likelihood of breakages for others) hence my post here. I'll try and think of some other work-arounds and submit a PR if I get something that works for my use-case; it's probably a lot easier to give feedback to a PR than a wall of text.