nosetesting modules for adherance to numpydoc docstring standards?

[pv]

Copied from numpy/numpy#2778

---

@rmcgibbo

I've always thought it would be cool to automatically to automatically validate (at least in a minimal way) my docstrings during build testing. (Because of issues like this, its not going to be perfect, but whatever.)

Well, I went ahead and did it. I'm wondering if there's any interest in the numpy community for adding this feature into the numpydoc codebase.

With code like this in the test suite:

# test.py
from numpydoc import DocStringFormatTester
import my_module_to_be_tested
TestDocstrings = DocStringFormatTester(my_module_to_be_tested)
[... rest of your test suite ...]

you get behavior like:

$ nosetests
[... your regular test results ...]
NumpyDoc: mymodule.function1 ... ok
NumpyDoc: mymodule.function2 ... ok
NumpyDoc: mymodule.class1.method1 ... ok
NumpyDoc: mymodule.class1.method2 ... ok

The mechanism is just a class factory that builds classes with names like "Test_XXX" so that they get discovered by nose. Perhaps it could be done more cleverly with a nose extension.

If there's any interest in this (Perhaps it's outside the scope of numpy/numpydoc) I will write it up for a PR.

[rmcgibbo]

Thanks for moving this over, and the expression of interest. I will bump
this on the todo list, but it's not a super high priority for me, so I
wouldn't hold your breath on the PR. If anyone else is interested, I'd be
happy for them to run with this idea. The code that I use currently that
implements this idea is at
https://github.com/rmcgibbo/mdtraj/blob/master/MDTraj/testing/docstrings.py.

That code is ostensibly licensed under the LGPL, but I'm happy to relicense
under the simplified BSD if anyone want's to take it and run with it.

On Mon, Feb 17, 2014 at 1:19 AM, Pauli Virtanen notifications@github.comwrote:

Copied from numpy/numpy#2778 numpy/numpy#2778

@rmcgibbo https://github.com/rmcgibbo

I've always thought it would be cool to automatically to automatically
validate (at least in a minimal way) my docstrings during build testing.
(Because of issues like thishttp://www.voidspace.org.uk/python/weblog/arch_d7_2009_05_16.shtml#e1090,
its not going to be perfect, but whatever.)

Well, I went ahead and did it. I'm wondering if there's any interest in
the numpy community for adding this feature into the numpydoc codebase.

With code like this in the test suite:

test.py

from numpydoc import DocStringFormatTester
import my_module_to_be_tested
TestDocstrings = DocStringFormatTester(my_module_to_be_tested)
[... rest of your test suite ...]

you get behavior like:

$ nosetests
[... your regular test results ...]
NumpyDoc: mymodule.function1 ... ok
NumpyDoc: mymodule.function2 ... ok
NumpyDoc: mymodule.class1.method1 ... ok
NumpyDoc: mymodule.class1.method2 ... ok

The mechanism is just a class factory that builds classes with names like
"Test_XXX" so that they get discovered by nose. Perhaps it could be done
more cleverly with a nose extension.

If there's any interest in this (Perhaps it's outside the scope of
numpy/numpydoc) I will write it up for a PR.

Reply to this email directly or view it on GitHubhttps://github.com//issues/13
.

[larsoner]

I have some code to do something like this already. If there is still interest I can try to make a PR at some point.

[jorisvandenbossche]

@Eric89GXL I think that would be very interesting (I would like to use it for pandas). If you have a link to the code, I would gladly test it

[larsoner]

We originally developed it over in VisPy:

https://github.com/vispy/vispy/blob/master/vispy/util/tests/test_docstring_parameters.py

But I think the more mature version now lives in MNE-Python:

https://github.com/mne-tools/mne-python/blob/master/mne/tests/test_docstring_parameters.py

It could probably be made more general and robust, but it could at least get you going

[larsoner]

Wherever this is done, it should ideally not duplicate effort with PyCQA/pydocstyle#129

[pv]

C.f. also scipy/tools/refguide_check.py

30.8.2016 21.52 "Eric Larson" notifications@github.com kirjoitti:

Wherever this is done, it should ideally not duplicate effort with
PyCQA/pydocstyle#129 PyCQA/pydocstyle#129

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#13 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AACI5h28wY3cpejnU8FeGcvRyY9vFieRks5qlHeRgaJpZM4BibHd
.

[jnothman]

We also recently merged something along these lines over at scikit-learn/scikit-learn#9206.

The other thing I'd like to see along these lines is that we often want the documentation of different functions to have shared descriptions of common parameters. Something like:

def assert_consistent_docs(objects,
                           include_params=None, exclude_params=None,
                           include_attribs=None, exclude_attribs=None
                           include_returns=None, exclude_returns=None):
    """

    Checks if types and descriptions of parameters, etc, are identical across
    objects. ``object``s may either be ``NumpyDocString`` instances or objects
    (classes, functions, descriptors) with docstrings that can be parsed as
    numpydoc.

    By default it asserts that any Parameters/Returns/Attributes entries having the
    same name among ``objects`` docstrings also have the same type
    specification and description (ignoring whitespace).

    ``include_*`` and ``exclude_*`` parameters here are mutually exclusive,
    and specify a whitelist or blacklist, respectively, of parameter, attribute or
    return value names.
    """
    pass

And a variant which checks for specific text in specific parameter / return value names:

def assert_param_docstring_matches(objects, section, param_name, regex):
    # handles cases like random_state that should have more-or-less the same description everywhere

[larsoner]

It sounds like a lot of packages could potentially make use of this functionality. Over in MNE, we dynamically generate docstrings sometimes so a linter like pydocstyle is of limited use for us.

I could try to make a PR that takes the scipy/tools/refguide_check.py or the MNE/sklearn/vispy code over if people want. I'd prefer to start with the latter because I'm more familiar with it, but I suppose I'd need to cross check both versions to get it right.