mangle_docstring is called on already-processed docstrings

[jnothman]

It seems that due to wrap_mangling_directives, even in our simple example code, we're visiting the same docstring multiple times, and running our mangling over an already-mangled docstring. This is creating especial rubbish with reference renaming, but we've certainly not tested the effect of this on any of our docscrape apparatus either!

With the current example.py and docs, we get calls to mangle_docstrings with:

what='module', name='example', len(lines)=9, hash(tuple(lines))=-8903261816696980551
what='function', name='example.foo', len(lines)=83, hash(tuple(lines))=-6998798167062068361
what='function', name="foo(var1, var2, long_var_name='hi')", len(lines)=108, hash(tuple(lines))=3694357165576974898

I show hashes and line counts to illustrate that we have mangled the dosctring, at least in some cases.

When I add a class Foobar and its method foobar, we get the following invocations of mangle_docstring:

what='module', name='example', len(lines)=9, hash(tuple(lines))=-475644803089738938, rubric in lines=False
what='class', name='example.Foobar', len(lines)=6, hash(tuple(lines))=-5539289288007919216, rubric in lines=False
what='method', name='example.Foobar.foobar', len(lines)=6, hash(tuple(lines))=7217615682760160437, rubric in lines=False
what='function', name='example.foo', len(lines)=83, hash(tuple(lines))=-8508781800898451264, rubric in lines=False
what='class', name='Foobar', len(lines)=39, hash(tuple(lines))=8692486620067943275, rubric in lines=True
what='method', name='example.Foobar.foobar', len(lines)=6, hash(tuple(lines))=7217615682760160437, rubric in lines=False
what='method', name='example.Foobar.foobar', len(lines)=22, hash(tuple(lines))=5869529305210585233, rubric in lines=False
what='function', name='Foobar.foobar()', len(lines)=5, hash(tuple(lines))=-8048095480324991988, rubric in lines=False
what='function', name="foo(var1, var2, long_var_name='hi')", len(lines)=108, hash(tuple(lines))=-7634163004091929948, rubric in lines=True

[jnothman]

Here there seem to be four manglings of the Foobar.foobar docstring, one of them operating over the pre-mangled docstring.

[jnothman]

This is not all due to wrap_mangling_directives, it seems. If I drop it, I'm still left with some duplicates:

what='module', name='example', len(lines)=9, hash(tuple(lines))=-3223048436244732897, rubric in lines=False
what='class', name='example.Foobar', len(lines)=6, hash(tuple(lines))=-5385718271852730703, rubric in lines=False
what='method', name='example.Foobar.foobar', len(lines)=6, hash(tuple(lines))=-6579204817804217871, rubric in lines=False
what='function', name='example.foo', len(lines)=83, hash(tuple(lines))=-176078909342747320, rubric in lines=False
what='method', name='example.Foobar.foobar', len(lines)=6, hash(tuple(lines))=-6579204817804217871, rubric in lines=False
what='method', name='example.Foobar.foobar', len(lines)=22, hash(tuple(lines))=-743157675148215057, rubric in lines=False

[jnothman]

The second processing of the foobar method appears to be harmless: it's called in processing the .. autosummary we include in the class docstring, when summarising the method dosctring. The third appears to be a bug in sphinx, and I will raise an issue there and reference ours.

The remaining unnecessary and harmful (double-processing) calls are due to wrap_mangling_directives, whose purpose I've not yet fathomed.

[jnothman]

Some tracing tells me that wrap_mangling_directives results in NumpyPythonDomain's directives being used to handle py:class, py:function, etc. Is that the intention, @pv?

Do these domains remain relevant? I can't see any reference to them in the numpy or scipy repositories currently. I see that you commented on their likely irrelevance in 2012.

Can we drop them without deprecation if they're unlikely used and causing weird behaviour (and duplicated work)??

[jnothman]

@astrofrog fwiw, merging #135 might also give multiplicative speedups if lots of time was spent in numpydoc

[jnothman]

Fixed in #135