Many new and bogus "nitpick" warnings when upgrading to sphinx 1.7.0

[njsmith]

Problem

When building my docs nitpicky mode enabled, with sphinx 1.7.0 (but not 1.6.7), I get tons of new warnings, many of which look obviously bogus.

Procedure to reproduce the problem

$ git clone https://github.com/python-trio/trio
$ cd trio
$ git checkout 317020537ecefa9d6c6214c3caf4011ca4cfb564
$ virtualenv -p python3.6 ve
$ ve/bin/pip install . -r ci/rtd-requirements.txt
$ cd docs
$ ../ve/bin/sphinx-build -E -n -nb html source build -v

Error logs / results

Running Sphinx v1.7.0
making output directory...
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 11 source files that are out of date
updating environment: 11 added, 0 changed, 0 removed
reading sources... [  9%] code-of-conduct
reading sources... [ 18%] contributing
reading sources... [ 27%] design
reading sources... [ 36%] glossary
reading sources... [ 45%] history
reading sources... [ 54%] index
reading sources... [ 63%] reference-core
reading sources... [ 72%] reference-hazmat
reading sources... [ 81%] reference-io
reading sources... [ 90%] reference-testing
reading sources... [100%] tutorial

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [  9%] code-of-conduct
writing output... [ 18%] contributing
writing output... [ 27%] design
writing output... [ 36%] glossary
writing output... [ 45%] history
writing output... [ 54%] index
writing output... [ 63%] reference-core
writing output... [ 72%] reference-hazmat
writing output... [ 81%] reference-io
writing output... [ 90%] reference-testing
writing output... [100%] tutorial

/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.Semaphore:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.run_sync_in_worker_thread:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.run_sync_in_worker_thread:: WARNING: py:class reference target not found: CapacityLimiter-like object
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.serve_tcp:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.serve_ssl_over_tcp:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.serve_ssl_over_tcp:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_all:6: WARNING: py:meth reference target not found: HalfCloseableStream.send_eof
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:3: WARNING: py:meth reference target not found: AsyncResource.aclose
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:3: WARNING: py:meth reference target not found: ReceiveStream.receive_some
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:3: WARNING: py:meth reference target not found: AsyncResource.aclose
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:3: WARNING: py:class reference target not found: SendStream
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:20: WARNING: py:class reference target not found: HalfCloseableStream
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:27: WARNING: py:class reference target not found: Stream
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:27: WARNING: py:class reference target not found: HalfCloseableStream
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:35: WARNING: py:meth reference target not found: SendStream.send_all
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.send_eof:35: WARNING: py:meth reference target not found: SendStream.wait_send_all_might_not_block
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.SocketStream.wait_send_all_might_not_block:13: WARNING: py:meth reference target not found: HalfCloseableStream.send_eof
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.open_tcp_listeners:: WARNING: py:class reference target not found: bytes-like
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.open_tcp_listeners:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.open_tcp_listeners:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.open_ssl_over_tcp_listeners:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/__init__.py:docstring of trio.open_ssl_over_tcp_listeners:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/ssl.py:docstring of trio.ssl.SSLStream:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/testing/__init__.py:docstring of trio.testing.MemorySendStream.get_data:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/testing/__init__.py:docstring of trio.testing.memory_stream_pump:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/socket.py:docstring of trio.socket.set_custom_hostname_resolver:: WARNING: py:class reference target not found: None
/tmp/repro-dir/trio/trio/socket.py:docstring of trio.socket.set_custom_socket_factory:: WARNING: py:class reference target not found: None
generating indices... genindex py-modindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 27 warnings.

If we run the exact same commands, but using sphinx==1.6.7, then the build completes successfully with no warnings.

Note that "CapacityLimiter-like object" and "bytes-like" are both listed in conf.py's nitpick_ignore list, so they definitely shouldn't be triggering warnings.

Also note all the error messages about "py:class reference target not found: None". The docs here don't attempt to refer to a class called "None". It's hard to know what is actually triggering this.

[njsmith]

Ah, the py:class reference target not found: None thing seems to be coming from lines like:

    Args:
      max_bytes (int or None): The maximum amount of data to transfer in this
          call, or None to transfer all available data.

(This is Google docstring format being processed via sphinx.ext.napolean.)

On 1.6.7, this was fine... it looks like it resolved via intersphinx to https://docs.python.org/3/library/constants.html#None
Is this an intentional change e.g. switching from using :py:obj: to :py:class:? If so I can add None to my nitpick_ignore list... except that's not working either :-)

[tk0miya]

The behavior was changed at #3927. At present, :type: and related fields are considered as class.
So please rewrite you nitpick_ignore settings to :py:class:.

In addition, autodoc uses docstrings of super class by default:
http://www.sphinx-doc.org/en/master/ext/autodoc.html?highlight=autodoc#confval-autodoc_inherit_docstrings
Some warnings are came from this options. So I think this should be also disabled.

Thanks,

[njsmith]

That does seem to have fixed it, thanks!

[apiraino]

Hello everyone,

I'm not sure this bug has been completely fixed and muting the warnings doesn't look to be the best option.

We noticed something weird when working on a PR for the jsonschema package.

I've created a small repo with a (hopefully meaningful) proof of concept. When building the docs with sphinx==1.6.7 the callable() is resolved. With sphinx==1.7.1 it doesn't.

The docs build environment is py27+py3 and sphinx apparently isn't able to link callable and functools.lru_cache, possibily others.

I couldn't get to the bottom of the issue, so I can't be more specific and I may be wrong somewhere.

I'd like your thoughts on the matter.

thanks!

[Julian]

So, I've only quickly glanced at #3927, so apologies if I've missed something obvious but @apiraino I think I understand what changed -- #3927 changed things so that only classes are matched, not all objects, and both callable and functools.lru_cache are functions, not classes, so in theory that's the intended behavior.

I think that is really impractical though -- in many cases you do not have (public) access to a physical type, and calling a function is the "way" to produce those kinds of objects.

#3927 definitely seems like a bug, but only matching classes seems wrong, it seems like it should match classes first but then fall back on everything to me. @tk0miya do you have any desire to reopen that ticket for discussion? (If so I bet we should move the discussion there)

[Julian]

(Callable itself is another good example -- there's collections.Callable, but imagine a world where it didn't exist, like the one pre-2.7. You'd have no way to correctly document the type of such a parameter).

EDIT: on second thought I guess I take this one back, because it's a bit abusive to say the type is the callable function, callable just happens to link to a somewhat-helpful thing, so yeah never mind this objection.