Interleave doctest examples w/ API ref docs.

[jgeewax]

Related to #1102

I just tried to walk a group of people through gcloud.pubsub, and of the few who found the docs under "Pubsub usage", almost everyone was frustrated by the examples being there rather than next to the method they were interested in. They couldn't find the specific thing they wanted quickly, whereas finding the right methods was pretty straight forward -- but then example code was hidden in a big long (and separate) page.

Ideally we would:

1. Move all the usage examples to live next to the methods they describe
2. Remove the pubsub-usage.rst file
3. Have test coverage of those docstring examples

(ie, https://googlecloudplatform.github.io/gcloud-python/stable/pubsub-topic.html has no example code snippets)

[dhermes]

FWIW I've never been a big fan of the *-usage docs.

IIRC you encouraged us to create them after @tseaver started the very first one for Pub/Sub (datastore and storage don't really have them).

[jgeewax]

Yes, the goal of the usage docs was "let's define the actions and the API surface" and debate the usage in those docs rather than in code that implements them -- which I still think we should do with new APIs...

The goal wasn't to hide examples in there and take them out of docstrings... Sorry if that was confusing.

[dhermes]

We never took them out, just for those packages they never went in as a result. The process was never really explicit in either direction.

[dhermes]

Also @jgeewax third item relies on #212 being done

[tseaver]

Making the example snippets testable requires being able to tie them to specific testsetup and testcleanup groups.

My inclination would be not to put the examples in the docstrings themselves, but to interleave them with the docstring-generated API reference docs. This would require making the API reference less DRY, as we would need to explicitly name each method (to pull in its docstring) before the example. In exchange, we would gain some benefits:

• We could explicitly reorder (or exclude) the method API reference entries.
• The Sphinx API source documents would become a logical place to inject other, too-verbose-for-the-docstring narrative.
• We would be able to use testsetup and testcleanup groups without trying to sync them between docstrings and the Sphinx source.

[dhermes]

I'd like to make that such a change would include #714 (e.g. an automated way to make sure all public functions, classes, methods, constants, or otherwise, are documented)

[theacodes]

It seems that we currently have a lot of usage snippets that are untested included in the docs. This is really unacceptable. I'd vote that until #1744 (testing for snippets) is resolved, that you guys instead link to the docs on cloud.google.com (where applicable) where samples are tested. Some products haven't been updated to use gcloud-python, but that gap should be closing this quarter for Datastore, Pub/Sub, Logging, BigQuery, and Storage.

We can't be putting untested samples in front of users.

/cc @omaray

[tseaver]

#2154 is really where this should be tracked as an ongoing issue.