-
Notifications
You must be signed in to change notification settings - Fork 68
Documenting UVCDAT
In CDAT, we use Sphinx to document our python code. Sphinx is a utility that lets you take documentation in your source code and generate references in a number of different formats (webpage, PDF, unix man pages, etc.).
Note: This wiki is intended for developers, so I will assume the reader has knowledge of how to set up a conda environment for whatever part of CDAT that they are working with.
After getting your cdat conda environment set up, $ conda install sphinx with that environment activated.
If there isn't some sort of documentation directory at the root
of your project, go ahead and make one. Inside of the documentation directory, run $ sphinx-quickstart. This will
take you through the sphinx setup process, and generate a conf.py and Makefile which will be used by Sphinx to generate
various formats of output for your documentation.
By default, Sphinx uses reStructuredText formatting to interpret the appropriate format for your documentation output. There are many built-in directives to help with the documentation process. I will discuss some of these that are used frequently in VCS, and how we use them. This can serve as a general "template" for other documentation in the cdat library.
These directives are used to document a function's parameters, returns, and their types.
:param: and :return: are for describing the details of the parameter/return (how it is used, formatting restrictions
on parameters, etc.). :type: and :rtype: should just contain a reference to the data type. Sphinx should
automatically provide a link to python builtins (like str) in the output.
docstring example:
"""
:param name: Name of created object
:type name: str
# ...
:returns: A VCS Line object
:rtype: vcs.line.Tl
"""If you include a reference to a class or type particular to your package in the return type or parameter type directives, when the output is built there will be a link the documentation for that.
Putting any text between two colons (i.e. :Words Words:) indicates that the text should be output in bold, with a colon.
In vcs, this is commonly used to denote a code example:
docstring example:
"""
:Example:
.. doctest:: canvas_boxfill
>>> a=vcs.init()
>>> a.show('boxfill') # Show all boxfills
*******************Boxfill Names List**********************
...
*******************End Boxfill Names List**********************
>>> box=a.getboxfill('quick') # Create instance of 'quick'
>>> arr=[range(10) for _ in range(10)] # data to plot
>>> a.boxfill(arr, box) # Plot array w/ box; default template
<vcs.displayplot.Dp ...>
>>> t = a.gettemplate('quick') # get quick template
>>> a.clear() # Clear VCS canvas
>>> a.boxfill(arr, box, t) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(box, arr, t) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(t, arr, box) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(t, box, arr) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(arr, 'polar', 'polar') # 'polar' template/boxfill
<vcs.displayplot.Dp ...>
>>> a.boxfill('polar', arr, 'polar') # 'polar' template/boxfill
<vcs.displayplot.Dp ...>
>>> a.boxfill('polar', 'polar', arr) # 'polar' template/boxfill
<vcs.displayplot.Dp ...>
"""We write code examples in doctest format. I will go over the details of this later.
The note directive is for any kind of side notes that you wish to make about the thing you are documenting. In VCS it is commonly used to note important tips on using the functions provided in the module:
docstring example:
"""
.. note::
As shown above, the data, 'template', and 'box' parameters can be provided in any order.
The 'template' and 'box' parameters can either be VCS template and boxfill objects,
or string names of template and boxfill objects.
The first string provided is assumed to be a template name. The second is assumed to be a
boxfill name.
"""For the note to display properly, there must be a blank line between the .. note:: declaration and its contents,
and its contents must be contained within an indented block, as pictured above.
Admonitions allow you to make a note-like directive with any text you want as the header. In VCS, this directive is used for a number of reasons. One of them is to indicate deprecated features:
docstring example:
"""
.. admonition:: VCS Scripts Deprecated
SCR scripts are no longer generated by this function
"""Admonitions require a block of indented text for the body, just like the note directive. The header text will be rendered
as whatever text you have following the '::', i.e. .. admonition:: Hello will have Hello as the header text.
To refer to a Python object/class/function inside of the package you are documenting, simply put
:(class|obj|func):`module.file.name` wherever you wish to link to the relevant source.
docstring example:
"""Given name, returns a :class:`vcs.unified1d.G1d` from vcs with that name.
Unlike other VCS 'get' functions, name cannot be None when calling get1d().
"""If you use the intersphinx extenstion, you can link to the documentation of external modules (i.e. :class:`numpy.ma.MaskedArray`
would provide a link to NumPy's MaskedArray class documentation).
Even if you don't use intersphinx, the RST directive for referring to documentation internal to your module is the same
(i.e. :py:class:`vcs.Canvas.Canvas` provides a link to the Canvas class in the Canvas module of VCS).
Sphinx has a lot of custom markup, and a solid RST Primer to introduce users to reStructuredText syntax. These sources contain much more detailed information on the subject than can be offered here.
Sphinx cares a great deal about how your documentation strings are formatted. Improper formatting is likely to affect the final output that Sphinx builds, so it's important to follow these guidelines when writing docstrings:
Lines over 80 characters are likely to extend past the styling of a webpage or the boundaries set up by sphinx's LaTeX output for making a PDF file. Limit lines to 80 characters, and the output should look good in any format.
If you use a directive within a docstring, it's important to clearly indicate where that directive ends by leaving a blank line between it and the rest of the documentation. If it is a directive that has a block of text attached to it (note, doctest, etc.), make sure you properly un-indent any text below it that should not be included in the directive.
If you're bringing documentation in from other sources (i.e. xmldocs.py in vcs), make sure that the tabbing in that documentation aligns with the same column as the file where you are importing it. Mismatched tabs can cause bad formatting in the final output.
It's a good practice to start the docstring text on the same line as the beginning of the dosctring, i.e.:
"""Start documentation ..."""versus
"""
Start documentation ...
"""In VCS, we put doctests in our docstrings because they allow us to show users simple examples of how to use various functions/classes, and those examples can be run through the doctest module to tell us whether they will run without errors.
docstring example:
"""
:Example:
.. doctest:: canvas_boxfill
>>> a=vcs.init()
>>> a.show('boxfill') # Show all boxfills
*******************Boxfill Names List**********************
...
*******************End Boxfill Names List**********************
>>> box=a.getboxfill('quick') # Create instance of 'quick'
>>> arr=[range(10) for _ in range(10)] # data to plot
>>> a.boxfill(arr, box) # Plot array w/ box; default template
<vcs.displayplot.Dp ...>
>>> t = a.gettemplate('quick') # get quick template
>>> a.clear() # Clear VCS canvas
>>> a.boxfill(arr, box, t) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(box, arr, t) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(t, arr, box) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(t, box, arr) # Plot w/ box and 'quick' template
<vcs.displayplot.Dp ...>
>>> a.boxfill(arr, 'polar', 'polar') # 'polar' template/boxfill
<vcs.displayplot.Dp ...>
>>> a.boxfill('polar', arr, 'polar') # 'polar' template/boxfill
<vcs.displayplot.Dp ...>
>>> a.boxfill('polar', 'polar', arr) # 'polar' template/boxfill
<vcs.displayplot.Dp ...>
"""As seen above, the doctest directive can be given a namespace on the first line (e.g. .. doctest:: doctest_name).
This puts the doctest into the named group of tests, so that when Sphinx runs the tests, any test setup or cleanup
specific to that group will be run in conjunction with those tests.
Doctests are meant to emulate the feeling of typing script into an interactive Python console, so that is how they are
formatted. The >>> must precede lines of test code, and if that code should cause something to be output to the
console, the expected output must go on the line beneath the test code.
doctest's ELLIPSES option allows you to spoof some of the expected output by using a ..., which is useful for
information that is determined at run-time (such as the memory addresses for the returned vcs.displayplot.Dps above).
This option is enabled by default if you are running your doctests via Sphinx, but if you need to run the doctests via
the commandline or in a script, you have to enable the option either in the doctest itself, or passed in as an argument
to the doctest function. See the documentation on doctest directives for more information.
Sphinx has detailed documentation on using the doctest extension.
You can use entries in sphinx's config.py to set up global setup/cleanup code needed for your doctest (for importing needed libraries or destroying objects you created in the tests).
To use the extension (once your config.py is set up correctly), all you have to do is run $ make doctest in your
documentation directory.
Make sure that your conda environment is active, that your current version of whatever CDAT library you are working
on is installed, and that you have Sphinx installed in your conda environment before you run $ make doctest
In VCS we found that Sphinx runs all doctests on the same instance of a Python console. While that might be fine for smaller packages, VCS is large, and we ended up with doctests that had a lot of conflicts when run sequentially on a single Python instance.
To fix this, we had to write a script to test individual modules in the package using doctest.testmod. In VCS we run a bash script that runs the python script above for each module in the package. It then runs a logging function that parses the doctest output to generate a markdown-formatted log file containing all of the errors encountered when running the doctests, and all of the locations in the module where there are no doctests. The latter bit of information is very helpful in determining where more documentation may be needed in the module.
Running doctests this way means that you don't necessarily have to put them in a Sphinx doctest directive (though doing so still gives them nice markup in the output). It also doesn't take into account the doctest namespacing. In VCS, we have still been using doctest namespacing to clearly delineate where doctests are, and in case we ever figure out a way to make Sphinx play nicely with our tests.
If you don't like the idea of typing :param:, :type:, :return:, etc. in all of your documentation,
you can use Sphinx's Napoleon extension and write your documentation in NumPy or Google format. These look to be much
more legible and easier to write in. Just don't forget to add API examples with doctests!