Skip to content

Commit

Permalink
documentation/python: initial support for generating Python stubs.
Browse files Browse the repository at this point in the history 
So far just aiming to produce valid code, skipping docstrings altogether.
Thus -- generating one file per module, and putting code that closely
resembles what's in the documentation, but in a way that is actual
Python, together with importing dependency modules.

To support incomplete or broken annotations, the tyoe_relative template
variable got specialized to type_quoted. In certain cases, such as base
classes for enums or (data / default) values, it's left as *_relative, as
in those cases no quoting is necessary. What isn't handled so far is
quoting forward references for types that were not yet listed in the
output.
  • Loading branch information
@mosra
mosra committed Sep 28, 2024
1 parent a797c7f commit 699abdd
Show file tree
Hide file tree
Showing 57 changed files with 2,862 additions and 251 deletions.
123 changes: 104 additions & 19 deletions doc/documentation/python.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,9 @@ Python docs
.. |wink| replace:: 😉

A modern, mobile-friendly Sphinx-alike Python documentation generator with a
first-class search functionality. Generated by inspecting Python modules and
using either embedded docstrings or external :abbr:`reST <reStructuredText>`
first-class search functionality and an ability to provide Python stubs for
IDE autocompletion and type checking. Generated by inspecting Python modules
and using either embedded docstrings or external :abbr:`reST <reStructuredText>`
files to populate the documentation.

One of the design goals is providing a similar user experience to the
Expand Down Expand Up @@ -124,6 +125,11 @@ fill it with the generated output. Open ``index.html`` to see the result.
- Can use both in-code docstrings and external :abbr:`reST <reStructuredText>`
files to describe the APIs, giving the user a control over the code size vs
documentation verbosity tradeoff
- Opt-in specialized behavior for understanding native bindings written with
`pybind11`_ and Python code using `attrs`_
- Ability to generate lightweight Python stubs for IDE autocompletion and
type checking that exactly matches the documentation, both for pure Python
and native bindings.

`Configuration`_
================
Expand Down Expand Up @@ -154,9 +160,15 @@ Variable Description
:py:`INPUT: str` Base input directory. If not set, config
file base dir is used. Relative paths are
relative to config file base dir.
:py:`OUTPUT: str` Where to save the output. Relative paths
are relative to :py:`INPUT`; if not set,
``output/`` is used.
:py:`OUTPUT: Optional[str]` Where to save the HTML output. Relative
paths are relative to :py:`INPUT`. If not
set, ``output/`` is used, if set to
:py:`None`, no HTML output is generated.
:py:`OUTPUT_STUBS: Optional[str]` Where to save generated Python stubs. See
`Stub generation`_ for more information.
Relative paths are relative to :py:`INPUT`.
If set to :py:`None`, no stubs are
generated, default is :py:`None`.
:py:`INPUT_MODULES: List[Any]` List of modules to generate the docs from.
Values can be either strings or module
objects. See `Module inspection`_ for more
Expand Down Expand Up @@ -285,6 +297,16 @@ Variable Description
module and class members. See
`Custom URL formatters`_ for more
information.
:py:`STUB_EXTENSION: str` Extension to use for generated Python
stub files. If not set, ``.pyi`` is used.
See `Stub generation`_ for more
information.
:py:`STUB_HEADER: str` Comment block or arbitrary other code to
put at the top generated Python stub files.
If not set, a default generic text is used.
If empty, no header is present in the files
at all. See `Stub generation`_ for more
information.
=================================== ===========================================

`Theme selection`_
Expand Down Expand Up @@ -875,6 +897,37 @@ attrs.*" in their docstring are implicitly hidden from the output if this
option is enabled. In order to show them again, override the docstring to
something meaningful.

`Stub generation`_
==================

By default, the tool produces just a HTML documentation. In many cases, and
especially when native bindings are involved, it's useful to provide so-called
*stubs* for the IDE in order for it to provide useful autocompletion and
perform type checking. Another use case is when the amount of code in the
actual implementation is too large for the IDE to handle.

Stubs can be opted in by specifying a path where to generate them in
:py:`OUTPUT_STUBS`. They can be generated either together with the HTML output,
or alone, if :py:`OUTPUT` is set to :py:`None` instead. A common setup in that
case is to create a second file, named ``conf-stubs.py``, that inherits
everything from ``conf.py`` but enables only the stub output:

.. code:: py
# Inherit everything from conf.py
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
from conf import *
OUTPUT = None
OUTPUT_STUBS = 'stubs/'
Now, when you run the script, the output directory will contain ``*.pyi`` files
that you can supply to your IDE.

.. code:: sh
./python.py path/to/conf-stubs.py
`Command-line options`_
=======================

Expand Down Expand Up @@ -1176,6 +1229,14 @@ Property Description
:py:`page.prefix_wbr` Fully-qualified symbol prefix for given
compound with trailing ``.`` with
:html:`<wbr/>` tag after every ``.``.
:py:`page.dependencies` List of :py:`(prefix, module)`
tuples with module dependencies. To
match relative type names, it's either
:py:`import module` if ``prefix`` is
empty, :py:`from prefix import *` if
``module`` is empty or
:py:`from prefix import module` if both
are non-empty.
:py:`page.modules` List of inner modules. See
`Module properties`_ for details.
:py:`page.classes` List of classes. See
Expand Down Expand Up @@ -1218,6 +1279,8 @@ Property Description
`Property properties`_ for details.
:py:`page.has_property_details` If there is at least one property with
full description block [3]_
:py:`page.has_members` If the class contains at least one
member or is completely empty
======================================= =======================================

Explicit documentation pages rendered with ``class.html`` have additional
Expand Down Expand Up @@ -1308,10 +1371,14 @@ Property Description
:py:`function.content` Detailed documentation, if any
:py:`function.type` Fully qualified function return type
annotation [2]_
:py:`function.type_relative` Like :py:`function.type`, but relative,
:py:`function.type_quoted` Like :py:`function.type`, but relative,
i.e. with a common prefix of the type and
containing module / class omitted, and with
parts quoted if type inspection failed to
match the annotation to an actual name
:py:`function.type_link` Like :py:`function.type`, but relative,
i.e. with a common prefix of the type and
containing module / class omitted
:py:`function.type_link` Like :py:`function.type_relative`, but with
containing module / class omitted, and with
cross-linked types
:py:`function.params` List of function parameters. See below for
details.
Expand All @@ -1328,12 +1395,18 @@ Property Description
:py:`function.return_value` Return value documentation. Can be empty.
:py:`function.has_details` If there is enough content for the full
description block [3]_
:py:`function.is_method` Set to :py:`True` if the function is a
class method, :py:`False` if it's
standalone.
:py:`function.is_classmethod` Set to :py:`True` if the function is
annotated with :py:`@classmethod`,
:py:`False` otherwise.
:py:`function.is_staticmethod` Set to :py:`True` if the function is
annotated with :py:`@staticmethod`,
:py:`False` otherwise.
:py:`function.is_overloaded` Set to :py:`True` if the function has
multiple overloads with the same name,
:py:`False` otherwise.
=================================== ===========================================

The :py:`function.params` is a list of function parameters and their
Expand All @@ -1346,10 +1419,14 @@ Property Description
=============================== ===============================================
:py:`param.name` Parameter name
:py:`param.type` Fully qualified parameter type annotation [2]_
:py:`param.type_relative` Like :py:`param.type`, but relative, i.e. with
:py:`param.type_quoted` Like :py:`param.type`, but relative, i.e. with
a common prefix of the type and containing
module / class omitted
:py:`param.type_link` Like :py:`param.type_relative`, but with
module / class omitted, and with parts quoted
if type inspection failed to match the
annotation to an actual name
:py:`param.type_link` Like :py:`param.type_relative`, but relative,
i.e. with a common prefix of the type and
containing module / class omitted, and with
cross-linked types
:py:`param.default` Default parameter value, if any. If
:py:`param.type` is an enum, is a
Expand Down Expand Up @@ -1402,11 +1479,15 @@ Property Description
:py:`property.id` Property ID [5]_
:py:`property.type` Fully qualified property getter return type
annotation [2]_
:py:`property.type_relative` Like :py:`property.type`, but relative, i.e.
:py:`property.type_quoted` Like :py:`property.type`, but relative, i.e.
with a common prefix of the type and containing
module / class omitted
:py:`property.type_link` Like :py:`property.type_relative`, but with
cross-linked types
module / class omitted, and with parts quoted
if type inspection failed to match the
annotation to an actual name
:py:`property.type_link` Like :py:`property.type`, but relative, i.e.
with a common prefix of the type and containing
module / class omitted, and with cross-linked
types
:py:`property.summary` Doc summary
:py:`property.content` Detailed documentation, if any
:py:`property.exceptions` List of exceptions raised when accessing this
Expand All @@ -1430,11 +1511,15 @@ Property Description
:py:`data.name` Data name
:py:`data.id` Data ID [5]_
:py:`data.type` Fully qualified data type annotation [2]_
:py:`data.type_relative` Like :py:`data.type`, but relative, i.e. with a
:py:`data.type_quoted` Like :py:`data.type`, but relative, i.e. with a
common prefix of the type and containing module
/ class omitted
:py:`data.type_link` Like :py:`data.type_relative`, but with
cross-linked types
/ class omitted, and with parts quoted
if type inspection failed to match the
annotation to an actual name
:py:`data.type_link` Like :py:`data.type`, but relative, i.e.
with a common prefix of the type and containing
module / class omitted, and with cross-linked
types
:py:`data.summary` Doc summary
:py:`data.content` Detailed documentation, if any
:py:`data.value` Data value representation
Expand Down
Loading

0 comments on commit 699abdd

Please sign in to comment.