sphinx.ext.autodoc: Typehints, aliases, and (internal/external) cross-references

[kratsg]

Describe the bug

For this issue, the code for the project is found here with conf.py and documentation rendered here.

In the code, we have lines like this for our typehints:

T = TypeVar("T", bound=NBitBase)

Tensor = Union["NDArray[np.number[T]]", "NDArray[np.bool_]"]
FloatIntOrBool = Literal["float", "int", "bool"]
...
...
    def astensor(
        self, tensor_in: ArrayLike, dtype: FloatIntOrBool = 'float'
    ) -> ArrayLike:
        """
        Convert to a NumPy array.
        Example:
            >>> import pyhf
            >>> pyhf.set_backend("numpy")
            >>> tensor = pyhf.tensorlib.astensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
            >>> tensor
            array([[1., 2., 3.],
                   [4., 5., 6.]])
            >>> type(tensor)
            <class 'numpy.ndarray'>
        Args:
            tensor_in (Number or Tensor): Tensor object
        Returns:
            `numpy.ndarray`: A multi-dimensional, fixed-size homogeneous array.
        """

which renders as

and

    def percentile(
        self,
        tensor_in: Tensor[T],
        q: Tensor[T],
        axis: None | Shape = None,
        interpolation: Literal[
            "linear", "lower", "higher", "midpoint", "nearest"
        ] = "linear",
    ) -> ArrayLike:
        r"""
        Compute the :math:`q`-th percentile of the tensor along the specified axis.
        Example:
            >>> import pyhf
            >>> pyhf.set_backend("numpy")
            >>> a = pyhf.tensorlib.astensor([[10, 7, 4], [3, 2, 1]])
            >>> pyhf.tensorlib.percentile(a, 50)
            3.5
            >>> pyhf.tensorlib.percentile(a, 50, axis=1)
            array([7., 2.])
        Args:
            tensor_in (`tensor`): The tensor containing the data
            q (:obj:`float` or `tensor`): The :math:`q`-th percentile to compute
            axis (`number` or `tensor`): The dimensions along which to compute
            interpolation (:obj:`str`): The interpolation method to use when the
             desired percentile lies between two data points ``i < j``:
                - ``'linear'``: ``i + (j - i) * fraction``, where ``fraction`` is the
                  fractional part of the index surrounded by ``i`` and ``j``.
                - ``'lower'``: ``i``.
                - ``'higher'``: ``j``.
                - ``'midpoint'``: ``(i + j) / 2``.
                - ``'nearest'``: ``i`` or ``j``, whichever is nearest.
        Returns:
            NumPy ndarray: The value of the :math:`q`-th percentile of the tensor along the specified axis.
        """

which renders as

And finally, in conf.py, we have the aliases defined such as 'ArrayLike': 'numpy.typing.ArrayLike'

_type_aliases_inverted = {
    'pyhf.typing': [
        'PathOrStr',
        'ParameterBase',
        'Parameter',
        'Measurement',
        'ModifierBase',
        'NormSys',
        'NormFactor',
        'HistoSys',
        'StatError',
        'ShapeSys',
        'ShapeFactor',
        'LumiSys',
        'Modifier',
        'Sample',
        'Channel',
        'Observation',
        'Workspace',
        'Literal',
    ],
    'numpy.typing': ['ArrayLike', 'DTypeLike', 'NBitBase', 'NDArray'],
}
autodoc_type_aliases = {
    item: f'{k}.{item}' for k, v in _type_aliases_inverted.items() for item in v
}

autodoc_typehints_format = 'fully-qualified'

Here are the issues

1. In astensor, Literal gets linked to typing.Literal instead of pyhf.typing.Literal (which is fine! but in other python versions, this comes from typing_extensions which is why we import from pyhf.typing which has the code for switching where it comes from) since the docs are rendered using python 3.10. In percentile, Literal does not get cross-referenced, but why?
2. In astensor, numpy.typing.ArrayLike is not getting cross-referenced at all, but it exists in the numpy intersphinx.
3. In percentile, Tensor is not getting cross-referenced (from the same module) and Shape is not getting cross-referenced (from pyhf.typing in the same package).

How to Reproduce

Recommend python 3.10+ (Sphinx does not correctly process UnionType in older python versions).

$ git clone https://github.com/scikit-hep/pyhf
$ cd pyhf
$ python -m pip install --upgrade pip setuptools wheel
$ python -m pip --quiet install --upgrade .[docs,test]
$ cd docs
$ make html
$ open _build/html/index.html

Expected behavior

Expected type aliases to be processed with intersphinx (but also internal cross-referencing working too).

Your project

https://pyhf--1969.org.readthedocs.build/en/1969

Screenshots

No response

OS

Mac 10.14.6 (18G9323)

Python version

Python 3.8.6

Sphinx version

5.1.1

Sphinx extensions

'sphinx.ext.autodoc', 'sphinx.ext.autosummary', 'sphinx.ext.coverage', 'sphinx.ext.mathjax', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode', 'sphinx.ext.githubpages', 'sphinx.ext.intersphinx', 'sphinxcontrib.bibtex', 'sphinx.ext.napoleon', 'sphinx_click.ext', 'nbsphinx', 'sphinx_issues', 'sphinx_copybutton', 'sphinx_togglebutton', 'xref',

Extra tools

No response

Additional context

Related:

• Intersphinx does not create references to external project when using import as alias #10151
• autoclass digests types from typehint but numpy types are not linked via intersphinx #10746
• docs: Add typehint aliases scikit-hep/pyhf#1969

[mscuthbert]

Agreed on the importance of this request.

Also it would be good if argument annotations were not typed with the same class as the argument name itself. It would mkae it much easier to give a different style to type declarations or to use a callback to fi the problems that @kratsg mentions abouve.

[cjw296]

Seems like there might be overlap between this issue and #10785