Boolean operation in NumPy-style type annotation raises error

[sisp]

Describe the bug

A NumPy-style type annotation of the form A or B (e.g. int or float) raises the following error when building MkDocs docs with mkdocstrings:

griffe: Failed to parse annotation from 'BoolOp' node at ...

This issue seems to be related to #82 where the comment

it's probably wrong to use boolean operations in annotations (a and b, a or b)

-- #82 (comment)

is incorrect according to https://numpydoc.readthedocs.io/en/latest/format.html#parameters.

To Reproduce

"""Docstring with boolean operation in NumPy-style type annotation.

Parameters
----------
value : int or float
    Some int or float value.
"""

Expected behavior
This docstring should parse fine.

System:

• griffe version: 0.22.0
• Python version: 3.9
• OS: Linux

[tristanlatr]

Parsing the numpy types can be quite hard because they allow free text as well as keywords, here is how we do it in pydoctor: https://github.com/twisted/pydoctor/blob/4176822d7d79e4a55bab6be229266a2c9c85d21b/pydoctor/napoleon/docstring.py#L159 the code is based on sphinx’s napoleon extension.

This code could be adapted to output markdown instead of restructuredtext.

[sisp]

/cc @pawamoy

[jmp75]

I bumped into the same issue ERROR - griffe: Failed to parse annotation from 'BoolOp' node with Google-style docstrings, with argument types (str or None). It is also worth mentioning that my build on readthedocs.org was reported as successful, yet I did not see the expected changes in the result. Only by looking at the raw logs did I notice that griffe reported several errors.

I can work around this issue by using (Optional[str]) instead of str or None

[tristanlatr]

The core of the issue is to assume the numpy types as parsable with ast.parse. Because they are not. We must handle them by natural language processing.

Here are a few examples:

List[str] or list(bytes), optional
{"html", "json", "xml"}, optional
list of int or float or None, default: None
complicated string or `strIO <twisted.python.compat.NativeStringIO>`

[pawamoy]

Types that are not valid Python (syntax error) are fine indeed. The issue arises when the type is valid Python but not a valid type. Rather than adding complex logic to parse complex Numpy types, we could consider simply ignoring BoolOp errors (as to not make the build fail). If users want automatic cross-reference, they should use valid types and not natural language. What do you all think?

[sisp]

@pawamoy Would ignoring BoolOp errors suffice for handling some other natural language non-type cases like the following (taken from numpydoc)?

Parameters
----------
dtype : data-type
iterable : iterable object
shape : int or tuple of int
files : list of str

[pawamoy]

• data-type: not sure, sub node probably not supported, will probably trigger a warning as well
• iterable object: syntax error, ignored
• int or tuple of int: syntax error, ignored
• list of str: syntax error, ignored

[pawamoy]

I've decided to use the DEBUG level for such log messages when parsing Numpydoc docstrings. This will prevent your builds from failing on errors or warnings. Other parsers are left untouched and will continue logging errors.

[sisp]

Thanks, @pawamoy! 🙏 NumPy-style parameter annotations work great now! 🙂

[pawamoy]

Ha, just realized it's you @sisp 😄Nice seeing you here and on Copier's repo!

[sisp]

Haha, yes, it's a small world it seems. 🤣 I recognized you a while ago on Copier's repo after we had been discussing here for a bit. 😆 mkdocstrings and the projects around it are awesome by the way! 🙏

[pawamoy]

Thanks! 😄

[dekromp]

Hi all,
just adding to this that griffe (v0.22.2) still raises an error if or is used in the Returns section.