Unable to build documentation for the ibis project #72

cpcloud · 2022-05-29T11:37:36Z

Describe the bug
A clear and concise description of what the bug is.

When running mkdocs build on master of ibis I get the following traceback:

Traceback (most recent call last):
  File "/nix/store/xybc4xyjrwda96sq975x5zsn3szjj46v-python3.10-mkdocs-1.3.0/bin/.mkdocs-wrapped", line 9, in <module>
    sys.exit(cli())
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/mkdocs/__main__.py", line 192, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/mkdocs/commands/build.py", line 292, in build
    _populate_page(file.page, config, files, dirty)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/mkdocs/commands/build.py", line 174, in _populate_page
    page.render(config, files)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/mkdocs/structure/pages.py", line 175, in render
    self.content = md.convert(self.markdown)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/markdown/core.py", line 264, in convert
    root = self.parser.parseDocument(self.lines).getroot()
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/markdown/blockparser.py", line 90, in parseDocument
    self.parseChunk(self.root, '\n'.join(lines))
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/markdown/blockparser.py", line 105, in parseChunk
    self.parseBlocks(parent, text.split('\n\n'))
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/markdown/blockparser.py", line 123, in parseBlocks
    if processor.run(parent, blocks) is not False:
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/mkdocstrings/extension.py", line 121, in run
    html, handler, data = self._process_block(identifier, block, heading_level)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/mkdocstrings/extension.py", line 195, in _process_block
    data: CollectorItem = handler.collect(identifier, options)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/handler.py", line 195, in collect
    unresolved, iterations = loader.resolve_aliases(only_exported=True, only_known_modules=True)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/griffe/loader.py", line 179, in resolve_aliases
    self.expand_wildcards(wildcards_module)
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/griffe/loader.py", line 254, in expand_wildcards
    self.expand_wildcards(member, only_known_modules, seen)  # type: ignore[arg-type]
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/griffe/loader.py", line 254, in expand_wildcards
    self.expand_wildcards(member, only_known_modules, seen)  # type: ignore[arg-type]
  File "/nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/griffe/loader.py", line 260, in expand_wildcards
    if new_member.name not in obj.members or obj[new_member.name].lineno < alias_lineno:
TypeError: '<' not supported between instances of 'NoneType' and 'int'

To Reproduce

Set up an ibis development environment (any of nix, conda or pip should reproduce)
Run mkdocs build

Expected behavior

I would expect the docs to build without error :)

System (please complete the following information):

griffe version:

❯ poetry show griffe
name         : griffe
version      : 0.19.3

Python version: 3.10
OS: NixOS (Linux)

Additional context

I dug around a bit with ipython --pdb -m mkdocs build.

It looks like griffe maybe isn't correctly tracking __all__ when it's computed instead of a constant hand-curated list which makes star imports (which we use sparingly in ibis) include aliases from other modules.

Here's the final stack frame in the traceback of that debugging session:

> /nix/store/4spdlhxmri6h2492n7iar1rk9l903pbw-python3-3.10.4-env/lib/python3.10/site-packages/griffe/loader.py(260)expand_wildcards()
    258
    259         for new_member, alias_lineno, alias_endlineno in expanded:
--> 260             if new_member.name not in obj.members or obj[new_member.name].lineno < alias_lineno:
    261                 obj[new_member.name] = Alias(
    262                     new_member.name, new_member, lineno=alias_lineno, endlineno=alias_endlineno

ipdb> new_member
<Alias('com', 'ibis.common.exceptions')>
ipdb> obj
<Module(PosixPath('ibis/expr/types/__init__.py'))>
ipdb> obj[new_member.name]
<Alias('com', 'ibis.expr.types.analytic.com')>
ipdb> obj[new_member.name].lineno is None
True
ipdb> pp obj.members
{'Analytic': <Alias('Analytic', 'ibis.expr.types.analytic.Analytic')>,
 'Column': <Alias('Column', 'ibis.expr.types.arrays.Column')>,
 'Exists': <Alias('Exists', 'ibis.expr.types.analytic.Exists')>,
 'Expr': <Alias('Expr', 'ibis.expr.types.analytic.Expr')>,
 'Iterable': <Alias('Iterable', 'ibis.expr.types.arrays.Iterable')>,
 'Scalar': <Alias('Scalar', 'ibis.expr.types.arrays.Scalar')>,
 'TYPE_CHECKING': <Alias('TYPE_CHECKING', 'ibis.expr.types.arrays.TYPE_CHECKING')>,
 'TopK': <Alias('TopK', 'ibis.expr.types.analytic.TopK')>,
 'V': <Alias('V', 'ibis.expr.types.arrays.V')>,
 'Value': <Alias('Value', 'ibis.expr.types.arrays.Value')>,
 'analytic': <Module(PosixPath('ibis/expr/types/analytic.py'))>,
 'annotations': <Alias('annotations', 'ibis.expr.types.arrays.annotations')>,
 'arrays': <Module(PosixPath('ibis/expr/types/arrays.py'))>,
 'binary': <Module(PosixPath('ibis/expr/types/binary.py'))>,
 'category': <Module(PosixPath('ibis/expr/types/category.py'))>,
 'collections': <Module(PosixPath('ibis/expr/types/collections.py'))>,
 'com': <Alias('com', 'ibis.expr.types.analytic.com')>,
 'core': <Module(PosixPath('ibis/expr/types/core.py'))>,
 'deprecated': <Alias('deprecated', 'ibis.expr.types.analytic.deprecated')>,
 'generic': <Module(PosixPath('ibis/expr/types/generic.py'))>,
 'geospatial': <Module(PosixPath('ibis/expr/types/geospatial.py'))>,
 'groupby': <Module(PosixPath('ibis/expr/types/groupby.py'))>,
 'inet': <Module(PosixPath('ibis/expr/types/inet.py'))>,
 'json': <Module(PosixPath('ibis/expr/types/json.py'))>,
 'literal': <Alias('literal', 'ibis.expr.types.arrays.literal')>,
 'logical': <Module(PosixPath('ibis/expr/types/logical.py'))>,
 'maps': <Module(PosixPath('ibis/expr/types/maps.py'))>,
 'numeric': <Module(PosixPath('ibis/expr/types/numeric.py'))>,
 'public': <Alias('public', 'ibis.expr.types.analytic.public')>,
 'relations': <Module(PosixPath('ibis/expr/types/relations.py'))>,
 'sortkeys': <Module(PosixPath('ibis/expr/types/sortkeys.py'))>,
 'strings': <Module(PosixPath('ibis/expr/types/strings.py'))>,
 'structs': <Module(PosixPath('ibis/expr/types/structs.py'))>,
 'temporal': <Module(PosixPath('ibis/expr/types/temporal.py'))>,
 'typing': <Module(PosixPath('ibis/expr/types/typing.py'))>,
 'uuid': <Module(PosixPath('ibis/expr/types/uuid.py'))>}

Here com is technically a member of ibis.expr.types.analytic but it's not part of __all__ (computed using @public), so ideally it isn't picked up.

The text was updated successfully, but these errors were encountered:

pawamoy · 2022-05-29T12:00:50Z

Thanks for the very detailed report!

Griffe does not support the public package indeed, which means it won't be able to infer __all__ correctly. That kind of support can be brought through an extension (and I think it would be easy to write: upon visiting classes or functions, if they are decorated with public or public.add, add them to objects exported by the module).

Now the issue here is more that the existing alias with name com in the ibis.expr.types module does not have a line number, when it should. I'll run my own debugging session to see if I forgot to set this attribute somewhere in the code 🙂

pawamoy · 2022-05-31T21:34:33Z

OK, your wildcard imports are a bit too wild for Griffe.

I was able to fix the issue with line numbers, and then faced multiple cyclic aliases errors. Your code works because Python, but doing inference is hard.

In ibis/backends/dask/execution/join.py:

-from ibis.backends.dask.execution import constants
+from ibis.backends.pandas.execution import constants

Importing from dask works, because there constants are wildcard imported from another dask submodule which imports it from the pandas backends.
Actually importing it from pandas reduces the indirections and helps Griffe.
Griffe can probably be improved here, with a smarter/more capable wildcard expanding algorithm.

Then more cyclic alias errors that I was able to suppress in different parts of the code.

Serialization still crashes with infinite recursion 😕 But you shouldn't need serialization for building your docs.

Now I have to admit that Griffe's data model is not perfect. It still considers submodules like members, when clearly it should not. In Python it seems that even if you define a member in a module using the same name as one of the module's submodules, you can still import from that submodule (import machinery and attribute access are different).

I'll have to refactor it. It promises to be a challenging task 😅

I'll post an update here when I feel like I can push a fix.

…d attribute Issue #72: #72 Issue #79: #79 Issue mkdocstrings/mkdocstrings#438: mkdocstrings/mkdocstrings#438

pawamoy · 2022-06-03T18:25:38Z

So the TypeError: '<' not supported between instances of 'NoneType' and 'int' should be fixed now, in 0.20.0, but you'll still have the cyclic alias errors. I'll leave this open until it's fixed 🙂

pawamoy · 2022-12-23T23:55:41Z

Hello @cpcloud, I've released quite a number of bugfixes, could you give Griffe another try? I'm having trouble setting up a dev environment for your ibis project, I've followed the docs, but am still missing dependencies.when running mkdocs build: ModuleNotFoundError: No module named 'google'.

cpcloud · 2023-01-04T19:12:55Z

Hey @pawamoy, really warms my heart to get this message in my inbox. I'll give it go and report back!

cpcloud · 2023-01-04T20:13:58Z

Ok, wow. This is really close.

Good news

Let's start with the good news: after a few changes I am able to build and render the API documentation using the latest version of mkdocstrings. It even found a bogus docstring and the error message for that was really really nice.

Loving the new bells and whistles, especially the interlinking of types in the Returns sections.

Not as good news, but hopefully fixable without a huge amount of work.

For some reason I can't render docs/api/config.md, I see this traceback running mkdocs build --strict:

Traceback

INFO     -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'include_dir': '', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '',
            'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO     -  [macros] - Found local Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Found external Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Extra variables (config file): ['version', 'project_name', 'team', 'social', 'support_levels']
INFO     -  [macros] - Extra filters (module): ['pretty']
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /home/cloud/src/ibis/site
WARNING  -  mkdocstrings_handlers: 3 aliases were still unresolved after 2 iterations
ERROR    -  Error reading page 'api/config.md': compile() arg 1 must be a string, bytes or AST object
Traceback (most recent call last):
  File "/nix/store/p3ldsxikv6206lm364b808dm59qykr48-python3.10-mkdocs-1.4.2/bin/.mkdocs-wrapped", line 9, in <module>
    sys.exit(cli())
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocs/__main__.py", line 250, in build_command
    build.build(cfg, dirty=not clean)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocs/commands/build.py", line 308, in build
    _populate_page(file.page, config, files, dirty)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocs/commands/build.py", line 181, in _populate_page
    page.render(config, files)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocs/structure/pages.py", line 270, in render
    self.content = md.convert(self.markdown)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/markdown/core.py", line 264, in convert
    root = self.parser.parseDocument(self.lines).getroot()
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/markdown/blockparser.py", line 90, in parseDocument
    self.parseChunk(self.root, '\n'.join(lines))
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/markdown/blockparser.py", line 105, in parseChunk
    self.parseBlocks(parent, text.split('\n\n'))
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/markdown/blockparser.py", line 123, in parseBlocks
    if processor.run(parent, blocks) is not False:
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings/extension.py", line 121, in run
    html, handler, data = self._process_block(identifier, block, heading_level)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings/extension.py", line 207, in _process_block
    rendered = handler.render(data, options)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/handler.py", line 254, in render
    return template.render(
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/templates/material/module.html", line 1, in top-level template code
    {% extends "_base/module.html" %}
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/templates/material/_base/module.html", line 58, in top-level template code
    {% include "children.html" with context %}
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/templates/material/children.html", line 1, in top-level template code
    {% extends "_base/children.html" %}
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/templates/material/_base/children.html", line 45, in top-level template code
    {% include "class.html" with context %}
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/templates/material/class.html", line 1, in top-level template code
    {% extends "_base/class.html" %}
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocstrings_handlers/python/templates/material/_base/class.html", line 72, in top-level template code
    <code>{% include "expression.html" with context %}</code>{% if not loop.last %}, {% endif %}
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/jinja2/environment.py", line 485, in getattr
    return getattr(obj, attribute)
  File "/nix/store/al6g1zbk8li6p8mcyp0h60d08jaahf8c-python3-3.10.9/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/griffe/dataclasses.py", line 142, in parsed
    return self.parse()
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/griffe/dataclasses.py", line 155, in parse
    return parse(self, parser or self.parser, **(options or self.parser_options))
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/griffe/docstrings/parsers.py", line 51, in parse
    return parsers[parser](docstring, **options)  # type: ignore[operator]
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/griffe/docstrings/numpy.py", line 693, in parse
    section, offset = reader(docstring, offset + 2, **options)  # type: ignore[operator]
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/griffe/docstrings/numpy.py", line 551, in _read_attributes_section
    annotation = parse_annotation(annotation, docstring, log_level=LogLevel.debug)  # type: ignore[arg-type]
  File "/nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/griffe/docstrings/utils.py", line 65, in parse_annotation
    code = compile(annotation, mode="eval", filename="", flags=PyCF_ONLY_AST, optimize=2)
TypeError: compile() arg 1 must be a string, bytes or AST object

I also cannot render docs/backends/Impala.md:

Traceback

❯ mkdocs build --strict
INFO     -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'include_dir': '', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '',
            'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO     -  [macros] - Found local Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Found external Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Extra variables (config file): ['version', 'project_name', 'team', 'social', 'support_levels']
INFO     -  [macros] - Extra filters (module): ['pretty']
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /home/cloud/src/ibis/site
WARNING  -  mkdocstrings_handlers: 3 aliases were still unresolved after 2 iterations
ERROR    -  mkdocstrings: ibis.backends.impala.Backend.table could not be found
ERROR    -  Error reading page 'backends/Impala.md':
ERROR    -  Could not collect 'ibis.backends.impala.Backend.table'

table is a method on the base class, not directly defined on impala.Backend, if that's useful to know

Similarly, I couldn't render any of the backend pages (under docs/backend) that reference a base class implementation of do_connect:

Traceback

❯ mkdocs build --strict
INFO     -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'include_dir': '', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '',
            'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO     -  [macros] - Found local Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Found external Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Extra variables (config file): ['version', 'project_name', 'team', 'social', 'support_levels']
INFO     -  [macros] - Extra filters (module): ['pretty']
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /home/cloud/src/ibis/site
WARNING  -  mkdocstrings_handlers: 3 aliases were still unresolved after 2 iterations
ERROR    -  mkdocstrings: ibis.backends.pandas.Backend.do_connect could not be found
ERROR    -  Error reading page 'backends/Pandas.md':
ERROR    -  Could not collect 'ibis.backends.pandas.Backend.do_connect'

After commented out the code related to 1, 2 and 3 and fixed the broken docstring I was able to render things successfully!

pawamoy · 2023-01-04T20:30:26Z

That's great! Thanks a lot for trying the new version and reporting back!
The TypeError is really weird, I'm looking at the code and there's no way something else than a string can be passed to compile 😆

For issues 2 and 3, could you re-run with MkDocs' -v option and paste the logs 🙏?

cpcloud · 2023-01-04T20:32:23Z

I dropped into pdb using this:

❯ python -m pdb -m mkdocs serve
> /nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/mkdocs/__main__.py(3)<module>()
-> from __future__ import annotations
(Pdb) c
INFO     -  Building documentation...
INFO     -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'include_dir': '', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '',
            'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO     -  [macros] - Found local Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Found external Python module 'main' in: /home/cloud/src/ibis
INFO     -  [macros] - Extra variables (config file): ['version', 'project_name', 'team', 'social', 'support_levels']
INFO     -  [macros] - Extra filters (module): ['pretty']
INFO     -  Cleaning site directory
WARNING  -  mkdocstrings_handlers: 3 aliases were still unresolved after 2 iterations
ERROR    -  Error reading page 'api/config.md': compile() arg 1 must be a string, bytes or AST object

And uncovered this:

> /nix/store/cwsg2cw6yk8yrybyyj0ybg99x6ldd9sl-python3-3.10.9-env/lib/python3.10/site-packages/griffe/docstrings/utils.py(65)parse_annotation()
-> code = compile(annotation, mode="eval", filename="", flags=PyCF_ONLY_AST, optimize=2)
(Pdb) annotation
Name(source='bool', full='bool')
(Pdb) type(annotation)
<class 'griffe.expressions.Name'>

Looks like some kind of custom griffe object is getting in there.

Hopefully that's useful!

cpcloud · 2023-01-04T20:47:29Z

Created some gists, because the logs are huge:

pawamoy · 2023-01-04T20:48:00Z

Thanks! Yes I was able to find this as well in a debugging session 🙂 (had the clever 🥸 idea to disable all plugins except mkdocstrings, this way I don't need to install/setup anything special to test).
And I just found why it happens: simple scoping issue that I wasn't seeing. The annotation value at this point is the one from the previous iteration (we're in a loop). Aaaaah, it's the simple things sometimes... 😫
So, this is a bug, and this will be fixed... right now!

pawamoy · 2023-01-04T20:57:29Z

About the other two: of course, inherited members are not yet supported 😅
As a workaround, you can reference the base backend do_connect and table attributes instead 😕
I've started working on inheritance support, but it's not easy, so it won't be ready soon. In the meantime I might have the time to add an option to fallback on introspection for specific objects. I'll keep you posted!

pawamoy · 2023-01-04T21:09:56Z

Also, the bug was triggered thanks to the missing type here: https://github.com/ibis-project/ibis/blob/master/ibis/config.py#L110 (interactive, in Attributes section of Repr).

Issue #72: #72

cpcloud · 2023-01-04T21:17:38Z

Is there any way to get rid of the

WARNING  -  mkdocstrings_handlers: 3 aliases were still unresolved after 2 iterations

?

We like to build with mkdocs build --strict to avoid introducing documentation regressions.

pawamoy · 2023-01-04T21:19:22Z

I'll change the level to DEBUG 👍

pawamoy · 2023-01-04T21:34:31Z

Griffe 0.25.3 and mkdocstrings-python 0.8.3 released 🚀

cpcloud · 2023-01-05T12:10:53Z

Thank you for all that you do!

cpcloud mentioned this issue May 29, 2022

Circular dependency with mkdocstrings-python-legacy mkdocstrings/mkdocstrings#376

Closed

pawamoy mentioned this issue May 31, 2022

Stub Files and Wildcard Expansion issues #79

Closed

pawamoy added a commit that referenced this issue Jun 3, 2022

fix: Don't crash when overwriting a submodule with a wildcard importe…

bfad1cc

…d attribute Issue #72: #72 Issue #79: #79 Issue mkdocstrings/mkdocstrings#438: mkdocstrings/mkdocstrings#438

pawamoy added bug Something isn't working griffe: wildcard-imports Related to wildcard imports labels Jun 9, 2022

cpcloud mentioned this issue Jun 15, 2022

chore(deps): update dependency mkdocstrings to >=0.17.0,<0.20.0 ibis-project/ibis#4010

Closed

1 task

pawamoy added a commit that referenced this issue Jan 4, 2023

fix: Fix parsing of annotations in Numpy attributes sections

18fa396

Issue #72: #72

pawamoy closed this as completed Jan 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to build documentation for the ibis project #72

Unable to build documentation for the ibis project #72

cpcloud commented May 29, 2022

pawamoy commented May 29, 2022

pawamoy commented May 31, 2022

pawamoy commented Jun 3, 2022

pawamoy commented Dec 23, 2022

cpcloud commented Jan 4, 2023

cpcloud commented Jan 4, 2023 •

edited

Loading

pawamoy commented Jan 4, 2023 •

edited

Loading

cpcloud commented Jan 4, 2023

cpcloud commented Jan 4, 2023

pawamoy commented Jan 4, 2023 •

edited

Loading

pawamoy commented Jan 4, 2023

pawamoy commented Jan 4, 2023

cpcloud commented Jan 4, 2023

pawamoy commented Jan 4, 2023

pawamoy commented Jan 4, 2023

cpcloud commented Jan 5, 2023

Unable to build documentation for the ibis project #72

Unable to build documentation for the ibis project #72

Comments

cpcloud commented May 29, 2022

pawamoy commented May 29, 2022

pawamoy commented May 31, 2022

pawamoy commented Jun 3, 2022

pawamoy commented Dec 23, 2022

cpcloud commented Jan 4, 2023

cpcloud commented Jan 4, 2023 • edited Loading

Good news

Not as good news, but hopefully fixable without a huge amount of work.

pawamoy commented Jan 4, 2023 • edited Loading

cpcloud commented Jan 4, 2023

cpcloud commented Jan 4, 2023

pawamoy commented Jan 4, 2023 • edited Loading

pawamoy commented Jan 4, 2023

pawamoy commented Jan 4, 2023

cpcloud commented Jan 4, 2023

pawamoy commented Jan 4, 2023

pawamoy commented Jan 4, 2023

cpcloud commented Jan 5, 2023

cpcloud commented Jan 4, 2023 •

edited

Loading

pawamoy commented Jan 4, 2023 •

edited

Loading

pawamoy commented Jan 4, 2023 •

edited

Loading