Submodule exclusion - differences between approaches

[tlgs]

Problem Description

Despite what the documentation suggests, when it comes to submodule exclusion specifying __all__ does not have the same effect as passing negative regular expression !patterns as part of the module specification.

Specifically, specifing submodules in __all__ will render the modules' docstrings, as well as generate the API Documentation section; the same is not true when using the alternative approach.

I uderstand the difference in approaches, just thought it would be worth pointing out so as to make the documentation clearer.
Related: Perhaps allow for documentation specific issues? They don't really fit in with the bugs/enhancement options currently in place.

Steps to reproduce the behavior:

1. Generate docs using pdoc foo; __all__ = ["bar"] in foo/__init__.py
2. Generate docs using pdoc foo !foo.baz; no specified __all__ in foo/__init__.py
3. Compare results

System Information

pdoc: 9.0.0
Python: 3.10.1
Platform: Windows-10-10.0.22000-SP0

[mhils]

I'm not sure if I understand what you are saying. Listing a module in __all__ has the opposite effect — it includes the module — of specifying a negative module specification pattern, which excludes the module. Put differently, you can exclude a module by not mentioning it in __all__. Reading through https://pdoc.dev/docs/pdoc.html#exclude-submodules-from-being-documented, we only refer back to __all__, but do not mention the negation.

So again, not sure if I get you correctly. But would it help if instead of

If you would like to exclude specific submodules from the documentation, the recommended way is to specify __all__ as shown in the previous section.

we would say

If you would like to exclude specific submodules from the documentation, the recommended way is to specify __all__ as shown in the previous section and not include the submodules there.

?

[tlgs]

Thanks for the speedy response.

I'll lay out a concrete example to showcase the differences I tried conveying above. I'll use tlgs/the-code-book which is a rather simple package/module that I'm using to try out pdoc. Its structure follows:

.
├── codebook
│   ├── __init__.py
│   ├── substitution.py
│   ├── transposition.py
│   └── utils.py
├── pyproject.toml
├── README.md
├── setup.cfg
└── UNLICENSE

I'll exemplify both scenarios below; hopefully they illustrate what I meant above.
Again, I understand why the outputs differ, it's just that from the current documentation I would infer that both scenarios would produce the same result.

Scenario 1

codebook/__init_.py:

__all__ = ["substitution", "transposition"]

pdoc codebook generates:

Scenario 2

Empty codebook/__init__.py.

pdoc codebook !codebook.utils generates:

[mhils]

Ah, right - thanks for clarifying! This is indeed a bit of a tricky edge case we have here. __all__ currently also includes a submodule's documentation into the current page, so it is a bit overworked by doing two things at once. I think we have broadly have three possible ways forward:

1. Don't include submodules in the main contents, only keep them in the menu.
2. If a module has no members on its down, list all submodules instead (this would list them in your second example)
3. Extend the documentation and explain the subtleties.

I'll have to think about what works best - comments are welcome! The benefit of 1) would be that it is consistent with the rest of pdoc and also simplifies the template logic quite a bit. I'm not sure how useful it is to have submodules listed as page contents after all. The benefit of 2) would be the empty page in your second example would actually have useful content. Then again, this very quickly gets messy and unwieldy for slightly larger packages. Having an empty page could actually encourage folks to write useful docs that fill the empty space. 🙊 I'm not a big fan of 3) - it feels like we just have unnecessary complexity here.

[tlgs]

Happy to see this is actually helpful and could improve pdoc's experience!

I believe 1) describes the behavior I was expecting. I agree, documenting a package's __init__.py is vastly underused and is a good place to give an overview of the project/solution.

[mhils]

I've pushed experimental changes for approach 1) to main, see CHANGELOG.md. 😃

[mhils]

This is now fixed in pdoc 10, thanks again! 🍰