Module documentation doesn't show up

[teenjuna]

🌍 Environment

• Your operating system and version: MacOS 11.5.2
• Your python version: 3.9.7
• How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: using brew
• Your Rust version (rustc --version): rustc 1.54.0 (a178d0322 2021-07-26)
• Your PyO3 version: 0.14.5
• Have you tried using latest PyO3 main (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")?: yes

💥 Reproducing

Repo containing minimal working example: https://github.com/teenjuna/pyo3-doc-bug

❯ cat src/lib.rs
use pyo3::prelude::*;

/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok((a + b).to_string())
}

/// This comment should appear in module docs but it doesn't!
#[pymodule]
fn pyo3_doc_bug(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
    Ok(())
}

❯ maturin develop
🔗 Found pyo3 bindings
🐍 Found CPython 3.9 at python
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s

❯ python
Python 3.9.7 (default, Sep  3 2021, 04:31:11)
[Clang 12.0.5 (clang-1205.0.22.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyo3_doc_bug
>>> print(pyo3_doc_bug.__doc__)
None

[mejrs]

I reproduced this, looks like it stopped working with pyo3 0.14.0. @teenjuna can you verify that it works with pyo3 version 0.13.2?

[mejrs]

Ah, I think I know what is happening...

This for example, will print all those doc strings:

import pyo3_doc_bug

print(pyo3_doc_bug.__doc__) # None
print(pyo3_doc_bug.pyo3_doc_bug.__doc__) # This comment should appear in module docs but it doesn't!
print(pyo3_doc_bug.sum_as_string.__doc__) # Formats the sum of two numbers as string.

I think that change happened in 355bd0c#diff-7a9651b82a479b31816fe42ad26b5d5c78b92e0745efc9ac9882c1518c4ba794 which removed module.setattr("__doc__", doc)?;

I guess that you can't define package docstrings with PyModuleDef? 🤔

cc @davidhewitt

[birkenfeld]

Something is wrong with maturin (or maturin develop specifically?). If I merely build the module with cargo build --release, and link the .so from target/release, everything is fine:

>>> import pyo3_doc_bug
>>> pyo3_doc_bug.__doc__
"This comment should appear in module docs but it doesn't!"
>>> pyo3_doc_bug.pyo3_doc_bug
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'pyo3_doc_bug' has no attribute 'pyo3_doc_bug'
>>> pyo3_doc_bug.sum_as_string.__doc__
'Formats the sum of two numbers as string.'

[davidhewitt]

I expect that maturin creates a folder pyo3_doc_bug with the shared library pyo3_doc_bug inside it?

Looks like this is related to #1509 ? maturin was presumably silently depending on re-exporting __doc__ via __all__.

I think #1509 is probably correct, so we just need to consider whether there are tools and examples that need to update if we want to have package-level docs automatically pulled out of the shared library...

[birkenfeld]

Why is maturin changing a single module into a package?

[davidhewitt]

Not sure. @messense might know?

[messense]

Not sure. @messense might know?

PyO3/maturin#558

[messense]

I've opened PyO3/maturin#639 to address this, please take a look.

[davidhewitt]

Thanks very much. I guess we should also update our example projects... I'll open a PR now.

[konstin]

Why is maturin changing a single module into a package?

For adding .pyi files and because it's more consistent, i.e. a package is always lib/pythonX.Y/site-packages/<package_name> and not whatever the abi tag is. This prevents insidious situation like having e.g. both my_module.cpython-38-x86_64-linux-gnu.so and my_module.abi3.so in your side packages.

[messense]

The fix is now released in maturin 0.11.5.