Python docstrings on the root module are insufficient

[rtyler]

Environment

Delta-rs version: 0.10.1

Binding: python

Environment: Linux/amd64

• Cloud provider:
• OS:
• Other:

---

Bug

What happened:

>>> help(deltalake)
Help on package deltalake:

NAME
    deltalake

PACKAGE CONTENTS
    _internal
    data_catalog
    exceptions
    fs
    schema
    table
    writer

FUNCTIONS
    rust_core_version()

DATA
    DataType = typing.Union[ForwardRef('PrimitiveType'), Forwar...orwardRe...

VERSION
    0.10.1

What you expected to happen:

The DeltaTable symbol is missing and I would expect it to show up here in the doc strings

How to reproduce it:

More details:

[rchojn]

Hi @rtyler can I take this one?

[ion-elgreco]

@gardnertechhq go ahead :) Happy to see some improvements on the python docs

[rchojn]

cool, any tip where to start? I see that rendering docs in sphinx

 sphinx-build -b html source build

return 32 warnings. Have you thought about sphinx linter, e.g in form of pre-commit hook?

[ion-elgreco]

@rchojn we don't use sphinx anymore, it's only mkdocs now

[rchojn]

Hey @ion-elgreco, if that's the case, should we decommission this part?

[ion-elgreco]

@rchojn eventually yes, @r3stl355 currently has put some redirects on the old docs

[rchojn]

thanks @ion-elgreco ! so I should be able to build mkdocs using this from delta-rs/python folder, right?

[rchojn]

I followed contributing guide and my make build-docs fails:

INFO    -  [macros] - Extra variables (config file): ['python_api_url', 'generator', 'social']
INFO    -  [macros] - Extra filters (module): ['pretty']
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /home/chojnar1/Desktop/personal/delta-rs/site
ERROR   -  mkdocstrings: With sys.path = ['/home/chojnar1/.pyenv/versions/3.8.10/bin', '/home/chojnar1/.pyenv/versions/3.8.10/lib/python38.zip', '/home/chojnar1/.pyenv/versions/3.8.10/lib/python3.8',
           '/home/chojnar1/.pyenv/versions/3.8.10/lib/python3.8/lib-dynload', '/home/chojnar1/.local/lib/python3.8/site-packages', '/home/chojnar1/.pyenv/versions/3.8.10/lib/python3.8/site-packages',
           '/home/chojnar1/Desktop/personal/delta-rs', '/home/chojnar1/Desktop/personal'], accessing 'deltalake' raises ModuleNotFoundError: No module named 'deltalake'
ERROR   -  Error reading page 'api/catalog.md':
ERROR   -  Could not collect 'deltalake.data_catalog.DataCatalog'

Aborted with a BuildError!
make: *** [Makefile:92: build-docs] Error 1

any tip?

[r3stl355]

You likely need to build delta. Try running make delelop before mkdocs

[rchojn]

Not sure about warnings, but make develop successfully passed:

   Compiling deltalake-azure v0.1.0 (/home/chojnar1/Desktop/personal/delta-rs/crates/deltalake-azure)
warning: variant `ManagedIdentity` is never constructed
  --> crates/deltalake-azure/src/config.rs:38:5
   |
30 | enum AzureCredential {
   |      --------------- variant in this enum
...
38 |     ManagedIdentity,
   |     ^^^^^^^^^^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: variant `Parse` is never constructed
 --> crates/deltalake-azure/src/error.rs:8:5
  |
6 | pub(crate) enum Error {
  |                 ----- variant in this enum
7 |     #[error("failed to parse config: {0}")]
8 |     Parse(String),
  |     ^^^^^
  |
  = note: `Error` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis

warning: variant `Parse` is never constructed
 --> crates/deltalake-gcp/src/error.rs:8:5
  |
6 | pub(crate) enum Error {
  |                 ----- variant in this enum
7 |     #[error("failed to parse config: {0}")]
8 |     Parse(String),
  |     ^^^^^
  |
  = note: `Error` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
  = note: `#[warn(dead_code)]` on by default

warning: `deltalake-azure` (lib) generated 2 warnings
warning: `deltalake-gcp` (lib) generated 1 warning
   Compiling deltalake v0.17.0 (/home/chojnar1/Desktop/personal/delta-rs/crates/deltalake)
   Compiling deltalake-python v0.15.1 (/home/chojnar1/Desktop/personal/delta-rs/python)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 31s
📦 Built wheel for abi3 Python ≥ 3.8 to /tmp/.tmp7AgSFZ/deltalake-0.15.1-cp38-abi3-linux_x86_64.whl
🛠 Installed deltalake-0.15.1

Unfortunately error persist and I cannot build the docs :/

[r3stl355]

Make sure you are running make develop followed by make build-docs in the same virtual environment. Ignore the build warnings for this, all should be good so long as you get Installed deltalake-...

[rchojn]

Thanks @r3stl355! Somehow after restarting terminal it starts to work, no updates were made from my side, magic 🪄