"Lexer is not generic" when making Python module a feature

[GuillaumeDesforges]

I'm trying to make very simple Python bindings to tweag/nickel, which works in GuillaumeDesforges/nickel@23234df

Now, if I try to make it optional (GuillaumeDesforges/nickel@6386585), so that one can build the library without the Python bindings, maturin develop fails:

(.venv) gdforj@kaguya:nickel$ RUST_BACKTRACE=1 maturin build
🔗 Found cffi bindings
🐍 Using CPython 3.10 at /home/gdforj/pro/tweag/nickel/.venv/bin/python3 to generate the cffi bindings
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
thread 'main' panicked at 'Lexer is not generic', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/cbindgen-0.24.3/src/bindgen/ir/opaque.rs:105:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
   2: <cbindgen::bindgen::ir::opaque::OpaqueItem as cbindgen::bindgen::ir::item::Item>::instantiate_monomorph
   3: cbindgen::bindgen::ir::ty::Type::add_monomorphs
   4: cbindgen::bindgen::library::Library::generate
   5: cbindgen::bindgen::builder::Builder::generate
   6: maturin::module_writer::write_cffi_module
   7: maturin::build_context::BuildContext::build_cffi_wheel
   8: maturin::build_context::BuildContext::build_wheels
   9: maturin::run
  10: maturin::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I use maturin version 0.14.8.

To reproduce, you can clone my branch https://github.com/GuillaumeDesforges/nickel/tree/python, then

$ nix develop
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install maturin==0.14.8
$ cargo build
# works
$ cargo build --features python-module
# works
$ maturin build
# fails

[messense]

maturin relies on cargo metadata output to determine which bindings you are using, so making pyo3 optional would hide it from cargo metadata by default so maturin detects the wrong bindings.

What you can do is adding a pyproject.toml and tell maturin that you want to enable the pyo3 feature.

[build-system]
requires = ["maturin>=0.14,<0.15"]
build-backend = "maturin"

[tool.maturin]
features = ["python-module"]

See also https://pyo3.rs/v0.17.3/faq#i-cant-run-cargo-test-or-i-cant-build-in-a-cargo-workspace-im-having-linker-issues-like-symbol-not-found-or-undefined-reference-to-_pyexc_systemerror

[messense]

maturin develop --features python-module and maturin build --features python-module should also work, but it's better to add it in pyproject.toml so you can properly install from source distribution.

[GuillaumeDesforges]

Thanks for the quick response! 🙏

I think this is what I did, I had added the pyproject.toml as suggested:
GuillaumeDesforges/nickel@6386585#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711

But maturin still failed. Is it a bug?

[GuillaumeDesforges]

Oh wait, that's not true, I've used cargo-extra-args not features, will try it.

[messense]

cargo-extra-args was removed in maturin 0.13, use features = ["python-module"] instead.

[GuillaumeDesforges]

It worked, thanks a lot!