How does testing fit into a Maturin module?

[chriskilding]

I'm writing a Maturin module (which will either be all-Rust or mixed Python/Rust), and want to add some unit tests to it.

In the pure Python world, this would be easy enough: create a test folder, put the test_*.py files in it, add pytest (or whatever framework) to the project dev dependencies, then install and run the tests with your favourite tool (pip, poetry, pipenv etc).

But what's the right way to put tests into a Maturin module?

Some considerations:

• To exercise the public API of the Python module, we'll want to write tests in Python + a Python test framework.
• To exercise the Rust functions inside the module, we'll want to either:
  • Write tests in Rust + a Rust test framework
  • Write tests in Python, but then we would need a way to control the visibility of exported Rust functions, so they can be visible for testing but not publicly visible outside the module.
• A standardised entry point to make the tests run might be needed. I don't have a view yet on whether Maturin should handle this (e.g. a maturin test command), or whether the test framework runner tools (e.g. poetry run pytest) should handle it.

[messense]

IMO it's pretty much the same as any C/C++ extensions. While you can write and run tests in Rust, I think it'd be better to run it via pytest/unitest or whatever Python test framework to exercise the actually use case (an importable Python extension module).

I don't believe a maturin test subcommand is needed, even setuptools has deprecated setup.py test.

[chriskilding]

Okay, the Python testing approach sounds like the way to go.

Is there a particular Python build tool that Maturin is designed for? The use of pyproject.toml suggests it was made with Poetry in mind, but (a) IIRC pyproject.toml is a thing in its own right (complete with a PEP standard) not tied to Poetry, and (b) when I try to use Maturin with Poetry I find I have to duplicate a lot of stuff from the project section into tool.poetry, tool.poetry.dependencies etc - because it seems Poetry can't read these properties from the project section.

[messense]

That's a problem of Poetry because it does not yet support PEP 621.

Is there a particular Python build tool that Maturin is designed for?

There isn't since maturin itself is a build tool. But you can use also pip or build to drive the process.

[messense]

To exercise the Rust functions inside the module

If the Rust functions are totally independent from Python, consider move it to a separate crate and use standard Rust tooling to build and test.

If that's too much trouble, you can also consider feature-gate your Python module. See PyO3/pyo3#2271

[messense]

Write tests in Python, but then we would need a way to control the visibility of exported Rust functions, so they can be visible for testing but not publicly visible outside the module.

You can sort of do this via cargo features, for example add a off-by-default testing feature and feature-gate code for testing to make them publicly visible only when testing feature is activated, then build the module with testing feature activated to run your tests.

[chriskilding]

This sounds... not as simple as it should be, given that testing is a fundamental of any Python module.

Could a Maturin Rust macro (or an extra visibility argument to the pyfunction / pymodule macros) be devised for this? So you can simply annotate your Rust function with it and you're ready to start testing?

[messense]

IMO maturin is a build backend, it should not care about how you expose your API.

[chriskilding]

You're right, pyfunction and pymodule come from PyO3's prelude, not Maturin 🤦 I'll ask the PyO3 people about it