Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace extension-module feature with PYO3_MAKE_EXTENSION_MODULE #1040

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Correct FFI definitions `Py_SetProgramName` and `Py_SetPythonHome` to take `*const` argument instead of `*mut`. [#1021](https://github.com/PyO3/pyo3/pull/1021)
- Rename `PyString::to_string` to `to_str`, change return type `Cow<str>` to `&str`. [#1023](https://github.com/PyO3/pyo3/pull/1023)
- Correct FFI definition `_PyLong_AsByteArray` `*mut c_uchar` argument instead of `*const c_uchar`. [#1029](https://github.com/PyO3/pyo3/pull/1029)
- Replace `extension-module` feature with `PYO3_MAKE_EXTENSION_MODULE` environment variable. See the migration guide for help with upgrading. [#1040](https://github.com/PyO3/pyo3/pull/1040)

### Removed
- Remove `PyString::as_bytes`. [#1023](https://github.com/PyO3/pyo3/pull/1023)
Expand Down
5 changes: 0 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,6 @@ nightly = []
# this is no longer needed internally, but setuptools-rust assumes this feature
python3 = []

# Use this feature when building an extension module.
# It tells the linker to keep the python symbols unresolved,
# so that the module can also be used with statically linked python interpreters.
extension-module = []

# The stable cpython abi as defined in PEP 384. Currently broken with
# many compilation errors. Pull Requests working towards fixing that
# are welcome.
Expand Down
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ crate-type = ["cdylib"]

[dependencies.pyo3]
version = "0.11.1"
features = ["extension-module"]
```

**`src/lib.rs`**
Expand Down
2 changes: 1 addition & 1 deletion build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -485,7 +485,7 @@ fn configure(interpreter_config: &InterpreterConfig) -> Result<String> {

check_target_architecture(interpreter_config)?;

let is_extension_module = env::var_os("CARGO_FEATURE_EXTENSION_MODULE").is_some();
let is_extension_module = env::var_os("PYO3_MAKE_EXTENSION_MODULE").is_some();
if !is_extension_module || cfg!(target_os = "windows") {
println!("{}", get_rustc_link_lib(&interpreter_config)?);
if let Some(libdir) = &interpreter_config.libdir {
Expand Down
1 change: 0 additions & 1 deletion examples/rustapi_module/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ edition = "2018"

[dependencies.pyo3]
path = "../../"
features = ["extension-module"]

[lib]
name = "rustapi_module"
Expand Down
2 changes: 1 addition & 1 deletion examples/word-count/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ edition = "2018"

[dependencies]
rayon = "1.0.2"
pyo3 = { path = "../..", features = ["extension-module"] }
pyo3 = { path = "../.." }

[lib]
name = "word_count"
Expand Down
28 changes: 4 additions & 24 deletions guide/src/building_and_distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,35 +8,15 @@ PyO3 uses a build script to determine the Python version and set the correct lin

Different linker arguments must be set for libraries/extension modules and binaries, which includes both standalone binaries and tests. (More specifically, binaries must be told where to find libpython and libraries must not link to libpython for [manylinux](https://www.python.org/dev/peps/pep-0513/) compliance).

Since PyO3's build script can't know whether you're building a binary or a library, you have to activate the `extension-module` feature to get the build options for a library, or it'll default to binary.

If you have e.g. a library crate and a profiling crate alongside, you need to use optional features. E.g. you put the following in the library crate:

```toml
[dependencies]
pyo3 = "0.6"

[lib]
name = "hyperjson"
crate-type = ["rlib", "cdylib"]

[features]
default = ["pyo3/extension-module"]
```

And this in the profiling crate:

```toml
[dependencies]
my_main_crate = { path = "..", default-features = false }
pyo3 = "0.6"
```
Since PyO3's build script can't know whether you're building a binary or a library, it defaults to building for binaries. To build a library, you should set the `PYO3_MAKE_EXTENSION_MODULE` environment variable to a nonempty value.

On Linux/macOS you might have to change `LD_LIBRARY_PATH` to include libpython, while on windows you might need to set `LIB` to include `pythonxy.lib` (where x and y are major and minor version), which is normally either in the `libs` or `Lib` folder of a Python installation.

## Distribution

There are two ways to distribute your module as a Python package: The old, [setuptools-rust](https://github.com/PyO3/setuptools-rust), and the new, [maturin](https://github.com/pyo3/maturin). setuptools-rust needs several configuration files (`setup.py`, `MANIFEST.in`, `build-wheels.sh`, etc.). maturin doesn't need any configuration files, however it does not support some functionality of setuptools such as package data ([pyo3/maturin#258](https://github.com/PyO3/maturin/issues/258)) and requires a rigid project structure, while setuptools-rust allows (and sometimes requires) configuration with python code.
There are two ways to distribute your module as a Python package:
- [`maturin`](https://github.com/pyo3/maturin) is the typical way to build and package PyO3 extension modules. It is deliberately opinionated about project structure, which allows it to need minimal configuration.
- [`setuptools-rust`](https://github.com/PyO3/setuptools-rust), in contrast, needs several configuration files (`setup.py`, `MANIFEST.in`, `build-wheels.sh`, etc.). This complexity comes at added flexibility, allowing configuration using python code.

## Cross Compiling

Expand Down
11 changes: 1 addition & 10 deletions guide/src/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,4 @@ PyO3 provides a struct [`GILOnceCell`] which works equivalently to `OnceCell` bu

## I can't run `cargo test`: I'm having linker issues like "Symbol not found" or "Undefined reference to _PyExc_SystemError"!

Currently, [#341](https://github.com/PyO3/pyo3/issues/341) causes `cargo test` to fail with linking errors when the `extension-module` feature is activated. For now you can work around this by making the `extension-module` feature optional and running the tests with `cargo test --no-default-features`:

```toml
[dependencies.pyo3]
version = "*"

[features]
extension-module = ["pyo3/extension-module"]
default = ["extension-module"]
```
If `PYO3_MAKE_EXTENSION_MODULE` environment variable is non-empty, PyO3 will not link against libpython. This will lead to errors like the above at link time.
5 changes: 3 additions & 2 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,9 @@
//!
//! ## Using Rust from Python
//!
//! PyO3 can be used to generate a native Python module.
//! PyO3 can be used to generate a native Python module. For a zero-configuration
//! package to aid with compiling and packaging the module, see
//! [`maturin`](https://github.com/PyO3/maturin).
//!
//! **`Cargo.toml`**
//!
Expand All @@ -53,7 +55,6 @@
//!
//! [dependencies.pyo3]
//! version = "0.11.1"
//! features = ["extension-module"]
//! ```
//!
//! **`src/lib.rs`**
Expand Down