note: perhaps two different versions of crate ndarray are being used?

[1138886114]

I'm learning to use rust to write WHL files for Python. When I first started running the official sample code, I encountered an error.

Cargo.toml

[lib]
name = "rust_ext"
crate-type = ["cdylib"]

[dependencies]
numpy = "0.13"
ndarray = "0.14"

[dependencies.pyo3]
version = "0.13"
features = ["extension-module"]

lib.rs

use ndarray::{ArrayD, ArrayViewD, ArrayViewMutD};
use numpy::{IntoPyArray, PyArrayDyn, PyReadonlyArrayDyn};
use pyo3::prelude::{pymodule, PyModule, PyResult, Python};

#[pymodule]
fn rust_ext(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
    // immutable example
    fn axpy(a: f64, x: ArrayViewD<'_, f64>, y: ArrayViewD<'_, f64>) -> ArrayD<f64> {
        a * &x + &y
    }

    // mutable example (no return)
    fn mult(a: f64, mut x: ArrayViewMutD<'_, f64>) {
        x *= a;
    }

    // wrapper of `axpy`
    #[pyfn(m, "axpy")]
    fn axpy_py<'py>(
        py: Python<'py>,
        a: f64,
        x: PyReadonlyArrayDyn<f64>,
        y: PyReadonlyArrayDyn<f64>,
    ) -> &'py PyArrayDyn<f64> {
        let x = x.as_array();
        let y = y.as_array();
        axpy(a, x, y).into_pyarray(py)
    }

    // wrapper of `mult`
    #[pyfn(m, "mult")]
    fn mult_py(_py: Python<'_>, a: f64, x: &PyArrayDyn<f64>) -> PyResult<()> {
        let x = unsafe { x.as_array_mut() };
        mult(a, x);
        Ok(())
    }

    Ok(())
}

The contents of the error report are as follows:

D:\Rust\pyo3\numpy_test>cargo build
   Compiling numpy_test v0.1.0 (D:\Rust\pyo3\numpy_test)
error[E0308]: mismatched types
  --> src\lib.rs:74:17
   |
74 |         axpy(a, x, y).into_pyarray(py)
   |                 ^ expected struct `ArrayBase`, found struct `ndarray::ArrayBase`
   |
   = note: expected struct `ArrayBase<ViewRepr<&f64>, Dim<IxDynImpl>>`
              found struct `ndarray::ArrayBase<ndarray::ViewRepr<&f64>, ndarray::dimension::dim::Dim<ndarray::dimension::dynindeximpl::IxDynImpl>>`
   = note: perhaps two different versions of crate `ndarray` are being used?

error[E0308]: mismatched types
  --> src\lib.rs:74:20
   |
74 |         axpy(a, x, y).into_pyarray(py)
   |                    ^ expected struct `ArrayBase`, found struct `ndarray::ArrayBase`
   |
   = note: expected struct `ArrayBase<ViewRepr<&f64>, Dim<IxDynImpl>>`
              found struct `ndarray::ArrayBase<ndarray::ViewRepr<&f64>, ndarray::dimension::dim::Dim<ndarray::dimension::dynindeximpl::IxDynImpl>>`
   = note: perhaps two different versions of crate `ndarray` are being used?

error[E0599]: no method named `into_pyarray` found for struct `ArrayBase<OwnedRepr<f64>, Dim<IxDynImpl>>` in the current scope
  --> src\lib.rs:74:23
   |
74 |         axpy(a, x, y).into_pyarray(py)
   |                       ^^^^^^^^^^^^ method not found in `ArrayBase<OwnedRepr<f64>, Dim<IxDynImpl>>`

error[E0308]: mismatched types
  --> src\lib.rs:81:17
   |
81 |         mult(a, x);
   |                 ^ expected struct `ArrayBase`, found struct `ndarray::ArrayBase`
   |
   = note: expected struct `ArrayBase<ViewRepr<&mut f64>, Dim<IxDynImpl>>`
              found struct `ndarray::ArrayBase<ndarray::ViewRepr<&mut f64>, ndarray::dimension::dim::Dim<ndarray::dimension::dynindeximpl::IxDynImpl>>`
   = note: perhaps two different versions of crate `ndarray` are being used?

warning: unused import: `IntoPyArray`
  --> src\lib.rs:49:13
   |
49 | use numpy::{IntoPyArray, PyArrayDyn, PyReadonlyArrayDyn};
   |             ^^^^^^^^^^^
   |
   = note: `#[warn(unused_imports)]` on by default

error: aborting due to 4 previous errors; 1 warning emitted

Some errors have detailed explanations: E0308, E0599.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `numpy_test`

To learn more, run the command again with --verbose.

[Equim-chan]

Try ndarray = "0.15"

[kngwyu]

Anyway I have to release 0.14...

[digitalillusions]

Hi, I also ran into this same problem when trying to compile some old code (couple of months old). For me the problem is that conflicting versions of ndarray exist when trying to use the IntoPyArray trait https://docs.rs/numpy/0.14.1/numpy/convert/trait.IntoPyArray.html#impl-IntoPyArray-for-ArrayBase%3COwnedRepr%3CA%3E%2C%20D%3E. I think the cleanest solution to this would be to either reexport the version of ndarray used by this crate, or to fix the version of ndarray for a specific version of this crate in order to guarantee future compatibility and predictable behavior.

As it stands, numpy will use a newer version of ndarray as it becomes available which will always break the compatibility with older versions and requiring projects to either use the same loose requirements on ndarray as this crate, or to continuously keep their version of ndarray up to date manually.

I think fixing the version of ndarray is the easiest fix, while reexporing is the most convenient (as long as it is clearly documented in the readme).

[adamreichold]

As it stands, numpy will use a newer version of ndarray as it becomes available which will always break the compatibility with older versions and requiring projects to either use the same loose requirements on ndarray as this crate, or to continuously keep their version of ndarray up to date manually.

While I agree that re-exporting ndarray would be wise as this crate's API is relatively tightly coupled to it, I am not sure the above is strictly correct. This crate currently declares the dependency

ndarray = ">= 0.13, < 0.16"

meaning that only versions 0.13.x, 0.14.x and 0.15.x of ndarray are applicable, not any newly released version.

If I understand things correctly, this is deliberately looser than the default ndarray = "0.13" which would only include version 0.13.x as versions 0.a and 0.b are considered incompatible for a != b by Cargo.

I think fixing the version of ndarray is the easiest fix,

I think this would go against the intention of the deliberately more loose version specification, i.e. allowing different versions of ndarray to be used with this crate. Hence, I think re-exporting or replicating the above version specification in downstream crates are preferable choices. Of course, mentioning them in the README would be nice.

[digitalillusions]

I think this would go against the intention of the deliberately more loose version specification, i.e. allowing different versions of ndarray to be used with this crate.

I understand that this might have been the intention in theory, but in practice this does not seem to work for (I may be doing something wrong though, still quite new to rust and cargo). I specify in my Cargo.toml

[dependencies]
ndarray = "0.14.0"  # ">=0.14.0, <0.15.0"
numpy = "0.13.0" # ">=0.13.0, <0.14.0"

which resolves to ndarray = "0.14.0" and numpy = "0.13.2", which is expected from this dependency resolution. Glossing over the fact that numpy = "0.13.2" and numpy = "0.13.0" have different version requirements on the ndarray crate, the ndarray dependency of numpy = "0.13.2" with ndarray = ">=0.13, <0.16" is resolved to ndarray = "0.15.2", resulting in conflicting versions when using ndarray and trying to call .into_pyarray from numpy.

It seems to me that while my requested version of ndarray is in the range of compatible versions of numpys requirements, conflicting versions are chosen nonetheless. Is there some way to avoid this?

[w1th0utnam3]

The Dependency Resolution page says

When multiple packages specify a dependency for a common package, the resolver attempts to ensure that they use the same version of that common package, as long as they are within a SemVer compatibility range.

This sounds a bit ambiguous but to me it seems like this does not mean that it looks for just any SemVer compatible subrange. Apparently it means the different ranges have to be entirely SemVer compatible for it to try to find a common version of the dependency.

So in the case of the ndarray ndarray = ">=0.13, <0.16" is incompatible with ndarray = "0.14.0" because the former permits versions which are incompatible with the latter so it resolves them separately.

EDIT: So this would mean, if you specify a range of SemVer breaking versions, the only way to unify them would be to specify the same identical range (or at least one that doesn't include more or fewer SemVer breaking versions). Maybe I'm wrong but I'm not sure how to explain the behavior observed by @digitalillusions otherwise.

[adamreichold]

So in the case of the ndarray ndarray = ">=0.13, <0.16" is incompatible with ndarray = "0.14.0" because the former permits versions which are incompatible with the latter so it resolves them separately.

Indeed, specifying

[dependencies]
numpy = "0.14"
ndarray = ">= 0.13, < 0.16"

does resolve to a single version for ndarray, namely 0.15.3 at this point in time:

> cargo tree
numpy-ndarray-test v0.1.0 (/home/adam/numpy-ndarray-test)
├── ndarray v0.15.3
│   ├── matrixmultiply v0.3.1
│   │   └── rawpointer v0.2.1
│   ├── num-complex v0.4.0
│   │   └── num-traits v0.2.14
│   │       [build-dependencies]
│   │       └── autocfg v1.0.1
│   ├── num-integer v0.1.44
│   │   └── num-traits v0.2.14 (*)
│   │   [build-dependencies]
│   │   └── autocfg v1.0.1
│   ├── num-traits v0.2.14 (*)
│   └── rawpointer v0.2.1
└── numpy v0.14.1
    ├── cfg-if v0.1.10
    ├── libc v0.2.102
    ├── ndarray v0.15.3 (*)
    ├── num-complex v0.4.0 (*)
    ├── num-traits v0.2.14 (*)
    └── pyo3 v0.14.5
        ├── cfg-if v1.0.0
        ├── libc v0.2.102
        └── parking_lot v0.11.2
            ├── instant v0.1.10
            │   └── cfg-if v1.0.0
            ├── lock_api v0.4.5
            │   └── scopeguard v1.1.0
            └── parking_lot_core v0.8.5
                ├── cfg-if v1.0.0
                ├── instant v0.1.10 (*)
                ├── libc v0.2.102
                └── smallvec v1.6.1
        [build-dependencies]
        └── pyo3-build-config v0.14.5
            └── once_cell v1.8.0

I agree, that this makes the version specification ndarray = ">=0.13, <0.16" less useful than probably originally intended.

Looking at ndarray's reverse dependencies, the plain ndarray = "0.15" seems preferable from an ecosystem perspective. Still, re-exporting would probably be a nice API courtesy in any case.