Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Eigen::Tensor & Eigen::TensorMap support #4201

Merged
merged 141 commits into from
Oct 18, 2022
Merged

Conversation

EthanSteinberg
Copy link
Collaborator

@EthanSteinberg EthanSteinberg commented Sep 27, 2022

Description

Adds Eigen::Tensor support to eigen.h.

Eigen::Tensor allows you to handle np.arrays with more than two dimensions, which is quite handy in many cases.

I also added tests for all the relevant behavior.

Note that the Eigen::TensorMap implementation could be better if we allow std::optional, but I'm not sure if we are allowed to require that in this codebase.

Closes #1377
Closes #2119

Suggested changelog entry:

pybind11/eigen/tensor.h`` adds converters to and from `Eigen::Tensor` and `Eigen::TensorMap`

@Skylion007
Copy link
Collaborator

Skimming this, the casters should probably be refactored to using the EigenProps helper struct? Unless there is a reason it's defining it's own helper structs?

@EthanSteinberg
Copy link
Collaborator Author

@Skylion007 I'm hesitant to merge the two since there wouldn't be that much shared code. Most of the EigenProps helper construct is dedicated to dealing with weird setups with partially dynamic matrix sizes, which Tensor doesn't do.

@EthanSteinberg
Copy link
Collaborator Author

@Skylion007 Can you do me a favor and trigger another run of the CI? I think I fixed the build errors on older versions of C++.

@Skylion007
Copy link
Collaborator

@lalaland Perhaps the common code snippits between the two should be declared in another class/struct that they inherit from then?

@EthanSteinberg
Copy link
Collaborator Author

@Skylion007 It turns out I wasn't even used the shared logic (which was just a mutable_data vs data call on array), so I deleted it. There is now currently no overlap between the two.

include/pybind11/eigen.h Outdated Show resolved Hide resolved
include/pybind11/eigen.h Outdated Show resolved Hide resolved
include/pybind11/eigen.h Outdated Show resolved Hide resolved
include/pybind11/eigen.h Outdated Show resolved Hide resolved
include/pybind11/eigen.h Outdated Show resolved Hide resolved
@EthanSteinberg
Copy link
Collaborator Author

@Skylion007 Thanks for the feedback! I think I addressed all of the issues you pointed out.

@rwgk
Copy link
Collaborator

rwgk commented Sep 28, 2022

Very quick high-level impression after looking for only a couple minutes: it would be much better IMO to add this code under a separate eigen_tensor.h. Additional though with less certainty: include eigen_tensor.h from eigen.h, or maybe do a more complete job, move the current implementation to eigen_matrix.h (if that name makes sense), and eigen.h includes both.

Background thought: in this PR it's obvious what's old (not tensor) and what's new (tensor), but once it's merged it's one big soup for everybody else.

@EthanSteinberg
Copy link
Collaborator Author

@rwgk Good and easy suggestion. Done.

@EthanSteinberg EthanSteinberg force-pushed the tensor branch 3 times, most recently from 3d8b19e to c034034 Compare September 28, 2022 20:47
@rwgk
Copy link
Collaborator

rwgk commented Sep 28, 2022

Something quick & trivial: to avoid the auto fixes, run pre-commit run --all-files before you commit & push.

The first time through it can take a couple minutes, after that it's less than 12 seconds on my machine.

Copy link
Collaborator

@Skylion007 Skylion007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more nits.

include/pybind11/detail/eigen_tensor.h Outdated Show resolved Hide resolved
include/pybind11/detail/eigen_tensor.h Outdated Show resolved Hide resolved
include/pybind11/detail/eigen_tensor.h Outdated Show resolved Hide resolved
tests/test_eigen_tensor.cpp Outdated Show resolved Hide resolved
}

handle result = array_t<typename Type::Scalar, compute_array_flag_from_tensor<Type>()>(
H::get_shape(*src), src->data(), parent_object)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parent_object gets converted to handle here. If it's not released, it will decrefed when it goes out of scope, which I think is slightly different behavior from what you had before.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a release right there in the next line? Anyways, I swapped these to release at the return statement instead.

@EthanSteinberg
Copy link
Collaborator Author

@rwgk Wasn't able to get that to work on my machine. I'll just squash this PR once all comments are resolved (so they won't clutter the git history).

@rwgk
Copy link
Collaborator

rwgk commented Sep 30, 2022

@rwgk Wasn't able to get that to work on my machine. I'll just squash this PR once all comments are resolved (so they won't clutter the git history).

It will be auto-squashed when we merge.

Please don't squash manually, that'd make it more difficult for reviewers to follow, and hard for reviewers and you to backtrack.

BTW: I may only get a chance to look carefully this weekend.

@EthanSteinberg
Copy link
Collaborator Author

Does anyone have any tips for setting up an environment to debug those windows pypy errors?

@rwgk
Copy link
Collaborator

rwgk commented Sep 30, 2022 via email

@rwgk
Copy link
Collaborator

rwgk commented Sep 30, 2022 via email

@EthanSteinberg
Copy link
Collaborator Author

@rwgk That might be a good idea. I just added the remaining tests for all the current expected behavior.

Although I'm not sure that clang's sanitizers will catch anything. The code is already clean according to valgrind.

@rwgk
Copy link
Collaborator

rwgk commented Sep 30, 2022

@rwgk That might be a good idea. I just added the remaining tests for all the current expected behavior.

Although I'm not sure that clang's sanitizers will catch anything. The code is already clean according to valgrind.

Cool, I'll try, thanks!

I don't have exact data, but from memory, we had cases where one or the other (valgrind, clang) didn't catch something, either way around.

@rwgk
Copy link
Collaborator

rwgk commented Oct 1, 2022

I had some fun resolving smart_holder merge conflicts, and hiccups with my usual Linux build environment (I'm using scons), but I got those resolved, too. But when trying to build with our internal toolchain I'm running into errors that seem related to the Eigen version. In my interactive standard Linux build the Eigen version is 3.4.0, but with our internal toolchain it is 3.4.90. That version was installed from this tar file:

https://gitlab.com/libeigen/eigen/-/archive/b3bf8d6a13585ff248c079402654647d298de60b/eigen-b3bf8d6a13585ff248c079402654647d298de60b.tar.gz

(This is the first time that I'm having issues with our internally used Eigen version not working while the pybind11 GitHub CI and my interactive standard Linux are fine.)

I'm pasting the error message below. I haven't tried to understand it yet, hoping you may understand it quicker because you are probably much more familiar with the Eigen code.

In file included from third_party/pybind11/tests/test_eigen_tensor.cpp:10:
In file included from third_party/pybind11/include/pybind11/eigen.h:13:
./third_party/pybind11/include/pybind11/detail/eigen_tensor.h:65:16: error: no viable conversion from returned value of type 'const Dimensions' (aka 'const DSizes<long, 3>') to function return type 'std::array<typename T::Index, T::NumIndices>' (aka 'array<long, T::NumIndices>')
        return f.dimensions();
               ^~~~~~~~~~~~~~
./third_party/pybind11/include/pybind11/detail/eigen_tensor.h:256:16: note: in instantiation of member function 'pybind11::detail::eigen_tensor_helper<Eigen::Tensor<double, 3, 0>>::get_shape' requested here
            H::get_shape(*src), src->data(), parent_object);
               ^
./third_party/pybind11/include/pybind11/detail/eigen_tensor.h:181:16: note: in instantiation of function template specialization 'pybind11::detail::type_caster<Eigen::Tensor<double, 3, 0>>::cast_impl<Eigen::Tensor<double, 3, 0>>' requested here
        return cast_impl(src, policy, parent);
               ^
./third_party/pybind11/include/pybind11/detail/type_caster_odr_guard.h:121:32: note: in instantiation of member function 'pybind11::detail::type_caster<Eigen::Tensor<double, 3, 0>>::cast' requested here
        return TypeCasterType::cast(
                               ^
third_party/pybind11/include/pybind11/pybind11.h:250:29: note: in instantiation of function template specialization 'pybind11::detail::type_caster_odr_guard<Eigen::Tensor<double, 3, 0>, pybind11::detail::type_caster<Eigen::Tensor<double, 3, 0>>>::cast<Eigen::Tensor<double, 3, 0> *>' requested here
                = cast_out::cast(std::move(args_converter).template call<Return, Guard>(cap->f),
                            ^
third_party/pybind11/include/pybind11/pybind11.h:101:9: note: in instantiation of function template specialization 'pybind11::cpp_function::initialize<(lambda at third_party/pybind11/tests/test_eigen_tensor.cpp:51:24), Eigen::Tensor<double, 3, 0> *, pybind11::name, pybind11::scope, pybind11::sibling, pybind11::return_value_policy>' requested here
        initialize(
        ^
third_party/pybind11/include/pybind11/pybind11.h:1154:22: note: in instantiation of function template specialization 'pybind11::cpp_function::cpp_function<(lambda at third_party/pybind11/tests/test_eigen_tensor.cpp:51:24), pybind11::name, pybind11::scope, pybind11::sibling, pybind11::return_value_policy, void>' requested here
        cpp_function func(std::forward<Func>(f),
                     ^
third_party/pybind11/tests/test_eigen_tensor.cpp:50:7: note: in instantiation of function template specialization 'pybind11::module_::def<(lambda at third_party/pybind11/tests/test_eigen_tensor.cpp:51:24), pybind11::return_value_policy>' requested here
    m.def(
      ^
third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/array:150:29: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const Dimensions' (aka 'const DSizes<long, 3>') to 'const std::array<long, 3> &' for 1st argument
struct _LIBCPP_TEMPLATE_VIS array
                            ^
third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/array:150:29: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const Dimensions' (aka 'const DSizes<long, 3>') to 'std::array<long, 3> &&' for 1st argument
In file included from third_party/pybind11/tests/test_eigen_tensor.cpp:10:
In file included from third_party/pybind11/include/pybind11/eigen.h:13:
In file included from ./third_party/pybind11/include/pybind11/detail/eigen_tensor.h:35:
In file included from third_party/eigen3/unsupported/Eigen/CXX11/Tensor:131:
./third_party/eigen3/unsupported/Eigen/CXX11/src/Tensor/TensorMap.h:99:26: error: no matching constructor for initialization of 'Dimensions' (aka 'DSizes<long, 3>')
      : m_data(dataPtr), m_dimensions(dimensions)
                         ^            ~~~~~~~~~~
./third_party/pybind11/include/pybind11/detail/eigen_tensor.h:138:17: note: in instantiation of function template specialization 'Eigen::TensorMap<Eigen::Tensor<double, 3, 0>, 0>::TensorMap<std::array<long, 3>>' requested here
        value = Eigen::TensorMap<Type>(const_cast<typename Type::Scalar *>(a.data()), shape);
                ^
./third_party/pybind11/include/pybind11/cast.h:1467:47: note: in instantiation of member function 'pybind11::detail::type_caster<Eigen::Tensor<double, 3, 0>>::load' requested here
        if ((... || !std::get<Is>(argcasters).load(call.args[Is], call.args_convert[Is]))) {
                                              ^
./third_party/pybind11/include/pybind11/cast.h:1445:50: note: in instantiation of function template specialization 'pybind11::detail::argument_loader<const Eigen::Tensor<double, 3, 0> &>::load_impl_sequence<0UL>' requested here
    bool load_args(function_call &call) { return load_impl_sequence(call, indices{}); }
                                                 ^
third_party/pybind11/include/pybind11/pybind11.h:229:33: note: in instantiation of member function 'pybind11::detail::argument_loader<const Eigen::Tensor<double, 3, 0> &>::load_args' requested here
            if (!args_converter.load_args(call)) {
                                ^
third_party/pybind11/include/pybind11/pybind11.h:101:9: note: in instantiation of function template specialization 'pybind11::cpp_function::initialize<(lambda at third_party/pybind11/tests/test_eigen_tensor.cpp:98:32), Eigen::Tensor<double, 3, 0>, const Eigen::Tensor<double, 3, 0> &, pybind11::name, pybind11::scope, pybind11::sibling>' requested here
        initialize(
        ^
third_party/pybind11/include/pybind11/pybind11.h:1154:22: note: in instantiation of function template specialization 'pybind11::cpp_function::cpp_function<(lambda at third_party/pybind11/tests/test_eigen_tensor.cpp:98:32), pybind11::name, pybind11::scope, pybind11::sibling, void>' requested here
        cpp_function func(std::forward<Func>(f),
                     ^
third_party/pybind11/tests/test_eigen_tensor.cpp:98:7: note: in instantiation of function template specialization 'pybind11::module_::def<(lambda at third_party/pybind11/tests/test_eigen_tensor.cpp:98:32)>' requested here
    m.def("round_trip_tensor", [](const Eigen::Tensor<double, 3> &tensor) { return tensor; });
      ^
./third_party/eigen3/unsupported/Eigen/CXX11/src/Tensor/TensorDimensions.h:251:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const std::array<long, 3>' to 'const Eigen::DSizes<long, 3>' for 1st argument
struct DSizes : array<DenseIndex, NumDims> {
       ^
./third_party/eigen3/unsupported/Eigen/CXX11/src/Tensor/TensorDimensions.h:251:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const std::array<long, 3>' to 'Eigen::DSizes<long, 3>' for 1st argument
./third_party/eigen3/unsupported/Eigen/CXX11/src/Tensor/TensorDimensions.h:268:30: note: candidate constructor not viable: no known conversion from 'const std::array<long, 3>' to 'const array<long, 3>' for 1st argument
  EIGEN_DEVICE_FUNC explicit DSizes(const array<DenseIndex, NumDims>& a) : Base(a) { }
                             ^
./third_party/eigen3/unsupported/Eigen/CXX11/src/Tensor/TensorDimensions.h:270:30: note: candidate constructor not viable: no known conversion from 'const std::array<long, 3>' to 'const long' for 1st argument
  EIGEN_DEVICE_FUNC explicit DSizes(const DenseIndex i0) {
                             ^
note: remaining 6 candidates omitted; pass -fshow-overloads=all to show them
2 errors generated.

.gitignore Show resolved Hide resolved
@rwgk
Copy link
Collaborator

rwgk commented Oct 18, 2022

@lalaland @Skylion007

Is there anything left here to do?

I put in a few hours of my time and REALLY would like to get this merged as soon as possible, so that I get a return on the investment. I'm sure this will get picked up quickly.

@EthanSteinberg
Copy link
Collaborator Author

As far as I know, I think I fixed everyone's comments and all tests are passing (except for a flake that someone should rerun).

I also need to update the docs, but I would prefer to do that in a separate PR.

@Skylion007
Copy link
Collaborator

Skylion007 commented Oct 18, 2022

@lalaland @Skylion007

Is there anything left here to do?

I put in a few hours of my time and REALLY would like to get this merged as soon as possible, so that I get a return on the investment. I'm sure this will get picked up quickly.

@rwgk @henryiii Were we not planning to put into the next major version release? We need an upgrade guide as this could break people's code that define their own Eigen::Tensor casters. Something to the effect of #include pybind11/eigen/matrix.h for only old eigen caster.

Copy link
Collaborator

@Skylion007 Skylion007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good aside from the few nits I pointed out.

@EthanSteinberg
Copy link
Collaborator Author

EthanSteinberg commented Oct 18, 2022

@Skylion007

Were we not planning to put into the next major version release? We need an upgrade guide as this could break people's code that define their own Eigen::Tensor casters.

This is a really good point. I'll modify this PR so that pybind11/eigen.h only includes eigen/matrix.h.

This way we can guarantee no breakages of existing code.

@Skylion007
Copy link
Collaborator

Skylion007 commented Oct 18, 2022

@Skylion007

Were we not planning to put into the next major version release? We need an upgrade guide as this could break people's code that define their own Eigen::Tensor casters.

This is a really good point. I'll refactor this so that pybind11/eigen.h only includes eigen/matrix.h.

This way we can guarantee no breakages of existing code.

Thoughts on this @rwgk @henryiii? I could go either way with having eigen.h define the eigen/matrix.h or not. We could also gradually change behavior eigen.h at the next major release, but just add in the refactor the new include file for now.

@rwgk
Copy link
Collaborator

rwgk commented Oct 18, 2022

Thoughts on this @rwgk @henryiii? I could go either way with having eigen.h define the eigen/matrix.h or not. We could also gradually change behavior eigen.h at the next major release, but just add in the refactor the new include file for now.

I'm not sure.

But what comes to mind: doing something special usually does both good & harm.

Until now I was thinking of eigen.h as a convenient way to include both eigen/matrix.h and eigen/tensor.h.

But now that that we have this discussion: is that even a good idea / what we want to promote?

IWYU seems far better in general.

That brings me to thinking of eigen.h as a header deprecated in 2.11 and removed in (e.g.) 2.13.

Which means we will/should never include eigen/tensor.h from eigen.h, now or in the future.

@rwgk
Copy link
Collaborator

rwgk commented Oct 18, 2022

Awesome, thanks everybody who commented here!

I looked at the CI failures: Centos 7 has been broken for a week already, the other failure is a github infrastructure hiccup (download failed). Good to ignore I'm sure.

@rwgk rwgk merged commit fab1eeb into pybind:master Oct 18, 2022
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label Oct 18, 2022
@rwgk rwgk changed the title First draft of Eigen::Tensor support Add Eigen::Tensor & Eigen::TensorMap support Oct 18, 2022
@rwgk
Copy link
Collaborator

rwgk commented Oct 19, 2022

@lalaland while integrating this PR into the smart_holder branch I noticed this block (somehow I missed it before):

try:
from pybind11_tests import eigen_tensor_avoid_stl_array as avoid
submodules += [avoid.c_style, avoid.f_style]
except ImportError:
pass

If something goes wrong (refactoring, new platform, etc.) it could easily go unnoticed that something is wrong. Is it ever expected that the import fails, given that the pybind11_tests.eigen_tensor in line 6 worked? If yes, we need to produce a warning that is not easily overlooked, but it would be ideal if we could remove the try - except entirely.

@EthanSteinberg
Copy link
Collaborator Author

eigen_tensor and eigen_tensor_avoid_stl_array are from two separate C++ files, so eigen_tensor_avoid_stl_array will sometimes be unimportable if that C++ file never got built.

You do get a warning from

pytest.importorskip("pybind11_tests.eigen_tensor_avoid_stl_array")
, but you are right that it is probably easily overlooked.

Do you want me to upgrade

pytest.importorskip("pybind11_tests.eigen_tensor_avoid_stl_array")
from a warning to a hard fail?

@rwgk
Copy link
Collaborator

rwgk commented Oct 19, 2022

sometimes be unimportable if that C++ file never got built.

In what situation(s) is that expected?

@EthanSteinberg
Copy link
Collaborator Author

In what situation(s) is that expected?

When you use DPYBIND11_TEST_OVERRIDE to only build a subset of the tests.

@rwgk
Copy link
Collaborator

rwgk commented Oct 19, 2022

In what situation(s) is that expected?

When you use DPYBIND11_TEST_OVERRIDE to only build a subset of the tests.

Ah, hm ... I feel we could make it a hard requirement that the two always have to be built together, then have a plain import without try-except, or maybe better, change the except branch to raise an informative message "you need none or both". It seems pretty unlikely to me that that will ever become limiting in some way. In contrast, being fooled into thinking all tests work when they don't is much worse (been there a few times; having to backtrack in some complicated refactoring for example can be extremely frustrating).

Alternatively, a very outstanding WARNING in the except branch might be enough to avoid accidents, although as someone used to working with gigantic systems, everything that is not a hard failure is often overlooked anyway. It's also pretty likely that we'll overlook such warnings in the CI logs until much later.


Something else minor I noticed:

// Similar to comments & pragma block in eigen_tensor.h. PLEASE KEEP IN SYNC.

That line seems to be outdated, I don't see the comment copied into tensor.h anymore (good; one place is enough). Could you please remove that line in matrix.h?

@rwgk
Copy link
Collaborator

rwgk commented Oct 19, 2022

I sent #4255, implementing what I suggested above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exposing Eigen::Tensor To Python Can you support Tensor datatype from Eigen library?
7 participants