Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

__radd__ etc don't get called if __add__ is also defined #844

Closed
davidhewitt opened this issue Mar 30, 2020 · 5 comments · Fixed by #1107
Closed

__radd__ etc don't get called if __add__ is also defined #844

davidhewitt opened this issue Mar 30, 2020 · 5 comments · Fixed by #1107
Assignees
Milestone

Comments

@davidhewitt
Copy link
Member

davidhewitt commented Mar 30, 2020

If we make a class like this:

#[pyclass]
#[derive(Debug)]
struct RhsArithmetic {}

#[pymethods]
impl RhsArithmetic {
    #[new]
    fn new() -> RhsArithmetic {
        RhsArithmetic {}
    }
}

#[pyproto]
impl PyNumberProtocol for RhsArithmetic {
    fn __mul__(lhs: PyRef<'p, Self>, rhs: &PyAny) -> PyResult<String> {
        Ok(format!("MUL"))
    }
    fn __rmul__(&self, other: &PyAny) -> PyResult<String> {
        Ok(format!("RMUL"))
    }
}

Then both 1 * rhs and rhs * 1 call the function in the tp_as_number.nb_mul slot. As of #839 this will be our __mul__ implementation above (and never __rmul__).

And specifically it will call it with the arguments in the same order as invoked. So from above, we will have __mul__(1, rhs) and __mul__(rhs, 1).

This is problematic, because __mul__ as defined above will raise a TypeError if the first argument is not Self.

We saw this behaviour reported on Gitter. All three of the below will currently be calling __mul__, which is why the second will TypeError:

>>> rhs * 1.0
'MUL'
>>> 1.0 * rhs # should call __rmul__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError
>>> rhs * rhs
'MUL'

I am thinking that instead of doing this, if both __mul__ and __rmul__ are defined we should be creating a wrapper function which calls the correct function depending on the type of the first argument.

The implementation of PyNumber_Add suggests that we should return Py_NotImplemented from this wrapper function if there is a type error, so that sq_concat is given a chance to be called also.

@programmerjake
Copy link
Contributor

Similar logic in PyNumber_Multiply

@davidhewitt
Copy link
Member Author

davidhewitt commented Mar 30, 2020

From a quick search I think the equivalent logic for the wrapper function for classes defined in Python is in this macro

@kngwyu
Copy link
Member

kngwyu commented Mar 31, 2020

This is difficult for our current protocol implementation, but after rewriting it without specialization, I think we can implement this.

@davidhewitt
Copy link
Member Author

I'm nominating this for 0.12.

@davidhewitt davidhewitt added this to the 0.12 milestone Jun 21, 2020
mvaled added a commit to mvaled/pyo3 that referenced this issue Aug 4, 2020
mvaled added a commit to mvaled/pyo3 that referenced this issue Aug 5, 2020
kngwyu added a commit that referenced this issue Aug 5, 2020
* Add (failing) tests for issue #1064

* Return NotImplemented when richcmp doesn't match the expected type.

* Fix tests that expect TypeError when richcmp returns NotImplemented.

- The python code 'class Other: pass; c2 {} Other()' was raising a NameError:
  c2 not found

- eq and ne never raise a TypeError, so I split the those cases.

* Return NotImplemented for number-like binary operations.

* Add dummy impl PyNumberProtocol for the test struct.

* Rework tests of NotImplemented.

* Make py_ternary_num_func return NotImplemented when type mismatches.

* Return NotImplement for type mismatches in binary inplace operators.

* Reduce boilerplate with `extract_or_return_not_implemented!`

* Extract common definition 'Other' into a function.

* Test explicitly for NotImplemented in the __ipow__ test.

* Add entry in CHANGELOG for PR #1072.

* Add the section 'Emulating numeric types' to the guide.

* Ensure we're returning NotImplemented in tests.

* Simplify the tests: only test we return NotImplemented.

Our previous test were rather indirect: were relying that Python
behaves correctly when we return NotImplemented.

Now we only test that calling a pyclass dunder method returns NotImplemented
when the argument doesn't match the type signature.  This is the expected
behavior.

* Remove reverse operators in tests of NotImplemented

The won't be used because of #844.

* Apply suggestions from code review

Co-authored-by: Yuji Kanagawa <yuji.kngw.80s.revive@gmail.com>

* Add a note about #844 below the list of reflected operations.

Co-authored-by: Yuji Kanagawa <yuji.kngw.80s.revive@gmail.com>
@davidhewitt
Copy link
Member Author

@kngwyu just a heads up; I'm assigning this to you after the discussion in #1072 so that others don't spend time taking a stab at this while you're working on an implementation :)

ravenexp added a commit to ravenexp/pyo3 that referenced this issue Apr 5, 2021
Supposedly resolved by PyO3#1107
ravenexp added a commit to ravenexp/pyo3 that referenced this issue Apr 5, 2021
Supposedly resolved by PyO3#1107

Fix a typo in the subsection header.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants