-
-
Notifications
You must be signed in to change notification settings - Fork 372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow user to customize how an attribute is compared (#435) #627
Conversation
The name |
Heyyy. :) Thanks for taking this on and sorry for the long time waiting. As a quick idea, we do have the precedence of passing a callable: In [1]: import attr
In [2]: @attr.s
...: class C:
...: x = attr.ib(repr=lambda v: f">>> { v } <<<")
...:
In [3]: C(42)
Out[3]: C(x=>>> 42 <<<) So I think we should stay consistent and not introduce a new argument. Does anything speak against something like this: from typing import Any
import abc
import attr
class Equality(abc.ABC):
@abc.abstractmethod
def eq(self, o1: Any, o2: Any):
pass
def ne(self, o1: Any, o2: Any):
return not self.eq(o1, o2)
class Order(abc.ABC):
@abc.abstractmethod
def lt(self, o1: Any, o2: Any):
pass
def le(self, o1: Any, o2: Any):
if o1 == o2:
return True
return self.lt(o1, o2)
# ...
class NumpyEquality(Equality):
def eq(self, o1: Any, o2: Any):
if type(o1) is not type(o2):
return NotImplemented
return (o1 == o2).all()
@attr.s
class C:
x = attr.ib(eq=NumpyEquality) ? Other than that: |
As it is, we have not used About the snippet you suggest above, I'm not sure how we would integrate this with the tuple comparison utilized in class NumpyEquality(Equality):
def eq(self, o1: Any, o2: Any):
if type(o1) is not type(o2):
return NotImplemented
return (o1 == o2).all() We took a slightly different approach: the comparison class wraps the attribute when we build the tuple: for a in attrs:
lines.append(" self.%s," % (a.name,))
others.append(" other.%s," % (a.name,))
if a.comparator:
cmp_name = "_%s_comparator" % (a.name,)
# Add the cmp class to the global namespace of the evaluated
# function, since local does not work in all python versions.
global_vars[cmp_name] = a.comparator
lines.append(" %s(self.%s)," % (cmp_name, a.name,))
others.append(" %s(other.%s)," % (cmp_name, a.name,))
else:
lines.append(" self.%s," % (a.name,))
others.append(" other.%s," % (a.name,)) Another question we have to answer is what kind of comparison functions or classes we provide off-the-shelf, and whether we enforce some sort of hierarchy or protocol. Comparing complex objects such as numpy arrays or pandas dataframes is a bit domain/application-specific, and I'm concerned that nothing we provide will be generic enough. With that in mind, we have been experimenting (here https://github.com/botant/attrs/pull/4) with a Usage would look like this: @attr.s
class C:
# case insensitive
name = attr.ib(eq=attr.comparators.using_key(lambda x: x.lower()))
# absolute value
value = attr.ib(eq=attr.comparators.using_key(abs))
# numpy array
data = attr.ib(eq=attr.comparators.using_function(np.array_equal)) What do you think? |
Merge attrs/master onto fork
@hynek I'm having second thoughts about reusing I'm trying to avoid something like this: @attr.s
class C:
x = attr.ib(eq=abs, order=abs)
y = attr.ib(eq=lambda s: s.lower(), order=lambda s: s.lower()) I would suggest we keep |
There are two approaches being discussed here. @hynek is proposing an API in which you define comparison, and @botant points our that you may have to express the same thing twice, which might be solved if the We could have a helper function that build an
Then @botant is proposing a new argument called
|
The API we're proposing is a bit different. The two use cases we're trying to address are:
In order to do this and keep the current @attr.s
class MyComparator:
value = attr.ib()
def __eq__ ...
def __lt__ ... This specification fits To facilitate the use of this API, we provide two functions that cover the use cases described above:
def __eq__(self, other):
return self.key(self.value) == self.key(other.value)
def __eq__(self, other):
return self.func(self.value, other.value) Example: @attr.s
class C:
# case insensitive
name = attr.ib(compare=attr.compare.using_key(lambda x: x.lower()))
# absolute value
value = attr.ib(compare=attr.compare.using_key(abs))
# numpy array
data = attr.ib(compare=attr.compare.using_function(np.array_equal)) In a nutshell, the API can be described like this:
Things we wanted achieved:
Things we didn't achieve:
Hope this makes our suggestion a bit clearer. Thanks! |
A few quick questions:
What Python versions are those? I'm happy to make this feature Python 3-only. I understand your concern about consistency between eq and order, and since you seem to come from the scientific community that makes a lot of sense, however I consider having I think this is something that could be solved either with a warning or some kind of syntactic sugar/helper class that combines both? But it shouldn't be mushed together again by default. Maybe we could keep I agree that we couldn't ever give people what everybody needs. I could live with not giving them anything at all, as long as we give them a good API to build it themselves. :). I have only glanced at the APIs but I think it has legs! It just needs to be split up with an helper. How would you (and this includes @wsanchez) feel about this: @attr.s
class C:
# Only change equality
x = attr.ib(eq=attr.eq.using_key(lambda x: x.lower()))
# Only change order
y = attr.ib(order=attr.order.using_key(abs))
# Change both
data = attr.ib(cmp=attr.cmp.using_function(np.array_equal)) This would give me the flexibility I want and you'd get safety/comfort ( |
# Add the cmp class to the global namespace of the evaluated
# function, since local does not work in all python versions. This should work on all versions from 2.7 to 3.8. We did try using Yes! I explicitly set How would you achieve that change in behaviour? A transition period where calls to What do you mean by:
This looks like a nice compromise between consistency and flexibility. x = attr.ib(eq=attr.eq.using_key(lambda x: x.lower()))
y = attr.ib(order=attr.order.using_key(abs))
data = attr.ib(cmp=attr.cmp.using_function(np.array_equal)) My only concern is that we're talking about potentially breaking changes, even though they would suit me. As a first-time contributor, I'd never have suggested that myself. I'm looking forward to seeing how an API change like this is managed. |
@hynek I'm good with that lat proposal from you, though I find the terms That is, if So I'd suggest |
@wsanchez >>> sorted("This is a test string from Andrew".split(), key=str.lower)
['a', 'Andrew', 'from', 'is', 'string', 'test', 'This'] And I agree the names I chose aren't great, but the truth is that it will be hard to pick intuitive and self-explanatory names given that this is a new API. Unfortunately we can't reuse a standard library name like in |
@botant I think so. I'm not going to object loudly based on the names, so the priority I'd suggest is what @hynek described above, though I do think that "normalizer" and "comparator" are much better than "key" and "function", as these are terms we can use in the documentation and it would be good for the API to match the terms we use in the docs. Better terms are welcome, but I don't think "key" and "function" make much sense. |
Cool, I'll get started then.
I'll rename the functions as you suggested.
Maybe we'll think of better names once all the pieces are in place...
…On Mon, 9 Mar 2020, 21:08 Wilfredo Sánchez Vega, ***@***.***> wrote:
@botant <https://github.com/botant> I think so. I'm not going to object
loudly based on the names, so the priority I'd suggest is what @hynek
<https://github.com/hynek> described above, though I do think that
"normalizer" and "comparator" are much better than "key" and "function", as
these are terms we can use in the documentation and it would be good for
the API to match the terms we use in the docs.
Better terms are welcome, but I don't think "key" and "function" make much
sense.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627?email_source=notifications&email_token=ABMJFPBBFPHO7QZKAMWVIHTRGVSHNA5CNFSM4KZQ2XI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOJCJNY#issuecomment-596780215>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPEAKUEWFU24K3544IDRGVSHNANCNFSM4KZQ2XIQ>
.
|
No, wait don’t. :) Normalizer and comparator may be technically correct, but it goes too far into We can still bikeshed over the names later (I would prefer |
@hynek: If we want I'll put down the paint bucket now. |
Can you give me quick examples of |
with_key and with_cmp is a lovely pair of names; if anything, they are
super easy to type :-)
…On Tue, 10 Mar 2020, 17:57 Wilfredo Sánchez Vega, ***@***.***> wrote:
@hynek <https://github.com/hynek>: characteristic wasn't all bad. ;-)
If we want stdlib-flavors for names, I'd say use key and cmp, eg: with_key
and with_cmp.
I'll put down the paint bucket now.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627?email_source=notifications&email_token=ABMJFPD4LSNIURIGBDFI4CTRGZ5P7A5CNFSM4KZQ2XI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOMPURI#issuecomment-597228101>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPHOOG2P6ITX66WIT3LRGZ5P7ANCNFSM4KZQ2XIQ>
.
|
|
lol: https://docs.python.org/3.8/library/functools.html#functools.cmp_to_key Yeah I think those two are good names. |
@botant just to be clear: do you need anything from us or are you good |
Hi Hynek, thanks for checking, and apologies for the radio silence.
The lockdown has been quite distracting and I haven't managed to do much
besides regular work and kid entertainment.
I'm going to dedicate a few hours to it this week, and hopefully you'll see
some updates soon.
Best regards,
Antonio
…On Sun, 29 Mar 2020, 06:59 Hynek Schlawack, ***@***.***> wrote:
@botant <https://github.com/botant> just to be clear: do you need
anything from us or are you good
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPCLZGQPOROYRPYQLJLRJ3PV5ANCNFSM4KZQ2XIQ>
.
|
No problem, just checking whether you're blocked unbeknownst to me. Stay healthy! |
Merge python-attrs/master onto botant/master
Hello @hynek and @wsanchez, hope you are well. Now that I've started working on this, I'm worried that the proposed API may lead to bugs very difficult to spot and debug. Consider the following code: @attr.s
class Test:
x = attr.ib(eq=attr.eq.using_key(str.lower))
y = attr.ib(order=attr.eq.using_key(abs)) That is clearly wrong. Surely the user wanted one of the following: y = attr.ib(eq=attr.eq.using_key(abs)) or y = attr.ib(order=attr.order.using_key(abs)) My concern is that a bug like this might be invisible to unit tests unless the user is actively testing for that and might show up at run time only. I see a few options:
@attr.s
class Test:
x = attr.ib(eq=True, cmp=attr.cmp.using_key(str.lower))
y = attr.ib(order=True, cmp=attr.cmp.using_key(abs)) In this case, we would preserve the current meaning of Looking forward to hearing what you think. |
Hi @hynek, Whether we go with this: @attr.s
class Test:
x = attr.ib(eq=True, cmp=attr.cmp.using_key(str.lower))
y = attr.ib(order=True, cmp=attr.cmp.using_key(abs)) Or this: @attr.s
class Test:
x = attr.ib(eq=attr.eq.using_key(str.lower))
y = attr.ib(order=attr.order.using_key(abs)) What are we doing about |
I am late to the party, and have clearly thought about this less than others earlier, so apologies if this comment is completely unhelpful -- But I don't see why things ended up needing helpers (i.e. why I see what the helpers/full API does -- they let someone override everything about equality even beyond the transformation of the attribute, including specifically whether a type check is first performed. Is that really a common enough case to support? (I'm asking a leading question obviously, and I think the answer is "no" and that those cases should just define I also see the back and forth about how cmp and order are really related (i.e. that usually you want to normalize them the same way), but if that's the primary reason for the helpers, I guess my vote if it counts would be for the API in the later comments, but without the helper, i.e. |
Hi Julian,
We'd like to support at least two use cases:
1) lower(a) == lower(b) (a.k.a normalisation function)
2) np.is_equal(a, b) (a.k.a custom equality function)
If all you pass to attr.ib( ) is a function, there is no way to know how it
is supposed to be used.
Also, there might be other use cases out there that do not fit either
pattern, so the API should be open enough to allow the user to provide
their own class supporting their exact use case.
…On Thu, 21 May 2020, 15:54 Julian Berman, ***@***.***> wrote:
I am late to the party, and have clearly thought about this less than
others earlier, so apologies if this comment is completely unhelpful --
But I don't see why things ended up needing helpers (i.e. why
eq=attr.cmp.with_key(str.lower) isntead of just eq=str.lower?
I see what the helpers/full API does -- they let someone override
everything about equality even beyond the transformation of the attribute,
including specifically whether a type check is first performed. Is that
really a common enough case to support? (I'm asking a leading question
obviously, and I think the answer is "no" and that those cases should just
define __eq__ fully manually, but just want to make sure I follow the
reasoning).
I also see the back and forth about how cmp and order are really related
(i.e. that usually you want to normalize them the same way), but if that's
the primary reason for the helpers, I guess my vote if it counts would be
for the API in the later comments, but without the helper, i.e. y =
attr.ib(order=True, cmp=abs), which triggers "generate an order, use the
same normalization as for cmp".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPHZW3ZEAT5QTXSLODDRSU6CPANCNFSM4KZQ2XIQ>
.
|
I saw that about numpy too, but that to me instinctively should be supported in the same API -- i.e. just via a key -- so someone should have a It's not a common thing outside of the numpy ecosystem, so I'd expect it be supported by allowing numpy things to be wrapped into objects that behave natively, and then you can just use the "normal" way. That may be my numpy-is-weird bias showing though in full disclosure :). |
I afraid I don't agree with that at all.
Both numpy/pandas and attrs are big components of the Python ecosystem.
Creating an equality API in attrs that excludes numpy because its
non-standard __eq__ behaviour feels like a waste of energy.
For full disclosure, I am a numpy user, and I'm also a bit annoyed about
the weird __eq__, but I can also see that it made sense in that particular
domain.
Ultimately, this is a decision for Hynek, Sanchez, etc.
…On Thu, 21 May 2020, 17:20 Julian Berman, ***@***.***> wrote:
I saw that about numpy too, but that to me instinctively should be
supported in the same API -- i.e. just via a key -- so someone should have
a array-to-eq wrapper thing that takes a numpy array and instead returns
an object whose __eq__ does the right thing (i.e. what any other python
object would do for equality).
It's not a common thing outside of the numpy ecosystem, so I'd expect it
be supported by allowing numpy things to be wrapped into objects that
behave natively, and then you can just use the "normal" way. That may be my
numpy-is-weird bias showing though in full disclosure :).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPBIHLIQVC2QQOWJ42TRSVIFPANCNFSM4KZQ2XIQ>
.
|
this is exactly what we need, just wanted to say thanks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's get this over the liiiineeee!
|
||
# cmp takes precedence due to bw-compatibility. | ||
if cmp is not None: | ||
warnings.warn(_CMP_DEPRECATION, DeprecationWarning, stacklevel=3) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- yes
- I think we do? It's still a derived attribute that can be used as syntactic sugar, to set eq and order at once – agreed?
src/attr/_make.py
Outdated
if a.eq_key: | ||
cmp_name = "_%s_key" % (a.name,) | ||
# Add the cmp class to the global namespace of the evaluated | ||
# function, since local does not work in all python versions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would strongly prefer to make this Python 3-only then. pip just dropped Python 2, we don't need to bend backward anymore when it comes to new features.
Unless the code becomes even more complicated? I guess I'll leave this to you, but I'm -0 on global namespace shenanigans.
Ping? 😇 |
Apologies, I'll address the questions later today!
…On Thu, 11 Feb 2021 at 06:54, Hynek Schlawack ***@***.***> wrote:
Ping? 😇
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPGRHN7247OUKQHZXYDS6N5LJANCNFSM4KZQ2XIQ>
.
|
No need for apologies! Just making sure we’re not both waiting for each other. :) |
Co-authored-by: Hynek Schlawack <hs@ox.cx>
On |
On
Do you know if there is an alternative? Can you confirm on your side? PS: |
What part? 😅
Just leave it then. We can look if we can improve the code later (narrator: they won't). |
Hey, I'm sorry but #760 added some conflicts. But I suspect it might actually simplify some of your code around eval? |
On
On globals: I did a bit more digging and I don't think it will ever work with locals, because
I think we're better off leaving it as is, after cleaning my nonsensical comment. I can only imagine that I had inverted the |
Ehh I'm not sure either anymore, but removing the property would break backward compatibility, so we definitely still need it. Or am I misunderstanding? |
Yes, I think we do need it. Or at least I don't want to be the one to take it off. 😃 |
Merged! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a few minor things that need to be done (a bit too keen on resolving! :D), but it's not worth to ping-pong them so I'll do it myself real quick.
So let's get this merged, literally on its first birthday! :D Thanks for staying on the ball!
Thank you everyone! It took a year to merge, but I think we've found a great solution! Now ~someone~ has to write some narrative docs. 🤪 |
Nice! I'll have a stab at the documentation.
…On Mon, Feb 22, 2021, 07:50 Hynek Schlawack ***@***.***> wrote:
Thank you everyone! It took a year to merge, but I think we've found a
great solution!
Now ~someone~ has to write some narrative docs. 🤪
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPAXXZBRODQGYQM6FILTAIED7ANCNFSM4KZQ2XIQ>
.
|
Wait, I’ll have to think about it how to integrate it first. I’ll ping you on the PR. :) |
Sure thing.
…On Mon, Feb 22, 2021, 10:31 Hynek Schlawack ***@***.***> wrote:
Wait, I’ll have to think about it how to integrate it first. I’ll ping you
on the PR. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABMJFPGUIETDXGY2UYTH4ITTAIXAVANCNFSM4KZQ2XIQ>
.
|
Apologies for resubmitting a PR. I had made a mess in my fork and this was a side-effect.
The idea of this PR is to add a
comparator
argument toattr.ib( )
, in which the user could provide a simple class implementing equality and ordering methods to suit their needs..pyi
).tests/typing_example.py
.docs/api.rst
by hand.@attr.s()
have to be added by hand too.versionadded
,versionchanged
, ordeprecated
directives. Find the appropriate next version in our__init__.py
file..rst
files is written using semantic newlines.changelog.d
.