-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
CLN/TST: delegate StringArray.fillna() to parent class + add tests #37987
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CLN/TST: delegate StringArray.fillna() to parent class + add tests #37987
Conversation
pandas/core/arrays/string_.py
Outdated
@@ -283,7 +283,8 @@ def __setitem__(self, key, value): | |||
super().__setitem__(key, value) | |||
|
|||
def fillna(self, value=None, method=None, limit=None): | |||
# TODO: validate dtype | |||
if not isinstance(value, str): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
np.str_?
its a weird usage, but there's no reason why a user couldn't pass a n NA fill value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added both, including the missing value
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
np.str_
is already be covered by str (it subclasses str):
In [13]: arr = np.array(["a"])
In [14]: type(arr[0])
Out[14]: numpy.str_
In [15]: isinstance(arr[0], str)
Out[15]: True
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah right. Got rid of the np.str_
check
0a85704
to
34e3d82
Compare
c5da53a
to
616a6c7
Compare
…ring-validate-fillna-args
Does this need a whatsnew? |
I don't think it is necessarily needed, it's only a change in the error message I think? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pandas/core/arrays/string_.py
Outdated
@@ -283,7 +283,10 @@ def __setitem__(self, key, value): | |||
super().__setitem__(key, value) | |||
|
|||
def fillna(self, value=None, method=None, limit=None): | |||
# TODO: validate dtype | |||
if value is not None and not ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should just be able to call _validate_setitem_value(value)
, maybe we can it it in the baseclass.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i would like this since it would allow us to implement a correct ExtensionBlock._can_hold_element, xref #36226
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should just be able to call
_validate_setitem_value(value)
, maybe we can it it in the baseclass.
Gave that a go - is the last commit what you meant? @jreback @jbrockmendel
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am -1 adding it in this PR (or at least adding it on the base class). If we add it to the base class, it needs to be part of the EA interface, documented that way, have a base extension test, have a fall back implementation, .. (and first discuss if we actually want it).
Also, we actually don't need to explicitly call the setitem validation here, since fillna
is already raising that exception because it uses the setitem implementation under the hood.
So we could also simply add the test to ensure fillna
raises the proper error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am -1 adding it in this PR (or at least adding it on the base class). If we add it to the base class, it needs to be part of the EA interface, documented that way, have a base extension test, have a fall back implementation, .. (and first discuss if we actually want it).
Ok! I'll open an issue
Also, we actually don't need to explicitly call the setitem validation here, since
fillna
is already raising that exception because it uses the setitem implementation under the hood.So we could also simply add the test to ensure
fillna
raises the proper error.
There was an existing test and I added more testcases here
…ring-validate-fillna-args
pandas/core/arrays/string_.py
Outdated
|
||
super().__setitem__(key, value) | ||
|
||
def fillna(self, value=None, method=None, limit=None): | ||
# TODO: validate dtype | ||
if value is not None: | ||
value = self._validate_setitem_value(value) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is actually not needed, as the setitem under the hood of fillna already calls it. So calling it here as well means we validate the value twice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right, reverted all the changes here
…ring-validate-fillna-args
I pared down this PR to just the fillna test for strings. Discussion on |
pandas/core/arrays/string_.py
Outdated
@@ -283,7 +283,6 @@ def __setitem__(self, key, value): | |||
super().__setitem__(key, value) | |||
|
|||
def fillna(self, value=None, method=None, limit=None): | |||
# TODO: validate dtype | |||
return super().fillna(value, method, limit) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this method need to be overriden here at all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it doesn't
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pushed that now
thanks @arw2019 |
black pandas
git diff upstream/master -u -- "*.py" | flake8 --diff