Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REGR: Replace changes the dtype of other columns #46636

Merged
merged 4 commits into from
May 25, 2022

Conversation

simonjayhawkins
Copy link
Member

@simonjayhawkins simonjayhawkins added Regression Functionality that used to work in a prior pandas version replace replace method labels Apr 4, 2022
@simonjayhawkins simonjayhawkins added this to the 1.4.3 milestone Apr 4, 2022
@jreback
Copy link
Contributor

jreback commented Apr 4, 2022

@jbrockmendel ok here?

nb = nb.copy()
putmask_inplace(nb.values, mask, value)
return [nb]
return [self]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably need self.copy() if not inplace?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, in #46404 (the cause of the latest regression), I reverted some of the code change in #44940 (the cause of the previous regression). In this PR I have reverted some more of the changes in #44940 that obviously (now!) should also have been included in the previous regression fix. I don't see the original code making a copy if not mask.any().

but yes, in #44940 the code path was changed to dispatch to Block.replace which does make a copy even if the replace is a no-op. So as a pure regression fix we could keep this as is, but also happy to do a copy for consistency with other replacements in 1.4.x

my biggest concern (need to check this though) is categorical since the special casing was also moved in #44940.

I will mark this a draft for now, (till we get closer to the next release) and see if we can get the issues on master sorted so that we can dispatch to Block.replace when the replacement value is None. (doing that might end up being difficult to backport though)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty actively trying to dis-engage, not gonna fuss about it either way. i trust your judgment here.

Copy link
Member Author

@simonjayhawkins simonjayhawkins Apr 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my biggest concern (need to check this though) is categorical since the special casing was also moved in #44940.

There is an issue raised for the categorical issue #46672, so can keep that out of this PR for now.

I will mark this a draft for now, (till we get closer to the next release) and see if we can get the issues on master sorted so that we can dispatch to Block.replace when the replacement value is None. (doing that might end up being difficult to backport though)

I guess no harm in getting this bit sorted and then doing that as a separate PR. (maybe after #46672 is sorted).

@jbrockmendel
Copy link
Member

one possibly-missed copy, otherwise seems reasonable

@simonjayhawkins simonjayhawkins marked this pull request as ready for review April 12, 2022 11:23
@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

@github-actions github-actions bot added the Stale label May 13, 2022
@simonjayhawkins
Copy link
Member Author

one possibly-missed copy, otherwise seems reasonable

done.

@mroeschke
Copy link
Member

Thanks @simonjayhawkins.

jreback pushed a commit that referenced this pull request May 26, 2022
…f other columns) (#47123)

Backport PR #46636: REGR: Replace changes the dtype of other columns

Co-authored-by: Simon Hawkins <simonjayhawkins@gmail.com>
@simonjayhawkins simonjayhawkins deleted the replace-regression branch May 26, 2022 09:35
yehoshuadimarsky pushed a commit to yehoshuadimarsky/pandas that referenced this pull request Jul 13, 2022
* REGR: Replace changes the dtype of other columns

* copy if not inplace
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Regression Functionality that used to work in a prior pandas version replace replace method
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Replace changes the dtype of other columns
4 participants