Skip to content

BUG: setting with iloc and no labels does not effect certain data types #34217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
rhshadrach opened this issue May 16, 2020 · 3 comments
Closed
3 tasks done
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves

Comments

@rhshadrach
Copy link
Member

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}, dtype="int64")
df.iloc[:, 0:1] = df.iloc[:, 0:1].astype("int32").values
print(df.dtypes)

Column "a" remains an int64 whereas it should be int32. This is similar to #33198, however the resolution there (taking the split path in _setitem_with_indexer) is not sufficient in this case where there are no labels. The issue appears to be the function

def maybe_convert_platform(values):
""" try to do platform conversion, allow ndarray or list here """
if isinstance(values, (list, tuple, range)):
values = construct_1d_object_array_from_listlike(values)
if getattr(values, "dtype", None) == np.object_:
if hasattr(values, "_values"):
values = values._values
values = lib.maybe_convert_objects(values)
return values

values enters here as int32 and gets converted to int64 in the call to maybe_convert_objects. Indeed, adding the line

values = values.astype(np.int32)

after the call to maybe_convert_objects gives the right output.

Should maybe_convert_objects take into account int32 vs int64, float32 vs float64, and others?

One last note: even if maybe_convert_objects returned int32 here, it would not resolve #33198.

@rhshadrach rhshadrach added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 16, 2020
@charlesdong1991 charlesdong1991 added Indexing Related to indexing on series/frames, not to indexes themselves and removed Needs Triage Issue that has not been reviewed by a pandas team member labels May 17, 2020
@charlesdong1991 charlesdong1991 added this to the Contributions Welcome milestone May 17, 2020
@phofl
Copy link
Member

phofl commented Nov 22, 2020

Can be fixed on top of #37798 and #37749

@mroeschke mroeschke added the Dtype Conversions Unexpected or buggy dtype conversions label Aug 7, 2021
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
@rhshadrach
Copy link
Member Author

@MarcoGorelli - while PDEP-6 was about upcasting, I think the idea is that when you set into an existing Series/column, the dtype does not change, is that right? That would make the current behavior okay and this issue can be closed.

@MarcoGorelli
Copy link
Member

yeah to be honest this looks expected to me - closing then, thanks for the ping!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

No branches or pull requests

5 participants