Skip to content

Commit

Permalink
Backport PR #47347 on branch 1.4.x (REGR: Regression in to_csv for ea…
Browse files Browse the repository at this point in the history
… dtype categorical) (#47388)

* Backport PR #47347: REGR: Regression in to_csv for ea dtype categorical

* inclusive -> closed

Co-authored-by: Patrick Hoefler <61934744+phofl@users.noreply.github.com>
Co-authored-by: Simon Hawkins <simonjayhawkins@gmail.com>
  • Loading branch information
3 people authored Jun 17, 2022
1 parent de071c6 commit f0af15c
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 1 deletion.
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.4.3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ including other versions of pandas.
Fixed regressions
~~~~~~~~~~~~~~~~~
- Fixed regression in :meth:`DataFrame.replace` when the replacement value was explicitly ``None`` when passed in a dictionary to ``to_replace`` also casting other columns to object dtype even when there were no values to replace (:issue:`46634`)
- Fixed regression in :meth:`DataFrame.to_csv` raising error when :class:`DataFrame` contains extension dtype categorical column (:issue:`46297`, :issue:`46812`)
- Fixed regression in representation of ``dtypes`` attribute of :class:`MultiIndex` (:issue:`46900`)
- Fixed regression when setting values with :meth:`DataFrame.loc` updating :class:`RangeIndex` when index was set as new column and column was updated afterwards (:issue:`47128`)
- Fixed regression in :meth:`DataFrame.nsmallest` led to wrong results when ``np.nan`` in the sorting column (:issue:`46589`)
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/internals/blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -2160,7 +2160,7 @@ def to_native_types(
**kwargs,
) -> np.ndarray:
"""convert to our native types format"""
if isinstance(values, Categorical):
if isinstance(values, Categorical) and values.categories.dtype.kind in "Mm":
# GH#40754 Convert categorical datetimes to datetime array
values = take_nd(
values.categories._values,
Expand Down
29 changes: 29 additions & 0 deletions pandas/tests/frame/methods/test_to_csv.py
Original file line number Diff line number Diff line change
Expand Up @@ -1333,3 +1333,32 @@ def test_to_csv_na_quoting(self):
)
expected = '""\n""\n'
assert result == expected

def test_to_csv_categorical_and_ea(self):
# GH#46812
df = DataFrame({"a": "x", "b": [1, pd.NA]})
df["b"] = df["b"].astype("Int16")
df["b"] = df["b"].astype("category")
result = df.to_csv()
expected_rows = [",a,b", "0,x,1", "1,x,"]
expected = tm.convert_rows_list_to_csv_str(expected_rows)
assert result == expected

def test_to_csv_categorical_and_interval(self):
# GH#46297
df = DataFrame(
{
"a": [
pd.Interval(
Timestamp("2020-01-01"),
Timestamp("2020-01-02"),
closed="both",
)
]
}
)
df["a"] = df["a"].astype("category")
result = df.to_csv()
expected_rows = [",a", '0,"[2020-01-01, 2020-01-02]"']
expected = tm.convert_rows_list_to_csv_str(expected_rows)
assert result == expected

0 comments on commit f0af15c

Please sign in to comment.