Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically keep categorical type on merge #24093

Open
Gerenuk opened this issue Dec 4, 2018 · 2 comments
Open

Automatically keep categorical type on merge #24093

Gerenuk opened this issue Dec 4, 2018 · 2 comments
Labels
Bug Categorical Categorical Data Type Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@Gerenuk
Copy link

Gerenuk commented Dec 4, 2018

Code Sample, a copy-pastable example if possible

import pandas as pd
dd1=pd.DataFrame({"a":pd.Series(["x"], dtype="category")})
dd2=pd.DataFrame({"a":pd.Series(["x", "y"], dtype="category")})
dd3=dd1.merge(dd2)
dd3.dtypes  # categorical turns into object type

Problem description

It would be nice if when merging on categorical columns, their categorical nature would remain by an automatic union_categoricals. Currently only categoricals with identical values remain categorical and merge on non-identical categoricals becomes object type.

@TomAugspurger
Copy link
Contributor

We have a similar open issue for pd.concat to union the categoricals. No one has implemented it yet.

@TomAugspurger TomAugspurger added Reshaping Concat, Merge/Join, Stack/Unstack, Explode Dtype Conversions Unexpected or buggy dtype conversions Categorical Categorical Data Type labels Dec 4, 2018
@ms7463
Copy link
Contributor

ms7463 commented Dec 7, 2018

I believe this would be relevant to this PR also. The feedback I got seemed to suggest that categorical with different dtypes should not be considered compatible for set operations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Categorical Categorical Data Type Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

No branches or pull requests

4 participants