Skip to content

category dtype coerced to object after pd.concat(axis = 1) #12007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Kodiologist opened this issue Jan 9, 2016 · 2 comments · Fixed by #29352
Closed

category dtype coerced to object after pd.concat(axis = 1) #12007

Kodiologist opened this issue Jan 9, 2016 · 2 comments · Fixed by #29352
Labels
Categorical Categorical Data Type good first issue Needs Tests Unit test(s) needed to prevent regressions
Milestone

Comments

@Kodiologist
Copy link
Contributor

import pandas as pd

x = pd.DataFrame(pd.Series(["a", "b", "c"], dtype = "category"))
print x.dtypes
df = pd.concat(
    [x, pd.Series([0, 1, 2], index = [0, 1, 3])],
    axis = 1)
print df.dtypes

This prints:

0    category
dtype: object
0     object
0    float64
dtype: object

Notice that category has unexpectedly changed to object, even though Categoricals can hold missing values (as code -1).

pd.show_versions() yields:

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-994-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.17.1
nose: None
pip: 1.5.6
setuptools: 18.4
Cython: None
numpy: 1.10.2
scipy: 0.16.1
statsmodels: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
Jinja2: None
@jreback
Copy link
Contributor

jreback commented Jan 9, 2016

xref #10409

yeh I think the logic in the reindex part is not exactly right.

if you'd like to have a look would be great!

@jreback jreback added Reshaping Concat, Merge/Join, Stack/Unstack, Explode Categorical Categorical Data Type labels Jan 9, 2016
@jreback jreback added this to the Next Major Release milestone Jan 9, 2016
@mroeschke
Copy link
Member

Looks fixed on master. Could use a test.

In [112]: df = pd.concat(
     ...:     [x, pd.Series([0, 1, 2], index = [0, 1, 3])],
     ...:     axis = 1)

In [113]: df.dtypes
Out[113]:
0    category
0     float64
dtype: object

In [115]: pd.__version__
Out[115]: '0.26.0.dev0+555.gf7d162b18'

@mroeschke mroeschke added good first issue Needs Tests Unit test(s) needed to prevent regressions and removed Bug Categorical Categorical Data Type Difficulty Intermediate Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Oct 14, 2019
@jbrockmendel jbrockmendel added the Categorical Categorical Data Type label Oct 16, 2019
@jreback jreback modified the milestones: Contributions Welcome, 1.0 Nov 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Categorical Categorical Data Type good first issue Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants