Skip to content

read_csv example tricks parser dtypes #4692

@hayd

Description

@hayd

Poster has an example which tricks read_csv into thinking column a is a int, but then throws it lots of strings (and it then infers 1s as strings).

import pandas as pd
df = pd.DataFrame({'a':['1']*100000 + ['X']*100000 + ['1']*100000, 'b':['b']*300000})
df.to_csv('test', sep='\t', index=False, na_rep='NA')
df2 = pd.read_csv('test', sep='\t')
print df2['a'].unique()

http://stackoverflow.com/questions/18471859/pandas-read-csv-dtype-inference-issue

I think this is rather an edge case tbh. :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugIO CSVread_csv, to_csvIO DataIO issues that don't fit into a more specific label

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions