Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dtypes): enhance dtype inferral #47

Open
jfsantos-ds opened this issue Sep 24, 2021 · 0 comments
Open

feat(dtypes): enhance dtype inferral #47

jfsantos-ds opened this issue Sep 24, 2021 · 0 comments
Labels
perf A code change that improves performance

Comments

@jfsantos-ds
Copy link
Contributor

Description of the feature

  • Enhance dtype detection for dataframe columns (infer_dtypes method in auxiliary.py)
  • Maintain support for user provided dtypes (override inference)

Acceptance criterias (if applicable)

  • Erroneous data like strings in otherwise numerical columns is not sufficient criteria to infer categorical
  • Optional: ordinal dtype detection (a column that is likely categorical but order inference rule is applicable)

Additional context

Currently inference is based on solely the dtypes that pandas automatically infered

@jfsantos-ds jfsantos-ds added the perf A code change that improves performance label Sep 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf A code change that improves performance
Projects
None yet
Development

No branches or pull requests

1 participant