-
-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
String dtype: honor mode.string_storage option (and change default to None) #59488
String dtype: honor mode.string_storage option (and change default to None) #59488
Conversation
Should I use |
Yea I prefer auto for string options like this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm ex changing None to auto
Thanks @jorisvandenbossche |
… None) (#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (pandas-dev#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
… None) (#59488) * String dtype: honor mode.string_storage option (and change default to None) * fix test + explicitly test default * use 'auto' instead of None
At the moment, the
pd.options.mode.string_storage
option defaults to"python"
, and theStringDtype()
constructor just ignores it when constructing the future default NaN-variant (i.e. the option is only used for creating the NA-variant), but uses the logic of "use"pyarrow"
if installed otherwise"python"
".Given this option exists, I think it makes sense to let it always be honored, so that a user can actually control this through the option (and also for testing this makes things a bit easier to check the behaviour in case pyarrow is not installed without uninstalling/installing it constantly).
In order to honor the option, I changed the default to
None
to mean this "use"pyarrow"
if installed otherwise"python"
" logic as default if constructing the NaN-variant.When constructing the NA-variant, for now
None
still gets translated to"python"
. I think we should also change the default for this dtype to use pyarrow if installed, but that's a change only for 3.0, I think (not to backport for 2.3), so therefore didn't include that in this PR.xref #54792