Skip to content

BUG fix for datetime null (closes #56853) #56869

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from

Conversation

NicoACloutier
Copy link

@NicoACloutier NicoACloutier commented Jan 14, 2024

Closes #56853

This is my first time contributing to this repo and I'm not super experienced with open source, sorry if the style isn't right or if there's some better way of doing it.

From what I can tell, the issue was caused by recurring calls to get_loc functions caused by the fact that when you pass in a datetime, it automatically converts subsequent insertions in the same level to datetime. If you don't supply one, it converts it to NaT, which causes an indexing error due to having a null value as an index, which gets you caught in endless recursion from try/except blocks. If you switch the datetime out for a float (or any other common type), the problem is avoided because the unfilled indexes are filled in will empty strings rather that null values. As far as I can tell, this problem only exists with datetime types.

It's kind of a hacky fix, but I figured it would just be easiest to check for null values in the indexes at the end of an insertion and replace them with empty strings. I tried fiddling around with the datetime stuff itself, but it just ended up messing other things up. Let me know if there's some other solution. I also added this as a test case, just to see whether it breaks.

@NicoACloutier
Copy link
Author

pre-commit.ci autofix

@NicoACloutier
Copy link
Author

pre-commit.ci autofix

@rhshadrach rhshadrach added Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate datetime.date stdlib datetime.date support labels Feb 6, 2024
@mroeschke
Copy link
Member

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

@mroeschke mroeschke closed this Feb 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug datetime.date stdlib datetime.date support Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Insert then delete column into MultiIndex with timestamps leads to RecursionError
3 participants