Skip to content

WIP warn if parsing with du_parse #47828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 20 commits into from

Conversation

MarcoGorelli
Copy link
Member

@MarcoGorelli MarcoGorelli commented Jul 23, 2022

@MarcoGorelli
Copy link
Member Author

MarcoGorelli commented Jul 23, 2022

The issue is that in

        df = DataFrame(
            [[1, 2], [3, 4]],
            columns=date_range("1/1/2013", "1/2/2013"),
            index=["A", "B"],
        )
        df.index.name = name

        with tm.assert_produces_warning(warn):
            result = df.reset_index()

, resetting the index will try to insert an element into a datetime index. hence, it'll try to parse the new element as a date, and this'll trigger the new warning

There's currently no way of catching multiple warnings, so perhaps that should be addressed first

EDIt: passing multiple warnings as a tuple works fine

@@ -238,6 +238,17 @@ cdef inline bint does_string_look_like_time(str parse_string):

return 0 <= hour <= 23 and 0 <= minute <= 59

from pandas.util._exceptions import find_stack_level
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there an option that avoids importing from outside libs here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, could set it explicitly each time - I'll do that once I've sorted out the tests

@mroeschke mroeschke added Datetime Datetime data dtype Warnings Warnings that appear or should be added to pandas labels Jul 25, 2022
@MarcoGorelli MarcoGorelli force-pushed the warn-on-du-parse branch 2 times, most recently from 8eab6e9 to 33fd5b6 Compare July 31, 2022 16:51
timedelta_index = pd.to_timedelta(np.arange(5), unit="s")
period_index = pd.period_range("2000/1/1", "2010/1/1/", freq="M")
period_index = pd.period_range("1/1/2000", "1/1/2010", freq="M")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pd.to_datetime(["2000/1/1", "2010/1/1"]) doesn't raise, but pd.period_range("2000/1/1", "2010/1/1/", freq="M") does

don't know why, need to investigate

@MarcoGorelli MarcoGorelli force-pushed the warn-on-du-parse branch 2 times, most recently from 6051071 to ee5c80c Compare August 5, 2022 08:32

def du_parse_with_warning(*args, **kwargs):
parsed = du_parse(*args, **kwargs)
warnings.warn(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we take a perf hit here?

@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

@github-actions github-actions bot added the Stale label Sep 12, 2022
@MarcoGorelli
Copy link
Member Author

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

still working on this

@MarcoGorelli
Copy link
Member Author

MarcoGorelli commented Sep 16, 2022

closing, as from #12585 (comment) I don't think this is the right solution

log.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype Warnings Warnings that appear or should be added to pandas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inconsistent date parsing of to_datetime
3 participants