-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Don't silently produce null values from invalid input to pl.datetime
and pl.date
#21013
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #21013 +/- ##
==========================================
+ Coverage 79.89% 79.96% +0.06%
==========================================
Files 1597 1597
Lines 228951 228964 +13
Branches 2618 2618
==========================================
+ Hits 182915 183080 +165
+ Misses 45437 45285 -152
Partials 599 599 ☔ View full report in Codecov by Sentry. |
pl.datetime
and pl.date
pl.datetime
and pl.date
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We really should return an error here, not a panic.
@ritchie46 I wasn't quite sure how to go about this, so let me know if it needs rework. Right now, we have an iterator that yields Instead, I opted to flag if an invalid value is observed during the array creation, and proceed as we used to (filling with nulls). Then I check in hindsight if an invalid value was observed, and at that point throw a |
}, | ||
// We have a valid date. | ||
|ns| Some(ns.num_days_from_ce() - EPOCH_DAYS_FROM_CE), | ||
) | ||
} else { | ||
None | ||
} | ||
}) | ||
.collect_trusted(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can return a result and then call try_collect_ca_with_dtype
, that's a bit cleaner.
Fixes #20977.
I am not sure if this is a breaking change or not.
Example of new behavior: