fix: Handle python_date_format in ExploreMixin #24068
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
This PR fixes an issue introduced in #20938 however it didn't surface until #22853 which resulted in a somewhat bad correctness regression as the resulting generated SQL was incorrect when filtering the temporal column, i.e., rather than queries being generated of the form,
they were being generated as:
which due to lexicographical ordering would wrongfully exclude any records where
ds
=2023-01-01
.The TL;DR is #20938 replicated logic from the
TableColumn
class however it did not include the logic which adhered to the Python date/time format meaning that columns which weren't handled by the DB engine specconvert_dttm
method were cast to an ISO 8601 formatted string adhering to theY-%m-%d %H:%M:%S.%f
format.This PR adds the missing logic and removes the now unused (per #22853)
TableColumn
methods to help improve the code hygiene and readability—having two or more instances of the same function makes the code very difficult to grok. @hughhhh I would suggest you perform a second pass to make sure all the legacy logic which was ported to theExploreMixin
has been removed.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TESTING INSTRUCTIONS
Added unit tests which would have hopefully helped discovered the issue.
ADDITIONAL INFORMATION