Better error handling when there are no crashes to process #1518
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
johnclary
commented
Aug 21, 2024
| return | ||
| if cli_args.s3_download and not extracts_todo: | ||
| # always short circuit if we find nothing in S3 | ||
| raise Exception("No extracts found in S3 bucket") |
There was a problem hiding this comment.
Eeek—I thought we already had a check like this in place but it must have dropped off during one of the later refactors
There was a problem hiding this comment.
I also could have sworn we were raising an exception already
johnclary
commented
Aug 21, 2024
|
|
||
| if not pdf_count: | ||
| raise IOError("No PDFs found in extract") |
There was a problem hiding this comment.
we don't need this check anymore since we're handling this elsewhere in the script
johnclary
requested review from
frankhereford,
chiaberry,
tillyw,
mddilley and
roseeichelmann
3 tasks
chiaberry
approved these changes
Aug 21, 2024
frankhereford
approved these changes
Aug 21, 2024
Comment on lines
+53
to
+58
|
|
||
| no_crashes_found = ( | ||
| True if cli_args.csv and records_processed["crashes"] == 0 else False | ||
| ) | ||
|
|
||
| if cli_args.pdf and not no_crashes_found: |
There was a problem hiding this comment.
Much more elegant than my hacky fix. Thank you!
4 tasks
|
Thank you for your reviews! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Associated issues
crashReportsdirectory atd-data-tech#18719This issue cropped up when processing ~30 daily CRIS extracts. It seems that occasionally, but not always, the
crashReportsdirectory is completely missing from the extracts with no crashes. The CSV files are always present.This script makes the following changes:
--csvand--pdfand (2) there were no crashes found in the CSV--csvand--pdf)Testing
I've left ~30 extracts in the dev inbox which you can use for testing.
--s3-downloadcommand:Ship list