Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ETL-312] Ignore specific non-severe Android validation errors #112

Merged
merged 4 commits into from
Jan 4, 2023

Conversation

philerooski
Copy link
Contributor

@philerooski philerooski commented Jan 3, 2023

  • Ignores Android validation errors as outlined in ETL-312.
  • Minor refactor of Github upload-and-deploy workflow.

@philerooski philerooski requested a review from a team as a code owner January 3, 2023 23:19
@philerooski philerooski temporarily deployed to develop January 4, 2023 00:00 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 00:02 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 00:02 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 00:04 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 00:05 — with GitHub Actions Inactive
Copy link
Member

@thomasyu888 thomasyu888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm - I will say that I do dislike the fact that we have many lines of conditions that return the same state (return false), BUT.... I can't really think of other ways to capture these conditions in a clean way that doesn't add some complexity to the code.

"'appName' is a required property" in metadata_errors and
"'files' is a required property" in metadata_errors):
return False
if "taskData.json" in validation_result["errors"]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit. Use .get(....) then you don't need the extra if statement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally, yes. But I'm checking for inequality (if not errors == ...) so unless I made the non-severe error the default return value of the get method then in the case that there were no taskData.json errors, the inequality would evaluate to true and is_expected_validation_error would prematurely return False. The way it's written now is more readable and less redundant imo than duplicating the non-severe errors in the arguments to get. Although, I think the not == syntax is a bit unconventional for single comparisons, so I will change that.

@thomasyu888
Copy link
Member

@rxu17 can you take a look at this too?

@philerooski philerooski temporarily deployed to develop January 4, 2023 20:33 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 20:34 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 20:34 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 20:37 — with GitHub Actions Inactive
@philerooski philerooski temporarily deployed to develop January 4, 2023 20:37 — with GitHub Actions Inactive
@philerooski philerooski merged commit 8d2e3f9 into main Jan 4, 2023
@philerooski philerooski deleted the etl-312 branch January 4, 2023 20:44
Comment on lines 386 to 390
if "motion.json" in validation_result["errors"]:
motion_errors = validation_result["errors"]["motion.json"]
if not motion_errors == [
"'type' is a required property"]:
return False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should motion.json be checking against the same error as taskData.json? According to BMC-432 it seems like each MotionRecord also has an extra property called type?

Also the ticket mentions stepPath as a required property as well for motion.json that is missing for some records, would that be considered a non-severe error we would want to let through?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right. The complete list of errors is tracked here: https://sagebionetworks.jira.com/browse/BMC-432

I'll need to think about which of these qualifies as non-severe, but at least one of them which I've left out does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants