-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix 13823: Validate and Parse gracefully IngestionPipeline #14461
Conversation
ingestion_pipeline = IngestionPipeline.parse_obj(config_dict) | ||
return ingestion_pipeline | ||
|
||
except ValidationError: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we take care of the serviceConnection
? Not only the sourceConfig
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe that's where we get also a lot of noise, for example when we configure an URI without the scheme or something
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as discussed, that was the wrong comment. We'll be adding it to the run_automations endpoint though
@@ -400,37 +406,84 @@ def parse_workflow_config_gracefully( | |||
raise ParsingConfigurationError("Uncaught error when parsing the workflow!") | |||
|
|||
|
|||
def parse_test_connection_request_gracefully( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing the method since it wasn't being used anywhere except for tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks
Quality Gate passed for 'open-metadata-airflow-apis'Kudos, no new issues were introduced! 0 New issues |
Quality Gate passed for 'open-metadata-ingestion'Kudos, no new issues were introduced! 0 New issues |
* fixed ingestion pipeline parsing * Added validation for automation workflow --------- Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Describe your changes:
Fixes #13823
Type of change:
Checklist:
Fixes <issue-number>: <short explanation>