-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[low-code connectors] fix so we don't display yaml when debug flag is turned off #15383
Conversation
@@ -42,9 +48,4 @@ def streams(self, config: Mapping[str, Any]) -> List[Stream]: | |||
def _read_and_parse_yaml_file(self, path_to_yaml_file): | |||
with open(path_to_yaml_file, "r") as f: | |||
config_content = f.read() | |||
parsed_config = YamlParser().parse(config_content) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how does moving this help? the debug
level was already set above it no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually when we construct the declarative source in _read_and_parse_yaml_file()
in source.py (here's how its implemented for source-sendgrid), this is done before we start entrypoint.py.launch()
. main.py for sengrid
So we don't actually know yet if the user specified --debug
since that gets interpreted in launch()
. When we move it to streams()
this gets called from entrypoint.py
after log level is set.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and the above debug level should be removed since its effectively a no-op, since it gets reassigned in entrypoint.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably add a todo to resolve this but nbd for this pr
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the debug level in the above code, so I don't think we'd need any follow ups. As it stands now the flow of the read command is:
- parse yaml
- save it to
self._source_config
- parse command line args in entrypoint
- assign logger level to DEBUG or INFO based on presence of
--debug
- handle
read
- emit debug message with parsed YAML if on
- process sync
What
For some reason I put the debug statement in the wrong place. It's poor DX to get debug statements when you don't have the
--debug
on.How
We store the parsed yaml on the declarative stream. We can just print it later in the
streams()
method and at this point the debug/info level has been properly set.Note: We don't need to do this for
check()
because check invokes thestreams()
method. Otherwise check would print it twice