Minor Cleanup of Structured Logging Module #4266
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nathaniel-may
force-pushed
the
logging-cosmetics
branch
from
6252d49
to
d4bf338
Compare
leahwicz
approved these changes
Nov 11, 2021
26 tasks
added 14 commits
November 16, 2021 09:45
nathaniel-may
force-pushed
the
logging-cosmetics
branch
from
ab25bfe
to
c5ca29f
Compare
5 tasks
emmyoop
approved these changes
Nov 16, 2021
There was a problem hiding this comment.
The adapter fixes you implemented made me realize we should have generic adapter types, similar to what's in test_types.py for new adapters(or anyone inclined to make more updates than the few lines). Seems off to have new adapters use the old logging format.
iknox-fa
reviewed
Nov 16, 2021
core/dbt/events/adapter_endpoint.py
Outdated
|
|
||
| fire_event(event) | ||
|
|
||
|
|
||
| def or_none(x: dict, key: str) -> Optional[Any]: |
There was a problem hiding this comment.
There's a builtin for this:
https://docs.python.org/3/library/stdtypes.html#dict.get
iknox-fa
reviewed
Nov 16, 2021
| # this class shouldn't be createable, but we can't make it an ABC because of a mypy bug | ||
| if type(self).__name__ == 'AdapterEventBase': | ||
| raise Exception( | ||
| 'attempted to create a message for AdapterEventBase which cannot be created' |
There was a problem hiding this comment.
Love the more explicit exception msg!
iknox-fa
suggested changes
Nov 16, 2021
|
@emmyoop if I'm understanding you correctly, I think that's exactly what we have already with the subclasses of |
emmyoop
approved these changes
Nov 16, 2021
iknox-fa
approved these changes
Nov 17, 2021
This was referenced Nov 17, 2021
Closed
iknox-fa
pushed a commit
that referenced
this pull request
Feb 8, 2022
* cleanup structured logging module * update adapter logger to preserve new-style logging formatting automatic commit by git-black, original commits: 719b202
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
1 - Cleaned up some of the code.
2- Fixed the adapter interface in here since post-cleanup it wasn't type checking. In the new system, the following logger call would have thrown a runtime exception before this change but is almost certainly present in the wild since it was found in our adapter tests:
logger.info("ran this sql: {}", sql)Rather than re-implement every way a python logger could be called, I just make the std lib logger construct the message component of the log line and hand it back with a stringio buffer.<- this strategy did not work because the std lib logger does not use new-style string formatting, only old-style.I dug into some logger libraries and found that this string formatting is really the only thing that
*argsis doing. Python prefers it this way so it can be done lazily which we are already doing via a function. I apply these arguments with new-style formatting viastr::formatto maintain compatibility with the former logbook logger.Checklist
CHANGELOG.mdand added information about my change