Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

change json override strategy #4396

Merged
merged 6 commits into from
Dec 2, 2021
Merged

change json override strategy #4396

merged 6 commits into from
Dec 2, 2021

Conversation

nathaniel-may
Copy link
Contributor

@nathaniel-may nathaniel-may commented Dec 2, 2021 •
edited
Loading

resolves #4392

Description

Previously we were overriding on a per field basis. However, some of the serialization problems came one step earlier when we called asdict. This allows concrete classes to define their own dict factory methods to override the serialization strategy earlier in the pipeline.

This is very manual and definitely not optimized, so we may want to explore using something like mashumaro instead after the initial release.

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change

@cla-bot cla-bot bot added the cla:yes label Dec 2, 2021
nathaniel-may
nathaniel-may commented Dec 2, 2021
View reviewed changes
core/dbt/events/functions.py
data[field] = f"JSON_SERIALIZE_FAILED: {type(value).__name__, 'NA'}"
log_line = dict()
try:
log_line = dataclasses.asdict(e, dict_factory=type(e).asdict)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the heart of the change.

@nathaniel-may nathaniel-may marked this pull request as draft December 2, 2021 18:36
@nathaniel-may nathaniel-may added the test all run tests for all python versions + systems label Dec 2, 2021
@nathaniel-may nathaniel-may marked this pull request as ready for review December 2, 2021 19:00
gshank
gshank approved these changes Dec 2, 2021
View reviewed changes
Copy link
Contributor

@gshank gshank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks fine to me. We should definitely look at other serialization options in the near future though.

@nathaniel-may nathaniel-may merged commit 74fbaa1 into main Dec 2, 2021
@nathaniel-may nathaniel-may deleted the nate/json-tweak branch December 2, 2021 20:04
leahwicz pushed a commit that referenced this pull request Dec 2, 2021
@leahwicz
change json override strategy (#4396)
af1a6d4
@leahwicz leahwicz mentioned this pull request Dec 2, 2021
Merged
4 tasks
leahwicz added a commit that referenced this pull request Dec 2, 2021
@leahwicz
change json override strategy (#4396) (#4403)
a4456fe 
Co-authored-by: Nathaniel May <nathaniel.may@fishtownanalytics.com>
iknox-fa pushed a commit that referenced this pull request Feb 8, 2022
@iknox-fa
change json override strategy (#4396)
6fd8349 
automatic commit by git-black, original commits:
  74fbaa1
  b2aea11
@aranke aranke mentioned this pull request May 20, 2024
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gshank gshank gshank approved these changes

@emmyoop emmyoop emmyoop approved these changes

@leahwicz leahwicz leahwicz approved these changes

@iknox-fa iknox-fa Awaiting requested review from iknox-fa

@jtcohen6 jtcohen6 Awaiting requested review from jtcohen6

Assignees
No one assigned
Labels
cla:yes test all run tests for all python versions + systems
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Cannot serialize _CachedRelation
4 participants
@nathaniel-may @gshank @emmyoop @leahwicz