Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schemas: fix handling of nullable fields #132

Merged
merged 11 commits into from
Nov 8, 2024

Conversation

hariso
Copy link
Contributor

@hariso hariso commented Nov 7, 2024

Description

Data unmarshaling is done in two phases: unmarshaling by Avro, which sometimes represents values with maps, and our own unmarshaling, where we substitute those maps with actual values. For example, "my_field": map[string]any{"int": 1} needs to be replaced with "my_fied": 1.

Value substitution needs to know what's the "parent" of a field, so that it can replace the value for it. Due to a bug, the value substitution for fields that are nullable (i.e. that have null in their union type) was triggered for all fields in the parent object. Because value substitution assumes that the values are in a special format (a map with just one key, where the key is the type, and the value is the actual value), we got an error, because the other fields aren't in that format.

The actually fix is this part, where we check the type of the field. The rest of the changes are mostly comments and a bit of extracting code into methods.

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@hariso hariso marked this pull request as ready for review November 7, 2024 19:30
@hariso hariso requested a review from a team as a code owner November 7, 2024 19:30
Copy link

@maha-hajja maha-hajja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! thank you for the fix

Copy link
Member

@lovromazgon lovromazgon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work 👍

schema/avro/union.go Show resolved Hide resolved
@hariso hariso enabled auto-merge (squash) November 8, 2024 12:20
@hariso hariso merged commit d237fb3 into main Nov 8, 2024
3 checks passed
@hariso hariso deleted the haris/schemas-fix-nullable-fields branch November 8, 2024 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants