-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(parsers.avro): Allow union fields to be specified as tags #16272
Conversation
Thanks so much for the pull request! |
!signed-cla |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks OK to me. I don't know that I can actually review it since I don't have write access to influxdata:master, and we definitely need some InfluxData eyes on it, but I (or, I presume Dan) would be happy to talk anyone from InfluxData through what we're doing and why.
I think it's funny that it's our own team that eventually demanded an answer to Angelo's question
I have a question that comes from @afausti -- what should the default behavior be if you explicitly mark an incoming field as both a tag and a field? The current code "does both" which ... I don't know what that does in practice. The right answer is "don't do that," of course...but should I catch it when building the parser and warn and leave it a tag but not add it as a field?
from #13945
Well, OK, not quite the same question, it's "what do we do if a field is not a simple type", but turning it into a string somehow is clearly the right answer and this seems like a pretty good way. |
Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip. 📦 Click here to get additional PR build artifactsArtifact URLs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome contribution @fajpunk! Thanks you!
@athornton thanks for the review!
I relabeled this as a feature as this is what it is in my view. @DStrand1 what do you think?
Summary
Flatten any fields specified in
avro_tags
before trying to convert their values to strings.Without flattening, any union fields included in
avro_tags
do not actually get converted into Influx tags. This warning is logged:2024-12-06T19:43:12Z W! [parsers.avro::file] Could not convert map[string:some_value] to string for tag "some_union_in_a_tag": type "map[string]interface {}" unsupported
Checklist
Related issues
resolves #16271