Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avro union fields listed in avro_tags do not get converted to tags #16271

Closed
fajpunk opened this issue Dec 6, 2024 · 0 comments · Fixed by #16272
Closed

Avro union fields listed in avro_tags do not get converted to tags #16271

fajpunk opened this issue Dec 6, 2024 · 0 comments · Fixed by #16272
Labels
bug unexpected problem or unintended behavior

Comments

@fajpunk
Copy link
Contributor

fajpunk commented Dec 6, 2024

Relevant telegraf.conf

[[ inputs.file ]]
  files = ["./testcases/union-nullable-tag/message.json"]
  data_format = "avro"

  avro_format = "json"
  avro_measurement = "Switch"
  avro_tags = ["switch_wwn", "some_union_in_a_tag"]
  avro_fields = ["up_time", "cpu_utilization", "memory_utilization", "statistics_collection_time"]
  avro_timestamp = "statistics_collection_time"
  avro_timestamp_format = "unix_ms"
  avro_union_mode = "nullable"
  avro_schema = '''
        {
                "namespace": "com.brocade.streaming",
                "name": "fibrechannel_switch_statistics",
                "type": "record",
                "version": "1",
                "fields": [
                        {"name": "some_union_in_a_tag", "type": ["null", "string"], "default": null, "doc": "Some union that is used in a tag"},
                        {"name": "switch_wwn", "type": "string", "doc": "WWN of the Physical Switch."},
                        {"name": "statistics_collection_time", "type": "long", "doc": "Epoch time when statistics is collected."},
                        {"name": "up_time", "type": "long", "doc": "Switch Up Time (in hundredths of a second)"},
                        {"name": "cpu_utilization", "type": ["null","float"], "default": null, "doc": "CPU Utilization in %"},
                        {"name": "memory_utilization", "type": ["null", "float"], "default": null, "doc": "Memory Utilization in %"}
                ]
        }
  '''

Logs from Telegraf

2024-12-06T19:43:12Z W! [parsers.avro::file] Could not convert map[string:some_value] to string for tag "some_union_in_a_tag": type "map[string]interface {}" unsupported

System info

docker.io/library/telegraf:1.30.2-alpine

Docker

No response

Steps to reproduce

  1. Configure telegraf with an avro input (binary, json, whatever, it doesn't matter)
  2. Configure telegraf with an avro schema that includes a field with a union type (schema can be specified in the config, or come from the schema registry, doesn't matter)
  3. Put that union field in avro_tags in the input configuration
  4. Try to ingest an avro-encoded message that matches that schema

Expected behavior

An InfluxDB metric is inserted with the value in avro_tags as a tag on the metric

Actual behavior

The value does not get added to the metric as a tag

Additional info

No response

@fajpunk fajpunk added the bug unexpected problem or unintended behavior label Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant