Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

socket listener collectd stops working on v1.33.1 #16455

Open
dmgeurts-mm opened this issue Jan 31, 2025 · 4 comments
Open

socket listener collectd stops working on v1.33.1 #16455

dmgeurts-mm opened this issue Jan 31, 2025 · 4 comments
Labels
bug unexpected problem or unintended behavior

Comments

@dmgeurts-mm
Copy link

Relevant telegraf.conf

[[inputs.socket_listener]]
  service_address = "udp://127.0.0.1:25826"
  data_format = "collectd"
  collectd_typesdb = ["/usr/share/collectd/types.db", "/usr/local/share/freeradius/radsniff_types.db"]
  collectd_security_level = "none"
  collectd_parse_multivalue = "split"

Logs from Telegraf

Apologies, I failed to collect debug logs as I encountered this on a production Radius server.

Nothing was logged in the systemd journal and when I verified output via stdin nothing was shown.

System info

Ubuntu 20.04

Docker

No response

Steps to reproduce

  1. Upgrade from <1.33.x to 1.33.1 and lo
  2. Spend a few hours checking everything but Telegraf
  3. Downgrade to v1.32.3 and metrics are back.
    ...

Expected behavior

Unaffected collectd processing

Actual behavior

No errors logged, and no output at all.

Additional info

No response

@dmgeurts-mm dmgeurts-mm added the bug unexpected problem or unintended behavior label Jan 31, 2025
@srebhan
Copy link
Member

srebhan commented Jan 31, 2025

@dmgeurts-mm do you have some more information? Without an error or at least some more specific symptoms it's hard to say anything... Was there some OOM or some other error?

@dmgeurts-mm
Copy link
Author

Sadly not, I can see if the secondary radius server receives the occasional query, to see if I can debug there.

@srebhan
Copy link
Member

srebhan commented Jan 31, 2025

Yeah it would be good to get some logging output of a sort. How often does this occur?

@dmgeurts-mm
Copy link
Author

When I update telegraf to the latest version, the socket_listener with collectd data format stops all output. So 100% loss of metrics without any errors or faults logged by Telegraf. I was looking at the logs as I expected the issue to be with radsniff or collectd. I don't recall seeing anything out of the ordinary.

Logs of Telegraf 1.33.1 starting:

Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Starting Telegraf 1.33.1 brought to you by InfluxData the makers of InfluxDB
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Available plugins: 236 inputs, 9 aggregators, 33 processors, 26 parsers, 63 outputs, 6 secret-stores
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Loaded inputs: cpu disk diskio exec kernel mem net netstat processes socket_listener swap system
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Loaded aggregators:
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Loaded processors:
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Loaded secretstores:
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Loaded outputs: file influxdb_v2 (2x)
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! Tags enabled: dc=*** host=rad01.***.domain.com
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"rad01.***.domain.com", Flush Interval:10s
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z W! [agent] The default value of 'skip_processors_after_aggregators' will change to 'true' with Telegraf v1.40.0! If you need the current default behavior, please explicitly set the option to 'false'!
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z W! DeprecationWarning: Value "false" for option "ignore_protocol_stats" of plugin "inputs.net" deprecated since version 1.27.3 and will be removed in 1.36.0: use the 'inputs.nstat' plugin instead for protocol stats
Jan 31 03:55:39 rad01 telegraf[388946]: 2025-01-31T02:55:39Z I! [inputs.socket_listener] Listening on udp://127.0.0.1:25826
Jan 31 03:55:39 rad01 systemd[1]: Started Telegraf.

And for 1.32.3 :

Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Starting Telegraf 1.32.3 brought to you by InfluxData the makers of InfluxDB
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Available plugins: 235 inputs, 9 aggregators, 32 processors, 26 parsers, 62 outputs, 6 secret-stores
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Loaded inputs: cpu disk diskio exec kernel mem net netstat processes socket_listener swap system
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Loaded aggregators:
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Loaded processors:
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Loaded secretstores:
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Loaded outputs: influxdb_v2 (2x)
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! Tags enabled: dc=*** host=rad01.***.domain.com
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"rad01.***.domain.com", Flush Interval:10s
Jan 31 04:06:51 rad01 systemd[1]: Started Telegraf.
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z W! DeprecationWarning: Value "false" for option "ignore_protocol_stats" of plugin "inputs.net" deprecated since version 1.27.3 and will be removed in 1.36.0: use the 'inputs.nstat' plugin instead for protocol stats
Jan 31 04:06:51 rad01 telegraf[401736]: 2025-01-31T03:06:51Z I! [inputs.socket_listener] Listening on udp://127.0.0.1:25826

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug unexpected problem or unintended behavior
Projects
None yet
Development

No branches or pull requests

2 participants