Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(logging): Implement early logging #15629

Merged
merged 11 commits into from
Jul 26, 2024
Merged

Conversation

srebhan
Copy link
Member

@srebhan srebhan commented Jul 16, 2024

Summary

This PR restructures the logging framework to implement "early" logging to collect log-messages that are issued before the actual logger is set-up. Currently, this is limited to non-fatal messages i.e. everything that survives loading and parsing of the config as well as configuring (but not initializing) the plugins.

Checklist

  • No AI generated code was used in this PR

Related issues

@telegraf-tiger telegraf-tiger bot added the chore label Jul 16, 2024
@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Jul 16, 2024
@srebhan srebhan assigned powersj and DStrand1 and unassigned srebhan Jul 16, 2024
Copy link
Member

@DStrand1 DStrand1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@DStrand1 DStrand1 removed their assignment Jul 17, 2024
Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mentioned this, but if I build and run by hand I see the loading config twice:

❯ ./telegraf --config config.toml
2024-07-18T16:19:12Z I! Loading config: config.toml
2024-07-18T16:19:12Z I! Loading config: config.toml
2024-07-18T16:19:12Z I! Starting Telegraf 1.32.0-ec5b839f brought to you by InfluxData the makers of InfluxDB

and with two configs:

❯ ./telegraf --config config.toml --config config.toml
2024-07-18T16:20:03Z I! Loading config: config.toml
2024-07-18T16:20:03Z I! Loading config: config.toml
2024-07-18T16:20:03Z W! Overlapping settings in multiple agent tables are not supported: may cause undefined behavior
2024-07-18T16:20:03Z I! Loading config: config.toml
2024-07-18T16:20:03Z I! Loading config: config.toml
2024-07-18T16:20:03Z W! Overlapping settings in multiple agent tables are not supported: may cause undefined behavior

The quiet and debug flags look fine.

Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got another one for you, it seems the debug agent output is now missing:

[agent]
  debug = true
  omit_hostname = true

[[inputs.mem]]

[[outputs.file]]

Specifically these lines:

2024-07-18T16:38:38Z D! [outputs.file]  Wrote batch of 1 metrics in 51.73µs
2024-07-18T16:38:38Z D! [outputs.file]  Buffer fullness: 0 / 10000 metrics

@srebhan
Copy link
Member Author

srebhan commented Jul 24, 2024

@DStrand1 and @powersj please give this another review as the code structure changed again... Sorry!

Copy link
Member

@DStrand1 DStrand1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great! Just one small comment

@@ -77,6 +77,7 @@ func (s *Shim) watchForShutdown(cancel context.CancelFunc) {
// Run the input plugins..
func (s *Shim) Run(pollInterval time.Duration) error {
if s.Input != nil {
fmt.Printf("input logger: %+v (%T)\n", s.log, s.log)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks line an unintentional printf?

@telegraf-tiger
Copy link
Contributor

Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My usual test cases seem to pass!

  • Basic config
  • Non-existent config
  • Two configs (same file)
  • HTTP config
  • Non-existent HTTP config
  • With debug enabled
  • With quiet enabled
  • Kafka's sarama debug messages
  • opcua client_trace

Run the above on both Linux and Windows

@powersj powersj merged commit ef41198 into influxdata:master Jul 26, 2024
27 checks passed
@github-actions github-actions bot added this to the v1.31.3 milestone Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants