Detecting when vector drops a large log line #22286
Unanswered
sean-brandenburg
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hey all, I'm working on adding some alerting for certain error conditions in vector. I have a file source writing to an s3 sink and want to detect when vector has dropped a log because the log line is larger than the source buffer.
I've been looking at the following internal_metrics "component_discarded_events_total", "buffer_discarded_events_total" and "events_discarded_total" on both the sink and source, but none of these seem to capture this case (I tested by appending a 16MB log line to the source file which is larger than the default 10MB buffer) and it none of these metrics were present when I checked the internal_metrics.
Wondering if there's another metric that might capture this or if there's some other way to detect when this happens. Thank you!
Beta Was this translation helpful? Give feedback.
All reactions