-
Notifications
You must be signed in to change notification settings - Fork 99
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
what is the reasonable setting for expiry-interval and flush-interval? #406
Comments
Hi It's possible, if the metrics are badly tagged (ie, with ephemeral data like a request id or a timestamp), then those will build up in memory over time. It won't drop a metric (unless it's due for expiry), but it could run out of memory and crash. If your system automatically restarts the process, you may not realize it occurred. There's an event emitted on startup which you can look for (there's also one on shutdown, however running out of memory will cause it to not be emitted). Also most of the metrics emitted by gostatsd are accumulated gauges, so you'll see them reset to 0. Generally speaking, my preference for By not persisting metrics, it puts the onus on the user to emit data if they want it to persist, conversely when you do persist, then users can't opt-out of it. It can also be confusing with timers - what value should be emitted for the upper value, when the count is 0? (#135 is related) Hope this is useful. |
Mmm, this is interesting. I haven't had the chance to look, but is this the default value for gostatsd? |
Hi,
I am not sure if this is gostatsd issue ( still investigate ), we have missing metrics and wonder if it is because of our expiry-interval (1h) and flush-interval (10s) setting. Could this cause gostatsd remove metric in some cases ? might be it running out of memory ?
thanks
The text was updated successfully, but these errors were encountered: