-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[0.9.3rc1] Invalid time stamp in graphite metric causes panic #3785
Comments
@pkittenis Thanks for the analysis, but I'm a little confused by your repro step.
An 11 digit timestamp in seconds, such as of 14,401,794,881, exceeds the maximum time range for InfluxDB: https://influxdb.com/docs/v0.9/guides/troubleshooting.html#time-ranges, but the write succeeds for me without a panic, using the HTTP endpoint. There are no points written, for which I've opened a new bug: #3789 |
Is
A panic only occurs when writing to the graphite plugin port, not influxdb's HTTP endpoint. No panic occurs when attempting to write invalid 11 digit timestamps on the HTTP end point, default port 8086. My config: [[graphite]]
enabled = true
bind-address = ":2003" Everything else default. Client feeding invalid metric line with 11 digit timestamp into port 2003. echo "test 1 14401794881" | nc localhost 2003 Influxdb output and stack trace. This is on a fresh db, nothing else done to it.
Hope that is clearer. |
If a timestamp was larger than the max epoch value was sent via graphite it would cause the timestamp to overflow when it was marshaled/unmarshaled back from the raft log. The overflow cause the shard group to get created with the wrong timestamp which cause a panic when writing the point. The panic was caused because the timestamp that were supposed to exists in a map created by MapShards did not actually exist so a nil ShardGroup was used. The change prevents creating the point with an invalid timestamp. Since graphite using a timestamp in seconds, the maximum range is known and can be prevented. This also adds a check for the minimum range as well. Fixes #3785
If a timestamp was larger than the max epoch value was sent via graphite it would cause the timestamp to overflow when it was marshaled/unmarshaled back from the raft log. The overflow cause the shard group to get created with the wrong timestamp which cause a panic when writing the point. The panic was caused because the timestamp that were supposed to exists in a map created by MapShards did not actually exist so a nil ShardGroup was used. The change prevents creating the point with an invalid timestamp. Since graphite using a timestamp in seconds, the maximum range is known and can be prevented. This also adds a check for the minimum range as well. Fixes #3785
Got the following stack trace from 0.9.3rc1.
Cause is invalid time stamp in graphite metric. Confirmed to happen on latest version from master and
0.9.2
as well.To replicate:
Where
localhost:2003
is graphite plugin tcp listen port.Stack trace:
The text was updated successfully, but these errors were encountered: