-
Notifications
You must be signed in to change notification settings - Fork 491
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd/tsdbrelay: enable version numbers, fix metadata, add error metrics #1859
Conversation
@@ -14,6 +15,8 @@ import ( | |||
"os" | |||
"strings" | |||
|
|||
version "bosun.org/_version" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@captncraig I'm curious why I had to put an alias here. Locally using go 1.6.3 it can't find the reference to version package and gives an error/removes the import if I try to use a default import or even the _ import. Scollector/Bosun seem to work fine without the alias.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't explain why. If it helps to remove the alias, go ahead, but it sounds like something odd may be going on with your go environment.
I'm going to test this on ny-gbraylx01, which uses tsdbrelay to forward to a local docker instance and an influxdb instance and relay to the primary bosun instance. @captncraig please review when you have time |
Hmm... Metadata is working now but it still doesn't always relay. I'll keep working on this. |
This approach isn't going to work. The reverse proxy generates a 500 response code if there is a connection issue to the primary target and immediately sends that to the client. So even if we still try forwarding the data points to the other endpoints the scollector client will have those same data points queued up to be retried, resulting in duplicate data. I think there is a simpler way to solve the issue of HA forwarding, so I'm going to bundle up the other commits and push those to master, since they fix a bunch of non forwarding related issues. |
This branch now has all the non-forwarding related changes. I plan on merging it tomorrow (testing on ny-gbraylx01 overnight) |
…. Also continue additional relay urls on failure.
Also:
removes runtime.GOMAXPROCS(runtime.NumCPU()) since that is now the default
tabs to spaces in docker.sh
updates docker image to use to go1.7.linux-amd64.tar.gz
fixes missing call to metadata.init
adds metrics for tracking errors from opentsdb/bosun
adds metrics for tracking relayed/errors from secondary relays.
This also changes the logic so that data points are always relayed to bosun and secondary relays instead of bailing when the primary opentsdb end point returns an error.Will keep thinking on this and open a separate PR.Fixes #1792