Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/tsdbrelay: enable version numbers, fix metadata, add error metrics #1859

Merged
merged 5 commits into from
Aug 18, 2016

Conversation

gbrayut
Copy link
Contributor

@gbrayut gbrayut commented Aug 11, 2016

Also:
removes runtime.GOMAXPROCS(runtime.NumCPU()) since that is now the default
tabs to spaces in docker.sh
updates docker image to use to go1.7.linux-amd64.tar.gz
fixes missing call to metadata.init
adds metrics for tracking errors from opentsdb/bosun
adds metrics for tracking relayed/errors from secondary relays.

This also changes the logic so that data points are always relayed to bosun and secondary relays instead of bailing when the primary opentsdb end point returns an error. Will keep thinking on this and open a separate PR.

Fixes #1792

@@ -14,6 +15,8 @@ import (
"os"
"strings"

version "bosun.org/_version"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@captncraig I'm curious why I had to put an alias here. Locally using go 1.6.3 it can't find the reference to version package and gives an error/removes the import if I try to use a default import or even the _ import. Scollector/Bosun seem to work fine without the alias.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't explain why. If it helps to remove the alias, go ahead, but it sounds like something odd may be going on with your go environment.

@gbrayut gbrayut changed the title cmd/tsdbrelay: enable version numbers cmd/tsdbrelay: enable version numbers, fix metadata, always relay Aug 16, 2016
@gbrayut
Copy link
Contributor Author

gbrayut commented Aug 16, 2016

I'm going to test this on ny-gbraylx01, which uses tsdbrelay to forward to a local docker instance and an influxdb instance and relay to the primary bosun instance.

@captncraig please review when you have time

@gbrayut
Copy link
Contributor Author

gbrayut commented Aug 16, 2016

Hmm... Metadata is working now but it still doesn't always relay. I'll keep working on this.

@gbrayut
Copy link
Contributor Author

gbrayut commented Aug 16, 2016

This approach isn't going to work. The reverse proxy generates a 500 response code if there is a connection issue to the primary target and immediately sends that to the client. So even if we still try forwarding the data points to the other endpoints the scollector client will have those same data points queued up to be retried, resulting in duplicate data.

I think there is a simpler way to solve the issue of HA forwarding, so I'm going to bundle up the other commits and push those to master, since they fix a bunch of non forwarding related issues.

@gbrayut gbrayut changed the title cmd/tsdbrelay: enable version numbers, fix metadata, always relay cmd/tsdbrelay: enable version numbers, fix metadata, add error metrics Aug 16, 2016
@gbrayut
Copy link
Contributor Author

gbrayut commented Aug 16, 2016

This branch now has all the non-forwarding related changes. I plan on merging it tomorrow (testing on ny-gbraylx01 overnight)

…. Also continue additional relay urls on failure.
@gbrayut gbrayut merged commit 8eedf8d into master Aug 18, 2016
@gbrayut gbrayut deleted the tsdbrelay branch August 18, 2016 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tsdbrelay doesn't support -version flag
2 participants