Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

receive: allow custom connection pooling #1966

Closed

Conversation

blockloop
Copy link
Contributor

@blockloop blockloop commented Jan 8, 2020

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

Receive: enable custom connection pool sizing. With the default configuration machines with very high load (over 500 requests per second) will cycle TCP connections too frequently and run out of available ports. See more here

Verification

Running a custom build in production. Also see #1955

@blockloop blockloop force-pushed the bjones/receive/connection-pool branch from d313c74 to b61fb0a Compare January 8, 2020 16:42
Copy link
Member

@squat squat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 awesome stuff

Copy link
Member

@squat squat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 awesome stuff

cmd/thanos/receive.go Show resolved Hide resolved
pkg/receive/handler.go Outdated Show resolved Hide resolved
Signed-off-by: Brett Jones <blockloop@users.noreply.github.com>
@blockloop blockloop force-pushed the bjones/receive/connection-pool branch from f5851b3 to 4904950 Compare January 8, 2020 17:38
@blockloop
Copy link
Contributor Author

blockloop commented Jan 8, 2020

I think the build needs to be kicked @bwplotka
Also, I addressed your change requests

@brancz
Copy link
Member

brancz commented Jan 8, 2020

Somewhat related, I was thinking we should switch the internal communications to use gRPC, which could eventually evolve to also be the externally accepting write request API. I’m mentioning this as that would probably make this obsolete. What do you think?

@blockloop
Copy link
Contributor Author

blockloop commented Jan 8, 2020

I agree. I think this should be gRPC streams all the way through. I have a branch on my fork with a lot of that work complete. https://github.com/blockloop/thanos/tree/bjones/receive/use-grpc

@squat
Copy link
Member

squat commented Jan 8, 2020 via email

@blockloop
Copy link
Contributor Author

@squat, I think that makes sense. Right now since forwarding and receiving are the same code that would require a bit of a change. I can change gears a bit and start on splitting away forwarding from receiving HTTP remote writes. That might be the better option. Ideally the gRPC endpoint would also accept remote writes externally as well.

@squat
Copy link
Member

squat commented Jan 8, 2020 via email

@blockloop
Copy link
Contributor Author

Yeah I think a RemoteWrite method needs to be added to store.TSDBStore though, right?

@blockloop
Copy link
Contributor Author

Maybe we should shift the conversation to #1710 and close this PR?

@squat
Copy link
Member

squat commented Jan 8, 2020

Yeah it’ll have to be a new service that has the store methods as well as the RemoteWrite method

@blockloop
Copy link
Contributor Author

Closing this in favor of #1970

@blockloop blockloop closed this Jan 9, 2020
@blockloop blockloop deleted the bjones/receive/connection-pool branch January 9, 2020 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants