-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use snappy compression for internal gRPC communication. #3368
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 17 files at r1.
Reviewable status: 2 of 17 files reviewed, all discussions resolved (waiting on @manishrjain)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 16 of 17 files at r1, 1 of 1 files at r2.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @danielmai and @manishrjain)
conn/pool.go, line 153 at r2 (raw file):
grpc.MaxCallRecvMsgSize(x.GrpcMaxSize), grpc.MaxCallSendMsgSize(x.GrpcMaxSize), grpc.UseCompressor((snappyCompressor{}).Name())),
You don't need the parentheses
Example: https://play.golang.org/p/T5DzEZhF_x8
conn/snappy.go, line 1 at r2 (raw file):
// Copyright 2017 The Cockroach Authors.
Is the copyright message right?
conn/snappy.go, line 25 at r2 (raw file):
) // NB: The encoding.Compressor implementation needs to be goroutine
nit: what is NB supposed to stand for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @manishrjain and @martinmr)
conn/pool.go, line 153 at r2 (raw file):
Previously, martinmr (Martin Martinez Rivera) wrote…
You don't need the parentheses
Nice catch. But I don't feel that it's necessary to change the source from CockroachDB.
conn/snappy.go, line 1 at r2 (raw file):
Previously, martinmr (Martin Martinez Rivera) wrote…
Is the copyright message right?
Yes. This file is from CockroachDB: https://github.com/cockroachdb/cockroach/blob/856ba9108f112f85d406bbe88d2208651859336e/pkg/rpc/snappy.go. It's the bare minimum needed to implement a gRPC compressor using snappy compression. Aside from the package name, everything else is the same.
conn/snappy.go, line 25 at r2 (raw file):
Previously, martinmr (Martin Martinez Rivera) wrote…
nit: what is NB supposed to stand for?
I'm not too sure, but it's probably this: https://en.wikipedia.org/wiki/Nota_bene. Basically, "pay attention". Again, I didn't change it from the original source.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @manishrjain)
Hey @danielmai, what happened to this PR? |
@campoy This PR in its current state would require all instances to use snappy compression for internal communication. Ideally, it'd be better if instances opted-in to use snappy compression for internal traffic if available to allow for rolling upgrades. This PR was created on the hunch that there'd be improvements for resource usage or performance with compressed payloads, but there was no investigation to confirm this. It'd be great if someone else could pick up where I left off here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 16 of 17 files at r1, 1 of 1 files at r2.
Reviewable status: complete! all files reviewed, all discussions resolved
Closes #3358.
This change is