A68 random subsetting #12377
A68 random subsetting #12377
Conversation
Zgoda91
force-pushed
the
A68_random_subsetting
branch
from
9178308 to
aefee00
Compare
Zgoda91
force-pushed
the
A68_random_subsetting
branch
from
aefee00 to
1870e6f
Compare
| resolvedAddresses.getLoadBalancingPolicyConfig(); | ||
|
|
||
| ResolvedAddresses subsetAddresses = filterEndpoints( | ||
| resolvedAddresses, config.subsetSize, new SecureRandom().nextLong()); |
There was a problem hiding this comment.
The same seed should be used for the life of the policy. When the addresses don't change much, we don't want the chosen subset to change much. If we change the seed every time, we'll always get vastly different results, even with the same input.
| resolvedAddresses.getLoadBalancingPolicyConfig(); | ||
|
|
||
| ResolvedAddresses subsetAddresses = filterEndpoints( | ||
| resolvedAddresses, config.subsetSize, new SecureRandom().nextLong()); |
There was a problem hiding this comment.
We're not cryptographically hashing, so Random is fine; no need for SecureRandom.
|
|
||
| Collections.sort(endpointWithHashList, new HashAddressComparator()); | ||
|
|
||
| ArrayList<EquivalentAddressGroup> addressGroups = new ArrayList<>(); |
There was a problem hiding this comment.
Pass subsetSize to the ArrayList constructor, to reduce the number of allocations and final memory use.
| * https://https://github.com/grpc/proposal/blob/master/A68-random-subsetting.md | ||
| */ | ||
| @Internal | ||
| public final class RandomSubsettingLoadBalancer extends LoadBalancer { |
There was a problem hiding this comment.
package-private? I don't see a need for it to be public.
| include ":grpc-util" | ||
| include ":grpc-opentelemetry" | ||
| include ":grpc-context-override-opentelemetry" | ||
| include ":grpc-third-party:zero-allocation-hashing" |
There was a problem hiding this comment.
We don't want to create a new artifact for this; that causes a lot of head-ache and lasts forever. xxhash is small enough that I'd sooner move it to grpc-core. The other option would be to use com.google.common.hash.Hashing, probably murmur3_128. xxhash has the zero-allocation implementation quite fitting for our use-case, but Guava's would probably be fine. We can always change it in the future.
implementing gRFC A65 grpc/proposal/pull/423
This change contains:
XxHash64library, so it can be shared betweenutilandxdsprojects. Proposed source directory is:third-party/zero-allocation-hashing.RandomSubsettingLoadBalancerandRandomSubsettingLoadBalancerProviderclasses and integrated them intoutilproject.random_subsettingLB policy and as well new envoy proto message.