Allocator throttled by default K8s Client requests per second

[markmandel]

What happened:

Allocation through the Allocator gRPC service is showing far lower performance than what was previously tested through a GameServerAllocation directly through the Kubernetes API. (#804 showed 80-160 QPS)

Instead this is showing 4-6 QPS, which is far lower

What you expected to happen:

Similar QPS as directly calling a GameServerAllocation when creating allocations through the gRPC API.

How to reproduce it (as minimally and precisely as possible):

https://github.com/googleforgames/agones/tree/master/test/load/locust-files

Anything else we need to know?:

I think I have a likely culprit for the slowdown through gRPC.

Through the controller we set the QPS and Burst values: https://github.com/googleforgames/agones/blob/master/cmd/controller/main.go#L137-L138

But we don't do the same thing on the allocator: https://github.com/googleforgames/agones/blob/master/cmd/allocator/main.go#L313-L332

Seems like we need a agones.allocator.apiServerQPSBurst and agones.allocator.apiServerQPSBurst on the Helm chart with similar values that are passed down into the Allocator service.

Environment:

• Agones version: 1.9.0
• Kubernetes version (use kubectl version): 1.16.0
• Cloud provider or hardware configuration: GKE
• Install method (yaml/helm): helm
• Troubleshooting guide log(s): N/A
• Others:

[markmandel]

/assign