Selenium Grid - Autoscaling misadventures

[Konrad-Smolko]

Hello,

I'm setting up a Selenium Grid on a K8S cluster with Keda via Helm chart (last updated on our side a week ago). I was unable to get satisfactory results with autoscaling, as it doesn't want to use the full extent of resources available.

Max replicas is set to 32, there's more than enough CPU/RAM to support that. We primarily use Chrome for testing, as Deployments. I ran the following tests via Python's selenium library:

• 50 Chrome tests - Grid scaled up to 32 Chrome nodes, no issues;
• 20 Chrome tests - Grid scaled up to 19 nodes, 1 test had to wait in queue;
• 5 Chrome tests - Grid scaled up to 3 nodes, 2 tests had to wait in queue.

Something's fishy, but okay. I asked our testers to run their tests on the Grid, and their results weren't good. However many tests they send to the Grid, half of them ends up waiting in queue. 18 Chrome tests - 9 nodes, 9 waiting. My testing using robotframework gave me results similar to the first Python tests, then testing with out testers' code gave me the 50-50 split in nodes and queue. The only thing I'm aware our testers do differently is connect to Selenium using sessions.

I'm at a loss as to what I could do to force the Grid to use all resources available. As far as I can tell from kubectl descriptions and logs - Grid, Keda and HPA are working as intended. Changing Keda's scaling strategy in the Helm chart between default and accurate didn't seem to affect the issue.

At this point I made the decision to give up on autoscaling entirely, at least until a working solution is found. I would be grateful for any input on this, and sorry if this isn't the right place to ask.

KS

[VietND96]

Are you using scalingType Deployment or Job?

[VietND96]

I missed to read the statement We primarily use Chrome for testing, as Deployments, so it is ScaledObject to scale Deployment

[VietND96]

I tried to fix this via #6368

[Konrad-Smolko]

I saw that your changes were merged, I'll test our deployment with autoscaling again and let you know how it went. Thanks!

[VietND96]

Yes, it merged. However, they haven't released it yet. In the meantime, you can use the image patched in docker-selenium proj for early testing https://github.com/SeleniumHQ/docker-selenium/tree/trunk/.keda