Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take into account Unhealthy GameServers when scaling #256

Closed
dgkanatsios opened this issue Jun 2, 2022 · 2 comments · Fixed by #272
Closed

Take into account Unhealthy GameServers when scaling #256

dgkanatsios opened this issue Jun 2, 2022 · 2 comments · Fixed by #272
Labels

Comments

@dgkanatsios
Copy link
Collaborator

In scenarios that we have Unhealthy GameServers, GameServerBuild controller does not take them into account when scaling.

@dgkanatsios dgkanatsios added the bug Something isn't working label Jun 2, 2022
@dgkanatsios
Copy link
Collaborator Author

dgkanatsios commented Jun 2, 2022

A repro:

A GameServerBuild became Unhealthy
A pod got evicted (because OOM), but the GameServer was not deleted because the GameServerBuild was Unhealthy and scaling was skipped
NodeAgent did not receive any heartbeats (since the Pod got evicted) so it marked the GameServer as Unhealthy
Increasing the CrashesToMarkUnhealthy did not incur deletion of the Unhealthy GameServer, since we're currently not doing that.
Scaling does not take Unhealthy into account, so eventually we ended up with an Unhealthy GameServer and less StandingBy than desired.

We should decide on what to do on these cases.

  • delete the Unhealthy GameServer
  • keep the Unhealthy GameServer alive for investigation

@dgkanatsios dgkanatsios added design and removed bug Something isn't working labels Jun 2, 2022
@dgkanatsios
Copy link
Collaborator Author

Related to #56

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant