Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing app creation timing issue #116

Merged
merged 22 commits into from
May 18, 2021
Merged

Conversation

arschles
Copy link
Collaborator

@arschles arschles commented Mar 29, 2021

As said in #60, when the operator starts up an app, there's a race between KEDA and the scaler. If the scaler isn't listening for connections by the time KEDA tries to establish one, KEDA will start retrying with an exponential backoff (see this code for example). In this case, depending on how long it takes the scaler to begin listening for connections, KEDA may not retry for a relatively long time, while meanwhile the app is not autoscaling.

This PR ensures that the scaler is ready to accept connections before creating the ScaledObject and instructing KEDA to begin establishing a connection(s)

Checklist

  • Commits are signed with Developer Certificate of Origin (DCO)
  • Implement readiness/liveness check endpoints on the scaler
  • Build tests for these checks
  • Make the operator create the scaler deployment with readiness/liveness checks
  • Ensure that waitForScaler checks that the scaler is ready according to liveness/readiness checks
  • A PR is opened to update the documentation on our docs repo

Fixes #60

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
the scaler uses the server stubs but the operator now needs to use the client, so it needs to move out of the scaler binary

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
@arschles arschles added this to the Alpha 2 milestone Mar 30, 2021
arschles and others added 13 commits March 31, 2021 14:18
Also centralizing config for the scaler with envconfig

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
…ancel/timeout

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
@arschles arschles marked this pull request as ready for review April 14, 2021 19:49
@arschles arschles enabled auto-merge (squash) May 14, 2021 15:27
@arschles arschles merged commit 22c9120 into kedacore:main May 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Timing issue when a new app is created
3 participants