[Question] Scaling the Query Frontend

[rp4fx12]

Hi, this question is sort of a two parter. 1. Do you guys have a slack or something to post questions? 2. The other question is regarding scaling out the query-frontend container. Regarding this portion of the docs: http://clipper.ai/tutorials/container_managers/#known-limitation-query-frontend-scaleout, would a rebalancing method mean that each query frontend would need to have at least one model container behind it? I'm guessing if I spin up a new query-frontend, I would need to launch it in such a way that it either terminates the connection to one of the pre-existing model containers or it spins up a fresh version of a model container. I'm also guessing I would need something to orchestrate that, like maybe some flask server or something else that sits as a separate pod to the query-frontend (or bake it into the mgmt frontend).

[rkooo567]

@simon-mo @RehanSD Could you guys answer this question?

[withsmilo]

1. We have a Slack workspace, but I don't have no idea about how to invite you.
2. We usually leave enough query-frontends already created and scale out the model containers as needed to avoid the problem.

[asauray]

I don't really understand your answer @withsmilo

(1) We agree that the query frontend is stateful right ? So do de we have a 1:1 relationship between a model and a query frontend ?

(2) If you add more instances of the query frontend, and at the same time, model instances connect only to one instance of the query frontend. How do you ensure that each query frontend has at least one connection to an instance of a model in that situation ?