-
Notifications
You must be signed in to change notification settings - Fork 233
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide --cluster-name flag to the OpenStack external CCM #1619
Conversation
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: xmudrii The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/cherry-pick release/v1.3 |
@xmudrii: once the present PR merges, I will cherry-pick it on top of release/v1.3 in a new PR and assign it to you. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
81c3603
to
38d4489
Compare
38d4489
to
b98e0f4
Compare
/hold |
LGTM label has been added. Git tree hash: 33f2780609a175dd4244954259db8b609001549b
|
b98e0f4
to
4a2ffb6
Compare
4a2ffb6
to
9f3b6e5
Compare
LGTM label has been added. Git tree hash: b157518036f9112f4ab105a1ebc704abda7198f7
|
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
9f3b6e5
to
00ee956
Compare
/retest |
LGTM label has been added. Git tree hash: ba3c60b91a295f5f2246b74ee434947abf3b0c3d
|
/hold cancel |
@xmudrii: #1619 failed to apply on top of branch "release/v1.3":
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
What this PR does / why we need it:
The OpenStack external CCM takes the
--cluster-name
flag which is used as a prefix for cloud resources. Currently, we're not providing this flag, so the CCM uses the default value which iskubernetes
. This can cause conflicts if there are multiple clusters in the same tenant, further causing some resources (e.g. Load Balancers) to be reused across clusters.For example, let's look at Octavia Load Balancers. If you create a Service named
nginx
in thedefault
namespace, the Octavia Load Balancer would be named askube_service_kubernetes_default_nginx
.If you create the same service in a different cluster running in the same tenant, OpenStack will just update the Load Balancer created earlier, instead of creating a new LB or failing. The Load Balancer would now point to the new cluster until the Service in the first cluster is not reconciled again (reconciliations happen automatically periodically). Once the Service in the first/old cluster is reconciled, the Load Balancer would now point to the first/old cluster until the Service in the new cluster is not reconciled again, and so on.
This is can be quite hard to debug and detect. The solution is to provide a unique value using the
--cluster-name
flag. In this case, the cloud resources (e.g. Load Balancer) would be named properly.However, you can't easily change the
--cluster-name
flag in a cluster that's already running. Doing that would cause OpenStack CCM to lose track of all Load Balancers, so all LBs would be recreated. That can be quite problematic and cause users' applications to go down. Therefore we need a backwards compatible approach.The cluster name is determined in the following way:
--cluster-name
flag set--cluster-name
flag on the OpenStack CCM. If there is, read the value and return it, otherwise don't set the OpenStack cluster name, in which case it defaults tokubernetes
If the operator wants to set the
--cluster-name
for clusters that don't have it set, they can change the OpenStack CCM DaemonSet to set the flag manually. It's recommended to set the value to the cluster name provided in the KubeOneCluster manifest, but it's not required. When runningkubeone apply
, KubeOne will read the value provided to the DaemonSet and will use it when reconciling the CCM deployment. Note that doing this will cause OpenStack CCM to effectively recreate all LBs. The old LBs need to be deleted manually.Fixes #1613
Does this PR introduce a user-facing change?:
/assign @kron4eg
cc: @qeqar