Initial OpenStack external CCM setup.

[ainmosni]

What this PR does / why we need it:
Adds support for the external OpenStack cloud controll manager.

As the internal OpenStack code in the controller manager
will be gone soon, we'll need to switch to the external cloud controller
manager, this PR adds the configuration for it.

* Adds support for the external OpenStack Cloud Control Manager.

[kubermatic-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign xmudrii
You can assign the PR to them by writing /assign @xmudrii in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kubermatic-bot]

Hi @ainmosni. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[xmudrii]

Hello @ainmosni,

thank you for the PR! However, before we can merge this, there are several things we need to take care of and decide about. The switch to the external CCMs is a little bit tough topic because there are some points that are still unclear.

You can find some of the discussion in #616

[ainmosni]

@xmudrii Yeah the migration will be painful but as there's no real development on the in-tree one and the in-tree code will be removed sooner rather later, it might be worth to pull the bandaid off sooner rather than later. I can definitely take a look at the OpenStack side to help with the migration options.

[kron4eg]

@xmudrii Yeah the migration will be painful but as there's no real development on the in-tree one and the in-tree code will be removed sooner rather later

AFAIK, timeline is set to 1.17.x

, it might be worth to pull the bandaid off sooner rather than later.

Indeed, that's why we create #616.

I can definitely take a look at the OpenStack side to help with the migration options.

There is a problem with openstack CCM. Its releases are bound to specific minor kubernetes versions. So it's not that easy to jump on it and we need to take into account older version (since we support k8s starting from 1.13).

[ainmosni]

Yeah I kept that in mind in this patch, AFAICS they only started to tag their images since 1.15, they used latest before that.

One thing that I didn't put in this patch but we should probably add is support for the cinder CSI plugin, so that PVCs keep working.

[kron4eg]

OK, I've got a confirmation that it's safe to use OS CCM 1.15 in 1.13, 1.14 and 1.15 clusters (but no 1.12, but since we don't support 1.12 anyway, we are safe).

[ainmosni]

@kron4eg Yeah that's what I gathered from their examples as well. It's a bit more complicated with cinder-csi (which isn't in this PR) as the CSI sidecar containers are k8s version specific. I was thinking about adding those, but changed my mind to keep the PR smaller.

[kron4eg]

It seems this PR lack the handling of in-tree VS out-of-tree cloud providers.

I mean, we have

kubeone/pkg/apis/kubeone/helpers.go

Line 59 in e05414f

func (p CloudProviderSpec) CloudProviderInTree() bool {

which currently hardcodes who is considered as in-tree.

So we need a way (which we currently don't have) to handle situation when in-tree cloud-provider is requested (in config) but also external: true. In such situation we believe we should resort to provider CCM (if available).

But all of this definitely is out of scope of this PR because it required refactoring.

[kubermatic-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[kron4eg]

/remove-lifecycle stale

[ivomarino]

Hi all, is this PR going to happen "soon" or what's the best actual solution on this topic? Thanks

[kron4eg]

@ivomarino yes, in the nearest future, I'd say. I'm working on integrating changes from this PR.

[kron4eg]

This PR is replaced by #820

@ainmosni thanks for initiating this!