cluster-api v1beta1 adoption ?

[jichenjc]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

I heard cluster-api v1beta1 is going to be out soon ... should we start to consider the adoption
and corresponding v1beta1 version of CAPO as well?

update:
https://kubernetes.io/blog/2021/10/08/capi-clusterclass-and-managed-topologies/ might be something
related as well, need further anaylsis
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

[pfisterer]

I guess that would be very good. I failed to try cluster-api the first time today because the released version v1.0.0 fails with an error when using cluster-api-provider-openstack. Just for your reference, this is the corresponding (closed) issue: kubernetes-sigs/cluster-api#5425

[mkjpryor]

The v1beta1 versions of the Cluster API core resources won’t adopt v1alpha4 resources from infrastructure providers. So this needs sorting one way or another, either by this provider moving to v1beta1 or by Cluster API core resources not expecting infrastructure resources to match their version.

[mkjpryor]

The current situation means we can’t upgrade the core Cluster API components to benefit from changes there.

[jichenjc]

from this kubernetes-sigs/cluster-api#5425 (comment)
seems yes, we need v1.0.x implementation for cluster-api now

The current situation means we can’t upgrade the core Cluster API components to benefit from changes there.

I don't fully understand this ... are you suggesting that even upgrade to v1beta1, CAPO still can't work with cluster-api 1.0.x ?

[mkjpryor]

@jichenjc When I tried Cluster API v1.0.0, the v1beta1 Cluster would not adopt a v1alpha4 OpenStackCluster as it’s infrastructure provider. The logs of the CAPI core provider were complaining about mismatching contracts because the version of the infrastructure cluster was not v1beta1.

[mkjpryor]

So all I’m saying is we now need to release a v1beta1 version to match the core resources.

I was kind of expecting the v1alpha4 CAPO resources to still work with the v1beta1 CAPI ones. It seems odd to require the infrastructure providers, which are independent of the core, to evolve at the same rate.

[jichenjc]

I was kind of expecting the v1alpha4 CAPO resources to still work with the v1beta1 CAPI ones. It seems odd to require the infrastructure providers, which are independent of the core, to evolve at the same rate.

usually provider version has some kind of coresponding vresion of CAPI version
looks CAPO need a new version to work with v1beta1 CAPI ,whatever the name v1beta1 or v1alpha5
we name it, need check the detail :)

[sbueringer]

@jichenjc Yes, you don't have to call it v1beta1. It can absolutely be v1alpha5 (and CAPO v0.5.0). The most important part is to specify the supported contract version (v1beta1 with CAPI v1.0.0) in the metadata.yaml and upgrade the dependencies to sigs.k8s.io/cluster-api@v1.0.0.

It's probably helpful to take a look at what other providers did and the doc here: https://cluster-api.sigs.k8s.io/developer/providers/v1alpha4-to-v1beta1.html

Usually it's:

• creating new CAPO types (as I said apiVersion does not have to match)
• create conversion webhooks
• upgrade CAPI dependency
• do everything else that's in the migration doc

[mkjpryor]

@sbueringer

Thanks for the clarification - all I observed is that the v1alpha4 resources didn’t work with the v1beta1 Cluster API resources with an error about contracts. If we just need to issue a new version with an updated supported contract value, that sounds sensible.

More generally, it would be good to know what would be needed for the OpenStack provider to go to v1beta1 as some people are nervous about using it in production until then.

[sbueringer]

More generally, it would be good to know what would be needed for the OpenStack provider to go to v1beta1 as some people are nervous about using it in production until then.

I can't really answer that. I think that's something that the CAPO community has to discuss. In general I think with v1beta1 comes a greater degree of API stability which the community should commit to.

[mkjpryor]

@sbueringer I didn’t expect you personally to answer it! It definitely needs to be a community discussion.

What I am saying is that the discussion needs to happen ASAP. I don’t think we want to be lagging too far behind and, in my experience so far, the provider is pretty reliable. If we just need to freeze the API and say this is v1beta1 then we do need to have a discussion about anything else that we want in the API to be able to do that.

We already have customers now saying “Cluster API is production ready - can we use it?” and we still have to say “well actually the OpenStack provider is still in alpha”, which they are less happy with.

We can’t stay in alpha forever so I think we as a community need to come up with a roadmap to beta ASAP. Ideally with a fairly short timeframe!

[chrischdi]

IMHO, going to v1beta1 or v1alpha5 is a matter of what kind of changes are maybe already planned to have.

As soon as we are on v1beta1 I think we should follow the "deprecate and remove in release + x" pattern.

Similar for "cut a v0.5.0" or "cut a 1.0.0".

However for capi compatibility I identified at least the following todos:

• create new CAPO types (either v1alpha5 or v1beta1)
• create conversion webhooks
• bump capi dependency

From migration docs:

• bump controller-runtime dependency to v0.10.+
• bump controller-tools version to v0.7.+
• check if not already done: Ensure your template resources support template.meta fields. Refer to the cluster and machine provider contract docs for more information. This is not required, but is recommended for consistency across the infrastructure providers as Cluster API graduates and opens up use cases where coordinating controllers can use labels and annotations from template infrastructure resources to do external provisioning or provide configuration information, e.g. IPAM support for vSphere / bare-metal.

[hidekazuna]

Sorry late to join discussion.

Current PRs which would break API are:

• some PRs related to port(Check trunk support before enabling trunk at port level #1014, ✨ Add tags to portOpts #1027,✨ Port rework #1028 ).
• Cinder type/AZ support(@mdbooth would propose https://kubernetes.slack.com/archives/CFKJB65G9/p1635160124014600?thread_ts=1635159518.012700&cid=CFKJB65G9).
• Add conditions(Implementation of Conditions feature in Openstack Machine Controller #972).

I understand we can release v1alpha5 using CAPI v1beta1 right now. But I think our code is not enough for v1beta1.
I propose

• release v1alpha5 using CAPI v1beta1 now
• After PRs above merged, release v1alpha6 and observe the release is good for v1beta1.
• If v1alpha6 is good for v1beta1, release v1beta1 which would almost relevant to v1alpha6

[jichenjc]

After PRs above merged, release v1alpha6 and observe the release is good for v1beta1.

I understand the v1alpahx to v1betax is mostly related to API stability (of course code quality/functions also important) , so might need some discussion and criteria on such decision

[chrischdi]

I'd be totally fine with going to v1alpha5 and then v1alpha6 too prior to going to beta :-)

[mkjpryor]

So would I (for now), as long as I can use the v1beta1 core resources.

I do think we should have a plan for getting to beta in the medium term though, e.g. O(12 months). We can't stay in alpha forever.

[mdbooth]

Thanks @hidekazuna for the change summary! Of those I think the only one which strictly needs a version bump is #1028, as all the rest are adding new fields with backwards-compatible defaults.

I would very much like to fully tidy up Networks/Ports before committing to a more stable API. Specifically I think we should aim to remove Networks entirely and keep only Ports.

We could also do with reducing the Instance API type to only the fields which are actually set, as this was formerly a combined Spec and Status type.

[mdbooth]

While we're at it, I just started working on RootVolume and I notice that it has a few problems:

• SourceType must be "image"
• DeviceType must be unset or "disk"
• ImageID must be set in RootVolume.SourceUUID and won't be taken from InstanceSpec.Image like non-boot from volume

This just seems like a collection of foot guns to me. I'd like to reduce RootVolume to:

• Size
• VolumeType
• AvailabilityZone

i.e. Remove the fields which the user can only reasonably use to break it[1], and don't require the user to specify the image differently for boot from volume.

This would also be a breaking change.

[1] That is assuming we don't have a use case for boot from volume snapshot? I can't think of one.

[hidekazuna]

Created cut a v0.5.0(#1029) from #1024 (comment). Thanks @chrischdi .

[jichenjc]

That is assuming we don't have a use case for boot from volume snapshot? I can't think of one.

snapshot seems not a big use case for us? mostly should be provisioning VM

[mdbooth]

Incidentally, a volume snapshot in this case would be used very much the same way as a glance image: it would just be stored in Cinder instead of Glance. However, I'm not aware of anybody using it this way. CoreOS specifically isn't designed to be used this way at all. I'm very happy not to maintain 'accidental' support for it, although I'd be happy to add it deliberately if somebody came to us with a use case.

I suspect that the only reason the API looks like this is that the implementation naively followed the Nova block device mapping API.

[jichenjc]

I suspect that the only reason the API looks like this is that the implementation naively followed the Nova block device mapping API.

I guess so and I agree with your suggestions above, thanks :)

[hidekazuna]

I am going to create v1alpha5 API with CAPI v1beta1(#1033). But now I am thinking that without creating new API version, only create v0.5.0 with CAPI v1beta1 is better option. We actively changing API now and our issue is that we need release with CAPI v1beta1. If we agree that release v0.5.0 only updating CAPI v1beta1, we will create release-0.4 branch, and create v0.5.0 release in the next week as much as possible.

[jichenjc]

So the question here is how to sync the changes from r-04 to 0.5 we will create soon
need some criteria on this? e.g some changes will be forward-port to 0.5 after it's settle down at r-0.4?

[hidekazuna]

If API version is the same, no reason to stay release-0.4, right?
If we release new minor version which is the same API version with the previous release, we maintain only new minor version.

[hidekazuna]

After all, I will do the followings:

Version	Branch name	CAPO API version	CAPI API version
0.4.0	release-0.4	v1alpha4	v1alpha4
0.5.0	release-0.5	v1alpha4	v1beta1
-	main	v1beta1	v1beta1

Steps:

1. Create release-0.4 branch
2. Merge ✨ Update CAPI to v1beta1 #1041
3. Create release-0.5 branch
4. Release 0.5.0 from release-0.5 branch
5. Merge ✨ Add v1beta1 API types #1047

I will start from 17th 00:00 UTC.

[jichenjc]

the approach sounds good, thanks :)

[jichenjc]

@hidekazuna given the great PR you made, I think we are ok to close this?

[hidekazuna]

@jichenjc Yes, let7s close this PR.