Questions about developing an operator that responds to as many Kubernetes versions as possible.

[smartkuk]

Type of question

Best practices
General operator-related help

Question

What did you do?

Hello.
There is an operator I developed based on the operator-sdk v0.17.2 version.
Based on this project, in other Kubernetes versions, the crd generated output is not correct, so I am thinking about creating a project for each Kubernetes version.
People who have had CRD problems have had other issue as well.

So I checked the docs and saw a guide saying that the operator-sdk version and the kubernetes version match.
So, to check which Kubernetes version operator-sdk is fully compatible with, I download the binary linked to the github release menu and run the operator-sdk version command. Can I judge the version by ?

$ ls -al
total 554496
drwxr-xr-x  2 root root     4096  3월  2 09:46 .
drwxr-xr-x 11 root root     4096  6월 24  2021 ..
-rwxrwxrwx  1 root root 89473879  5월 12  2021 argocd
lrwxrwxrwx  1 root root       22  5월 18  2021 GitAhead -> /opt/GitAhead/GitAhead
-rwxr-xr-x  1 root root 45113344  9월  9 14:34 helm3
-rwxrwxrwx  1 root root 32138779  8월 27  2021 kubebuilder
-rwxrwxrwx  1 root root 48541136  5월 14  2021 kubectl-argo-rollouts
lrwxrwxrwx  1 root root       35  4월 29  2021 libreoffice7.1 -> /opt/libreoffice7.1/program/soffice
lrwxrwxrwx  1 root root       35  2월 28 16:18 operator-sdk -> /usr/local/bin/operator-sdk-v0.19.4
-rwxr-xr-x  1 root root 69896879  8월 30  2021 operator-sdk-v0.17.2
-rwxr-xr-x  1 root root 76794439  2월 28 11:49 operator-sdk-v0.19.4
-rwxr-xr-x  1 root root 68108655  3월  2 09:28 operator-sdk-v1.4.0
-rwxr-xr-x  1 root root 68108368  3월  2 09:28 operator-sdk-v1.4.2
-rwxr-xr-x  1 root root 69589068  3월  2 09:26 operator-sdk-v1.9.2
$  operator-sdk-v0.17.2 version
operator-sdk version: "v0.17.2", commit: "0258db0119e8e18e15d035532427c329fce1e871", kubernetes version: "unknown", go version: "go1.13.10 linux/amd64"
$  operator-sdk-v0.19.4 version
operator-sdk version: "v0.19.4", commit: "125d0dfcc71fef4f9d7e2a42b1354cb79ffdee03", kubernetes version: "v1.18.2", go version: "go1.13.15 linux/amd64"
$  operator-sdk-v1.4.0 version
operator-sdk version: "v1.4.0", commit: "67f9c8b888887d18cd38bb6fd85cf3cf5b94fd99", kubernetes version: "1.19.4", go version: "go1.15.5", GOOS: "linux", GOARCH: "amd64"
$  operator-sdk-v1.4.2 version
operator-sdk version: "v1.4.2", commit: "4b083393be65589358b3e0416573df04f4ae8d9b", kubernetes version: "1.19.4", go version: "go1.15.5", GOOS: "linux", GOARCH: "amd64"
$ operator-sdk-v1.9.2 version
operator-sdk version: "v1.9.2", commit: "319e77f2fff57c6c862bddd64ecf9c562f2d2161", kubernetes version: "1.20.2", go version: "go1.16.6", GOOS: "linux", GOARCH: "amd64"

I want to install and test the operator I developed from version 1.17 to the most recent version of Kubernetes.

Please note that this inquiry is the result of processing with Google Translate.

What did you expect to see?

What did you see instead? Under which circumstances?

Environment

Operator type:

Kubernetes cluster type:

$ operator-sdk version

$ go version (if language is Go)

$ kubectl version

Additional context

[camilamacedo86]

Hi @smartkuk,

The versions of k8s deps used in the CLI or scaffolded by it is not exactly what will drive the supportability statement of your Operator. Note that k8s API has some API removals, see: https://kubernetes.io/docs/reference/using-api/deprecation-guide/

Then, for example, if your Operator is using APIs which were removed in the K8s version 1.22 that means that your project will not work on this one and upper versions. So, let's use the CRD API as an example:

If your project is using the API v1 for the CRDs then, your project can work on from k8s versions 1.16. However, you will be unable to make a project mainly integrated with OLM that works in versions < 1.16 and upper or equals 1.22 (Openshft 4.9).

You might be able to support more versions if you try to create logics that will work accordingly with the cluster versions, or better, the APIs available on the cluster. See: Why not couple an Operator's logic to a specific Kubernetes platform?

If your project is integrated with OLM (If you are distributing your project via OperatorHub.io, Openshift, etc):

OLM will try to apply the CRDs to initialized the Operator. Note that the API v1 for CRDs was introduced with the k8s version 1.16 and in its release 1.22, the v1beta1 was removed. Then, technically it is not achievable to build a bundle that supports
both Kubernetes versions e.g 1.22 and 1.15 in this case.

However, if you publish new versions where they have any logic to create resources on runtime according to cluster apis that would be possible and work with OLM assuming that:

• a) All manifests shipped on the bundle does not use the removed APIs (it might only work for specific scenarios where you create the resources via the code and they are resources that are not used to initialize the solutions such as the CRD)
• b) Your distribution must be workable on the previous and upper versions where your solution will be distributed

If you are distributing your solution via Openshift Catalogs such as RedHat Community

We also recommend, for those who would like to still ship bundles that use these removed APIs on 1.22/OCP 4.9:

• a) Configure the OCP version where the bundle should be shipped accordingly by using OCP Labels. It should have a range that does not contain 4.9 and upper versions.
• b) Add the max OCP version annotation = 4.8 to not allow the Operator users to upgrade their cluster from 4.8 to 4.9 with a version installed that does not work on 4.9+.
• c) Create two bundles versions where one can only be shipped to be published on OCP versions < 4.9 and another one that does not use these APIs targeting the OCP upper versions >= 4.9.

For further information see: https://docs.openshift.com/container-platform/4.8/operators/operator_sdk/osdk-working-bundle-images.html#osdk-control-compat_osdk-working-bundle-images

However, it is important to highlight that all OCP/OLM versions which are actually supported would work with the upper versions of these APIs. (See that OCP 4.4 is no longer supported at all)

[camilamacedo86]

Hi @smartkuk,

It shows that further and detailed information was provided above already. So, I am closing this one as sorted out. However, please, feel free to re-open or raise new issues as you see fit.