From d4867573fc1781ff8ac4aab85722af1c9c6fccd7 Mon Sep 17 00:00:00 2001 From: clux <sszynrae@gmail.com> Date: Tue, 31 May 2022 20:06:42 +0100 Subject: [PATCH 01/10] Add information on supported k8s versions and how to pick them factored out of #923 Signed-off-by: clux <sszynrae@gmail.com> --- docs/kubernetes-version.md | 71 ++++++++++++++++++++++++++++++++++++++ includes/abbreviations.md | 2 ++ mkdocs.yml | 1 + 3 files changed, 74 insertions(+) create mode 100644 docs/kubernetes-version.md diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md new file mode 100644 index 0000000..4a5bda0 --- /dev/null +++ b/docs/kubernetes-version.md @@ -0,0 +1,71 @@ +## Compatibility + +Our Kubernetes version compatibility is similar to the strategy employed by [client-go](https://github.com/kubernetes/client-go#compatibility-matrix) and can interoperate well under a wide range of target Kubernetes versions defined by a **soft minimum** (MINK8SV) and a **soft maximum** (MAXK8SV): + +| kube version | MINK8SV | MAXK8SV | Generated Source | +| -------------- | --------- | -------- | ----------------- | +| `0.48.0` | `1.16` | `1.20` | [k8s-openapi@0.11.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0110-2021-01-23) | +| `0.57.0` | `1.17` | `1.21` | [k8s-openapi@0.12.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0120-2021-06-15) | +| `0.66.0` | `1.18` | `1.22` | [k8s-openapi@0.13.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0131-2021-10-08) | +| `0.67.0` | `1.19` | `1.23` | [k8s-openapi@0.14.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0140-2022-01-23) | +| `0.73.0` | `1.20` | `1.24` | [k8s-openapi@0.15.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0150-2022-05-22) | + + + The **maximum** indicates how much of the latest api surface we support, and can be in general be exceeded. The **minimum** indicates the lower bound of our testing range (4 versions from the maximu - matching [Kubernetes support range](https://endoflife.date/kubernetes). + + The MAXK8SV is shown as a readme badge: + + > [![Kubernetes 1.24](https://img.shields.io/badge/K8s-1.24-326ce5.svg)](https://kube.rs/kubernetes-version) +## Picking Versions + +Given a `kube` versions, you must pick a **target Kubernetes version** from the available ones in the generated source that is used by that kube version. + +E.g. if using [`kube@0.73.0`](https://docs.rs/kube/0.73.0/kube/), we see its generated source is [`k8s-openapi@0.15.0`](https://docs.rs/k8s-openapi/0.15.0/k8s_openapi/), which exports the [following version features](https://docs.rs/crate/k8s-openapi/0.15.0/features). + +You can find the MAXK8SV from this feature list and pick this as your target. In this case the maximally supported version feature is `v1_24`. + +By default; you **SHOULD** pick the MAXK8SV as your target version even when running against older clusters. The **exception** is if you are programming explicitly against apis that have been removed in newer versions. + +With [k8s-pb], we plan on [doing this automatically](https://github.com/kube-rs/k8s-pb/issues/10). + +See below for details on a skew between your cluster and your target version. + +## Version Skew + +How kube version skew interacts with clusters is largely determined by how [Kubernetes deprecates api versions upstream](https://kubernetes.io/docs/reference/using-api/deprecation-policy/). + +Consider the following outcomes when picking **target versions** based on your **cluster version**: + +1. if `target version == cluster version` (cluster in sync with kube), then: + * kube has api parity with cluster + * rust structs are all queryable via kube +2. if `target version > cluster version` (cluster behind kube), then: + * kube has more recent api features than the cluster supports + * recent rust api structs might not work with the cluster version yet + * [deprecated](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)/alpha apis might have been removed from rust structs ⚡ +3. if `target version < cluster version` (cluster ahead of kube), then: + * kube has less recent api features than the cluster supports + * recent kubernetes resources might not have rust struct counterparts + * [deprecated](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)/alpha apis might have been removed from the cluster ⚡ + +Kubernetes takes a long time to remove deprecated apis (unless they alpha or beta apis), so the **acceptable distance** from your **cluster** version actually **depends** on what **apis you target**. + +In particular, when using your own **custom** or **stable official** api resources - where exceeding the range will have **little impact**. + +**If** you are **targeting deprecated/alpha** apis on the other hand, then you should **pick** a target version **in sync with your cluster**. Note that alpha apis may vanish or change significantly in a single release, and is not covered by any guarantees. + +As a result; relying on alpha apis will make the amount of **upgrades required** to an application **more frequent**. To alleviate this; consider using api [discovery] to **match on available api versions** rather than writing code against each Kubernetes version. + +## Exceeding The Range + +We recommend developers stay within the supported version range for the best experience, but it is **technically possible** to operate outside the bounds of this range (by picking older `k8s-openapi` features, or by running against older clusters). + +!!! warning "Untested Version Combinations" + + While exceeding the supported version range **can** work: **we do not test** kube's functionality **outside this version range**. + +In minor skews, both kube and Kubernetes will share a large functioning API surface, while relying on deprecated apis to fill the gap. However, the **further you stray** from the range you are **increasingly likely** to encounter rust structs that doesn't work against your cluster, or miss support for resources entirely. + + +--8<-- "includes/abbreviations.md" +--8<-- "includes/links.md" diff --git a/includes/abbreviations.md b/includes/abbreviations.md index e3b7e53..8436ed4 100644 --- a/includes/abbreviations.md +++ b/includes/abbreviations.md @@ -2,3 +2,5 @@ *[WIP]: Work In Progress *[gRPC]: Google Remote Procedure Call *[MSRV]: Minimum Supported Rust Version +*[MINK8SV]: Minimum Supported Kubernetes Version +*[MAXK8SV]: Maximum Supported Kubernetes Version diff --git a/mkdocs.yml b/mkdocs.yml index 9a8ed42..6ad654b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -53,6 +53,7 @@ nav: - security.md - release-process.md - rust-version.md + - kubernetes-version.md - architecture.md - adopters.md From bbeaa456d2f910d5717a190aebb9006e14565f3e Mon Sep 17 00:00:00 2001 From: clux <sszynrae@gmail.com> Date: Tue, 31 May 2022 22:40:19 +0100 Subject: [PATCH 02/10] set to 5 versions if we are going to put minimum Signed-off-by: clux <sszynrae@gmail.com> --- docs/kubernetes-version.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index 4a5bda0..4530294 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -4,18 +4,21 @@ Our Kubernetes version compatibility is similar to the strategy employed by [cli | kube version | MINK8SV | MAXK8SV | Generated Source | | -------------- | --------- | -------- | ----------------- | -| `0.48.0` | `1.16` | `1.20` | [k8s-openapi@0.11.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0110-2021-01-23) | -| `0.57.0` | `1.17` | `1.21` | [k8s-openapi@0.12.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0120-2021-06-15) | -| `0.66.0` | `1.18` | `1.22` | [k8s-openapi@0.13.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0131-2021-10-08) | -| `0.67.0` | `1.19` | `1.23` | [k8s-openapi@0.14.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0140-2022-01-23) | -| `0.73.0` | `1.20` | `1.24` | [k8s-openapi@0.15.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0150-2022-05-22) | +| `0.48.0` | `1.15` | `1.20` | [k8s-openapi@0.11.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0110-2021-01-23) | +| `0.57.0` | `1.16` | `1.21` | [k8s-openapi@0.12.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0120-2021-06-15) | +| `0.66.0` | `1.17` | `1.22` | [k8s-openapi@0.13.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0131-2021-10-08) | +| `0.67.0` | `1.18` | `1.23` | [k8s-openapi@0.14.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0140-2022-01-23) | +| `0.73.0` | `1.19` | `1.24` | [k8s-openapi@0.15.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0150-2022-05-22) | +The MINK8SV is listed in our README as a badge: - The **maximum** indicates how much of the latest api surface we support, and can be in general be exceeded. The **minimum** indicates the lower bound of our testing range (4 versions from the maximu - matching [Kubernetes support range](https://endoflife.date/kubernetes). +> [![Tested against Kubernetes 1.19 and above](https://img.shields.io/badge/MINK8SV-1.19-326ce5.svg)](https://kube.rs/kubernetes-version) - The MAXK8SV is shown as a readme badge: +The **minimum** indicates the lower bound of our testing range, and the **maximum** indicates how much of the latest api surface we support, and this range covers **5 Kubernetes releases**. + +This policy is intended to match **stable channel support** within **major cloud providers**. +Compare with: [EKS](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html), [AKS](https://docs.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-kubernetes-release-calendar), [GKE](https://cloud.google.com/kubernetes-engine/docs/release-notes-stable), [upstream Kubernetes](https://endoflife.date/google-kubernetes-engine). - > [![Kubernetes 1.24](https://img.shields.io/badge/K8s-1.24-326ce5.svg)](https://kube.rs/kubernetes-version) ## Picking Versions Given a `kube` versions, you must pick a **target Kubernetes version** from the available ones in the generated source that is used by that kube version. @@ -62,10 +65,9 @@ We recommend developers stay within the supported version range for the best exp !!! warning "Untested Version Combinations" - While exceeding the supported version range **can** work: **we do not test** kube's functionality **outside this version range**. + While exceeding the supported version range is likely to work for most api resources: **we do not test** kube's functionality **outside this version range**. In minor skews, both kube and Kubernetes will share a large functioning API surface, while relying on deprecated apis to fill the gap. However, the **further you stray** from the range you are **increasingly likely** to encounter rust structs that doesn't work against your cluster, or miss support for resources entirely. - --8<-- "includes/abbreviations.md" --8<-- "includes/links.md" From e2b1671f7aeab82eecd7f031f82e19c46522c504 Mon Sep 17 00:00:00 2001 From: clux <sszynrae@gmail.com> Date: Tue, 31 May 2022 22:50:15 +0100 Subject: [PATCH 03/10] avoid using MINK8SV and MAXK8SV and just use MK8SV and latest Signed-off-by: clux <sszynrae@gmail.com> --- docs/kubernetes-version.md | 24 ++++++++++++------------ includes/abbreviations.md | 3 +-- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index 4530294..b236f9e 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -1,20 +1,20 @@ ## Compatibility -Our Kubernetes version compatibility is similar to the strategy employed by [client-go](https://github.com/kubernetes/client-go#compatibility-matrix) and can interoperate well under a wide range of target Kubernetes versions defined by a **soft minimum** (MINK8SV) and a **soft maximum** (MAXK8SV): +Our Kubernetes version compatibility is similar to the strategy employed by [client-go](https://github.com/kubernetes/client-go#compatibility-matrix) and can interoperate well under a wide range of target Kubernetes versions defined by a **soft minimum** (MK8SV) and the current **latest** available Kubernetes feature version. -| kube version | MINK8SV | MAXK8SV | Generated Source | -| -------------- | --------- | -------- | ----------------- | -| `0.48.0` | `1.15` | `1.20` | [k8s-openapi@0.11.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0110-2021-01-23) | -| `0.57.0` | `1.16` | `1.21` | [k8s-openapi@0.12.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0120-2021-06-15) | -| `0.66.0` | `1.17` | `1.22` | [k8s-openapi@0.13.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0131-2021-10-08) | -| `0.67.0` | `1.18` | `1.23` | [k8s-openapi@0.14.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0140-2022-01-23) | -| `0.73.0` | `1.19` | `1.24` | [k8s-openapi@0.15.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0150-2022-05-22) | +| kube version | MINK8SV | Latest | Generated Source | +| -------------- | --------- | ------- | ----------------- | +| `0.48.0` | `1.15` | `1.20` | [k8s-openapi@0.11.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0110-2021-01-23) | +| `0.57.0` | `1.16` | `1.21` | [k8s-openapi@0.12.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0120-2021-06-15) | +| `0.66.0` | `1.17` | `1.22` | [k8s-openapi@0.13.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0131-2021-10-08) | +| `0.67.0` | `1.18` | `1.23` | [k8s-openapi@0.14.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0140-2022-01-23) | +| `0.73.0` | `1.19` | `1.24` | [k8s-openapi@0.15.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0150-2022-05-22) | -The MINK8SV is listed in our README as a badge: +The MK8SV is listed in our README as a badge: > [![Tested against Kubernetes 1.19 and above](https://img.shields.io/badge/MINK8SV-1.19-326ce5.svg)](https://kube.rs/kubernetes-version) -The **minimum** indicates the lower bound of our testing range, and the **maximum** indicates how much of the latest api surface we support, and this range covers **5 Kubernetes releases**. +The **minimum** indicates the lower bound of our testing range, and the **latest** indicates how much of the latest api surface we support, and this range covers **5 Kubernetes releases**. This policy is intended to match **stable channel support** within **major cloud providers**. Compare with: [EKS](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html), [AKS](https://docs.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-kubernetes-release-calendar), [GKE](https://cloud.google.com/kubernetes-engine/docs/release-notes-stable), [upstream Kubernetes](https://endoflife.date/google-kubernetes-engine). @@ -25,9 +25,9 @@ Given a `kube` versions, you must pick a **target Kubernetes version** from the E.g. if using [`kube@0.73.0`](https://docs.rs/kube/0.73.0/kube/), we see its generated source is [`k8s-openapi@0.15.0`](https://docs.rs/k8s-openapi/0.15.0/k8s_openapi/), which exports the [following version features](https://docs.rs/crate/k8s-openapi/0.15.0/features). -You can find the MAXK8SV from this feature list and pick this as your target. In this case the maximally supported version feature is `v1_24`. +You can find the latest supported from this feature list and pick this as your target. In this case the latest supported version feature is `v1_24`. -By default; you **SHOULD** pick the MAXK8SV as your target version even when running against older clusters. The **exception** is if you are programming explicitly against apis that have been removed in newer versions. +By default; you **SHOULD** pick the latest as your target version even when running against older clusters. The **exception** is if you are programming explicitly against apis that have been removed in newer versions. With [k8s-pb], we plan on [doing this automatically](https://github.com/kube-rs/k8s-pb/issues/10). diff --git a/includes/abbreviations.md b/includes/abbreviations.md index 8436ed4..cca1531 100644 --- a/includes/abbreviations.md +++ b/includes/abbreviations.md @@ -2,5 +2,4 @@ *[WIP]: Work In Progress *[gRPC]: Google Remote Procedure Call *[MSRV]: Minimum Supported Rust Version -*[MINK8SV]: Minimum Supported Kubernetes Version -*[MAXK8SV]: Maximum Supported Kubernetes Version +*[MK8SV]: Minimum Supported Kubernetes Version From 17273a14c8d3ffd536d092b394a464eb941cfd00 Mon Sep 17 00:00:00 2001 From: clux <sszynrae@gmail.com> Date: Tue, 31 May 2022 23:00:17 +0100 Subject: [PATCH 04/10] clean up last references to MINK8SV and make a clear little policy blob Signed-off-by: clux <sszynrae@gmail.com> --- docs/kubernetes-version.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index b236f9e..180ad32 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -2,19 +2,23 @@ Our Kubernetes version compatibility is similar to the strategy employed by [client-go](https://github.com/kubernetes/client-go#compatibility-matrix) and can interoperate well under a wide range of target Kubernetes versions defined by a **soft minimum** (MK8SV) and the current **latest** available Kubernetes feature version. -| kube version | MINK8SV | Latest | Generated Source | -| -------------- | --------- | ------- | ----------------- | -| `0.48.0` | `1.15` | `1.20` | [k8s-openapi@0.11.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0110-2021-01-23) | -| `0.57.0` | `1.16` | `1.21` | [k8s-openapi@0.12.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0120-2021-06-15) | -| `0.66.0` | `1.17` | `1.22` | [k8s-openapi@0.13.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0131-2021-10-08) | -| `0.67.0` | `1.18` | `1.23` | [k8s-openapi@0.14.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0140-2022-01-23) | -| `0.73.0` | `1.19` | `1.24` | [k8s-openapi@0.15.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0150-2022-05-22) | +| kube version | MK8SV | Latest | Generated Source | +| -------------- | ------- | ------- | ----------------- | +| `0.48.0` | `1.15` | [`1.20`](https://kubernetes.io/blog/2020/12/08/kubernetes-1-20-release-announcement/) | [k8s-openapi@0.11.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0110-2021-01-23) | +| `0.57.0` | `1.16` | [`1.21`](https://kubernetes.io/blog/2021/04/08/kubernetes-1-21-release-announcement/) | [k8s-openapi@0.12.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0120-2021-06-15) | +| `0.66.0` | `1.17` | [`1.22`](https://kubernetes.io/blog/2021/08/04/kubernetes-1-22-release-announcement/) | [k8s-openapi@0.13.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0131-2021-10-08) | +| `0.67.0` | `1.18` | [`1.23`](https://kubernetes.io/blog/2021/12/07/kubernetes-1-23-release-announcement/) | [k8s-openapi@0.14.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0140-2022-01-23) | +| `0.73.0` | `1.19` | [`1.24`](https://kubernetes.io/blog/2022/05/03/kubernetes-1-24-release-announcement/) | [k8s-openapi@0.15.0](https://github.com/Arnavion/k8s-openapi/blob/master/CHANGELOG.md#v0150-2022-05-22) | The MK8SV is listed in our README as a badge: -> [![Tested against Kubernetes 1.19 and above](https://img.shields.io/badge/MINK8SV-1.19-326ce5.svg)](https://kube.rs/kubernetes-version) +> [![Tested against Kubernetes 1.19 and above](https://img.shields.io/badge/MK8SV-1.19-326ce5.svg)](https://kube.rs/kubernetes-version) -The **minimum** indicates the lower bound of our testing range, and the **latest** indicates how much of the latest api surface we support, and this range covers **5 Kubernetes releases**. +The **minimum** indicates the lower bound of our testing range, and the **latest** is the Kubernetes version selectable as a target version, indicating how much of the latest api surface we support. + +!!! note "Minimum Kubernetes Version Policy" + + The Minimum Supported Kubernetes Version (MK8SV) is set as **5 releases below** the **latest** Kubernetes version. This policy is intended to match **stable channel support** within **major cloud providers**. Compare with: [EKS](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html), [AKS](https://docs.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-kubernetes-release-calendar), [GKE](https://cloud.google.com/kubernetes-engine/docs/release-notes-stable), [upstream Kubernetes](https://endoflife.date/google-kubernetes-engine). From f7fac37ee07e0e3ddefbd42d652707863c8b1ffb Mon Sep 17 00:00:00 2001 From: clux <sszynrae@gmail.com> Date: Sat, 18 Jun 2022 13:02:17 +0100 Subject: [PATCH 05/10] add note about our special abstractions Signed-off-by: clux <sszynrae@gmail.com> --- .vscode/settings.json | 6 +++++- docs/kubernetes-version.md | 20 +++++++++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/.vscode/settings.json b/.vscode/settings.json index bec166c..df916fe 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -5,7 +5,11 @@ "editor.lineHeight": 24, "workbench.colorTheme": "Seti Modified", "[markdown]": { - "editor.quickSuggestions": true + "editor.quickSuggestions": { + "comments": "on", + "strings": "on", + "other": "on" + } }, "foam.openDailyNote.directory": "./docs/proposals", "foam.openDailyNote.fileExtension": "md", diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index 180ad32..9da0483 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -63,7 +63,7 @@ In particular, when using your own **custom** or **stable official** api resourc As a result; relying on alpha apis will make the amount of **upgrades required** to an application **more frequent**. To alleviate this; consider using api [discovery] to **match on available api versions** rather than writing code against each Kubernetes version. -## Exceeding The Range +## Outside The Range We recommend developers stay within the supported version range for the best experience, but it is **technically possible** to operate outside the bounds of this range (by picking older `k8s-openapi` features, or by running against older clusters). @@ -73,5 +73,23 @@ We recommend developers stay within the supported version range for the best exp In minor skews, both kube and Kubernetes will share a large functioning API surface, while relying on deprecated apis to fill the gap. However, the **further you stray** from the range you are **increasingly likely** to encounter rust structs that doesn't work against your cluster, or miss support for resources entirely. +## Special Abstractions + +In a small number of cases, kube provides abstractions on top of certain api resources that are not managed along with the generated sources. For these cases we currently __track the source__ and remove when Kubernetes removes them. + +This only affects a small number of special resources such as `CustomResourceDefinition`, `Event`, `Lease`, `AdmissionReview`. + +### Example + +The `CustomResourceDefinition` resource at `v1beta1` was [removed](https://kubernetes.io/docs/reference/using-api/deprecation-guide/) in Kubernetes `1.22`: + +> The apiextensions.k8s.io/v1beta1 API version of CustomResourceDefinition is no longer served as of v1.22. + +Their replacement; in `v1` was released in Kubernetes `1.16`. + +Kube had special support for both versions of `CustomResourceDefinition` from `0.26.0` up until [`0.72.0`](https://github.com/kube-rs/kube-rs/releases/tag/0.72.0) when kube supported structs from Kubernetes >= 1.22. + +This special support took the form of the proc macro [CustomResource] and [associated helpers](https://docs.rs/kube/latest/kube/core/crd/index.html) that allowing pinning the crd version to `v1beta1` up until its removal. It is now `v1` only. + --8<-- "includes/abbreviations.md" --8<-- "includes/links.md" From 22533f06f8db3664452bd2ec89ae704d550c721e Mon Sep 17 00:00:00 2001 From: Eirik A <sszynrae@gmail.com> Date: Sun, 19 Jun 2022 06:54:06 +0100 Subject: [PATCH 06/10] Update docs/kubernetes-version.md Co-authored-by: kazk <kazk.dev@gmail.com> Signed-off-by: Eirik A <sszynrae@gmail.com> --- docs/kubernetes-version.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index 9da0483..b0624fa 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -45,7 +45,7 @@ Consider the following outcomes when picking **target versions** based on your * 1. if `target version == cluster version` (cluster in sync with kube), then: * kube has api parity with cluster - * rust structs are all queryable via kube + * Rust structs are all queryable via kube 2. if `target version > cluster version` (cluster behind kube), then: * kube has more recent api features than the cluster supports * recent rust api structs might not work with the cluster version yet From 2dbb0464577c147341813d589c47101657eb3667 Mon Sep 17 00:00:00 2001 From: Eirik A <sszynrae@gmail.com> Date: Sun, 19 Jun 2022 06:54:12 +0100 Subject: [PATCH 07/10] Update docs/kubernetes-version.md Co-authored-by: kazk <kazk.dev@gmail.com> Signed-off-by: Eirik A <sszynrae@gmail.com> --- docs/kubernetes-version.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index b0624fa..5498563 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -48,7 +48,7 @@ Consider the following outcomes when picking **target versions** based on your * * Rust structs are all queryable via kube 2. if `target version > cluster version` (cluster behind kube), then: * kube has more recent api features than the cluster supports - * recent rust api structs might not work with the cluster version yet + * recent Rust api structs might not work with the cluster version yet * [deprecated](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)/alpha apis might have been removed from rust structs ⚡ 3. if `target version < cluster version` (cluster ahead of kube), then: * kube has less recent api features than the cluster supports From 884386545cdfc58b21a6b31be76318eaf35a7854 Mon Sep 17 00:00:00 2001 From: Eirik A <sszynrae@gmail.com> Date: Sun, 19 Jun 2022 06:54:19 +0100 Subject: [PATCH 08/10] Update docs/kubernetes-version.md Co-authored-by: kazk <kazk.dev@gmail.com> Signed-off-by: Eirik A <sszynrae@gmail.com> --- docs/kubernetes-version.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index 5498563..5028841 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -49,7 +49,7 @@ Consider the following outcomes when picking **target versions** based on your * 2. if `target version > cluster version` (cluster behind kube), then: * kube has more recent api features than the cluster supports * recent Rust api structs might not work with the cluster version yet - * [deprecated](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)/alpha apis might have been removed from rust structs ⚡ + * [deprecated](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)/alpha apis might have been removed from Rust structs ⚡ 3. if `target version < cluster version` (cluster ahead of kube), then: * kube has less recent api features than the cluster supports * recent kubernetes resources might not have rust struct counterparts From 698d1f2505d27f8f3cb060d261f8d55a76efc68e Mon Sep 17 00:00:00 2001 From: Eirik A <sszynrae@gmail.com> Date: Sun, 19 Jun 2022 06:54:25 +0100 Subject: [PATCH 09/10] Update docs/kubernetes-version.md Co-authored-by: kazk <kazk.dev@gmail.com> Signed-off-by: Eirik A <sszynrae@gmail.com> --- docs/kubernetes-version.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index 5028841..2a341bd 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -52,7 +52,7 @@ Consider the following outcomes when picking **target versions** based on your * * [deprecated](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)/alpha apis might have been removed from Rust structs ⚡ 3. if `target version < cluster version` (cluster ahead of kube), then: * kube has less recent api features than the cluster supports - * recent kubernetes resources might not have rust struct counterparts + * recent Kubernetes resources might not have Rust struct counterparts * [deprecated](https://kubernetes.io/docs/reference/using-api/deprecation-policy/)/alpha apis might have been removed from the cluster ⚡ Kubernetes takes a long time to remove deprecated apis (unless they alpha or beta apis), so the **acceptable distance** from your **cluster** version actually **depends** on what **apis you target**. From 196ef5f177d6503c233135ebc14ccca4bda76c6f Mon Sep 17 00:00:00 2001 From: Eirik A <sszynrae@gmail.com> Date: Sun, 19 Jun 2022 06:54:32 +0100 Subject: [PATCH 10/10] Update docs/kubernetes-version.md Co-authored-by: kazk <kazk.dev@gmail.com> Signed-off-by: Eirik A <sszynrae@gmail.com> --- docs/kubernetes-version.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kubernetes-version.md b/docs/kubernetes-version.md index 2a341bd..4e2c0a5 100644 --- a/docs/kubernetes-version.md +++ b/docs/kubernetes-version.md @@ -71,7 +71,7 @@ We recommend developers stay within the supported version range for the best exp While exceeding the supported version range is likely to work for most api resources: **we do not test** kube's functionality **outside this version range**. -In minor skews, both kube and Kubernetes will share a large functioning API surface, while relying on deprecated apis to fill the gap. However, the **further you stray** from the range you are **increasingly likely** to encounter rust structs that doesn't work against your cluster, or miss support for resources entirely. +In minor skews, both kube and Kubernetes will share a large functioning API surface, while relying on deprecated apis to fill the gap. However, the **further you stray** from the range you are **increasingly likely** to encounter Rust structs that doesn't work against your cluster, or miss support for resources entirely. ## Special Abstractions