Test & Document Kubernetes Version and API Support

[shaneutt]

Problem Statement

We need an authoritative source of truth as to which k8s versions are "supported" by which version of KIC.

Proposed solution

We call a version supported if:

• tests defined in Test KIC against multiple k8s versions #1542 for that version pass
• there are no known API incompatibilities between the respective: KIC version and the k8s version

The version compatibility matrix lives here.

Acceptance Criteria

• e2e testing added to verify versions 1.17-1.21 on production quality cluster deployment (GKE)
• starting with KIC 2.0, each version marked as "supported" is covered by automated integration tests (Test KIC against multiple k8s versions #1542)
• Releases are blocked (by manual process or by CI/automation) if the criteria above are not met.
  • Possible solutions: add an entry to the release checklist, or add CI/automation that blocks a release if the criteria are not met.
• optional: expose the compatibility test results in a visible place to end-users
  • Possible solution: link from changelog/release notes to the CI results page

Issues

• GKE Support kubernetes-testing-framework#32
• GKE Existing Cluster Loader kubernetes-testing-framework#66
• Disable GCE Ingress on GKE Clusters kubernetes-testing-framework#68
• Knative ingress tests don't work on GKE #1617
• Knative ingress tests don't work on GKE #1617
• Legacy Ingress for KIC 2.0 #1614

Pull Requests

• [feat] GKE Cluster Support kubernetes-testing-framework#59
• fix: support clusterIP services for proxy kubernetes-testing-framework#64
• [feat] enable gke clusters from existing kubernetes-testing-framework#67
• fix: increase the default for proxy requests #1610
• [feat] add gke release testing #1618

NOTES

Original spike: #1043

[mflendrich]

Instead of targetting 2.0 specifically, maybe we could do it once and for all by:

• introducing automated testing for the k8s versions promised to be supported Implement regression tests for the entire constellation of supported configs  #1053
• updating our release procedure to always update docs
  ?

I don't see why we'd need to treat the 2.0 release in any special manner.

[shaneutt]

Instead of targetting 2.0 specifically, maybe we could do it once and for all by:

* introducing automated testing for the k8s versions promised to be supported #1053

* updating our release procedure to always update docs
  ?

I don't see why we'd need to treat the 2.0 release in any special manner.

Ok I'm persuaded we don't need to do this specifically in 2.0 scope (though I would like to for documentation's sake). I've moved this out fo the milestone and reworded it for later grooming, and we can add it as a stretch goal for 2.0 later if we like.

[shaneutt]

actually for now I'll just close this and bookmark it for myself and perhaps we can come back to it at a later time if we feel so inclined.

[shaneutt]

Re-open context: we're going to check requirements with @eskibars and then groom this for a later scope.

[mflendrich]

@mflendrich to deduplicate version support issues we have (they exist in several places)

[shaneutt]

While it doesn't resolve this, the following commit does add the version options we need for this in the KTF: Kong/kubernetes-testing-framework@b45df3e

[shaneutt]

With the above changes to KTF we can prove version compatibility using our integration testing suite, I would like to see this be the way in which we move forward, and document results. Additionally I think we should make some hard calls to NOT support Kubernetes older than v1.16

[shaneutt]

The v2.0.0-alpha.3 release and the successful workflow thereof is the validation of the CI portions of this PR:

• https://github.com/Kong/kubernetes-ingress-controller/releases/tag/v2.0.0-alpha.3
• https://hub.docker.com/layers/kong/kubernetes-ingress-controller/2.0.0-alpha.3/images/sha256-94e68bc67ae1003011f7b016b2c511c98c849dfb4241d487afac578ad35acac0?context=explore
• https://github.com/Kong/kubernetes-ingress-controller/actions/runs/1092174311

Now Kong/docs.konghq.com#3081 has been made to be the last PR needed to finally resolve this issue by adding the relevant documentation.

I'm considering the optional part of the AC resolved by virtue of our CI workflows being publicly available for releases.

[shaneutt]

As per a recent @Kong/team-k8s discussion the remaining work (which is all documentation) does not need to block the beta release here. I'm separating the remaining documentation work into its own issue.