Reduce the Kubernetes version support matrix #3750

CecileRobertMichon · 2018-08-27T20:59:13Z

What this PR does / why we need it: Deprecates Kubernetes versions except for the two latest patches of each minor release while maintaining support for scale and upgrade from these versions. Exception is version 1.6.9 which we need to keep support for because of API v20160930.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

If applicable:

documentation
unit tests
tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

CecileRobertMichon · 2018-08-27T20:59:43Z

pkg/api/common/versions.go

+	"1.7.12":         false,
+	"1.7.13":         false,
+	"1.7.14":         false,
+	"1.7.15":         false,
 	"1.7.16":         true,


Should we keep 1.7.16?

Do we know if any acs-engine users still provision 1.7 clusters? Otherwise, I'd say get rid of it except that 1.7.x is still supported in AKS currently. (I assume we can't remove Kubernetes versions that AKS still expects.)

mboersma · 2018-08-27T23:14:14Z

Should we keep the last two patch versions for each minor? That's the slightly more conservative policy that I've heard mentioned as a goal for AKS, for example.

CecileRobertMichon · 2018-09-05T17:47:17Z

pkg/api/vlabs/validate.go

@@ -862,16 +862,6 @@ func (w *WindowsProfile) Validate(orchestratorType string) error {
 func (k *KubernetesConfig) Validate(k8sVersion string, hasWindows bool) error {
 	// number of minimum retries allowed for kubelet to post node status
 	const minKubeletRetries = 4
-	// k8s versions that have cloudprovider backoff enabled
-	var backoffEnabledVersions = common.AllKubernetesSupportedVersions
-	// at present all supported versions allow for cloudprovider backoff


at present all supported versions allow for cloudprovider backoff --> this validation is a no-op, remove it for now

codecov · 2018-09-05T22:10:43Z

Codecov Report

Merging #3750 into master will increase coverage by 0.01%.
The diff coverage is 45.45%.

@@            Coverage Diff             @@
##           master    #3750      +/-   ##
==========================================
+ Coverage   55.48%   55.49%   +0.01%     
==========================================
  Files         108      108              
  Lines       16140    16138       -2     
==========================================
+ Hits         8955     8956       +1     
- Misses       6420     6421       +1     
+ Partials      765      761       -4

CecileRobertMichon · 2018-09-05T22:24:06Z

Back-compat tests:

Scaling an existing cluster with now deprecated version should succeed: https://jenkins.azure-containers.io/view/acs-engine%20ad-hoc/job/k8s-scale/53/console ✔️
Upgrading an existing cluster with now deprecated version to supported versions should succeed: https://jenkins.azure-containers.io/view/acs-engine%20ad-hoc/job/k8s-upgrade/148/ ✔️
Upgrading an existing cluster with now deprecated version to deprecated versions should fail: https://jenkins.azure-containers.io/view/acs-engine%20ad-hoc/job/k8s-upgrade/149/ ✔️
Deploying a cluster with now deprecated version should fail: https://jenkins.azure-containers.io/view/acs-engine%20ad-hoc/job/k8s-deployment/274/ ✔️

CecileRobertMichon · 2018-09-05T22:38:53Z

Error message when trying to deploy a cluster with a deprecated version:

2018/09/05 22:34:02 Output:FATA[0000] error loading API model in generateCmd: error parsing the api model: the following OrchestratorProfile configuration is not supported: OrchestratorType: "Kubernetes", OrchestratorRelease: "", OrchestratorVersion: "1.10.0". Please use one of the following versions: [1.6.9 1.7.15 1.7.16 1.8.14 1.8.15 1.9.9 1.9.10 1.10.6 1.10.7 1.11.1 1.11.2 1.12.0-alpha.1 1.12.0-beta.0]

CecileRobertMichon · 2018-09-05T22:41:15Z

Error message when trying to upgrade to deprecated version:
FATA[0003] error loading existing cluster: Upgrading to version 1.8.0 is not supported. To see a list of available upgrades, use 'acs-engine orchestrators --orchestrator kubernetes --version 1.7.0'

$ bin/acs-engine orchestrators --orchestrator kubernetes --version 1.7.0
{
  "orchestrators": [
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.7.0",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.7.15"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.7.16"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.8.14"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.8.15"
        }
      ]
    }
  ]
}```

weinong

LGTM. However, it would require some changes in RP as we moved apiloader into RP code base

CecileRobertMichon · 2018-09-06T16:37:54Z

Orchestrators command output:

bin/acs-engine orchestrators --orchestrator kubernetes
{
  "orchestrators": [
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.6.9",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.7.15"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.7.16"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.7.15",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.7.16"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.8.14"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.8.15"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.7.16",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.8.14"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.8.15"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.8.14",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.8.15"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.9.9"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.9.10"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.8.15",
      "default": true,
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.9.9"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.9.10"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.9.9",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.9.10"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.10.6"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.10.7"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.9.10",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.10.6"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.10.7"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.10.6",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.10.7"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.11.1"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.11.2"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.10.7",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.11.1"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.11.2"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.11.1",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.11.2"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.12.0-alpha.1"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.12.0-beta.0"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.11.2",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.12.0-alpha.1"
        },
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.12.0-beta.0"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.12.0-alpha.1",
      "upgrades": [
        {
          "orchestratorType": "",
          "orchestratorVersion": "1.12.0-beta.0"
        }
      ]
    },
    {
      "orchestratorType": "Kubernetes",
      "orchestratorVersion": "1.12.0-beta.0"
    }
  ]
}

jackfrancis · 2018-09-06T22:05:54Z

pkg/api/apiloader.go

@@ -251,14 +251,14 @@ func (a *Apiloader) LoadContainerServiceForAgentPoolOnlyCluster(

 		// use defaultKubernetesVersion arg if no version was supplied in the request contents
 		if managedCluster.Properties.KubernetesVersion == "" && defaultKubernetesVersion != "" {
-			if !common.AllKubernetesSupportedVersions[defaultKubernetesVersion] {
+			if !common.IsSupportedKubernetesVersion(defaultKubernetesVersion, isUpdate, false) {


nit: instead of passing in the literal false create a hasWindows var with the value false and pass in hasWindows for readability.

jackfrancis · 2018-09-06T22:06:03Z

pkg/api/apiloader.go

 				return nil, IsSSHAutoGenerated, a.Translator.Errorf("The selected orchestrator version '%s' is not supported", defaultKubernetesVersion)
 			}
 			managedCluster.Properties.KubernetesVersion = defaultKubernetesVersion
 		}

 		// verify orchestrator version
-		if len(managedCluster.Properties.KubernetesVersion) > 0 && !common.AllKubernetesSupportedVersions[managedCluster.Properties.KubernetesVersion] {
+		if len(managedCluster.Properties.KubernetesVersion) > 0 && !common.IsSupportedKubernetesVersion(managedCluster.Properties.KubernetesVersion, isUpdate, false) {


jackfrancis · 2018-09-06T22:06:09Z

pkg/api/apiloader.go

@@ -289,7 +289,7 @@ func (a *Apiloader) LoadContainerServiceForAgentPoolOnlyCluster(

 		// use defaultKubernetesVersion arg if no version was supplied in the request contents
 		if managedCluster.Properties.KubernetesVersion == "" && defaultKubernetesVersion != "" {
-			if !common.AllKubernetesSupportedVersions[defaultKubernetesVersion] {
+			if !common.IsSupportedKubernetesVersion(defaultKubernetesVersion, isUpdate, false) {


jackfrancis · 2018-09-06T22:06:23Z

pkg/api/apiloader.go

@@ -300,7 +300,7 @@ func (a *Apiloader) LoadContainerServiceForAgentPoolOnlyCluster(
 		}

 		// verify orchestrator version
-		if len(managedCluster.Properties.KubernetesVersion) > 0 && !common.AllKubernetesSupportedVersions[managedCluster.Properties.KubernetesVersion] {
+		if len(managedCluster.Properties.KubernetesVersion) > 0 && !common.IsSupportedKubernetesVersion(managedCluster.Properties.KubernetesVersion, isUpdate, false) {


jackfrancis · 2018-09-06T22:11:18Z

/lgtm pending consideration of nits

acs-bot · 2018-09-06T22:11:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon, jackfrancis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [CecileRobertMichon,jackfrancis]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

only keep latest patches

02ba08c

ghost assigned CecileRobertMichon Aug 27, 2018

ghost added the in progress label Aug 27, 2018

acs-bot added the approved label Aug 27, 2018

jackfrancis removed the approved label Aug 27, 2018

CecileRobertMichon added do-not-merge/hold and removed in progress labels Aug 27, 2018

acs-bot added the size/L label Aug 27, 2018

CecileRobertMichon commented Aug 27, 2018

View reviewed changes

fix testdata apimodels

d7a4e60

ghost added the in progress label Aug 27, 2018

acs-bot added the approved label Aug 27, 2018

Cecile Robert-Michon added 2 commits August 27, 2018 14:50

fix cloud provider backoff versions

e06bc53

remove superfluopus cloud provider check

e07421b

Cecile Robert-Michon added 4 commits August 27, 2018 18:13

support 2 patches per minor version

fed59e6

add isSupportedKubernetesVersion function

7d2deab

replace version checks

9264a58

re-add 1.6.9 support

f99195f

CecileRobertMichon commented Sep 5, 2018

View reviewed changes

fix unit tests

229ebad

weinong approved these changes Sep 5, 2018

View reviewed changes

CecileRobertMichon removed the do-not-merge/hold label Sep 6, 2018

jackfrancis reviewed Sep 6, 2018

View reviewed changes

use hasWindows var

25bf25b

jackfrancis merged commit 849e5fd into Azure:master Sep 6, 2018

ghost removed the in progress label Sep 6, 2018

mboersma mentioned this pull request Oct 4, 2018

Remove Kubernetes 1.7.x from supported versions #3952

Closed

3 tasks

CecileRobertMichon deleted the deprecate-k8s-versions branch November 6, 2018 19:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce the Kubernetes version support matrix #3750

Reduce the Kubernetes version support matrix #3750

CecileRobertMichon commented Aug 27, 2018 •

edited

Loading

CecileRobertMichon Aug 27, 2018

mboersma Aug 27, 2018

mboersma commented Aug 27, 2018

CecileRobertMichon Sep 5, 2018

jackfrancis Sep 6, 2018

codecov bot commented Sep 5, 2018

CecileRobertMichon commented Sep 5, 2018 •

edited

Loading

CecileRobertMichon commented Sep 5, 2018 •

edited

Loading

CecileRobertMichon commented Sep 5, 2018 •

edited

Loading

weinong left a comment

CecileRobertMichon commented Sep 6, 2018 •

edited

Loading

jackfrancis Sep 6, 2018

jackfrancis Sep 6, 2018

jackfrancis Sep 6, 2018

jackfrancis Sep 6, 2018

jackfrancis commented Sep 6, 2018

acs-bot commented Sep 6, 2018

Reduce the Kubernetes version support matrix #3750

Reduce the Kubernetes version support matrix #3750

Conversation

CecileRobertMichon commented Aug 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mboersma commented Aug 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Sep 5, 2018

Codecov Report

CecileRobertMichon commented Sep 5, 2018 • edited Loading

CecileRobertMichon commented Sep 5, 2018 • edited Loading

CecileRobertMichon commented Sep 5, 2018 • edited Loading

weinong left a comment

Choose a reason for hiding this comment

CecileRobertMichon commented Sep 6, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jackfrancis commented Sep 6, 2018

acs-bot commented Sep 6, 2018

CecileRobertMichon commented Aug 27, 2018 •

edited

Loading

CecileRobertMichon commented Sep 5, 2018 •

edited

Loading

CecileRobertMichon commented Sep 5, 2018 •

edited

Loading

CecileRobertMichon commented Sep 5, 2018 •

edited

Loading

CecileRobertMichon commented Sep 6, 2018 •

edited

Loading