Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kops should check that maxSize >= minSize in instance groups. #4276

Closed
elisiano opened this issue Jan 15, 2018 · 0 comments
Closed

Kops should check that maxSize >= minSize in instance groups. #4276

elisiano opened this issue Jan 15, 2018 · 0 comments

Comments

@elisiano
Copy link
Contributor

While creating an additional instance group I mistakenly inverted min and max sizes.

Kops did not complain and kept trying to create it although aws was spitting out the error.

------------- BUG REPORT TEMPLATE --------------------

  1. What kops version are you running? The command kops version, will display
    this information.
$ kops version
Version 1.8.0
  1. What Kubernetes version are you running? kubectl version will print the
    version if a cluster is running or provide the Kubernetes version specified as
    a kops flag.
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.1", GitCommit:"3a1c9449a956b6026f075fa3134ff92f7d55f812", GitTreeState:"clean", BuildDate:"2018-01-04T20:00:41Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.5", GitCommit:"cce11c6a185279d037023e02ac5249e14daa22bf", GitTreeState:"clean", BuildDate:"2017-12-07T16:05:18Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
  1. What cloud provider are you using?
    AWS

  2. What commands did you run? What is the simplest way to reproduce this issue?
    Create an instance group with minSize higher than maxSize.

kops create ig testig

edit the yaml so that maxSize is smaller than minSize. (ie: 2 and 5 respectively).
then just deploy that with kops update cluster --name $NAME --yes

  1. What happened after the commands executed?
    Kops did not complain and tried to create the ig. Then when AWS correctly pushed back on not creating an autoscaling group with the wrong limits, kops would just keep retrying (admittedly I hit CTRL+C before waiting for any timeout).
W0112 17:34:27.745662   86520 executor.go:109] error running task "AutoscalingGroup/testig.$clustername" (9m58s remaining to succeed): error creating AutoscalingGroup: ValidationError: Max bound, 2, must be greater than or equal to min bound, 5
apiVersion: kops/v1alpha2
	status code: 400, request id: bf7ca677-f7e8-11e7-8428-fb0b1ec19d3e
I0112 17:34:27.745692   86520 executor.go:124] No progress made, sleeping before retrying 1 failed task(s)
I0112 17:34:37.749998   86520 executor.go:91] Tasks: 132 done / 133 total; 1 can run
W0112 17:34:38.189608   86520 executor.go:109] error running task "AutoscalingGroup/testig.$clustername" (9m47s remaining to succeed): error creating AutoscalingGroup: ValidationError: Max bound, 2, must be greater than or equal to min bound, 5
	status code: 400, request id: c5b29d31-f7e8-11e7-b3a5-25efbdb3be2a
I0112 17:34:38.189648   86520 executor.go:124] No progress made, sleeping before retrying 1 failed task(s)
I0112 17:34:48.190326   86520 executor.go:91] Tasks: 132 done / 133 total; 1 can run
W0112 17:34:49.007471   86520 executor.go:109] error running task "AutoscalingGroup/testig.$clustername" (9m37s remaining to succeed): error creating AutoscalingGroup: ValidationError: Max bound, 2, must be greater than or equal to min bound, 5
	status code: 400, request id: cc280ee5-f7e8-11e7-9486-31a2a6721f89
  1. What did you expect to happen?
    Kops warn the user that the sizes are incorrect instead of attempting to create the instance group.

  2. Please provide your cluster manifest. Execute
    kops get --name my.example.com -oyaml to display your cluster manifest.
    You may want to remove your cluster name and other sensitive information.

$ kops get --name $NAME -oyaml | sed -e 's/real_name/\$NAME/g' -e 's/ami-.*/ami-XXXXXXXX/'
apiVersion: kops/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: 2018-01-10T15:16:23Z
  name: $NAME
spec:
  api:
    loadBalancer:
      type: Public
  authorization:
    rbac: {}
  channel: stable
  cloudProvider: aws
  configBase: s3://$NAME-kops-state-store/$NAME
  encryptionConfig: false
  etcdClusters:
  - etcdMembers:
    - encryptedVolume: true
      instanceGroup: master-eu-west-1a
      name: a
    - encryptedVolume: true
      instanceGroup: master-eu-west-1b
      name: b
    - encryptedVolume: true
      instanceGroup: master-eu-west-1c
      name: c
    name: main
    version: 3.1.11
  - etcdMembers:
    - encryptedVolume: true
      instanceGroup: master-eu-west-1a
      name: a
    - encryptedVolume: true
      instanceGroup: master-eu-west-1b
      name: b
    - encryptedVolume: true
      instanceGroup: master-eu-west-1c
      name: c
    name: events
    version: 3.1.11
  iam:
    allowContainerRegistry: true
    legacy: false
  kubernetesApiAccess:
  - 0.0.0.0/0
  kubernetesVersion: 1.8.5
  masterInternalName: api.internal.$NAME
  masterPublicName: api.$NAME
  networkCIDR: 172.21.0.0/16
  networking:
    calico: {}
  nonMasqueradeCIDR: 100.64.0.0/10
  sshAccess:
  - 0.0.0.0/0
  subnets:
  - cidr: 172.21.32.0/19
    name: eu-west-1a
    type: Private
    zone: eu-west-1a
  - cidr: 172.21.64.0/19
    name: eu-west-1b
    type: Private
    zone: eu-west-1b
  - cidr: 172.21.96.0/19
    name: eu-west-1c
    type: Private
    zone: eu-west-1c
  - cidr: 172.21.0.0/22
    name: utility-eu-west-1a
    type: Utility
    zone: eu-west-1a
  - cidr: 172.21.4.0/22
    name: utility-eu-west-1b
    type: Utility
    zone: eu-west-1b
  - cidr: 172.21.8.0/22
    name: utility-eu-west-1c
    type: Utility
    zone: eu-west-1c
  topology:
    bastion:
      bastionPublicName: bastion.$NAME
    dns:
      type: Public
    masters: private
    nodes: private

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2018-01-10T15:16:24Z
  labels:
    kops.k8s.io/cluster: $NAME
  name: bastions
spec:
  image: ami-XXXXXXXX
  machineType: t2.micro
  maxSize: 1
  minSize: 1
  nodeLabels:
    kops.k8s.io/instancegroup: bastions
  role: Bastion
  subnets:
  - utility-eu-west-1a
  - utility-eu-west-1b
  - utility-eu-west-1c

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2018-01-10T15:16:24Z
  labels:
    kops.k8s.io/cluster: $NAME
  name: master-eu-west-1a
spec:
  image: ami-XXXXXXXX
  machineType: t2.medium
  maxSize: 1
  minSize: 1
  nodeLabels:
    kops.k8s.io/instancegroup: master-eu-west-1a
  role: Master
  subnets:
  - eu-west-1a

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2018-01-10T15:16:25Z
  labels:
    kops.k8s.io/cluster: $NAME
  name: master-eu-west-1b
spec:
  image: ami-XXXXXXXX
  machineType: t2.medium
  maxSize: 1
  minSize: 1
  nodeLabels:
    kops.k8s.io/instancegroup: master-eu-west-1b
  role: Master
  subnets:
  - eu-west-1b

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2018-01-10T15:16:25Z
  labels:
    kops.k8s.io/cluster: $NAME
  name: master-eu-west-1c
spec:
  image: ami-XXXXXXXX
  machineType: t2.medium
  maxSize: 1
  minSize: 1
  nodeLabels:
    kops.k8s.io/instancegroup: master-eu-west-1c
  role: Master
  subnets:
  - eu-west-1c

---

apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
  creationTimestamp: 2018-01-10T15:16:26Z
  labels:
    kops.k8s.io/cluster: $NAME
  name: nodes
spec:
  image: ami-XXXXXXXX
  machineType: t2.medium
  maxSize: 5
  minSize: 2
  nodeLabels:
    kops.k8s.io/instancegroup: nodes
  role: Node
  subnets:
  - eu-west-1a
  - eu-west-1b
  - eu-west-1c
  1. Please run the commands with most verbose logging by adding the -v 10 flag.
    Paste the logs into this report, or in a gist and provide the gist link here.
    N/A

  2. Anything else do we need to know?
    Not really

------------- FEATURE REQUEST TEMPLATE --------------------

  1. Describe IN DETAIL the feature/behavior/change you would like to see.
    Kops should do a sanity check on maxSize >= minSize and in case it fails it should output a meaningful error to the user.

  2. Feel free to provide a design supporting your feature request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant