Validate cloud labels

[olemarkus]

Since moving to LaunchTemplates, cloudLabels cannot have empty values anymore. If one tries to, kops will just loop endlessly with AWS errors when trying to create the launch template.

/cc @rifelpet

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: olemarkus
To complete the pull request process, please assign rifelpet
You can assign the PR to them by writing /assign @rifelpet in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hakman]

/lgtm

[hakman]

/lgtm

[rifelpet]

This issue: hashicorp/terraform-provider-aws#11164 makes me wonder if it is specifically "tag on create" that doesn't allow empty values, and if ec2.CreateTags with a launch template resourceID would succeed :( I haven't tested it yet but I'll try to today unless anyone else has already.

If thats the case I can open a support ticket to get more info to confirm this is expected behavior and maybe we can add a workaround in Kops (only pass nonempty-value tags to ec2.CreateLaunchTemplate() and let AddAWSTags()'s ec2.CreateTags() add the remaining tags)

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[olemarkus]

/test pull-kops-verify-staticcheck

[rifelpet]

/hold while we do some investigation first.

[rifelpet]

hm so this works for me:

(replace sg-12345 with a valid one)
aws ec2 create-launch-template --launch-template-name TemplateForAutoScaling --version-description AutoScalingVersion1 --launch-template-data '{"NetworkInterfaces":[{"DeviceIndex":0,"AssociatePublicIpAddress":true,"Groups":["sg-12345"],"DeleteOnTermination":true}],"ImageId":"ami-b42209de","InstanceType":"m4.large","TagSpecifications":[{"ResourceType":"instance","Tags":[{"Key":"foo-instance","Value":""}]},{"ResourceType":"volume","Tags":[{"Key":"foo-volume","Value":""}]},{"ResourceType":"launch-template","Tags":[{"Key":"foo-lt","Value":""]],"BlockDeviceMappings":[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":100}}]}' --debug

this creates a launch template with value-less tags assigned to the instance, volume, and launch template itself.

[rifelpet]

Ah, i was able to repro the issue, turns out i was specifying the tag in the wrong place.

aws ec2 create-launch-template --launch-template-name TemplateForAutoScaling3 --version-description AutoScalingVersion1 --launch-template-data '{"NetworkInterfaces":[{"DeviceIndex":0,"AssociatePublicIpAddress":true,"Groups":["sg-12345"],"DeleteOnTermination":true}],"ImageId":"ami-b42209de","InstanceType":"m4.large","BlockDeviceMappings":[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":100}}]}' --region us-east-1 --debug --tag-specifications 'ResourceType=launch-template,Tags=[{Key=lt-tag,Value=""}]'

An error occurred (InvalidParameterValue) when calling the CreateLaunchTemplate operation: Tag keys and tag values cannot be empty. Ensure that each tag has a key and value pair and try again.

I'll open a support case and find out if that's intended.

[hakman]

I didn't notice this before the change to "tag on create.
It is a PITA, as it will affect all people using cluster autoscaler.

[rifelpet]

I contacted AWS support and they're raising an issue with an internal team. No ETA on more info or a fix, so we may want to consider reverting the tag-on-create for now.

[hakman]

I contacted AWS support and they're raising an issue with an internal team. No ETA on more info or a fix, so we may want to consider reverting the tag-on-create for now.

I think that would be a good idea for now.

[k8s-ci-robot]

@olemarkus: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[olemarkus]

I suppose this one should just be closed then?

[k8s-ci-robot]

@olemarkus: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-kops-e2e-k8s-containerd	0ac26e5	link	/test pull-kops-e2e-k8s-containerd

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[hakman]

AWS already deployed the fix in us-east-1 and will continue the rollout to the rest of the regions this week or the next one.