Support ASG to SQS

[foriequal0]

It seems that ASG LifecycleHook can directly send events to SQS: (See NotificationTargetARN https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_LifecycleHook.html)

Is there any special reason to use EventBridge?

[bwagner5]

Good point! Unfortunately, the current implementation does not support the payload body of ASG sending it directly, since it's different from EventBridge. I think it would be an excellent feature to support though.

For reference here is the direct ASG-to-SQS event payload:

{
  "LifecycleHookName": "test-to-sqs",
  "AccountId": "000000000000",
  "RequestId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx",
  "LifecycleTransition": "autoscaling:EC2_INSTANCE_TERMINATING",
  "AutoScalingGroupName": "my-asg,
  "Service": "AWS Auto Scaling",
  "Time": "2021-01-28T20:06:18.130Z",
  "EC2InstanceId": "i-00000000000",
  "LifecycleActionToken": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx"
}

And here is the EventBridge one that is currently supported:

{
  "version": "0",
  "id": "782d5b4c-0f6f-1fd6-9d62-ecf6aed0a470",
  "detail-type": "EC2 Instance-terminate Lifecycle Action",
  "source": "aws.autoscaling",
  "account": "0000000000",
  "time": "2020-07-01T22:19:58Z",
  "region": "us-east-1",
  "resources": [
    "arn:aws:autoscaling:us-east-1:0000000000:autoScalingGroup:26e7234b-03a4-47fb-b0a9-2b241662774e:autoScalingGroupName/testt1.demo-0a20f32c.kops.sh"
  ],
  "detail": {
    "LifecycleActionToken": "0befcbdb-6ecd-498a-9ff7-ae9b54447cd6",
    "AutoScalingGroupName": "testt1.demo-0a20f32c.kops.sh",
    "LifecycleHookName": "cluster-termination-handler",
    "EC2InstanceId": "i-0633ac2b0d9769723",
    "LifecycleTransition": "autoscaling:EC2_INSTANCE_TERMINATING"
  }
}

[patstrom]

Is it only the data format that differs? Or are there other, larger concerns?

I am looking to deploy this project in the queue-processor mode and it would be nice to do it with as little complexity as possible. If EventBridge isn't strictly necessary I would prefer to not utilize (and pay for) it. Depending on how big of a patch this would require I could give it a go.

[foriequal0]

It seems that EventBridge exists to funnel all events in the account into a single global queue channel.
ASG to SQS only sends two kind of events: autoscaling:EC2_INSTANCE_LAUNCHING, autoscaling:EC2_INSTANCE_TERMINATING. No Spot ITN, no scheduled events, no rebalance recommendation with ASG to SQS alone, and it seems that there is no way to listen EC2 status changes.
To use it alone without EventBridge, we should make a IMDS+Queue hybrid processor. I think it'll be more complicated.

[JnMik]

Hmm.. I guess that must be why I have these errors ignoring event due to error error="Event source () is not supported :

Source:

aws-node-termination-handler/pkg/monitor/sqsevent/sqs-monitor.go

Line 126 in 081fa73

return nil, fmt.Errorf("Event source (%s) is not supported", event.Source)

I have ASG > Terminate hook > SQS and I've passed the SQS Queue URL to the NTH (v1.13.0)

[Sreddy-gs]

as @JnMik mentioned i have same error logs showing up

2021/07/22 14:11:47 ??? Started watching for interruption events
2021/07/22 14:11:47 ??? Kubernetes AWS Node Termination Handler has started successfully!
2021/07/22 14:11:47 ??? Started watching for event cancellations
2021/07/22 14:11:47 ??? Started monitoring for events event_type=SQS_TERMINATE
2021/07/22 14:20:05 WRN ignoring event due to error error="Event source () is not supported"
2021/07/22 14:20:05 ??? There was a problem monitoring for events error="All of the waiting queue events could not be processed" event_type=SQS_TERMINATE

we are testing the NTH first time and i don't see a way to test we followed exact steps shown here https://github.com/aws/aws-node-termination-handler#infrastructure-setup , we did try increasing and decreasing the spot and on demand asg , but whenever a terminate message is received in the SQS we get above mentioned error and until we delete the message the NTH pod keeps restarting, we are having issues to test NTH Queue model

Opened a issue regarding the same #468

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want this issue to never become stale, please ask a maintainer to apply the "stalebot-ignore" label.