Skip to content

Commit

Permalink
Merge rebuy-de/aws-nuke (#6)
Browse files Browse the repository at this point in the history
* Add support for Athena WorkGroups and NamedQueries (rebuy-de#464)

* Upgrade to aws sdk version 1.28.12 (rebuy-de#466)

* Upgrade to aws sdk version 1.28.12

* added eks fargate profile resource (rebuy-de#469)

* added eks fargate profile resource

* added eks nodegroup as resource (rebuy-de#468)

* added eks nodegroup as resource

* add route53 health check resource (rebuy-de#471)

* Delete all record sets of the type "NS" except for the default "NS" settings (rebuy-de#473)

* fixed rrs behaviour on NS deletion

* Handle pagination 101+ route53 healthchecks (rebuy-de#472)

* Handle 101+ route53

* add cloudformation stackset support (rebuy-de#475)

* feat: Add OwnerId Prop to EC2TGW (rebuy-de#483)

* rebuy-de#430 - cloudformation support for disabling termination protection (rebuy-de#481)

* rebuy-de#430 - cloudformation support for disabling termination protection

* Disable Security Hub (rebuy-de#484)

* Disable Security Hub

See rebuy-de#452

Add the ability to detect regions with Security Hub enabled and disable it if a
"hub" resource is returned.

* Rename property of SecurityHub hub Arn

* Replace Regex in S3Object sample to recursive nuke (rebuy-de#486)

Replace the wildcard to ** to allow recurside sublevel directories in
the nuke process.

Co-authored-by: Bruno Paiuca <bruno.paiuca@wildlifestudios.com>

Co-authored-by: Jan Michalowsky <sejamich@googlemail.com>
Co-authored-by: matthagenbuch <7414485+matthagenbuch@users.noreply.github.com>
Co-authored-by: Brian <Brian-Williams@users.noreply.github.com>
Co-authored-by: Tyler Southwick <tyler.southwick@nike.com>
Co-authored-by: Ryan Whelan <4249368+rwhelan@users.noreply.github.com>
Co-authored-by: Jeff <jscarter3@gmail.com>
Co-authored-by: Greg Bailey <gbailey@lxpro.com>
Co-authored-by: Bruno Paiuca <bn.paiuca@gmail.com>
Co-authored-by: Bruno Paiuca <bruno.paiuca@wildlifestudios.com>
  • Loading branch information
10 people authored Apr 6, 2020
1 parent a2fe279 commit 3378684
Show file tree
Hide file tree
Showing 13 changed files with 716 additions and 24 deletions.
2 changes: 1 addition & 1 deletion config/example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ accounts:
- "s3://my-bucket"
S3Object:
- type: "glob"
value: "s3://my-bucket/*"
value: "s3://my-bucket/**"
Route53HostedZone:
- property: Name
type: "glob"
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/rebuy-de/aws-nuke
go 1.13

require (
github.com/aws/aws-sdk-go v1.25.0
github.com/aws/aws-sdk-go v1.28.12
github.com/fatih/color v1.7.0
github.com/golang/mock v1.3.1
github.com/mattn/go-colorable v0.1.2 // indirect
Expand Down
11 changes: 3 additions & 8 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/aws/aws-sdk-go v1.25.0 h1:MyXUdCesJLBvSSKYcaKeeEwxNUwUpG6/uqVYeH/Zzfo=
github.com/aws/aws-sdk-go v1.25.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.28.12 h1:VeTvzq8Wy89P+1YjtsBpB/SlHhec/U9bpY2ux4hZ1I0=
github.com/aws/aws-sdk-go v1.28.12/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
Expand Down Expand Up @@ -49,7 +49,6 @@ github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnIn
github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
Expand All @@ -58,22 +57,18 @@ github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190926025831-c00fd9afed17 h1:qPnAdmjNA41t3QBTx2mFGf/SD1IoslhYu7AmdsVzCcs=
golang.org/x/net v0.0.0-20190926025831-c00fd9afed17/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894 h1:Cz4ceDQGXuKRnVBDTS23GTn/pU5OE2C0WrNTOYK1Uuc=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a h1:aYOabOQFp6Vj6W1F80affTUvO9UxmJRx8K0gsfABByQ=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262 h1:qsl9y/CJx34tuA7QCPNp86JNJe4spst6Ff8MjvPUdPg=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
5 changes: 3 additions & 2 deletions pkg/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ type Nuke struct {

type FeatureFlags struct {
DisableDeletionProtection struct {
RDSInstance bool `yaml:"RDSInstance"`
EC2Instance bool `yaml:"EC2Instance"`
RDSInstance bool `yaml:"RDSInstance"`
EC2Instance bool `yaml:"EC2Instance"`
CloudformationStack bool `yaml:"CloudformationStack"`
} `yaml:"disable-deletion-protection"`
}

Expand Down
23 changes: 22 additions & 1 deletion resources/cloudformation-stack.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,16 @@ package resources

import (
"errors"
"strings"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/cloudformation"
"github.com/aws/aws-sdk-go/service/cloudformation/cloudformationiface"
"github.com/rebuy-de/aws-nuke/pkg/config"
"github.com/rebuy-de/aws-nuke/pkg/types"
"github.com/sirupsen/logrus"
"strings"
)

const CLOUDFORMATION_MAX_DELETE_ATTEMPT = 3
Expand Down Expand Up @@ -50,6 +53,11 @@ type CloudFormationStack struct {
svc cloudformationiface.CloudFormationAPI
stack *cloudformation.Stack
maxDeleteAttempts int
featureFlags config.FeatureFlags
}

func (cfs *CloudFormationStack) FeatureFlags(ff config.FeatureFlags) {
cfs.featureFlags = ff
}

func (cfs *CloudFormationStack) Remove() error {
Expand All @@ -59,6 +67,19 @@ func (cfs *CloudFormationStack) Remove() error {
func (cfs *CloudFormationStack) removeWithAttempts(attempt int) error {
if err := cfs.doRemove(); err != nil {
logrus.Errorf("CloudFormationStack stackName=%s attempt=%d maxAttempts=%d delete failed: %s", *cfs.stack.StackName, attempt, cfs.maxDeleteAttempts, err.Error())
if cfs.featureFlags.DisableDeletionProtection.CloudformationStack {
awsErr, ok := err.(awserr.Error)
if ok && awsErr.Code() == "ValidationError" &&
awsErr.Message() == "Stack ["+*cfs.stack.StackName+"] cannot be deleted while TerminationProtection is enabled" {
_, err = cfs.svc.UpdateTerminationProtection(&cloudformation.UpdateTerminationProtectionInput{
EnableTerminationProtection: aws.Bool(false),
StackName: cfs.stack.StackName,
})
if err != nil {
return err
}
}
}
if attempt >= cfs.maxDeleteAttempts {
return errors.New("CFS might not be deleted after this run.")
} else {
Expand Down
158 changes: 158 additions & 0 deletions resources/cloudformation-stackset.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
package resources

import (
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/cloudformation"
"github.com/aws/aws-sdk-go/service/cloudformation/cloudformationiface"
"github.com/rebuy-de/aws-nuke/pkg/types"
"github.com/sirupsen/logrus"
"strings"
"time"
)

func init() {
register("CloudFormationStackSet", ListCloudFormationStackSets)
}

func ListCloudFormationStackSets(sess *session.Session) ([]Resource, error) {
svc := cloudformation.New(sess)

params := &cloudformation.ListStackSetsInput{}
resources := make([]Resource, 0)

for {
resp, err := svc.ListStackSets(params)
if err != nil {
return nil, err
}
for _, stackSetSummary := range resp.Summaries {
resources = append(resources, &CloudFormationStackSet{
svc: svc,
stackSetSummary: stackSetSummary,
sleepDuration: 10 * time.Second,
})
}

if resp.NextToken == nil {
break
}

params.NextToken = resp.NextToken
}

return resources, nil
}

type CloudFormationStackSet struct {
svc cloudformationiface.CloudFormationAPI
stackSetSummary *cloudformation.StackSetSummary
sleepDuration time.Duration
}

func (cfs *CloudFormationStackSet) findStackInstances() (map[string][]string, error) {
accounts := make(map[string][]string)

input := &cloudformation.ListStackInstancesInput{
StackSetName: cfs.stackSetSummary.StackSetName,
}

for {
resp, err := cfs.svc.ListStackInstances(input)
if err != nil {
return nil, err
}
for _, stackInstanceSummary := range resp.Summaries {
if regions, ok := accounts[*stackInstanceSummary.Account]; !ok {
accounts[*stackInstanceSummary.Account] = []string{*stackInstanceSummary.Region}
} else {
accounts[*stackInstanceSummary.Account] = append(regions, *stackInstanceSummary.Region)
}
}

if resp.NextToken == nil {
break
}

input.NextToken = resp.NextToken
}

return accounts, nil
}

func (cfs *CloudFormationStackSet) waitForStackSetOperation(operationId string) error {
for {
result, err := cfs.svc.DescribeStackSetOperation(&cloudformation.DescribeStackSetOperationInput{
StackSetName: cfs.stackSetSummary.StackSetName,
OperationId: &operationId,
})
if err != nil {
return err
}
logrus.Infof("Got stackInstance operation status on stackSet=%s operationId=%s status=%s", *cfs.stackSetSummary.StackSetName, operationId, *result.StackSetOperation.Status)
if *result.StackSetOperation.Status == cloudformation.StackSetOperationResultStatusSucceeded {
return nil
} else if *result.StackSetOperation.Status == cloudformation.StackSetOperationResultStatusFailed || *result.StackSetOperation.Status == cloudformation.StackSetOperationResultStatusCancelled {
return fmt.Errorf("unable to delete stackSet=%s operationId=%s status=%s", *cfs.stackSetSummary.StackSetName, operationId, *result.StackSetOperation.Status)
} else {
logrus.Infof("Waiting on stackSet=%s operationId=%s status=%s", *cfs.stackSetSummary.StackSetName, operationId, *result.StackSetOperation.Status)
time.Sleep(cfs.sleepDuration)
}
}
}

func (cfs *CloudFormationStackSet) deleteStackInstances(accountId string, regions []string) error {
logrus.Infof("Deleting stack instance accountId=%s regions=%s", accountId, strings.Join(regions, ","))
regionsInput := make([]*string, len(regions))
for i, region := range regions {
regionsInput[i] = aws.String(region)
fmt.Printf("region=%s i=%d\n", region, i)
}
result, err := cfs.svc.DeleteStackInstances(&cloudformation.DeleteStackInstancesInput{
StackSetName: cfs.stackSetSummary.StackSetName,
Accounts: []*string{&accountId},
Regions: regionsInput,
RetainStacks: aws.Bool(true), //this will remove the stack set instance from the stackset, but will leave the stack in the account/region it was deployed to
})

fmt.Printf("got result=%v err=%v\n", result, err)

if result == nil {
return fmt.Errorf("got null result")
}
if err != nil {
return err
}

return cfs.waitForStackSetOperation(*result.OperationId)
}

func (cfs *CloudFormationStackSet) Remove() error {
accounts, err := cfs.findStackInstances()
if err != nil {
return err
}
for accountId, regions := range accounts {
err := cfs.deleteStackInstances(accountId, regions)
if err != nil {
return err
}
}
_, err = cfs.svc.DeleteStackSet(&cloudformation.DeleteStackSetInput{
StackSetName: cfs.stackSetSummary.StackSetName,
})
return err
}

func (cfs *CloudFormationStackSet) Properties() types.Properties {
properties := types.NewProperties()
properties.Set("Name", cfs.stackSetSummary.StackSetName)
properties.Set("StackSetId", cfs.stackSetSummary.StackSetId)

return properties
}

func (cfs *CloudFormationStackSet) String() string {
return *cfs.stackSetSummary.StackSetName
}
Loading

0 comments on commit 3378684

Please sign in to comment.