Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack Removal documentation #4209

Merged
merged 24 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions docs/Catalog/RemovingStacks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
<!-- markdownlint-disable-next-line first-line-h1 -->
drernie marked this conversation as resolved.
Show resolved Hide resolved
If you ever need to delete a Quilt stack from your AWS account, you should know
that:

- Your data in S3 buckets remains accessible.
- You can continue using the Quilt SDK for reading and writing packages.
- Stack-managed resources (Athena tables, user database, audit logs) will be affected.
- The deletion behavior varies between CloudFormation and Terraform deployments.

> **Consider Pausing Instead:** To temporarily suspend the stack, you can shut
> down ECS services to reduce costs. Note that some charges may continue even
> when ECS is inactive, so the only way to completely eliminate all costs is to
> delete the stack.
drernie marked this conversation as resolved.
Show resolved Hide resolved

---

## I. Impact Assessment and Preparation

drernie marked this conversation as resolved.
Show resolved Hide resolved
### What May Be Deleted

- Custom configurations specific to the Quilt stack (e.g., IAM roles, resource
mappings, permissions).
- Analytics data in stack-managed buckets.
- Audit logs and Athena querying setups.
- User database configurations.

### Recommended Actions

- Export analytics data you need to keep.
- Save important audit logs.
- Document existing Athena configurations if you'll need to recreate them.
- If using Terraform, use it to capture a snapshot of the user database.

---

## II. Stack Deletion Process

drernie marked this conversation as resolved.
Show resolved Hide resolved
### Using CloudFormation

1. **Start the Deletion Process**
- Navigate to the **CloudFormation** console in your AWS region.
- Select your stack under **CloudFormation > Stacks**.
- Click **Delete Stack** to begin the process.

2. **Empty or Ignore Non-Deleted Resources**
- CloudFormation attempts to delete all stack-managed resources.
- If a resource cannot be deleted (e.g., non-empty S3 bucket):
- Other independent resources will still be deleted.
- The stack enters the **DELETE_FAILED** state.
- Failed resources remain intact, including non-empty S3 buckets.
- If you wish to remove those resources:
- Back up any important files.
- Remove all files from the bucket via the S3 console or CLI.
- Retry the deletion process.
- Alternatively, you can leave them undeleted, though that may require manual
cleanup later.

### Using Terraform

1. **Start the Deletion Process**
- Open a terminal in your Terraform configuration directory.
- Run `terraform destroy`.

2. **Empty or Retain Non-Deleted Resources**
- Terraform stops at the first resource it cannot delete (e.g., non-empty S3
buckets).
- Dependencies of the failed resource are preserved.
- To resolve, back up important files and clear the bucket contents.
- Alternatively, configure Terraform to retain the resource by marking it
for retention in the configuration. For example:

```hcl
resource "aws_s3_bucket" "logical_bucket_name" {
bucket = "actual-bucket-name"
lifecycle {
prevent_destroy = true
}
}
```
drernie marked this conversation as resolved.
Show resolved Hide resolved
drernie marked this conversation as resolved.
Show resolved Hide resolved

- Once addressed, resume deletion by re-running `terraform destroy`.

---

## III. Final Steps

- Verify deletion of all non-retained resources.
- Confirm that the stack is no longer listed in CloudFormation or Terraform.
- Check for any orphaned resources or residual costs using the **AWS Cost
Explorer**.

---

## IV. After Stack Deletion

- Your S3 bucket data remains intact and accessible.
- The Quilt SDK continues to function for package operations.
- Recreate any necessary Athena/Glue configurations if needed.
- Rebuild the user database, if needed and using Terraform.
drernie marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
* **Advanced**
* [Athena](advanced-features/athena.md)
* [Elasticsearch](walkthrough/working-with-elasticsearch.md)
* [Removing Stacks](Catalog/RemovingStacks.md)

## Quilt Platform Administrator

Expand Down