Possibility of "adopting" existing AWS resources

[marcindulak]

Cloudformation supports importing existing resources https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import.html

A parallel issue is opened in aws/aws-cdk-rfcs#52

[mhausenblas]

Thanks @marcindulak and can you flesh out your proposal a little, please?

[marcindulak]

It's just a question about ASO that had to be asked at some point.

By "adopting" I mean starting to manage, using ASO, the already existing AWS resources. The term "adopting" is used by pulumi https://www.pulumi.com/blog/adopting-existing-cloud-resources-into-pulumi

An example use case:

• create an s3 bucket by hand
• use the ASO s3 resource to "adopt" the bucket, without changing any of the bucket parameters in AWS. If the state of the ASO s3 resource differs from the state in AWS the application of the ASO resource should fail
• from now on, use the ASO s3 resource to manage the bucket instead of doing this by hand. If the state of the ASO s3 resource differs from the state in AWS the AWS state should be updated

[mhausenblas]

Thanks for the clarification @marcindulak. Adopting of existing AWS resources is out of scope, at least for now. I will keep this issue open as a reminder, though for if and when we revisit this topic again.

[hikhvar]

Adopting existing resources should be a first class feature. I have two use cases for adopting resources:

1. Migrating existing infrastructure to the aws-controller-k8s. Recreation on migration is simple for resources like Route53 records, security group rules, etc. However, it will be more complicated if you have to manage stateful services like a RDS database or a S3 bucket. Imagine an already deployed application. They may have non-trivial amount of data in those services. Without an "adopt resources" feature, I have to migrate the data to migrate my service to aws-controller-k8s. Depending on the nature of the data, this migration is a non trivial task.

2. The second use case is disaster recovery. Imaging you have to rebuild the k8s cluster but the databases/s3 buckets are still present. Since you lost the state of the cluster, you have to combine the already existing AWS resources with the aws-controller-k8s CRDs.

[Vlaaaaaaad]

As discussed on Twitch for Containers Day, posting some thoughts here: adopting/importing resources is also vital for handling failures, and upgrades.

ACK holds its state in etcd, and while etcd is stable( and backed up, and replicated, and so on), it can be broken. Importing resources would help a lot in case of disasters or when restoring backups is too lengthy.

Another use-case is cell-based architectures or blue-green deployments for cluster versions. Treating clusters as cattle will become very difficult if ACK and its resources cannot be easily moved between clusters. Having a very coddled cluster because it manages "that critical S3 bucket" is... not ideal. On the other hand, this could be a feature: ACK only manages short-lived resources and it should not be used for anything else.

Having an import ability would also help migrating to ACK from CF, TF, CDK, or whatever else. At this time ACK can only handle "new" resources.

I can see 2 major import scenarios:

• Automated imports AKA "I want all CDK resources from the X cluster/namespace to be moved to cluster/namespace Y". When I am moving ACK from a cluster to another, I want all resources to be moved. This would mean ACK would transfer its state from 1 cluster to another.
• manual imports AKA "I want to move this S3 bucket from cluster A to cluster B". This would be a "special" case for one-off imports/breakages.

Does the CDK set any labels on resources? I am thinking the ALB Ingress Controller / AWS VPC CNI model of using "ownership" labels could maybe work here too + some "taint" commands that stop the reconciler loop on specific sets of resources.

EDIT: #82 is also considering labels for lifecycle management

[RedbackThomson]

@Vlaaaaaaad @hikhvar @marcindulak I'd love to invite you to all provide feedback on #639. It's an evolving design for an implementation strategy for this issue.

[RedbackThomson]

Runtime tag v0.1.0 contains the new AdoptedResourceReconciler - https://github.com/aws-controllers-k8s/runtime/releases/tag/v0.1.0

[jaypipes]

@RedbackThomson remaining items on this task: user docs and e2e tests?

[RedbackThomson]

There are other elements of the proposal that have not been touched, yet, either - such as the validating webhooks. But yes for this version, e2e tests and docs.

[RedbackThomson]

The e2e test framework has been a pending PR for a while - aws-controllers-k8s/test-infra#64

[ack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale

[a-hilaly]

/remove-lifecycle stale

[vijtrip2]

This is implemented now

[bwhaley]

Is there any documentation for importing/adopting resources?

[jaypipes]

Is there any documentation for importing/adopting resources?

Hi @bwhaley! @RedbackThomson is actually working on documentation of AdoptedResource in this sprint. Expect to see some docs shortly. Thanks for your patience!