Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[aws-ec2] Add way to filter VPC subnets by tags #10366

Open
2 tasks
jpSimkins opened this issue Sep 15, 2020 · 11 comments
Open
2 tasks

[aws-ec2] Add way to filter VPC subnets by tags #10366

jpSimkins opened this issue Sep 15, 2020 · 11 comments
Labels
@aws-cdk/aws-ec2 Related to Amazon Elastic Compute Cloud effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1

Comments

@jpSimkins
Copy link

jpSimkins commented Sep 15, 2020

A way to filter VPC subnets by tags. This is important to allow proper setup of systems with the CDK that have not been built with the CDK. No all systems are built with the best practices as these tend the change as the years go on anyways. Moving from Terraform, the VPC's are not setup according to AWS standards (compared to the CDK) and this has some serious consequences as to "fix" the issue would require rebuilding system. That's not ideal or really necessary if we had some control over which subnets to use VIA a filter.

Use Case

I am running into issues with codepipeline due to VPC subnets not being able to be filtered by a tag.
I am coming from Terraform, having built most of our VPCs with: https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/2.25.0

This creates the subnets like:

olyott-sand-db-us-east-1a
olyott-sand-db-us-east-1b
olyott-sand-db-us-east-1c
olyott-sand-elasticache-us-east-1a
olyott-sand-elasticache-us-east-1b
olyott-sand-elasticache-us-east-1c
olyott-sand-private-us-east-1a
olyott-sand-private-us-east-1b
olyott-sand-private-us-east-1c
olyott-sand-public-us-east-1a
olyott-sand-public-us-east-1b
olyott-sand-public-us-east-1c

The issues I am having is that the db and elasticache subnets seem to be set to private. This leaves me with issues in the pipeline as this will build and deploy ECS to the DB and the cache subnets, which will always fail. These subnets behave as isolated but they are not identified as isolated.

It's unrealistic to expect replacing all existing PROD VPCs with the new (proper) VPC using CDK. Especially since I cannot truly import like we can in Terraform. This leaves me in a state where I cannot use the CDK for codepipelines on any existing system. As the builds and deploys will work, at best. half the time.

What we need is a way to filter the subnets by the tag name, or any other tag for more control.

Proposed Solution

I am new to the CDK but perhaps a filterBy(tag, regexPattern|string) to the VPC for subnets.
What I would like to do is use it like:

const _privateSubnets = vpc.privateSubnets.filterByTag('Name', /*(private)*/);

I tried to see if I could build this myself but I don't see the metric possible to filter by. Perhaps a CloudFormation limitation?

  • 👋 I may be able to implement this feature request
  • ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

@jpSimkins jpSimkins added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Sep 15, 2020
@github-actions github-actions bot added the @aws-cdk/aws-ec2 Related to Amazon Elastic Compute Cloud label Sep 15, 2020
@rix0rrr
Copy link
Contributor

rix0rrr commented Sep 16, 2020

If your issue is that types are incorrectly recognized, you can use the aws-cdk:subnet-type tag to correct those. See here:

https://docs.aws.amazon.com/cdk/api/latest/docs/aws-ec2-readme.html#importing-an-existing-vpc

A combination of aws-cdk:subnet-type and aws-cdk:subnet-name should allow you enough flexibility to achieve most VPC selections you need. If that's not sufficient for you, #10112 will help once it lands.

@rix0rrr rix0rrr closed this as completed Sep 16, 2020
@moelholm
Copy link

If your issue is that types are incorrectly recognized, you can use the aws-cdk:subnet-type tag to correct those. See here:

https://docs.aws.amazon.com/cdk/api/latest/docs/aws-ec2-readme.html#importing-an-existing-vpc

A combination of aws-cdk:subnet-type and aws-cdk:subnet-name should allow you enough flexibility to achieve most VPC selections you need. If that's not sufficient for you, #10112 will help once it lands.

That solution primarily works for subnets managed by CDK. I have a situation where the VPC was created by others. And now I want to filter by the name Tag. I believe this issue is still relevant. I can work a fix?

@buttnomaan9
Copy link

Any update on this?

@alexjfisher
Copy link

@rix0rrr Would you consider reopening this? #10112 didn't help as the tags aren't available to filter by.

@okonon
Copy link

okonon commented Oct 19, 2021

I am also interested in this. I have vpc created using terraform with 2 private subnets and 2 db subnets. I am looking for a way to filter out db subnets out of all matched subnets with type SubnetType.PRIVATE_WITH_NAT

@njlynch
Copy link
Contributor

njlynch commented Dec 21, 2021

The SubnetFilters only operate on the ISubnet interface, for which tags are not currently available (reasonably). This is still not supported, so let's open it back up to track.

@njlynch njlynch reopened this Dec 21, 2021
@njlynch njlynch added effort/medium Medium work item – several days of effort p2 and removed needs-triage This issue or PR still needs to be triaged. labels Dec 21, 2021
@rumesh-athu
Copy link

Here is the fix which works for me.

Firstly, Add a new tag to each subnet group as below.

Tag

<KEY | VALUE>

For Private subnets

<subnet-groupname | private>

For DB subnets

<subnet-groupname | db>

Please follow the same for other subnets groups as well

Secondly, Use ec2.Vpc.fromLookup to retrieve VPC object as below

use above new subnet tag KEY in subnetGroupNameTag

import { Vpc } from "aws-cdk-lib/aws-ec2";

const vpc = Vpc.fromLookup(this, "vpc", {
  vpcId: "vpc-1234567890",
  subnetGroupNameTag: "subnet-groupname",
});

Thirdly, Add subnetGroupName to vpc.selectSubnet

use above new subnet tag VALUE in subnetGroupName

vpc.selectSubnets({
  subnetGroupName: "private",
  availabilityZones: [az],
  onePerAz: true,
}),

Referred documents
VpcLookupOptions
SubnetSelection

Note:
Please run cdk context --clear to clear the cdk.context.json prior to verify this code

@rix0rrr rix0rrr removed their assignment Feb 9, 2022
@HCharlie
Copy link

HCharlie commented Mar 10, 2023

is there any progress on this issue? It would be super helpful to have a feature to select subnet resources based on tags.

I am having a similar issue with this, currently, I am trying to create a Sagemaker domain that requires a list of subnet_ids, and I am trying to derive the subnets based on the tag value, but I failed to find a solution till now.
Is there some workaround for this?

@yo-ga
Copy link
Contributor

yo-ga commented Apr 7, 2023

@HCharlie Our workaround solution is using AWS SDK. We filter the subnets by the specific tags and export those IDs in SDK. After that, we use them with ec2.SubnetFilter.byIds in CDK.

@github-actions github-actions bot added p1 and removed p2 labels May 28, 2023
@github-actions
Copy link

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.

@joewoz
Copy link

joewoz commented Oct 9, 2024

Perhaps the simplest solution here is just supporting wildcards / regex in the subnetGroupName property of vpc.selectSubnets(). If you then set the subnetGroupNameTag to "Name" in your VPC lookup, it would then make a group for each subnet. Implementation might be easier said that done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-ec2 Related to Amazon Elastic Compute Cloud effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1
Projects
None yet
Development

No branches or pull requests