Add namevaluesfilters package for resource filtering code

[ewbankkit]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

A number of AWS services, for example EC2 and RDS, support generic attribute filtering in APIs' resource Describe calls. The code to manipulate the data structures associated with resource filtering is nearly identical across services and is very amenable to code generation.

Following on from this comment, it make senses to implement such generators in a new namevaluesfilters package.

One possible interface is:

type NameValuesFilters map[string][]string

func New(m map[string][]string) NameValuesFilters

func (NameValuesFilters) AutoscalingFilters() []*autoscaling.Filter
func (NameValuesFilters) DatabasemigrationserviceFilters() []*databasemigrationservice.Filter
func (NameValuesFilters) DocdbFilters() []*docdb.Filter
func (NameValuesFilters) Ec2Filters() []*ec2.Filter
func (NameValuesFilters) FsxFilters() []*fsx.Filter
func (NameValuesFilters) ImagebuilderFilters() []*imagebuilder.Filter
func (NameValuesFilters) LicensemanagerFilters() []*licensemanager.Filter
func (NameValuesFilters) NeptuneFilters() []*neptune.Filter
func (NameValuesFilters) RdsFilters() []*rds.Filter
func (NameValuesFilters) Route53resolverFilters() []*route53resolver.Filter

Definition of Done

• namevaluesfilters code generation for all relevant services with passing unit tests
• Documentation on use of the generated code
• Instructions in the Contributing Guide on how to add namevaluesfilters support for new services
• Refactor at least one EC2 service data source to use the new pattern. Passing acceptance tests
• Refactor at least one non-EC2 service data to use the new pattern. Passing acceptance tests
• Open linked issue(s) to complete the refactoring work, ideally one per AWS service
• Open linked issue to use namevaluefilters in generated GetTags code: internal/keyvaluetags: Create {SERVICE}GetTag generator, support EC2 list/get, use in aws_ec2_tag implementation #13745, provider: Support arbitrary additional tag data, implement keyvaluetags and ignore_tags support in aws_autoscaling_group resource, return empty strings with key-only tags instead of panicking #13868
• Look at aws/internal/service/ec2/filter.go and open a linked issue if there is an opportunity for simplification/reuse/refactoring

Related:

• Refactor Resource Tagging Code to use keyvaluetags Package #10688
• Add 'filter' block to EC2 service data source that are currently missing this #11994

[ewbankkit]

The code for data sources such as the aws_vpc data source could be changed to something like

	// Filters based on attributes.
	filters := namevaluesfilters.New(map[string]string{
		"internet-gateway-id": d.Get("internet_gateway_id").(string),
	})
	// Filters based on keyvalue tags.
	filters.Add(namevaluesfilters.Ec2Tags(keyvaluetags.New(d.Get("tags").(map[string]interface{})).IgnoreAws().IgnoreConfig(ignoreTagsConfig).Map()))
	// Filters based on the custom filtering "filter" attribute.
	filters.Add(d.Get("filter").(*schema.Set))

	req.Filters = filters.Ec2Filters()

[ewbankkit]

Data sources (other than for EC2 resources) that can use this functionality:

• aws_autoscaling_groups
• aws_db_cluster_snapshot
• aws_db_event_categories
• aws_db_instance
• aws_db_snapshot
• aws_route53_resolver_rule
• aws_route53_resolver_rules

[bflad]

Looks like AWS Go SDK v1.33.5 includes similar functionality for service/secretsmanager 🎉

[ewbankkit]

Yes, it's effectively the same functionality although the Name field is called Key but the code generator can handle that.
I'll add the aws_secretsmanager_secret data source to the list above.
Requires #14000.
Announcement.

[ewbankkit]

I decided not to add the secretsmanager service to the generated code list as the fact that the name field is Key implies we cannot use the standard

		Elem: &schema.Resource{
			Schema: map[string]*schema.Schema{
				"name": {
					Type:     schema.TypeString,
					Required: true,
				},

				"values": {
					Type:     schema.TypeSet,
					Required: true,
					Elem: &schema.Schema{
						Type: schema.TypeString,
					},
				},
			},
		},

definition for the filter attribute.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.