More flexibility on AMI and subnet filtering

[multani]

We are using Kitchen conjointly with Packer to build our images. Although Packer builds the final image, Kitchen + kitchen-ec2 makes the development workflow much much easier.

There are 2 things that we are relying on from Packer which isn't currently fully supported by kitchen-ec2:

• subnet_filter allows to pass complete queries to search the subnet to create the instance in. We are currently relying on the mapPublicIpOnLaunch filter, which allows us to find any subnet which can automatically associate a public IP address to an instance.
• source_ami_filter allows us to find either a public AMI made by a well-known account (we are using Debian Buster) but also to find one our "base" image, which is private and owned by our own account.

Thus, we have this type of filter for example:

...
      "source_ami_filter": {
        "filters": {
          "architecture": "x86_64",
          "tag:Name": "base",
          "tag:Version": "v5"
        },
        "most_recent": true,
        "owners": [
          "self"
        ]
      },
      "subnet_filter": {
        "filters": {
          "mapPublicIpOnLaunch": "true"
        },
        "most_free": true
      },
...

This type of filtering is currently not available in kitchen-ec2, where the allowed filter are much more specific/restrictive.

I would like to know if you were open to a breaking change on the configuration format that would allow filtering options similar to what Packer offers (also similar to what the AWS API actually proposes), which would open up more flexibility on Kitchen-side.

The idea would be to support filters like this in Kitchen:

For subnets (this would map more closely to the DescribeSubnets API, where somehow mapPublicIpOnLaunch is not documented 🤔 ):

subnet_filter:
  filters:
    tag:Name: 'example-subnet-name'
    mapPublicIpOnLaunch: 'true'

For AMIs (also closer to the DescribeImages API):

image_search:
  filters:
    tag:Name: foobar
    # name: ubuntu/images/*/ubuntu-*-14.04*
    # owner-id: "099720109477"
  owners:
    - self

I made an initial draft to implement that here, I would be very happy to improve that and propose a PR to merge this upstream if this there's a chance for it 👍

[multani]

Any interest in that functionality?

[marcparadise]

Thanks for taking the time on that PR and this writeup @multani. I'm going to forward this along to our product team for further consideration. Leaving this untagged for now so it shows up on our triage list and doesn't get forgotten.

[deric4]

@marcparadise this would be a great enhancement and would reduce the amount pre-processing/erb needed to reuse templates across iam boundaries.

Support for referencing the aws public ssm parameters for the default AMIs would be a great value add as well. Never know whats actually getting pulled in 🙃