AWS S3 Endpoints now support Interface type

[eriepasquare]

It appears that AWS released (at least in our accounts) a second endpoint type for S3 Endpoints... until now S3 Endpoints only supported Gateway, but now they support an S3 endpoint of type Interface. Because of this, the vpc module is throwing the following error:

Error: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service

The data source for s3 endpoint needs to include an additional filter to ensure a single record is returned:

terraform-aws-vpc/vpc-endpoints.tf

Lines 4 to 8 in a78cee9

	data "aws_vpc_endpoint_service" "s3" {
	count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0
	service = "s3"
	}

Should now include:

data "aws_vpc_endpoint_service" "s3" {
  count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0

  service = "s3"
  service_type = "Gateway"
}

[rogersd]

Also related: #445

[michaelhelmick]

@rogersd thanks for this. Literally, just had a colleague trying to implement something and came across this error. Will have him try it (judging by the 👍🏻) looks like it should work.

[vadanx]

Yep. This broke our pipelines. Exact same problem.

[123BLiN]

yep, but fix does not work with old aws provider looks like?

Error: "service_type": this field cannot be set

  on .terraform/modules/vpc/vpc-endpoints.tf line 4, in data "aws_vpc_endpoint_service" "s3":
   4: data "aws_vpc_endpoint_service" "s3" {

[123BLiN]

hey @antonbabenko minimal AWS provider version should be increased probably as well, but I've not discovered correct version yet, it is definetly doesn't work with 2.70 (see above)

[pib]

They added service_type in 3.10.0: https://github.com/hashicorp/terraform-provider-aws/blob/main/CHANGELOG.md#3100-october-09-2020

[antonbabenko]

Good point, I will update the required version constraint now.

[antonbabenko]

#574 => v2.70.0 has been just released.

[dgonzalez]

So that means it is broken for providers prior to 3.10?

[antonbabenko]

Yes, data-source was not available in prior versions.

There are two workarounds we can have in the code:

1. Don't use data-source aws_vpc_endpoint_service but require users to provide service_name explicitly.
2. Require users to create S3 VPC endpoint resources outside of the module if they can't upgrade to 3.10+.

PS: 3.10 has been released 5 months ago.

What do you think about this?

[dgonzalez]

sounds reasonable. Will need to look into our codebase as this can be tricky but...

[marpada]

@antonbabenko thank you for being awesome!

[michaelhelmick]

sounds reasonable. Will need to look into our codebase as this can be tricky but...

Agreed. Upgrading seems to be non-trivial, you will have to remove the VPC endpoint resource from Terraform code and the tfstate then apply changes, upgrade your Terraform version, fix any syntax issues, upgrade your AWS provider version, fix any issues and then re-add the VPC endpoint resource.

Also, not mentioned, looks like AWS provider 3.10 requires Terraform version >= 0.12.21

Hoping this helps anybody trying to figure out how to overcome this issue.

[dgonzalez]

It turns out it is a big issue for us. We have several third party modules and we are upgrading from terraform 12.x which is causing mayhem. We have elevated the request to AWS to remove the PrivateLink interface to S3 so we can plan again otherwise we are effectively blocked on the infrastructure side.

[dgonzalez]

I dont see any obvious option to remove the PrivateLink interface from the UI.

[dgonzalez]

Why do you need to remove the box endpoint and then readd it?

…

On Wed, Feb 3, 2021, 5:05 PM Mike Helmick ***@***.***> wrote: sounds reasonable. Will need to look into our codebase as this can be tricky but... Agreed. Upgrading seems to be non-trivial, you will have to remove the VPC endpoint resource from Terraform code and the tfstate then apply changes, upgrade your Terraform version, fix any syntax issues, upgrade your AWS provider version, fix any issues and then re-add the VPC endpoint resource. Also, not mentioned, looks like AWS provider 3.10 requires Terraform version >= 0.12.21 Hoping this helps anybody trying to figure out how to overcome this issue. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#571 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA6IOTWAKDWN45KTF7DH7LS5F663ANCNFSM4W7JZWEA> .

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.