New Resource: aws_route53_vpc_association_authorization

[RyanJarv]

I've been sitting on this for a bit hoping the cross account aws_route53_zone_association issue would sort it self out (#384). A few people have expressed interest in this even though the association will need to be done outside of Terraform so I think it's fine to just have a note in the docs about this issue.

Much of the code is copied from aws_route53_zone_association since the logic is pretty similar.

Related Issues/PRs

hashicorp/terraform#10208
hashicorp/terraform#12553 (PR before the repo split)
#384

[tobyclemson]

We're running into exactly this problem and it would be great to get this merged in.

[aggallim]

Is this on someone's todo list? Would love to have this functionality before I start off down the route of automating it using the aws cli!

[bevanbennett]

Thumbs up for this. About half our terraform apply runs now always report an error "NotAuthorizedException: The VPC: vpc-XXXXXXX has not authorized to associate with your hosted zone." even when we authorize it explicitly or have already associated manually.

[ewbankkit]

Any ideas on what is required to get this merged?

[ewbankkit]

@RyanJarv I took the liberty of playing a bit with your code, see here - no substantive changes, mainly just a rename of aws_route53_vpc_association_authorization to aws_route53_zone_association_authorization and some code cleanup to match other recent changes.
In testing I hit the issue I describe here which is an artifact of the way the Route 53 API works - No zone association resource with an ID is exposed so we have to query the zone resource and don't have permissions to do that in the account that didn't create the zone.

[RyanJarv]

@ewbankkit Awesome, thanks! I will try to take a look at your changes soon and get them added to this PR.

Since the error you mentioned is expected in that use case, and it's likely more people will run into it I'm wondering if we can/should add a message regarding this when that error is raised.

I would also be interested if you have any ideas on how to approach that issue. It's been a while since I looked at this but it may be worth seeing if any other resources have dealt with something similar since.

[ewbankkit]

@RyanJarv Please feel free to merge my changes with yours if appropriate.
I spent a while thinking about how to deal with this cross-account association access issue and couldn't come up with any solution other than not managing the actual association in Terraform which completely defeats the purpose of this PR 😄.
It's a fundamentally a problem with the design of the Route53 API.
I opened a discussion forum item:

• https://forums.aws.amazon.com/thread.jspa?threadID=271250

and will follow up with our account team.

[bflad]

@ewbankkit / @RyanJarv sorry for the long maintainer silence on this PR. Can you reach out to me if/when its ready to be looked at or if you have specific questions? Thanks!

[smastrorocco]

Any chance this is getting merged soon? Running into scenario where I need this. Any workaround in meantime?

[RyanJarv]

@smastrorocco I would +1 the AWS forms thread at https://forums.aws.amazon.com/thread.jspa?threadID=271250 if you need this.

This is really something that needs to be fixed by AWS.

Haven't really messed around with this, but your best bet for now is to probably to shell out and run something outside of terraform.

[RyanJarv]

Hey @ewbankkit, wondering if you ever heard anything back from your AWS account team on this.

[ewbankkit]

No, I never heard anything back.

[lorengordon]

Any further progress on this? The approach mentioned by @JoshiiSinfield in the associated issue would work for me.

#384 (comment)

[wobondar]

Gents @ewbankkit @RyanJarv , is there any chance to get it merged? Thanks!

[FransUrbo]

LGTM

[FransUrbo]

As mentioned elsewhere, this should probably be aws_route53_zone_association_authorization. Even better, would probably be aws_route53_zone_association_accepter (depending on what it actually do - I'm not a fan of Go! :).

This because we already have:

aws_vpc_peering_connection
aws_vpc_peering_connection_accepter

So having the r53 resources being named

aws_route53_zone_association
aws_route53_zone_association_accepter

makes sense. Pretty if nothing else :D

[zdoherty]

aws_route53_vpc_association_authorization would match the name of the API call that creates it, though:

https://docs.aws.amazon.com/Route53/latest/APIReference/API_CreateVPCAssociationAuthorization.html

[FransUrbo]

Should this possibly be vpc_id and something like peer_zone_id (much like aws_vpc_peering_connection does it)?

[zdoherty]

In the case of aws_vpc_peering_connection, the peer prefix is needed to disambiguate between the two VPC IDs in the arguments. Since there's only one zone_id in these arguments, I feel like just zone_id would be sufficient.

[rfarro]

Is there any movement on this? We have been needing to create the authorizations with aws cli commands. Here is an example on how we accomplish this with our transit account. If we could get this piece in terraform it would solve a world of problems with our Route53 module

resource "null_resource" "route53_association" {

  provisioner "local-exec" {
    command = "aws route53 create-vpc-association-authorization --profile ${var.account_alias} --hosted-zone-id ${element(aws_route53_zone.private_hosted_zone.*.id, count.index)} --vpc VPCRegion=${data.terraform_remote_state.transit_peer_east.region},VPCId=${data.terraform_remote_state.transit_peer_east.vpc_id}"
  }

  provisioner "local-exec" {
    command = "aws route53 associate-vpc-with-hosted-zone --profile transit --hosted-zone-id ${element(aws_route53_zone.private_hosted_zone.*.id, count.index)} --vpc VPCRegion=${data.terraform_remote_state.transit_peer_east.region},VPCId=${data.terraform_remote_state.transit_peer_east.vpc_id}"
  }
}

[feraudet]

Any chance to see aws_route53_vpc_association_authorization merged soon ?

[nunofernandes]

I'm also blocked in this.. Any ideas/timeline on when it will be merged?

[hdpoe]

Just throwing out there that we my organization is also facing a block because of this. I also feel that the Terraform docs should be updated to indicated that cross account zone association is not currently possible as the current wording led to me spending no small amount of time trying various configurations of provider, vpc_id and hosted zone_id.

[bluemalkin]

It's pretty obvious that it's not possible to do cross aws account resources without manual approval. Vpc peering requires manual approval between accounts.

[bevanbennett]

It's pretty obvious that it's not possible to do cross aws account resources without manual approval. Vpc peering requires manual approval between accounts.

This is provably untrue. The only issue is that the request and approval need to be executed with different credentials. There is zero requirement for "manual" anything and this can be automated today with the CLI. The solution here would be to either split this resource into two pieces (association_authorization and association) or to allow us to specify a separate provider to perform the authorization than the one being used for the association. Splitting it into two resources is probably cleanest.

[drmaciej]

It's pretty obvious that it's not possible to do cross aws account resources without manual approval. Vpc peering requires manual approval between accounts.

This is provably untrue. The only issue is that the request and approval need to be executed with different credentials. There is zero requirement for "manual" anything and this can be automated today with the CLI. The solution here would be to either split this resource into two pieces (association_authorization and association) or to allow us to specify a separate provider to perform the authorization than the one being used for the association. Splitting it into two resources is probably cleanest.

That is correct, peering can be done with Terraform without any manual intervention.

[bluemalkin]

@maciejdrozdzowski peering can be done with Terraform within the same AWS account yes, not with a different account, you'd still need to accept that separately with another terraform state or manually.

[drmaciej]

@bluemalkin that's how it was in my case - it was a cross-account peering, with different state files. The fact that it needed no manual steps was a pleasant surprise.

[lorengordon]

@bluemalkin You can indeed accept the VPC peering request in the same terraform state, even between accounts. You just need to use multiple providers, one with a credential to the requester, and one with a credential to the accepter.

provider "aws" {}

provider "aws" {
  alias = "peer"
}

resource "aws_vpc_peering_connection" "this" {
  peer_owner_id = "${var.peer_owner_id}"
  peer_vpc_id   = "${var.peer_vpc_id}"
  vpc_id        = "${var.vpc_id}"
}

resource "aws_vpc_peering_connection_accepter" "this" {
  provider = "aws.peer"

  vpc_peering_connection_id = "${aws_vpc_peering_connection.this.id}"
  auto_accept               = true
}

[hdpoe]

I apologize that my comment was unclear. There is currently not a way to use native terraform aws resources to associate a vpc in one account with the private hosted zone of a vpc in another account. I'll make an update to clarify what was intended to be communicated.

[rfarro]

I just want to throw out that we are testing getting away from hosting our own DHCP options set and using unbound as our DNS traffic cop. AWS has recently released route 53 resolver https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resolver-getting-started.html which allows you to create outbound/inbound endpoint so that you can get answers from both onprim server and aws servers.

[ewbankkit]

• https://aws.amazon.com/premiumsupport/knowledge-center/private-hosted-zone-different-account/

[aeschright]

Hi @RyanJarv ! Do you have a minute to resolve the merge conflict? Unfortunately the PR has been sitting for a while. We also need the output from your test run, which you can add to the PR description like so:

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccXXX'

...

[RyanJarv]

Want to note there is a lot of comments here that are distractions from the real issue. This is likely because discussion about it got split up between several issue making it fairly confusing to follow.

About to look into this again to see if anything has changed. But, from my memory (which granted is probably poor since this issue is fairly old now), the issue is mainly that the API around these resources doesn't map well to how terraform manages resources...

It is certainly possible to hack something together, I simply chose not to do that here especially considering this seems to be an upstream issue IMO

Here's a bit more info on the API issue (posted this in another issue prior): https://blog.ryanjarv.sh/2019/05/24/backdooring-route53-with-cross-account-dns.html

Granted I'm not sure if this is still the case anymore. Will update this comment shortly.

Apologies about the poor communication.

Edit 1: Forgot my original plan was to merge this despite it not working with route53_hosted_zone_association. I’ll go ahead and just rebase this soon if nothing else.

[aeschright]

@RyanJarv We've been trying to sort out whether the API will map to how Terraform works too, so I'd be interested in any thoughts you might have on that given the cli workflow linked above.

[RyanJarv]

@aeschright This seems fine to me. With the exception of using —profile instead of connecting to nodes in different accounts this is how I’ve done it in the past.

Also saw this today which may be helpful https://medium.com/@dalethestirling/managing-route53-cross-account-zone-associations-with-terraform-e1e45de8f3ea.

Edit: If your wondering how that might map to terraform it’s important to remember resources validate the current state before they execute. So the CLI process is simply skipping that step.

Edit 2: Actually gonna think about this some more. This PR doesn’t really take in to account the fact you may want to delete the authorization afterwards, unsure if this matters or how it should be handled.

[aeschright]

After internal discussion we've decided not to go forward with the PR, for the reasons described above. There doesn't seem to be a way to do this through the API that will be effective within Terraform's design -- we're especially thinking about the need to delete the authorization afterward. I'm going to make sure the cli info is visible on the issue ticket. Thanks so much for your work on this!

[RyanJarv]

Thanks @aeschright, that makes sense to me.

FYI, I cleaned up my post around the API issue a bit (https://blog.ryanjarv.sh/2019/05/24/backdooring-route53-with-cross-account-dns.html), so if anyone is concerned about this I recommend mentioning it to AWS support. There is an internal ticket on it somewhere, it's been about 2 years without any action as far as I know, but I'm sure if more people bring it up they will fix it.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!