Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS: Certificate not found even though it was just created #2499

Closed
nathanielks opened this issue Jun 25, 2015 · 17 comments
Closed

AWS: Certificate not found even though it was just created #2499

nathanielks opened this issue Jun 25, 2015 · 17 comments

Comments

@nathanielks
Copy link
Contributor

Related: #2490

I deleted the certificate using AWS's cli. The output shows it was created but then when it goes to create the ELB, AWS says that the certificate doesn't exist.

Here are the logs.

@phinze
Copy link
Contributor

phinze commented Jun 26, 2015

Thanks for the report - we've seen this issue in several different places. We'll have to see if there's something we can do after the certificate create to ensure it's ready to be attached to an ELB.

@nathanielks
Copy link
Contributor Author

👍

@mitchellh
Copy link
Contributor

I'm playing with this now. The AWS logs say that it can take up to 15 minutes for certificates to propagate, which would indeed be a really sad thing. I'm trying to find a way for us to detect this since the API doesn't seem to expose it.

@rgabo
Copy link

rgabo commented Jul 1, 2015

Just ran into the same issue. awscli is quite misleading because aws iam list-server-certificates immediately lists the certificate that was just created, even though when referenced in other services (ELB in our case), it will throw an error. Thanks for looking into this, Mitchell.

@xuwang
Copy link
Contributor

xuwang commented Jul 27, 2015

I have the same issue, one workaroud is to wait for a while in local provisioner :

resource "aws_iam_server_certificate" "mycert" {
  name = "mycert"
  certificate_body = "${file("mycert.pem")}"
  private_key = "${file("mycert-key.pem")}"

  provisioner "local-exec" {
    command = <<EOF
        echo # Sleep 10 secends so that mycert is propagated by aws iam service
        echo # See https://github.com/hashicorp/terraform/issues/2499 (terraform ~v0.6.1)
        sleep 10
EOF
  }
}

10 seconds wait time seems to work for me. May a terraform function can be introduced to handle the need for external waiting? e.g. depend_on with a waiting time.

@farridav
Copy link

This is affecting me too, and I also got he same output from aws iam list-server-certificates ..Just posted on these 2 related issues #3275 and #3412 .. I will try the temporary sleep fix.. but keen to hear if there is a proper solution I can use 👍

@catsby
Copy link
Contributor

catsby commented Nov 12, 2015

I've submitted #3898 as a patch for #3275 and #3412. Unfortunately it is largely a bandaid. IAM resources are notoriously eventually consistent for our experiences; API calls return "success" but follow up calls that reference the ARN return 404s :/

@catsby
Copy link
Contributor

catsby commented Nov 20, 2015

#3898 was merged, so I'm going to close this.
Let me know if you're still seeing this on the master branch, otherwise the fix will rollout in the next release (should be next week)

@catsby catsby closed this as completed Nov 20, 2015
@nathanielks
Copy link
Contributor Author

👍

@nathanielks
Copy link
Contributor Author

@catsby Finally had the opportunity to test #3898. Having upgraded to 0.6.8, I'm unfortunately still seeing the error. Adding a 10 second sleep to the certificate "resolves" the issue though ¯_(ツ)_/¯

@catsby catsby added the waiting-response An issue/pull request is waiting for a response from the community label Dec 7, 2015
@catsby
Copy link
Contributor

catsby commented Dec 7, 2015

Adding a 10 second sleep to the certificate "resolves" the issue though

Can you elaborate on how you're doing this? The retry logic in #3898 should retry the creation for up to 1 minute, so, I'm not sure where a sleep for 10 seconds is helping, that a retry for 1 minute wouldn't also fix. Do you see the warning "[WARN] Error creating ELB Listener with SSL Cert, retrying" in the logs?

Or is this for deletion? This issue has crossed some wires 😄

@nathanielks
Copy link
Contributor Author

Sure can, @catsby! Here's the whole resources for funzies:

resource "aws_iam_server_certificate" "domains" {                                                                                                               
    count = "${var.load_balancer_count}"                                                                                                                        

    name = "${var.environment}-${element(split(" ",var.certificate_names), count.index)}"                                                                       

    certificate_body = "${file(format("%s/%s/%s.%s", var.certificates_base, var.environment, element(split(" ",var.certificate_names), count.index), "crt"))}"  
    certificate_chain = "${file(format("%s/%s/%s.%s", var.certificates_base, var.environment, element(split(" ",var.certificate_names), count.index), "chain"))}
    private_key = "${file(format("%s/%s/%s.%s", var.certificates_base, var.environment, element(split(" ",var.certificate_names), count.index), "key"))}"       

    provisioner "local-exec" {                                                                                                                                  
        command = "sleep 10"                                                                                                                                    
    }                                                                                                                                                           
}                                                                                                                                                               

It's on creation. I'm not sure I'll have time to re-run everything, but I'll do that as soon as I can and get back with you on the ELB Listener warning!

@catsby
Copy link
Contributor

catsby commented Dec 11, 2015

@nathanielks I attempted to reproduce with this:

but I can't. Let me know what I'm missing

@jurajseffer
Copy link

I think I'm hitting this in terraform v0.10.8 when trying to assign a newly saved IAM server certificate to an ALB listener. Terraform gives me an ARN for aws_iam_server_certificate but it fails to apply the change to listeners with an AWS error saying the certificate with given ARN cannot be found. When I manually look at the certificates available via AWS console, I can see it being there and subsequent plan and apply works fine so it looks like a problem with eventual consistency.

@cloudvant
Copy link

Yeah, I'm hitting this in terraform v0.10.8 as well. It works on macOS and Ubuntu 16, but fails on Centos/Jenkins running on EC2.

@mattchilds1
Copy link

mattchilds1 commented Apr 19, 2018

It seems I'm experiencing the same thing as @jurajseffer . The only different thing I'm doing is pulling the cert in from another AWS account with a separate provider.

Terraform returns the correct ARN, pass's it into the ELB Listener but then fails with the certificate not found error.

@catsby

EDIT: Apologies, I've now seen that AWS don't support using of certificates across trusted accounts. A change to get a different error message could still be good though?

@ghost
Copy link

ghost commented Apr 3, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

10 participants