-
Notifications
You must be signed in to change notification settings - Fork 9.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ECR creation sometimes fails with Resource '...' does not have attribute 'repository_url' for variable '...' #3849
Comments
I believe the workaround to be really simple: diff --git a/aws/resource_aws_ecr_repository.go b/aws/resource_aws_ecr_repository.go
index 3a244743..92ca0b2a 100644
--- a/aws/resource_aws_ecr_repository.go
+++ b/aws/resource_aws_ecr_repository.go
@@ -65,7 +65,13 @@ func resourceAwsEcrRepositoryCreate(d *schema.ResourceData, meta interface{}) er
d.Set("arn", repository.RepositoryArn)
d.Set("registry_id", repository.RegistryId)
- return resourceAwsEcrRepositoryRead(d, meta)
+ d.Set("name", repository.RepositoryName)
+
+ repositoryUrl := buildRepositoryUrl(repository, meta.(*AWSClient).region)
+ log.Printf("[INFO] Setting the repository url to be %s", repositoryUrl)
+ d.Set("repository_url", repositoryUrl)
+
+ return nil
}
func resourceAwsEcrRepositoryRead(d *schema.ResourceData, meta interface{}) error { |
See issue hashicorp#3849 In some cases, AWS can answer with an error 400 RepositoryNotFoundException on te second DescribeRepositories request right after the repository was successfully created. The resource end up with repository_url not there (can cause errors if it is used elsewhere) and the tfstate says that the ECR does not exists while it was in fact created. We do not need to perform this request, and can avoid this error.
Will be testing the following change and report back: v1.7.1...squarescale:v1.7.1-squarescale |
@mildred thanks for the report. We currently handle situations like these in other resources, like |
The response from AWS tells us that it is possible that DescribeRepositories does not always return up to date answers:
|
@bflad in the code you linked, there is a hardcoded timeout of two minutes. I believe this is not correct because AWS can rate limit any API request, especially when we perform many reads (the reads are the most throttled) and it can lead to strange failures. See #3128 for an example of issue regarding timeouts |
When creating an ECR repository, creation can be successful but it may not be available right away through DescribeRepositories API request. We add a retry mechanism until the repository appears instead of marking the repository aas not existing. Fixes hashicorp#3849
When creating an ECR repository, creation can be successful but it may not be available right away through DescribeRepositories API request. We add a retry mechanism until the repository appears instead of marking the repository as not existing. Fixes hashicorp#3849
Hi there, I have some ECR creation/deletion in a CI Pipeline (with an s3 backend) and I face the some job failure because of "RepositoryAlreadyExistsException" during the |
The fix for this has been merged and will release with version 1.40.0 of the AWS provider, likely middle of this week. 👍 |
This has been released in version 1.40.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks! |
We have a problem with the resource
aws_ecr_repository
and by looking at the debug log and the terraform code, we found what the problem was. The problem comes from the fact that when creating an ECR repository, terraform is executing two API requests:resourceAwsEcrRepositoryCreate
line 47 ofresource_aws_ecr_repository.go
resourceAwsEcrRepositoryRead
line 71 of the same fileThe first request succeeds with AWS, but the second, for reasons not yet understood (we have a support request opened with AWS) returns with a 400 error RepositoryNotFoundException.
Thus, the
resourceAwsEcrRepositoryRead
function fails and the attributerepository_url
is never set on the resource state.Moreover, in the tfstate, the ECR resource is marked as not existing (while it has in fact been created by AWS) and a subsequent terraform apply will try to create the ECR repository, and failing because a repository of the same name already exists.
As I see it, it is primarily a problem of the AWS API not returning the correct response (it says the repository does not exists while in fact it has been created). Perhaps this has to do with a consistency model of the AWS API not fully detailed on their API docs. In which case, we might need to be prepared to handle stale data in API responses.
I'll keep you updated with the replies from the AWS support regarding this issue.
In the meantime, perhaps the provider code can be simplified. After a successful CreateRepository I do not see the necessity to run DescribeRepositories. perhaps we could omit the second call completely and this would work around the API inconsistency.
Terraform Version
Affected Resource(s)
Terraform Configuration Files
We have much more terraform resources, but the problems happens when adding the above snippet to an up to date infrastructure and we run terraform apply. Only the ECR resource is being created.
Output
This is our full output of
terraform apply
of a pre-computed terraform plan file:Debug Output
Relevant part of the debug output:
Expected Behavior
The error
output.ECR: Resource 'aws_ecr_repository.client-registry-984561098034cf1a7ba7d64b520feaf7aeed7a81' does not have attribute 'repository_url' for variable 'aws_ecr_repository.client-registry-984561098034cf1a7ba7d64b520feaf7aeed7a81.repository_url'
should not happenActual Behavior
A terraform error for perfectly valid code
Steps to Reproduce
This is really difficult to reproduce, and it doesn't happen every time on out environment. Basically, it cannot be reproduced easily.
Important Factoids
We have a support request with AWS and we asked them why the DescribeRepositories API request is returning with an error 400 telling the ECR does not exist
References
None public
The text was updated successfully, but these errors were encountered: