Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_cloudfront_function PreconditionFailed: The request failed because it didn't meet the preconditions in one or more request-header fields. #19529

Closed
hougholi opened this issue May 26, 2021 · 20 comments · Fixed by #19697
Labels
bug Addresses a defect in current functionality. service/cloudfront Issues and PRs that pertain to the cloudfront service.
Milestone

Comments

@hougholi
Copy link

hougholi commented May 26, 2021

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v0.13.5
Provider v3.42.0

Affected Resource(s)

  • aws_cloudfront_function

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_cloudfront_function" "cloudfront_security_headers" {
  name    = "MyFunction"
  runtime = "cloudfront-js-1.0"
  publish = true
  code    = file("${path.module}/../../src/index.js")
}

Debug Output

https://gist.github.com/hougholi/0f099da4f04599fbe26e38e0568b1f45

Expected Behavior

CloudFront function code should be updated.

Actual Behavior

Error: error updating CloudFront Function (dev-cloudfront-security-headers) configuration : PreconditionFailed: The request failed because it didn't meet the preconditions in one or more request-header fields.
status code: 412, request id: 38cb6e1b-b74a-4f1c-bb91-0c385e39c88d

Steps to Reproduce

Update CloudFront function code and run terraform apply.

Important Factoids

The function already exists in AWS, this error only occurs when updating the code and trying to run terraform apply

@ghost ghost added the service/cloudfront Issues and PRs that pertain to the cloudfront service. label May 26, 2021
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label May 26, 2021
@ewbankkit ewbankkit removed the needs-triage Waiting for first response or review from a maintainer. label May 27, 2021
@ewbankkit
Copy link
Contributor

This error is returned because the ETag value in state isn't equal to the current value at AWS (because the function was updated outside Terraform):

func resourceAwsCloudFrontFunctionUpdate(d *schema.ResourceData, meta interface{}) error {
conn := meta.(*AWSClient).cloudfrontconn
etag := d.Get("etag").(string)
if d.HasChanges("code", "comment", "runtime") {
input := &cloudfront.UpdateFunctionInput{
FunctionCode: []byte(d.Get("code").(string)),
FunctionConfig: &cloudfront.FunctionConfig{
Comment: aws.String(d.Get("comment").(string)),
Runtime: aws.String(d.Get("runtime").(string)),
},
Name: aws.String(d.Id()),
IfMatch: aws.String(etag),
}
log.Printf("[INFO] Updating Cloudfront Function: %s", d.Id())
output, err := conn.UpdateFunction(input)
if err != nil {
return fmt.Errorf("error updating CloudFront Function (%s) configuration : %w", d.Id(), err)
}
etag = aws.StringValue(output.ETag)
}

@ewbankkit ewbankkit added bug Addresses a defect in current functionality. thinking labels May 27, 2021
@ewbankkit ewbankkit self-assigned this May 27, 2021
@ewbankkit ewbankkit removed the thinking label Jun 1, 2021
@ewbankkit ewbankkit removed their assignment Jun 1, 2021
@tomhaynes
Copy link

@hougholi why was this closed? I'm seeing the same issue.

@hougholi hougholi reopened this Jun 2, 2021
@hougholi
Copy link
Author

hougholi commented Jun 2, 2021

I worked around it by deleting the CloudFront function and letting Terraform recreate it.

@tomhaynes
Copy link

Thanks for reopening 👍 - yes I did the same, but it is still worth fixing I think - ideally manual changes would be detected and reset by terraform - they are ignored at the moment, I guess that's a separate issue though

@fularac
Copy link

fularac commented Jun 2, 2021

Same here. I would expect it to work again after running terraform refresh but the same precondition error occurs.

It appears that there's a bug with etag value for the LIVE & DEVELOPMENT stages of the function and having publish = true. Each stage has separate etag value and when the function is refreshed with publish = true the etag stored is the value of the LIVE function. I can see in the state of a cloudfront function not manually modified (but one that has been updated via terraform) the etag matches the function inDEVELOPMENT stage. In fact the etag of LIVE & DEVELOPMENT stages match.

@hougholi
Copy link
Author

hougholi commented Jun 3, 2021

I now have this issue with a function in production - so I can't delete it and let Terraform recreate. So yeah it's worth fixing.

@richard-underwood
Copy link

I now have this issue with a function in production - so I can't delete it and let Terraform recreate. So yeah it's worth fixing.

Another workaround is to remove it from the state file (terraform state rm) and add it back in (terraform import) before updating.

@holstvoogd
Copy link

I have the same issue with origin-access-identities

@GideonPARANOID
Copy link

We've got the same issue, leading to an inconsistent build (occasional reruns work) which blocks us releasing anything with CloudFront Functions.

@alexjurkiewicz
Copy link
Contributor

I am interested in fixing this issue, it causes problems for us. I can see a few ways forward:

  1. When modifying a function, if there is a failure due to etag mismatch, silently pull the latest etag and retry
  2. Require the user to run terraform refresh to update the etag

If a maintainer could confirm the preferred approach, I will code it up.

@simontabor
Copy link

(Not a maintainer but throwing in my 2c @alexjurkiewicz)

I'd prefer option 1, because on other resources Terraform attempts to reset any changes (manual or otherwise) to the terraform representation of the infrastructure. For example, if I manually modify the desired number of tasks on an ECS service, terraform apply will set it back to the desired_count.

If we had to run terraform refresh every time those changes were made, we'd end up running a refresh before every plan/apply ("just in case"), which adds complexity that isn't needed for other resources

PS. just spotted in the docs that terraform refresh is deprecated

@Rondineli
Copy link

Rondineli commented Jul 8, 2021

Running for the same exception. I am fixing it by running with publish=false then after applied the code, run it back with publish=true it seems tf is getting lost dealing with trying publish it before saves the code.

@fularac
Copy link

fularac commented Jul 9, 2021

From: https://www.terraform.io/docs/cli/commands/refresh.html

You shouldn't typically need to use this command, because Terraform automatically performs the same refreshing actions as a part of creating a plan in both the terraform plan and terraform apply commands. This command is here primarily for backward compatibility, but we don't recommend using it because it provides no opportunity to review the effects of the operation before updating the state.

Performing a plan/apply will refresh your state. The problem here is that with publish=true the etag retrieved is for the published (LIVE) version of the cloudfront function. If for some reason the etag of the published version and the unpublished (DEVELOPMENT) version does not match, the request to update the published function fails. I've fixed this in #19697 and I've been using a local version of the provider in production since I opened the PR in our production environment with no issues.

A separate issue that causes a similar precondition errors happens if a plan will modify both the distribution & cloudfront function that is published. The cloudfront function being published actually modifies the etag of the distribution. Which is where @alexjurkiewicz first proposal would be useful as it would help alleviate inter-resource etag conflicts. You can see this already being done for cloudfront distribution deletions

// Refresh our ETag if it is out of date and attempt deletion again.
if isAWSErr(err, cloudfront.ErrCodeInvalidIfMatchVersion, "") {
getDistributionInput := &cloudfront.GetDistributionInput{
Id: aws.String(d.Id()),
}
var getDistributionOutput *cloudfront.GetDistributionOutput
log.Printf("[DEBUG] Refreshing CloudFront Distribution (%s) ETag", d.Id())
getDistributionOutput, err = conn.GetDistribution(getDistributionInput)
if err != nil {
return fmt.Errorf("error refreshing CloudFront Distribution (%s) ETag: %s", d.Id(), err)
}
if getDistributionOutput == nil {
return fmt.Errorf("error refreshing CloudFront Distribution (%s) ETag: empty response", d.Id())
}
deleteDistributionInput.IfMatch = getDistributionOutput.ETag
_, err = conn.DeleteDistribution(deleteDistributionInput)
}

@ewbankkit ewbankkit self-assigned this Jul 11, 2021
@DJAlPee
Copy link

DJAlPee commented Aug 4, 2021

Running for the same exception. I am fixing it by running with publish=false then after applied the code, run it back with publish=true it seems tf is getting lost dealing with trying publish it before saves the code.

Thanks, this helped to resolve the error in my case.

I have two functions in the same (CDK) terraform stack. Applying with publish = false did only work for one function. On a second apply (without any additional changes) it fixed the second function. Apply with no publish config (default true) works now again.

@ohmer
Copy link

ohmer commented Sep 6, 2021

Hi there,

Can reproduce this with Terraform 1.0.2/1.0.5 and provider version 3.56.0.
Anyone working a fix? ;-)

@tsposato
Copy link

This is still occurring - It's not an issue with Lambda@Edge functions so I'm assuming the same sort of rules etc. should apply here too?

@github-actions github-actions bot added this to the v3.65.0 milestone Nov 8, 2021
@github-actions
Copy link

This functionality has been released in v3.65.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@Raffaello
Copy link

I have just experiencing this happening with AWS provider v3.73.
it could be related to terraform version too or it is not fully fixed?

Error: error updating CloudFront Distribution (***): PreconditionFailed: The request failed because it didn't meet the preconditions in one or more request-header fields.
	status code: 412

the weird thing is re-applying might fix the error.
but worried that might happen sometimes, intermittently and it would be just annoying.

@fularac
Copy link

fularac commented Jan 28, 2022

Error: error updating CloudFront Distribution

Your failure is when updating the cloudfront distribution, not the cloudfront function. Theres a separate issue I described in this comment #19529 (comment). I'll create a bug report on it.

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/cloudfront Issues and PRs that pertain to the cloudfront service.
Projects
None yet