Terraform is showing changes to lambdas when there is no code change

[PaulF2022-55]

Hi there,

1. We are creating AWS lambda function with nodejs-14 code.
2. We deployed this lambda function using CICD pipe line, wherein we first build a zip file using application using yarn build and deployed on the zip file using terraform apply.
3. Recently after upgrading to new AWS provider 4.2.0 and terraform version 0.15.4

resource "aws_lambda_function" "lambda" {
  filename         = var.filename
  function_name    = var.function_name
  role             = aws_iam_role.lambda.arn
  handler          = "index.handler"
  runtime          = var.runtime
  publish          = true
  source_code_hash = filebase64sha256(var.filename)
  memory_size      = var.memory_size
  timeout          = var.timeout
  environment {
    variables = var.environment_variables
  }
}

Even when there is no change in lambda function still terraform plan/apply shows and applies changes in zip file and tries to redeploy lambda

Expected Behavior

No change should be applied to lambda function if there is no change in lambda function code

Actual Behavior

~ resource "aws_lambda_function" "lambda" {
        id                             = "***************"
      ~ last_modified                  = "2022-03-08T08:45:32.000+0000" -> (known after apply)
      ~ qualified_arn                  = "*******" -> (known after apply)
      ~ source_code_hash               = "4OqR0j99yrVKKzaTV++/nDb7Zeo2zu0dkAer8vVT9Pb4=" -> "j2hxf3m/7tvFCdyx5N7JZ3abGihWekp4kMhLwRnuBaT8="
        tags                           = {}
      ~ version                        = "1318" -> (known after apply)
        # (16 unchanged attributes hidden)
        # (2 unchanged blocks hidden)
    }
        # module.console_auth_aad_search_get_lambda.aws_lambda_function.lambda will be updated in-place
  ~ resource "aws_lambda_function" "lambda" {
        id                             = "***************"
      ~ last_modified                  = "2022-03-08T08:45:46.000+0000" -> (known after apply)
      ~ qualified_arn                  = "***************" -> (known after apply)
      ~ source_code_hash               = "wsPHQGqSEeJnbQnmg8/RtbeVUwkeQJrw1A0HofhsKr6CA=" -> "StcGIIxJqmMHGZiFXl3nfHSYhlqiYLFvEQGqy6LCJOQ="
        tags                           = {}
      ~ version                        = "143" -> (known after apply)
        # (16 unchanged attributes hidden)
        # (2 unchanged blocks hidden)
    }

[jbardin]

Hi @PaulF2022-55,

The plan indicates that the source code has changed since the last apply, reflected in the change to the source_code_hash attribute. Can you show a minimal complete example to reproduce the issue?

Thanks!

[PaulF2022-55]

Hi @jbardin ,

our problem is that even if there is no change in code terraform is showing change in zip file hash.

Steps to reproduce

1. yarn build (this builds my application zip file )
2. terraform plan
3. terraform apply

  ~ resource "aws_lambda_function" "lambda" {
        id                             = "********"
      ~ last_modified                  = "2022-03-22T06:19:00.722+0000" -> (known after apply)
      ~ qualified_arn                  = "**********" -> (known after apply)
      ~ source_code_hash               = "6e5I+6+8q5MP5807W51FOztLLcMtIAdeB+pAMefDkgk=" -> "e9ee48fbafbcab930fe7cd3b5b9d453b3b4b2dc32d20075e07ea4031e7c39209"
        tags                           = {}
      ~ version                        = "1" -> (known after apply)
        # (16 unchanged attributes hidden)
        # (2 unchanged blocks hidden)
    }

4. terraform apply (This still shows changes in zip file hash)

  # module.submissions_lambda.aws_lambda_function.lambda will be updated in-place
  ~ resource "aws_lambda_function" "lambda" {
        id                             = "*********"
      ~ last_modified                  = "2022-03-22T07:21:56.000+0000" -> (known after apply)
      ~ qualified_arn                  = "**************" -> (known after apply)
      ~ source_code_hash               = "6e5I+6+8q5MP5807W51FOztLLcMtIAdeB+pAMefDkgk=" -> "e9ee48fbafbcab930fe7cd3b5b9d453b3b4b2dc32d20075e07ea4031e7c39209"
        tags                           = {}
      ~ version                        = "1" -> (known after apply)
        # (16 unchanged attributes hidden)
        # (2 unchanged blocks hidden)
    }

[jbardin]

Hi @PaulF2022-55,

Thanks for the additional details. The fact that the second apply operation is making the exact same change as the first implies there is an error in the configuration (or possibly a misbehaving resource) causing module.submissions_lambda.aws_lambda_function.lambda to get a stale value during the first apply.

In order to troubleshoot this further, we are going to need a minimal, complete example to reproduce the issue. Can you supply the configuration you are using to generate these plans?

[PaulF2022-55]

Hi @jbardin
Below is the the entire tf file you requested. It is the file I use to create the lambda function. Hope it helps you to figure out the issue. Please let me know if you need anything.

resource "aws_lambda_function" "lambda" {
  filename         = var.filename
  function_name    = var.function_name
  role             = aws_iam_role.lambda.arn
  handler          = "index.handler"
  runtime          = var.runtime
  publish          = true
  source_code_hash = filebase64sha256(var.filename)
  memory_size      = var.memory_size
  timeout          = var.timeout
  environment {
    variables = var.environment_variables
  }
}

resource "aws_iam_role" "lambda" {
  name = "${var.function_name}-lambda"
  assume_role_policy = jsonencode({
    Version : "2012-10-17",
    Statement : [
      {
        Action : "sts:AssumeRole",
        Principal : { Service : "lambda.amazonaws.com" },
        Effect : "Allow",
        Sid : ""
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "basic_execution_role" {
  role       = aws_iam_role.lambda.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "permissions" {
  count = var.iam_permissions == null ? 0 : 1
  name  = "Permissions"
  role  = aws_iam_role.lambda.name
  policy = jsonencode(
    {
      Version : "2012-10-17",
      Statement : var.iam_permissions
  })
}

resource "aws_lambda_permission" "api" {
  count         = length(var.api_execute_arns)
  statement_id  = "AllowExecutionFromAPIGateway_${count.index}"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = var.api_execute_arns[count.index]
}

resource "aws_lambda_permission" "cognito" {
  count         = var.cognito_integration ? 1 : 0
  statement_id  = "AllowExecutionFromCognito"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda.function_name
  principal     = "cognito-idp.amazonaws.com"
  source_arn    = var.cognito_user_pool_arn
}

resource "aws_cloudwatch_log_group" "lambda" {
  name = "/aws/lambda/${aws_lambda_function.lambda.function_name}"
}

[jbardin]

Hi @PaulF2022-55,

Thanks for the added info. This does not look like an issue with Terraform, since the source_code_hash attribute is being updated but the provider is returning the old value for some reason. I found a similar issue in the aws provider repository, which is probably due to the same root cause: hashicorp/terraform-provider-aws#17989

I'm going to close the issue here, and we can follow the aws issue for updates.

Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.