Cannot run plan without setting envvars.

[rquadling]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.24
+ provider.aws v2.56.0
+ provider.cloudflare v2.5.1
+ provider.github v2.6.1
+ provider.null v2.1.2
+ provider.random v2.2.1
+ provider.template v2.1.2

Affected Resource(s)

Credential failure.

Terraform Configuration Files

terraform {
  backend "s3" {
    bucket         = "management-state-bucket"
    acl            = "private"
    encrypt        = true
    region         = "eu-west-1"
    dynamodb_table = "terraform_locks"
    key            = "devops.tfstate"
    profile        = "terraformer"
  }
}

provider "aws" {
  profile = "terraformer"
  region  = "eu-west-1"
  assume_role {
    role_arn = "arn:aws:iam::SNIP:role/OrganizationAccountAccessRole"
  }
}

provider "aws" {
  alias   = "use1"
  profile = "terraformer"
  region  = "us-east-1"
  assume_role {
    role_arn = "arn:aws:iam::SNIP:role/OrganizationAccountAccessRole"
  }
}

My ~/.aws/credentials has ...

[terraformer]
aws_secret_key_id=AKIA****VT
aws_secret_access_key=j****Z

My ~/.aws/config has ...

[profile terraformer]
region = eu-west-1

Debug Output

$ TF_LOG=TRACE terraform init
2020/04/14 16:47:59 [INFO] Terraform version: 0.12.24  
2020/04/14 16:47:59 [INFO] Go runtime version: go1.12.13
2020/04/14 16:47:59 [INFO] CLI args: []string{"/usr/local/Cellar/tfenv/1.0.2/versions/0.12.24/terraform", "init"}
2020/04/14 16:47:59 [DEBUG] Attempting to open CLI config file: /Users/richardquadling/.terraformrc
2020/04/14 16:47:59 Loading CLI configuration from /Users/richardquadling/.terraformrc
2020/04/14 16:47:59 [INFO] CLI command args: []string{"init"}
2020/04/14 16:47:59 [TRACE] ModuleInstaller: installing child modules for . into .terraform/modules
Initializing modules...
2020/04/14 16:47:59 [DEBUG] Module installer: begin docs_website
2020/04/14 16:47:59 [TRACE] ModuleInstaller: Module installer: docs_website <nil> already installed in modules/cf_s3_website
2020/04/14 16:47:59 [TRACE] modsdir: writing modules manifest to .terraform/modules/modules.json

Initializing the backend...
2020/04/14 16:47:59 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value 4080641221
2020/04/14 16:47:59 [TRACE] Preserving existing state lineage "2c911f2d-2ddb-db78-022a-8c59f40ecfe9"
2020/04/14 16:47:59 [TRACE] Preserving existing state lineage "2c911f2d-2ddb-db78-022a-8c59f40ecfe9"
2020/04/14 16:47:59 [TRACE] Meta.Backend: working directory was previously initialized for "s3" backend
2020/04/14 16:47:59 [TRACE] Meta.Backend: using already-initialized, unchanged "s3" backend configuration
2020/04/14 16:47:59 [INFO] Setting AWS metadata API timeout to 100ms
2020/04/14 16:48:00 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2020/04/14 16:48:00 [INFO] Attempting to use session-derived credentials

Error: No valid credential sources found for AWS Provider.
	Please see https://terraform.io/docs/providers/aws/index.html for more information on
	providing credentials for the AWS Provider

✘-1 2020-04-14 16:49:15 richardquadling@Richards-MacBook-Pro:~/dev/inf-devops $  [master|✚ 1]$ 

Expected Behavior

It to work.

Actual Behavior

The above error.

Steps to Reproduce

1. terraform init

Important Factoids

My state files for all of my accounts is in a separate account that terraformer has access to, which is why the providers have an role_arn.

If I run AWS_ACCESS_KEY=AKIA****VT AWS_SECRET_ACCESS_KEY=j****Z terraform init things work.

Adding the TF_LOG=TRACE to that command and I see ...

2020/04/14 16:53:29 [INFO] Terraform version: 0.12.24  
2020/04/14 16:53:29 [INFO] Go runtime version: go1.12.13
2020/04/14 16:53:29 [INFO] CLI args: []string{"/usr/local/Cellar/tfenv/1.0.2/versions/0.12.24/terraform", "init"}
2020/04/14 16:53:29 [DEBUG] Attempting to open CLI config file: /Users/richardquadling/.terraformrc
2020/04/14 16:53:29 Loading CLI configuration from /Users/richardquadling/.terraformrc
2020/04/14 16:53:29 [INFO] CLI command args: []string{"init"}
2020/04/14 16:53:29 [TRACE] ModuleInstaller: installing child modules for . into .terraform/modules
Initializing modules...
2020/04/14 16:53:29 [DEBUG] Module installer: begin docs_website
2020/04/14 16:53:29 [TRACE] ModuleInstaller: Module installer: docs_website <nil> already installed in modules/cf_s3_website
2020/04/14 16:53:29 [TRACE] modsdir: writing modules manifest to .terraform/modules/modules.json

Initializing the backend...
2020/04/14 16:53:29 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value 4080641221
2020/04/14 16:53:29 [TRACE] Preserving existing state lineage "2c911f2d-2ddb-db78-022a-8c59f40ecfe9"
2020/04/14 16:53:29 [TRACE] Preserving existing state lineage "2c911f2d-2ddb-db78-022a-8c59f40ecfe9"
2020/04/14 16:53:29 [TRACE] Meta.Backend: working directory was previously initialized for "s3" backend
2020/04/14 16:53:29 [TRACE] Meta.Backend: using already-initialized, unchanged "s3" backend configuration
2020/04/14 16:53:29 [INFO] Setting AWS metadata API timeout to 100ms
2020/04/14 16:53:30 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2020/04/14 16:53:30 [INFO] AWS Auth provider used: "EnvProvider"
2020/04/14 16:53:30 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2020/04/14 16:53:30 [DEBUG] [aws-sdk-go] DEBUG: Request sts/GetCallerIdentity Details:
...

If I use AWS_PROFILE=terraformer with or without the AWS_SDK_LOAD_CONFIG envvar, still no joy.

I must be missing something really obvious but I've no idea what it is.

I can run aws CLI no problem with the --profile terraformer option.

On the surface it looks like the profile named in the backend and providers is not being used and/or the credentials file is not being used.

[matthewhembree]

Using AWS_PROFILE with AWS_SHARED_CREDENTIALS_FILE exhibits the same issues.

[cajund]

Hoping this will help narrow it down:

provider "aws" {
  region = var.aws_region[terraform.workspace]
  profile = var.aws_profile[terraform.workspace]
}

The profile is picked up when using local state, but when transferring state to S3, the env var is required. In other words, I was able to create the S3 and Dynamo DB table required for state management with no problem, but moving the state to S3 via terraform init gave the ResourceNotFound error.

Thanks,

[bflad]

Hi folks 👋 The issues reported here with the Terraform S3 Backend should be resolved as of Terraform 0.13.0-beta2, including the errant error messaging saying there is an issue with the "Terraform AWS Provider", which was previously hardcoded into the shared authentication code between the Terraform S3 Backend and Terraform AWS Provider. If you are still having trouble after trying that version or later of Terraform, please do note that we will be applying very similar improvements and bug fixes to the Terraform AWS Provider in version 3.0.0, releasing in a few weeks.

[bflad]

The authentication changes applied to the Terraform S3 Backend as part of Terraform CLI 0.13.0-beta2 have also been merged into the Terraform AWS Provider and will release with version 3.0.0, likely in the next two weeks. Please follow the v3.0.0 milestone for tracking the progress of that release. If you are still having trouble after updating when its released, please file a new issue. Thanks!

[ghost]

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!