AWS Provider in 0.4.0- Credentials can't be used in modules #1380

zxjinn · 2015-04-03T15:52:46Z

Hey guys, I appreciate all your hard work. 0.4.0 looks like a really great release! I downloaded it excitedly yesterday and ran into an issue I couldn't resolve. So I made a tiny POC to show the problem. I hope I'm not missing something really small and dumb here :person_frowning:.

File structure and contents

tree

$ tree
.
├── awsprovider-poc.tf
└── modules
    └── awsprovider-poc
        └── main.tf
2 directories, 2 files

awsprovider-poc.tf

variable "module_access_key" {}
variable "module_region" {}
variable "module_secret_key" {}
module "poc" {
  source     = "modules/awsprovider-poc"
  access_key = "${var.module_access_key}"
  region     = "${var.module_region}"
  secret_key = "${var.module_secret_key}"
}

modules/awsprovider-poc/main.tf

variable "access_key" {}
variable "secret_key" {}
variable "region" {}
provider "aws" {
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  region     = "${var.region}"
}
resource "aws_vpc" "040-awsprovider-poc" {
    cidr_block = "10.0.0.0/16"
    tags {
        Name = "0.4.0-awsprovider-poc"
    }
}

Using 0.3.7

$ terraform get
Get: file:///Users/<username>/code/terraform/awsprovider-poc/modules/awsprovider-poc
$ terraform plan -var 'module_access_key=<actual key>' -var 'module_secret_key=<actual secret>' -var 'module_region=us-west-2' -module-depth=1 -refresh=true
Refreshing Terraform state prior to plan...
...
+ module.poc.aws_vpc.040-awsprovider-poc
    cidr_block:                "" => "10.0.0.0/16"
    default_network_acl_id:    "" => "<computed>"
    default_security_group_id: "" => "<computed>"
    enable_dns_hostnames:      "" => "<computed>"
    enable_dns_support:        "" => "<computed>"
    main_route_table_id:       "" => "<computed>"
    tags.#:                    "" => "1"
    tags.Name:                 "" => "0.4.0-awsprovider-poc"

So everything works as expected. Note that this works if I put the key, region, and secret key in all possible locations: in terraform.tfvars, pass it on the command line (as in the example), hard code it in the aws-provider.tf file, or hard code it in the modules/aws-provider-poc/main.tf file.

Using 0.4.0

$ terraform get
Get: file:///Users/<username>/code/terraform/awsprovider-poc/modules/awsprovider-poc
$ terraform plan -var 'module_access_key=<actual key>' -var 'module_secret_key=<actual secret>' -var 'module_region=us-west-2' -module-depth=1 -refresh=true
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * provider.aws: "access_key": required field is not set
  * provider.aws: "secret_key": required field is not set
  * provider.aws: "region": required field is not set

Just the opposite of 0.3.7: this does not work if I put the variables in all possible locations: using terraform.tfvars (in the root of the folder, not in the modules folder), passing it on the command line (as in the example), hard coding the values in aws-provider-poc.tf, or hard coding them in the modules/aws-provider-poc/main.tf file. I made a gist showing the output with TF_LOG=1. Thanks guys, Terraform is amazing!

The text was updated successfully, but these errors were encountered:

phinze · 2015-04-03T15:59:00Z

Hey there, sorry for the trouble, and thanks for the detailed report!

Let me take a look and see what's up.

phinze · 2015-04-03T16:08:45Z

Hmm everything seems to be working fine from my side. Let me detail out my environment / process and hopefully we can figure out where the discrepancy is.

$ tree
.
├── awsprovider-poc.tf
└── modules
    └── awsprovider-poc
        └── main.tf

2 directories, 2 files

$ cat variable "module_access_key" {}
variable "module_region" {}
variable "module_secret_key" {}
module "poc" {
  source     = "modules/awsprovider-poc"
  access_key = "${var.module_access_key}"
  region     = "${var.module_region}"
  secret_key = "${var.module_secret_key}"
}

$ cat modules/awsprovider-poc/main.tf
variable "access_key" {}
variable "secret_key" {}
variable "region" {}
provider "aws" {
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  region     = "${var.region}"
}
resource "aws_vpc" "040-awsprovider-poc" {
    cidr_block = "10.0.0.0/16"
    tags {
        Name = "0.4.0-awsprovider-poc"
    }
}

$ terraform version
Terraform v0.4.0

$ terraform plan -var "module_access_key=$AWS_ACCESS_KEY_ID" -var "module_secret_key=$AWS_SECRET_ACCESS_KEY" -var "module_region=$AWS_REGION" -module-depth=1 -refresh=true
Refreshing Terraform state prior to plan...


The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

+ module.poc.aws_vpc.040-awsprovider-poc
    cidr_block:                "" => "10.0.0.0/16"
    default_network_acl_id:    "" => "<computed>"
    default_security_group_id: "" => "<computed>"
    enable_dns_hostnames:      "" => "<computed>"
    enable_dns_support:        "" => "<computed>"
    main_route_table_id:       "" => "<computed>"
    tags.#:                    "" => "1"
    tags.Name:                 "" => "0.4.0-awsprovider-poc"

See any differences between that and your environment?

phinze · 2015-04-03T17:33:23Z

Figured it out! I had env variables set that the provider was using.

Unsetting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY lets me reproduce the issue. Digging in now! 👍

(This also means that setting those env vars should function as a workaround for you.)

zxjinn · 2015-04-03T17:36:28Z

I just deleted everything I could find related to terraform:

0.3.7 version in my PATH
0.4.0 version in my Downloads folder
pre-0.4.0 version I'd compiled for testing (it also didn't work, it was version dc4abb4)
~/.terraform.d directory

I then re-downloaded/reinstalled the bins for 0.4.0 from the download site then put them in my PATH.

I then copy/pasted your output and made a new directory with your output in it (for absolute/complete separation) and got the same output as before 😦

I moved over to a co-workers machine and had him try out the same POC with the same code, his output is in my updated gist https://gist.github.com/zxjinn/3bf821620e784ac709bb#file-coworker

Notable facts about my system (from facter):

operatingsystem => Darwin
operatingsystemrelease => 14.1.0
macosx_productname => Mac OS X
macosx_productversion => 10.10.2
macosx_productversion_major => 10.10
productname => MacBookPro11,3

I even compiled the latest 0.4.1 version 0cff04a and still have the same problem.

phinze · 2015-04-03T17:39:05Z

Wow - thanks for putting in all that work to help isolate the issue! Now it's my turn to work on getting it solved for you. 😀

zxjinn · 2015-04-03T17:42:59Z

I just tried the environment variable setting and unfortunately it did work for me! Glad I could assist in helping you find this. I'm digging into it so much because most of what I am working on today relies on Terraform and I wanted to use the newest version!

$ export AWS_REGION=us-west-2
$ export AWS_ACCESS_KEY_ID=<actual key>
$ export AWS_SECRET_ACCESS_KEY=<actual secret>
terraform plan -var "module_access_key=$AWS_ACCESS_KEY_ID" -var "module_secret_key=$AWS_SECRET_ACCESS_KEY" -var "module_region=$AWS_REGION" -module-depth=1 -refresh=true
...
+ module.poc.aws_vpc.040-awsprovider-poc
    cidr_block:                "" => "10.0.0.0/16"
    default_network_acl_id:    "" => "<computed>"
    default_security_group_id: "" => "<computed>"
    enable_dns_hostnames:      "" => "<computed>"
    enable_dns_support:        "" => "<computed>"
    main_route_table_id:       "" => "<computed>"
    tags.#:                    "" => "1"
    tags.Name:                 "" => "0.4.0-awsprovider-poc"

$ terraform version
Terraform v0.4.1-dev (0cff04a37ffa89db8f24b338c292e5630768dafe)

zxjinn · 2015-04-03T17:43:31Z

s/unfortunately/fortunately

mitchellh · 2015-04-03T19:40:19Z

@phinze Nothing is popping out as an exact solution for this for me, but it is almost certainly in the graph config merge logic stuff, which is reasonably advanced. Some pointers of where to look:

Print out the config that is being merged/seen in evaltree_provider.go
Figure out if the module variables are interpolating properly
It might be the case that for some reason the module variables are just blank, and so the provider is being configured with blank. I don't know if this is the case, but it could be.
This is a really easy context test, we should just make it a test case. Use p.ValidateFn and config.CheckSet(["foo"]) to validate the fields are set, check the value to check non-empty values.

The only tingling I'm getting is that I'm really confident this is a subgraph variable merge issue. The way that works:

Context function is called to set module variables/interpolate
Get the module variables for interpolation
Provider config is executed, merges parent configs (this is that new eval node I made recently)
Provider config is done/validated

It is somewhere in there, there is a lot of data flowing so somethign is happening.

phinze · 2015-04-03T19:47:01Z

Thanks for the pointers @mitchellh - time to dive in. 🏊

solarce · 2015-04-06T23:36:17Z

Looks like I am running into this as well, when trying to test out our zookeeper template with the changes I made to solarce/tf_aws_asg@9485be2 to use split()

I normally have the AWS_* stuff set in my environment, except I don't set a REGION by default, so when I unset AWS_ACCESS_KEY and AWS_SECRET_KEY I begin to get the same errors as previously reported

thq-d-bburt01 福 ~/code/lookout/terraform/tf_zookeeper ➤ e52d8c0|testing-split-bburton⚡
6218 ± : unset AWS_ACCESS_KEY

thq-d-bburt01 福 ~/code/lookout/terraform/tf_zookeeper ➤ e52d8c0|testing-split-bburton⚡
6219 ± : terraform plan -var-file environments/terraform-bburton.tfvars --module-depth=1
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * provider.aws: "region": required field is not set
  * provider.aws: "access_key": required field is not set

thq-d-bburt01 福 ~/code/lookout/terraform/tf_zookeeper ➤ e52d8c0|testing-split-bburton⚡
6220 ± : unset AWS_SECRET_KEY

thq-d-bburt01 福 ~/code/lookout/terraform/tf_zookeeper ➤ e52d8c0|testing-split-bburton⚡
6221 ± : terraform plan -var-file environments/terraform-bburton.tfvars --module-depth=-1
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * provider.aws: "access_key": required field is not set
  * provider.aws: "secret_key": required field is not set
  * provider.aws: "region": required field is not set

Additionally, doing export AWS_REGION="us-west-2" gets me back in business for the time being and I've shared this with my team

johnrengelman · 2015-04-09T13:51:05Z

I'm encountering this same issue if I try to delete a module from a plan that's already been applied. Running terraform plan results in

terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * provider.aws: "region": required field is not set

phinze · 2015-04-09T14:05:11Z

Hey folks - just landed what should be the fix for this. If you have the ability to build from master, it'd be great if you could take it for a spin to double check that your issue is solved. Either way, it should come out in v0.4.1 this week.

johnrengelman · 2015-04-09T14:11:00Z

Still getting the error in my situation.

//main.tf

provider "aws" {
  region = "us-east-1"
}

module "child" {
  name = "foo"
}

//child/child.tf
resource "aws_instance" "server" {
 //....
}

$ terraform apply
....

Then comment out module "child" in main.tf and run terraform plan...

 $GOPATH/bin/terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * provider.aws: "region": required field is not set

My system:

$GOPATH/bin/terraform --version
Terraform v0.4.1-dev (c0196d948d942a32a46fb9ef203076d3be134e4d)

phinze · 2015-04-09T14:12:46Z

@johnrengelman Thanks for double checking. Will take a look at that scenario and follow up.

johnrengelman · 2015-04-09T14:19:12Z

Yup. Do you want a separate ticket?

phinze · 2015-04-09T14:20:40Z

@johnrengelman actually yeah that'd be great! 👍

johnrengelman · 2015-04-09T14:53:09Z

Moved to #1447

zxjinn · 2015-04-09T17:23:16Z

Thanks a lot guys, just pulled the latest master and it appears as if it's fixed. With the same TF config as above:

$ terraform plan -var 'module_access_key=<actual key>' -var 'module_secret_key=<actual secret>' -var 'module_region=us-west-2' -module-depth=1 -refresh=true -out=out.tfplan
...
+ module.poc.aws_vpc.040-awsprovider-poc
    cidr_block:                "" => "10.0.0.0/16"
    default_network_acl_id:    "" => "<computed>"
    default_security_group_id: "" => "<computed>"
    enable_dns_hostnames:      "" => "<computed>"
    enable_dns_support:        "" => "<computed>"
    main_route_table_id:       "" => "<computed>"
    tags.#:                    "" => "1"
    tags.Name:                 "" => "0.4.0-awsprovider-poc"

$ terraform apply out.tfplan
module.poc.aws_vpc.040-awsprovider-poc: Creating...
  cidr_block:                "" => "10.0.0.0/16"
  default_network_acl_id:    "" => "<computed>"
  default_security_group_id: "" => "<computed>"
  enable_dns_hostnames:      "" => "<computed>"
  enable_dns_support:        "" => "<computed>"
  main_route_table_id:       "" => "<computed>"
  tags.#:                    "" => "1"
  tags.Name:                 "" => "0.4.0-awsprovider-poc"
module.poc.aws_vpc.040-awsprovider-poc: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

$ terraform plan -var 'module_access_key=<actual key>' -var 'module_secret_key=<actual secret>' -var 'module_region=us-west-2' -module-depth=1 -refresh=true -out=out.tfplan -destroy
Refreshing Terraform state prior to plan...

module.poc.aws_vpc.040-awsprovider-poc: Refreshing state... (ID: vpc-93eb53f6)
...
Path: out.tfplan

- module.poc.aws_vpc.040-awsprovider-poc

$ terraform apply out.tfplan
aws_vpc.040-awsprovider-poc: Destroying...
aws_vpc.040-awsprovider-poc: Destruction complete

Apply complete! Resources: 0 added, 0 changed, 1 destroyed.

Funny enough $ terraform --version returns Terraform v0.4.1 looks like I pulled at the right time!

Thanks again everyone! 👏 🎆

jtopper · 2015-04-09T18:09:32Z

I've also pulled and tried this today because I hit a similar bug. My use case is that I want to use different AWS credentials inside >1 invocation of a module, in order to create resources across multiple Amazon accounts in one run of the tool, and manage ordering between them.

If the module tries to create just one resource (eg. a VPC), this works fine. If I also try to create multiple subnets, these all fail with

* Error creating subnet: could not find credentials configuration.
module.test_vpc.aws_subnet.public_b: Error: 1 error(s) occurred:

I'm happy to provide you with a failing use case, or more details.

mitchellh · 2015-04-09T18:21:29Z

@jtopper Can you open another issue with a full repro case? I'd be happy to take a look.

adelamarre · 2015-05-22T08:44:09Z

Hi, i run into the same issue. Here is my context :

Terraform v0.5.2
ProductName:    Mac OS X
ProductVersion: 10.10.3
BuildVersion:   14D136

and here is the console output when i run : terraform plan

There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * provider.aws: "secret_key": required field is not set
  * provider.aws: "region": required field is not set
  * provider.aws: "access_key": required field is not set

maint.tf

module "network" {
    source = "../modules/network"
    aws_region      = "${var.aws_region}"
    aws_access_key  = "${var.aws_access_key}"
    aws_secret_key  = "${var.aws_secret_key}"
}
...

variables.tf

variable "aws_region" {}
variable "aws_secret_key" {}
variable "aws_access_key" {}

terraform.tfvars

aws_region      = "eu-west-1"
aws_access_key  = "+++++++++++++++"
aws_secret_key  = "+++++++++++++++++++++++++++++++"

../modules/network/main.tf

provider "aws" {
    access_key  = "${var.aws_access_key}"
    secret_key  = "${var.aws_secret_key}"
    region      = "${var.aws_region}"
}

../modules/network/variables.tf

variable "aws_region" {}
variable "aws_secret_key" {}
variable "aws_access_key" {}

phinze · 2015-05-26T18:10:45Z

Thanks for the report @adelamarre - reopening so we can recheck this.

phinze · 2015-05-27T11:39:21Z

@adelamarre just followed your instructions to set up the same scenario locally (both with 0.5.2 and master), and everything seems to be working AOK here.

Perhaps you can double check on your side?

zxjinn · 2015-05-27T23:36:54Z

@phinze @adelamarre I just followed your instructions with v0.5.0, an old compiled version of v0.5.1-dev I had from some previous testing (72afc6b) and the latest master (88f72aa) and all of them were just fine for me.
Not sure what to tell you 😦

phinze · 2015-05-27T23:40:07Z

Thanks @zxjinn! Okay with two instances of it working I'm going re-close this - @adelamarre can you open up a new issue if you are still seeing a problem?

ghost · 2020-05-02T02:21:00Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

phinze added bug provider/aws labels Apr 3, 2015

phinze self-assigned this Apr 3, 2015

phinze mentioned this issue Apr 3, 2015

Google Cloud Provider attributes vs. modules usage in 0.4.0 #1375

Closed

mitchellh mentioned this issue Apr 9, 2015

terraform: don't validate providers outside of modules if we don't have to #1437

Merged

phinze closed this as completed in #1437 Apr 9, 2015

johnrengelman mentioned this issue Apr 9, 2015

AWS provider configuration not available in delete module #1447

Closed

mitchellh mentioned this issue Apr 9, 2015

terraform: don't prune, but disable, inherited configs [GH-1447] #1452

Merged

jtopper mentioned this issue Apr 9, 2015

Using multiple AWS credentials across modules failing #1461

Closed

phinze reopened this May 26, 2015

phinze closed this as completed May 27, 2015

8interactive mentioned this issue Jun 23, 2015

AWS Provider in v0.5.3 - Credentials can't be used in modules #2445

Closed

ghost locked and limited conversation to collaborators May 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AWS Provider in 0.4.0- Credentials can't be used in modules #1380

AWS Provider in 0.4.0- Credentials can't be used in modules #1380

zxjinn commented Apr 3, 2015

phinze commented Apr 3, 2015

phinze commented Apr 3, 2015

phinze commented Apr 3, 2015

zxjinn commented Apr 3, 2015

phinze commented Apr 3, 2015

zxjinn commented Apr 3, 2015

zxjinn commented Apr 3, 2015

mitchellh commented Apr 3, 2015

phinze commented Apr 3, 2015

solarce commented Apr 6, 2015

johnrengelman commented Apr 9, 2015

phinze commented Apr 9, 2015

johnrengelman commented Apr 9, 2015

phinze commented Apr 9, 2015

johnrengelman commented Apr 9, 2015

phinze commented Apr 9, 2015

johnrengelman commented Apr 9, 2015

zxjinn commented Apr 9, 2015

jtopper commented Apr 9, 2015

mitchellh commented Apr 9, 2015

adelamarre commented May 22, 2015

phinze commented May 26, 2015

phinze commented May 27, 2015

zxjinn commented May 27, 2015

phinze commented May 27, 2015

ghost commented May 2, 2020

AWS Provider in 0.4.0- Credentials can't be used in modules #1380

AWS Provider in 0.4.0- Credentials can't be used in modules #1380

Comments

zxjinn commented Apr 3, 2015

File structure and contents

tree

awsprovider-poc.tf

modules/awsprovider-poc/main.tf

Using 0.3.7

Using 0.4.0

phinze commented Apr 3, 2015

phinze commented Apr 3, 2015

phinze commented Apr 3, 2015

zxjinn commented Apr 3, 2015

phinze commented Apr 3, 2015

zxjinn commented Apr 3, 2015

zxjinn commented Apr 3, 2015

mitchellh commented Apr 3, 2015

phinze commented Apr 3, 2015

solarce commented Apr 6, 2015

johnrengelman commented Apr 9, 2015

phinze commented Apr 9, 2015

johnrengelman commented Apr 9, 2015

phinze commented Apr 9, 2015

johnrengelman commented Apr 9, 2015

phinze commented Apr 9, 2015

johnrengelman commented Apr 9, 2015

zxjinn commented Apr 9, 2015

jtopper commented Apr 9, 2015

mitchellh commented Apr 9, 2015

adelamarre commented May 22, 2015

phinze commented May 26, 2015

phinze commented May 27, 2015

zxjinn commented May 27, 2015

phinze commented May 27, 2015

ghost commented May 2, 2020