Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terraform with local AWS services #3608

Closed
groeney opened this issue Mar 2, 2018 · 17 comments
Closed

Terraform with local AWS services #3608

groeney opened this issue Mar 2, 2018 · 17 comments
Labels
enhancement Requests to existing resources that expand the functionality or scope. provider Pertains to the provider itself, rather than any interaction with AWS.

Comments

@groeney
Copy link

groeney commented Mar 2, 2018

Hi there,

Terraform Version

Terraform v0.11.2
+ provider.aws v1.10.0

Affected Resource(s)

Everything

Terraform Configuration Files

provider "aws" {
  region = "ap-southeast-2"

  endpoints {
    dynamodb = "http://localhost:4569/"
  }
}

resource "aws_dynamodb_table" "poc_table" {
  name = "${terraform.workspace}.ProofOfConcept"
  read_capacity = 5
  write_capacity = 5
  hash_key = "UserId"

  attribute {
    name = "UserId"
    type = "S"
  }
}

OR

provider "aws" {
  access_key = "123"
  secret_key = "123"
  region = "ap-southeast-2"

  endpoints {
    dynamodb = "http://localhost:4569/"
  }
}

resource "aws_dynamodb_table" "poc_table" {
  name = "${terraform.workspace}.ProofOfConcept"
  read_capacity = 5
  write_capacity = 5
  hash_key = "UserId"

  attribute {
    name = "UserId"
    type = "S"
  }
}

Debug Output

https://gist.github.com/j-groeneveld/b0af35f4f7ab518bbb8cd413b3281e3e
OR
https://gist.github.com/j-groeneveld/43aa1df229e50ef8e06e64028351f2a9

Expected Behavior

Terraform ignores missing credentials and uses the local Dynamo DB endpoint http://localhost:4569.

Actual Behavior

Terraform complains about invalid or incorrect credentials.

Steps to Reproduce

  1. terraform plan

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

  • 1059 -- related to a different issue but notice that the aws creds are missing in the config fie. For anyone else that is running into Terraform on local issues I am also experiencing issues DescribeTimeToLive and ListTagsOfResource which should be fixed with recent localstack PR #599.

I have seen multiple examples of missing or bogus credentials for the aws provider. Perhaps credentials are now required in a new Terraform release but I haven't seen anything explicitly addressing this change in requirement.

@bflad bflad added enhancement Requests to existing resources that expand the functionality or scope. provider Pertains to the provider itself, rather than any interaction with AWS. labels Mar 5, 2018
@theladyjaye
Copy link

Using this with localstack as well I don't seem to hit this error

Terraform v0.11.3
+ provider.aws v1.10.0

Not to say that it doesn't exist, but I'm very cautious now after reading this issue.

@groeney
Copy link
Author

groeney commented Mar 6, 2018

This issue has nothing to do with localstack per se. The issue arrises for me whether I am using localstack or not -- but for the sake of this issue we should disregard localstack altogether.

Do you mind sharing your Terraform config config.tf and output from terraform plan?

@theladyjaye
Copy link

Provider Config

provider "aws" {
  region     = "us-west-2"

  endpoints {
        dynamodb = "http://localhost:4569"
    }
}

Test Resource:

resource "aws_dynamodb_table" "mytable" {
    name           = "mytable"
    read_capacity  = 5
    write_capacity = 5
    hash_key       = "Id"
    range_key      = "Timestamp"

    attribute {
        name = "Id"
        type = "S"
    }

    attribute {
        name = "Timestamp"
        type = "S"
    }
}

terrafrom plan

13:05 $ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_dynamodb_table.mytable
      id:                        <computed>
      arn:                       <computed>
      attribute.#:               "2"
      attribute.3292831115.name: "Id"
      attribute.3292831115.type: "S"
      attribute.423918437.name:  "Timestamp"
      attribute.423918437.type:  "S"
      hash_key:                  "Id"
      name:                      "mytable"
      range_key:                 "Timestamp"
      read_capacity:             "5"
      stream_arn:                <computed>
      stream_label:              <computed>
      stream_view_type:          <computed>
      write_capacity:            "5"


Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

@groeney
Copy link
Author

groeney commented Mar 6, 2018

Hmm odd. After replicating your exact configuration in a new dir and running a fresh terraform init:

13:25:46 ▶ terraform --version
Terraform v0.11.3
+ provider.aws v1.10.0

I still get the following error:

13:25:41 ▶ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

Error: Error running plan: 1 error(s) occurred:

* provider.aws: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

I'm not sure what is going on here.

@aventurella would you mind sharing the OS you are working on? Here is mine:

13:35:11 ▶ echo $OSTYPE
darwin16

Next step is probably to find where in the source code this error is being thrown -- I will try do this when I have a spare moment in the next week.

Appreciate the help so far :-)

@theladyjaye
Copy link

I don't know if this will help.. but I do have a ~/.aws dir from running aws configure with the awscli installed

@groeney
Copy link
Author

groeney commented Mar 6, 2018

Great.

Now can you confirm that the credentials under [default] section in ~/.aws/credentials are in fact valid credentials?

cat ~/.aws/credentials

Should yield something that looks like the following

...
[default]
aws_access_key_id = redacted
aws_secret_access_key = redacted
...

My main question here is to see whether dummy credentials work or whether in fact you need a valid AWS IAM for this to work.

@theladyjaye
Copy link

theladyjaye commented Mar 6, 2018

Bingo...

I just changed [default] -> [default-1] so it wouldn't find it.. Got your error.

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


Error: Error refreshing state: 1 error(s) occurred:

* provider.aws: No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

Additionally, if I suffix the credentials under [default] with 111 or anything to make them be not the right thing, it will also fail.

So it would seem terraform, when given a new url for an AWS resource, still does some preliminary check with the credentials. That would make sense in terms of, you may only have done 1 new url for an AWS service. I don't know why anyone would only do one. Nevermind, I do. I had a client once that had their own S3 implementation, but still could use AWS for other things.

@groeney
Copy link
Author

groeney commented Mar 8, 2018

Thanks @aventurella

For everyone else interested in debugging this I have created a test repo with the above setup -- https://github.com/j-groeneveld/terraform-providers-3608.

@iadknet
Copy link

iadknet commented Apr 17, 2018

@j-groeneveld I was just able to get this working by including skip_credentials_validation = true

variable "local_dynamo" {
  description = "Endpoint for local dynamodb"
  default = "http://localhost:4569"
}

variable "local_s3" {
  description = "Endpoint for local s3"
  default = "http://localhost:4572"
}

provider "aws" {
  region = "us-east-1"
  skip_credentials_validation = true
  endpoints {
    dynamodb = "${var.local_dynamo}"
    s3 = "${var.local_s3}"
  }
}

@Ghazgkull
Copy link

@iadknet Terraform doesn't appear to be honoring the skip_credential_validation flag. I've opened an Issue with details: hashicorp/terraform#18696

@iadknet
Copy link

iadknet commented Aug 16, 2018

@Ghazgkull Ah, I realize now why it was working for me.

I thought it was because I introduced the skip_credentials_validation flag, but I also happened to switch to using the S3 backend at about the same time, which requires valid credentials. D'oh.

You're right, if I remove the s3 backend, and do not inject valid credentials, then skip_credentials_validation is not honored.

@Ghazgkull
Copy link

Yeah, we went through the same mystery phase of "Why is it working for some people but not others???"

@cdowey-r7
Copy link

I have been able to work around this by also setting:
skip_requesting_account_id=true

@stefansundin
Copy link
Contributor

I am not sure if dynamodb-local is supposed to work without sending credentials. I can't get it to work with the aws cli either.

$ aws dynamodb list-tables --no-sign-request --endpoint-url http://localhost:8000

An error occurred (MissingAuthenticationToken) when calling the ListTables operation: Request must contain either a valid (registered) AWS access key ID or X.509 certificate.

And I think that error is coming from dynamodb-local, not the cli.

Also, removing the credentials from ~/.aws/credentials gives me this error (obviously from the cli):

Unable to locate credentials. You can configure credentials by running "aws configure".

Further, accessing dynamodb-local with different credentials (different AWS account ids I am guessing) gives me different "namespaces" of sorts. I can only list my dynamodb-local table using the aws cli if I use the same --profile that I configured terraform to use.

@Ghazgkull
Copy link

This issue should probably be resolved as a duplicate of #5584

That issue explains a functional workaround in the latest version of the terraform aws provider.

@groeney
Copy link
Author

groeney commented Nov 19, 2018

Resolved as a duplicate of #5584

@groeney groeney closed this as completed Nov 19, 2018
bflad added a commit that referenced this issue Mar 27, 2019
Historically, we have treated endpoint customization as an optional enhancement when adding service clients to the provider. Over time, the necessity of service endpoint customization has changed including:

* FIPS 140-2 compliant endpoints
* AWS C2S and SC2S potential usage
* AWS Snowball potential usage
* Local testing solutions such as LocalStack
* Rare incorrect default endpoint information provided by the AWS Go SDK

This change enables all existing service clients to support endpoint customization and going forward we can include this support by default. If the AWS Go SDK provides a better customization method in the future or defines environment variable support, we can revisit this in the future. We may also choose to further optimize endpoint handling via new provider configurations (e.g. use all known FIPS 140-2 endpoints).

Endpoint service naming is based on existing configuration. Where we do not have a prior configuration, we opt for using the AWS CLI service naming minus any hyphens. We also deprecate our own naming for `kinesis_analytics` and `r53` with `kinesisanalytics` and `route53` respectively for consistency.

References:

* https://aws.amazon.com/compliance/fips/
* https://aws.amazon.com/federal/us-intelligence-community/
* https://aws.amazon.com/snowball/
* https://docs.aws.amazon.com/cli/latest/index.html
* https://localstack.cloud/
* #7735
* #8007
* #7985
* #4967
* #4670
* #3941
* #3888
* #3608
pull bot pushed a commit to KernelPanicAUS/terraform-provider-aws that referenced this issue Apr 1, 2019
Historically, we have treated endpoint customization as an optional enhancement when adding service clients to the provider. Over time, the necessity of service endpoint customization has changed including:

* FIPS 140-2 compliant endpoints
* AWS C2S and SC2S potential usage
* AWS Snowball potential usage
* Local testing solutions such as LocalStack
* Rare incorrect default endpoint information provided by the AWS Go SDK

This change enables all existing service clients to support endpoint customization and going forward we can include this support by default. If the AWS Go SDK provides a better customization method in the future or defines environment variable support, we can revisit this in the future. We may also choose to further optimize endpoint handling via new provider configurations (e.g. use all known FIPS 140-2 endpoints).

Endpoint service naming is based on existing configuration. Where we do not have a prior configuration, we opt for using the AWS CLI service naming minus any hyphens. We also deprecate our own naming for `kinesis_analytics` and `r53` with `kinesisanalytics` and `route53` respectively for consistency.

References:

* https://aws.amazon.com/compliance/fips/
* https://aws.amazon.com/federal/us-intelligence-community/
* https://aws.amazon.com/snowball/
* https://docs.aws.amazon.com/cli/latest/index.html
* https://localstack.cloud/
* hashicorp#7735
* hashicorp#8007
* hashicorp#7985
* hashicorp#4967
* hashicorp#4670
* hashicorp#3941
* hashicorp#3888
* hashicorp#3608
@ghost
Copy link

ghost commented Apr 2, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 2, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement Requests to existing resources that expand the functionality or scope. provider Pertains to the provider itself, rather than any interaction with AWS.
Projects
None yet
Development

No branches or pull requests

7 participants