Support (and test functionality) for AWS alternatives?

[radeksimko]

It seems that more people are asking for support of AWS-like alternatives - some supported/provided directly by AWS, some not, e.g.:

• Eucalyptus (EC2 and probably a lot more)
• Hologram (Fake EC2 metadata API) - related: Fix IAM issue when using mock credential provider #5014
• Local DynamoDB and/or Fake Dynamo - related: add validate_credentials option to provider-aws #6533
• Fake SQS
• Fake SNS
• Fake S3
• EB Local

The obvious issue is that alternatives won't be as much battle-tested as real APIs hence some false positives/negatives may occur. If we decide to run nightly acceptance tests against such endpoints, these should be completely separated from current setup and fixing bugs in those should probably has lower priority than real endpoints - i.e. we don't want to break integration with real AWS API to make something work with an alternative.

I still think it's worth supporting these though because it can lower the bar for potential contributors wanting to contribute tested code as they will be able to test it against local endpoints which is not going to cost 💰 💰 in most cases. Yes, there might be false positives/negatives, but it's IMO still better than untested code.

I personally think that developers should have access to a real AWS environment and be able to create real resources (which may cost money, yes - but that's about trust and potential IAM limitations and/or account limits). There will always be services which cannot be "mocked" locally - e.g. ECS, Lambda, CloudFront, EFS, RDS, ... and alternative implementations will inevitably lack behind the real official API.

I'm opening this issue to track interest and also to gather opinions from other maintainers.

[rsrchboy]

This would be nice -- for remote state, if nothing else. e.g. I'm trying to use DreamHost Objects (S3) with the S3 remote state backend and failing miserably (and confusingly), even while other tools (e.g. s3cmd) work w/o issue.

[vrenjith]

We were already using terraform up to version 0.6.7 to provision in our inhouse EC2 compliant cloud with some restrictions on the regions that can be used. The latest version of terraform (0.7.0) has some additional checks with AWS which causes it to completely fail.

[radeksimko]

@vrenjith can you be a bit more specific about what checks are causing you troubles? (except the mentioned region name check)

[vrenjith]

@radeksimko This is what is happening with Terraform with the latest version 0.7.3

⋊> ~/p/S/mon-devlab-tr on master ⨯ terraform apply                                                                                                      
* RequestError: send request failed
caused by: Post https://sts.amazonaws.com/: dial tcp 54.239.29.25:443: i/o timeout

This was not happening till version 0.6.15

[vrenjith]

Just figured out that terraform has a new argument for provider to skip_credentials_validation and this solves the above issue (terraform trying to connect to https://sts.amazonaws.com/).

[thebrianlopez]

Was able to get this working by adding the following:

+provider "aws" {
+  skip_credentials_validation = true
+  max_retries = 1
+  skip_metadata_api_check = true
+}

[hamstah]

Hi,
I'm working on implementing a custom IAM/STS and I've run into this issue as well.
Only IAM can be overwritten in the config at the moment but it would be great to be able to change all the endpoints like we can in boto or awscli.

I have a patch to allow specifying a custom STS endpoint which I'm still testing but could send a PR for.

[vrenjith]

Have a custom build in case anyone needs to skip the Validation of regions (I just commented that validation function)
https://github.com/vrenjith/terraform/releases/tag/vCustom-0.1

[sgentzen]

I'm coming across this and it's potentially a showstopper for my terraform implementation. If there was a skip_region_verify or something similar so it trusts that I know what I'm doing with my region name, it would at least get me to the point where I can feed endpoints in.

[vrenjith]

Currently we are using a custom terraform with that function commented out.

[Cinderhaze]

I am in a similar boat. I have a rebuilt version of terraform with custom endpoints compiled into it, so the region validation passes, but it would be great if there was a skip_region_validation parameter. I also wish there was an eaiser way to set the CHECKPOINT_DISABLE. We may have to look to setting it's default in our rebuilt version of terraform.

It would also be great if there was some consistency added to the endpoints map and the hard coded dynamodb_endpoint/kinesis_endpoint parameters. I would love for them to be accepted up into the endpoints map, instead of having to have two different 'styles' in place in the code

[vrenjith]

We can close this as terraform now supports the flag skip_region_validation

provider "aws" {
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  insecure  = true
  skip_credentials_validation = true
  skip_region_validation = true
  endpoints {
    ec2 = "${var.ec2}"
    elb = "${var.elb}"
    iam = "${var.iam}"
  }
  region = "${var.region}"
}

[radeksimko]

@vrenjith Agreed, thanks for the ping.

It would also be great if there was some consistency added to the endpoints map and the hard coded dynamodb_endpoint/kinesis_endpoint parameters.

@Cinderhaze That is certainly something we can address, see #11768

Overall - based on the number of 👍 on this issue (1) - I feel that most folks have what they need in order to use terraform w/ AWS alternatives, so I'm going to close this in 2 weeks from now. This is just to give time to anyone who may want to jump in and disagree by presenting specific suggestions.

Running nightly acceptance tests against such endpoints is unlikely to happen from our side as we expect such tests to be very flaky and the main focus is real APIs for now.

[jason-riddle]

Atlassian has https://github.com/atlassian/localstack if someone needs to simulate multiple AWS services. I haven't used it myself but it looks promising.

[ebarault]

from the doc, it seems that provider AWS does not yet support lambda and apigateway endpoints customization in the nested endpoints block.

e.g.

provider "aws" {
  endpoints {
    s3 = "http://localhost:4572"
    # lambda = ...
    # apigateway = ...
  }
}

It would be nice to match localstack service coverage !

Are there plan to add the ability to use custom endpoints for these 2 ?

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.