Incompatible with stepfunctions-local / localstack

[atbentley]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.7
+ provider.aws v2.6.0

Affected Resource(s)

• aws_sfn_state_machine

Terraform Configuration Files

provider "aws" {
  access_key                  = "mock_access_key"
  secret_key                  = "mock_secret_key"
  region                      = "eu-central-1"
  s3_force_path_style         = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  version = "2.6.0"

  endpoints {
    stepfunctions  = "http://localhost:4585"
  }
}

resource "aws_sfn_state_machine" "state_machine" {
  name     = "machine"
  role_arn = "arn:aws:iam::123123123123:role/some_role"

  definition = <<EOF
{
  "StartAt": "finish",
  "States": {
    "finish": {
      "Type": "Succeed"
    }
  }
}
EOF
}

Debug Output

https://gist.github.com/atbentley/ca44043bb0abf55b913aaad0205f381d

Expected Behavior

Terraform should successfully create a state machine in stepfunctions-local / localstack, and then successfully exit.

It is expected to work since the endpoint configuration for stepfunctions exists to interact with localstack / etc.

Actual Behavior

Terraform successfully creates the statemachine, then after creation requests the tags for the state machine and then fails since stepfunctions-local does not implement the list-tags-for-resource endpoint of stepfunctions.

Steps to Reproduce

1. Install localstack
2. SERVICES=lambda,stepfunctions localstack start --docker
3. terraform init
4. terraform apply
5. aws stepfunctions --region eu-central-1 --endpoint http://localhost:4585 list-state-machines

Important Factoids

References

This issue introduces the stepfunctions endpoint configuration so that terraform can be used with stepfunctions-local or localstack: #7735

This issue introduces support for resource tags on state machines: #6971

localstack (wraps stepfunctions-local): https://github.com/localstack/localstack

stepfunctions-local: https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local.html

• Feature request: AWS Step Functions tags #6971
• Provide endpoint for AWS step functions #7735

[bflad]

Hi @atbentley 👋 Thanks for reporting this and sorry for the trouble here.

While we support customization of service endpoints in the Terraform AWS Provider, their usage is only regularly integration tested against standard AWS environments (e.g. AWS Standard and AWS GovCloud (US)) and support outside those is best effort at the current time. The Custom Service Endpoints Configuration Guide currently has the following note to try and relay this information:

NOTE: Support for connecting the Terraform AWS Provider with custom endpoints and AWS compatible solutions is offered as best effort. Individual Terraform resources may require compatibility updates to work in certain environments. Integration testing by HashiCorp during provider changes is exclusively done against default AWS endpoints at this time.

If we can improve the above documentation on this manner, please do let us know.

That said, we do want to improve our compatibility with LocalStack and this seems like it should be fairly straightforward to fix the support in the aws_sfn_state_machine resource by ignoring that error here:

https://github.com/terraform-providers/terraform-provider-aws/blob/b05c213aae2a04df3ecb27c5ff2be867681c3bf9/aws/resource_aws_sfn_state_machine.go#L141-L143

I have submitted this fix, which I verified against LocalStack using the configuration provided above: #8467

There are some complexities to enable LocalStack with our current acceptance testing framework which generally uses environment variables, however there is no such environment variables for endpoint configurations. We would prefer AWS introduced a standard configuration method (environment variable convention, AWS configuration file, etc.) for customizing multiple endpoints instead of implementing our own system if possible. Hopefully this situation will improve over time and we will think about other ways we can achieve LocalStack testing.

If you do find other cases where the Terraform AWS Provider is not able to work correctly with LocalStack, please feel free to submit additional issues. Thanks again.

[atbentley]

Hey @bflad, I totally get and respect the product not wanting to support multiple targets other than AWS. I also think the documentation is totally clear here. I just wanted to point out that the sfn endpoint configuration change and the sfn tag change seem to conflict with eachother in that one was done to allow localstack and the other had broken that support.

In any case, I'm thankful for you submitting a fix for this :)

[bflad]

The change to bypass the UnknownOperationException error for the aws_sfn_state_machine Terraform resource with the ListTagsForResource API call has been merged and will release with version 2.9.0 of the Terraform AWS Provider, likely later this week.

[alejandrocena]

Hi all, Thank you @bflad.

Using that recent released Version I came up with this:
ListTagsForResource <= Unsupported Operation: 'ListTagsForResource'

Here I have my repo wich ends throwing that error. To saw it, you need to add DEBUG=1 before start Localstack.

https://github.com/alejandrocena/aws-localstack-terraform-stepfunctions-lambda/releases/tag/BUG-ListTagsForResource

Localstack on debug mode Throws me:

019-05-07 13:19:29.487: [500] ListTagsForResource <= Unsupported Operation: 'ListTagsForResource'
2019-05-07 13:19:29.543: [500] ListTagsForResource <= Unsupported Operation: 'ListTagsForResource'
2019-05-07 13:19:29.625: [500] ListTagsForResource <= Unsupported Operation: 'ListTagsForResource'

[bflad]

Hi @alejandrocena 👋 While we cannot debug your LocalStack environment, the logging output you show appears to be the desired behavior with the Terraform resource, even on version 2.9.0 of the Terraform AWS Provider. Terraform will always attempt to make the ListTagsForResource API call, but we ignore the unsupported operation error within Terraform and refresh the tags as empty in the Terraform state when this occurs. We prefer this setup so no additional code change is needed if/when that API call is implemented.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!