Import does not work for resources accessible from endpoint different from region

[begault]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.13

• provider.aws v2.4.0

Affected Resource(s)

• aws_opsworks_custom_layer
• aws_opsworks_rails_app_layer
• aws_opsworks_xxx_layer
• aws_opsworks_instance

Terraform Configuration Files

resource "aws_opsworks_stack" "app" {
  name                         = "${var.opsworks_stack_name}"
  region                       = "eu-west-1"
  service_role_arn             = "${aws_iam_role.opsworks.arn}"
  default_instance_profile_arn = "${aws_iam_instance_profile.opsworks.arn}"
  use_custom_cookbooks         = true
  use_opsworks_security_groups = true
  manage_berkshelf             = false
  berkshelf_version            = ""
  hostname_theme               = "Europe_Cities"
  default_os                   = "Amazon Linux 2018.03"
  default_root_device_type     = "ebs"
  default_ssh_key_name         = "${var.ec2_default_keypair}"
  color                        = "rgb(45, 114, 184)"

  ...

  lifecycle {
    ignore_changes = ["custom_json"]
  }
}

resource "aws_opsworks_rails_app_layer" "rails_app" {
  name = "Rails App Server"
  short_name = "rails-app"
  stack_id                = "${aws_opsworks_stack.app.id}"
  auto_assign_elastic_ips = false
  auto_assign_public_ips  = true
  auto_healing            = true
}

resource "aws_opsworks_custom_layer" "rails_console" {
  name       = "For Rails Consoles"
  short_name = "rails console"
  stack_id   = "${aws_opsworks_stack.app.id}"
}

provider "aws" {
  region = "eu-west-1"
}

Debug Output

https://gist.github.com/begault/a65cef85542bc19e4569904f2bdc443e

Panic Output

Expected Behavior

The API endpoint for resources can be overridden and the host can be: Host: opsworks.us-east-1.amazonaws.com even if the default region is eu-west-1

Actual Behavior

When using Opsworks console, I get the following url : .../opsworks/home?region=eu-west-1&endpoint=us-east-1/....
This is due to Opsworks history. All API endpoints are pointing to us-east-1, even if the instances are in eu-west-1.

When I want to import a layer or another type of resource, I get the following error:

2019/04/02 14:40:44 [DEBUG] plugin: waiting for all plugin processes to complete...
Error: aws_opsworks_custom_layer.rails_app (import id: 00000000-0000-0000-0000-000000000000): 1 error(s) occurred:

* import aws_opsworks_custom_layer.rails_app result: 00000000-0000-0000-0000-000000000000: import aws_opsworks_custom_layer.rails_app (id: 00000000-0000-0000-0000-000000000000): Terraform detected a resource with this ID doesn't
exist. Please verify the ID is correct. You cannot import non-existent
resources using Terraform import.

When checking the endpoint of the aws API request, I notice that the endpoint region used is the default region given and cannot be overridden independently.

See: https://gist.github.com/begault/a65cef85542bc19e4569904f2bdc443e#file-terraform_layers_output-txt-L231

Where the host used is: Host: opsworks.eu-west-1.amazonaws.com

The import task does not find the resource as it's not looking to the right endpoint.

Steps to Reproduce

This bug is hard to reproduce as you need to have an Opsworks architecture which has been created before the endpoint regions opening.

Important Factoids

References

• #0000

[bflad]

Hi @begault 👋 Sorry you are running into trouble here. By chance, you may be able to work around this once version 2.5.0 of the Terraform AWS Provider is released later this week. We added support for customizing all service endpoints in #8096 which was merged yesterday.

I believe you should be able to setup something like the following once this is released:

provider "aws" {
  region = "eu-west-1"

  endpoints {
    opsworks = "https://opsworks.us-east-1.amazonaws.com"
  }
}

If you are working with new and old OpsWorks resources, I would recommend creating a separate provider alias (see also: the Terraform documentation on multiple provider instances) to handle those special older resources just to ensure new resources do not wind up in us-east-1, e.g.

provider "aws" {
  alias = "opsworks-us-east-1"
  region = "eu-west-1"

  endpoints {
    opsworks = "https://opsworks.us-east-1.amazonaws.com"
  }
}

resource "aws_opsworks_rails_app_layer" "rails_app" {
  provider = "aws.opsworks-us-east-1"

  # ... other configuration ...
}

If you do use provider aliases, just don't forget the terraform import -provider argument so Terraform knows to use the special provider for import.

[bflad]

The endpoint customization I mentioned above has been released in version 2.5.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

Would definitely like to know if it helps in anyway.

[begault]

Hello @bflad

Thank you very much for your valuable help.
I've chosen to wait the release of the endpoint feature to continue my setup.

I've tried today the new feature.
Unfortunately, I meet a new error:

here is my provider.tf file:

provider "aws" {
  version = ">= 2.5.0"
  region = "eu-west-1"

  endpoints {
    opsworks = "https://opsworks.us-east-1.amazonaws.com"
  }
}

Here is the error:

aws_opsworks_rails_app_layer.rails_app: aws_opsworks_rails_app_layer.rails_app: InvalidSignatureException: Credential should be scoped to a valid region, not 'eu-west-1'.

I've seen that in the changelog of AWS provider the terraform 0.11.13 is not compatible anymore with the new feature las update. (https://github.com/terraform-providers/terraform-provider-aws/blob/master/CHANGELOG.md#250-april-05-2019)

Do you think my problem is related & I need to wait for the terraform 0.12 version as well to be released ?

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.