Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

d/aws_dx_connection: Adding new data source for Direct Connect connection #9705

Closed
wants to merge 6 commits into from

Conversation

robh007
Copy link
Contributor

@robh007 robh007 commented Aug 9, 2019

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Changes proposed in this pull request:
Adds a new data resource for DX connections: aws_dx_connection

Lookup DX connections from a name.

example

data "aws_dx_connection" "example" {
  name = "example"
}
  • Adds data source aws_dx_connection

Output from acceptance testing:

$ DX_LOCATION=TCSH make testacc TEST=./aws/ TESTARGS='-run=TestAccDataSourceAwsDxConnection_Basic'                                                                                                   master * ] 6:09 pm
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccDataSourceAwsDxConnection_Basic -timeout 120m
=== RUN   TestAccDataSourceAwsDxConnection_Basic
=== PAUSE TestAccDataSourceAwsDxConnection_Basic
=== CONT  TestAccDataSourceAwsDxConnection_Basic
--- PASS: TestAccDataSourceAwsDxConnection_Basic (24.28s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       24.326s

...

@robh007 robh007 requested a review from a team August 9, 2019 16:41
@ghost ghost added size/L Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. provider Pertains to the provider itself, rather than any interaction with AWS. service/directconnect Issues and PRs that pertain to the directconnect service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Aug 9, 2019
Read: dataSourceAwsDxConnectionRead,

Schema: map[string]*schema.Schema{
"id": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to define an explicit id attribute here.

Computed: true,
},
"jumbo_frame_capable": {
Type: schema.TypeString,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

schema.TypeBool to match aws_dx_connection resource?

connection := connections[0]

d.SetId(aws.StringValue(connection.ConnectionId))
d.Set("state", aws.StringValue(connection.ConnectionState))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to use aws.StringValue() for attributes set with d.Set. Pointer dereferencing is dome automatically.
e.g. d.Set("state", connection.ConnectionState).

d.Set("state", aws.StringValue(connection.ConnectionState))
d.Set("location", aws.StringValue(connection.Location))
d.Set("bandwidth", aws.StringValue(connection.Bandwidth))
d.Set("jumbo_frame_capable", strconv.FormatBool(aws.BoolValue(connection.JumboFrameCapable)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If jumbo_frame_capable is schema.TypeBool then we can just do
d.Set("jumbo_frame_capable", connection.JumboFrameCapable)

)

func TestAccDataSourceAwsDxConnection_Basic(t *testing.T) {
key := "DX_LOCATION"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. The acceptance tests for the corresponding hard-code the location to EqSe2, assuming that the AWS region is us-west-2. It's good to challenge that assumption.
Maybe call DescribeLocations for the current test region and pick one at random rather than requiring an environment variable be set?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added #9708 to track adding new data sources to make this easier.

"name": {
Type: schema.TypeString,
Required: true,
},
Copy link
Contributor

@ewbankkit ewbankkit Aug 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add "tags": tagsSchemaComputed() (and query for tags) as DX Connection now support resource tags.
Maybe set an "arn" attribute as well?

@ghost ghost added size/XL Managed by automation to categorize the size of a PR. and removed size/L Managed by automation to categorize the size of a PR. labels Aug 11, 2019
@robh007
Copy link
Contributor Author

robh007 commented Aug 11, 2019

Hi @ewbankkit thanks for the feedback. I've updated the PR with your suggestions.

#9705 (comment)
With regards to the above comment, I've adding something to the test which takes a random location from the returned DX locations within a given region. The only thing I'm not sure if it's a good approach or not in the way I've setup the AWS connection. I used a function called sharedClientForRegion as I don't think the provider configuration is available at the time I want to set the DX Location.

I've added tags as an optional attribute & included a test too.

Passing test output below.

=== CONT TestAccDataSourceAwsDxConnection_Basic
=== CONT TestAccDataSourceAwsDxConnection_tags
--- PASS: TestAccDataSourceAwsDxConnection_tags (42.39s)
--- PASS: TestAccDataSourceAwsDxConnection_Basic (45.49s)
PASS
ok github.com/terraform-providers/terraform-provider-aws/aws 48.539s

@ewbankkit
Copy link
Contributor

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccDataSourceAwsDxConnection_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccDataSourceAwsDxConnection_ -timeout 120m
=== RUN   TestAccDataSourceAwsDxConnection_Basic
2019/08/12 09:48:11 [INFO] Building AWS auth structure
2019/08/12 09:48:11 [INFO] Setting AWS metadata API timeout to 100ms
2019/08/12 09:48:16 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2019/08/12 09:48:16 [INFO] AWS Auth provider used: "EnvProvider"
2019/08/12 09:48:16 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/08/12 09:48:16 [DEBUG] Trying to get account information via sts:GetCallerIdentity
Testing Connections in DX Location: SNAP=== PAUSE TestAccDataSourceAwsDxConnection_Basic
=== RUN   TestAccDataSourceAwsDxConnection_tags
2019/08/12 09:48:17 [INFO] Building AWS auth structure
2019/08/12 09:48:17 [INFO] Setting AWS metadata API timeout to 100ms
2019/08/12 09:48:23 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2019/08/12 09:48:23 [INFO] AWS Auth provider used: "EnvProvider"
2019/08/12 09:48:23 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/08/12 09:48:23 [DEBUG] Trying to get account information via sts:GetCallerIdentity
Testing Connections in DX Location: ECPO1=== PAUSE TestAccDataSourceAwsDxConnection_tags
=== CONT  TestAccDataSourceAwsDxConnection_Basic
=== CONT  TestAccDataSourceAwsDxConnection_tags
--- PASS: TestAccDataSourceAwsDxConnection_tags (43.91s)
--- PASS: TestAccDataSourceAwsDxConnection_Basic (45.00s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	51.398s

func testAccAwsDxConnectionLocation() (*string, error) {
var region string

if _, ok := os.LookupEnv("AWS_DEFAULT_REGION"); ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use testAccGetRegion() here (which is the same logic).

@ewbankkit
Copy link
Contributor

I'll try and get #9708 done today but that shouldn't hold up a review of this PR - I can refactor the acceptance tests for this data source and the corresponding resource depending on the merge order.

@bflad bflad added the new-data-source Introduces a new data source. label Aug 13, 2019
@devonbleak
Copy link
Contributor

I'm getting a dependency error with this as-is:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccDataSourceAwsDxConnection_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccDataSourceAwsDxConnection_ -timeout 120m
go: github.com/golangci/golangci-lint@v1.17.1 requires
        github.com/go-critic/go-critic@v0.0.0-20181204210945-1df300866540: invalid pseudo-version: does not match version-control timestamp (2019-05-26T07:48:19Z)
make: *** [testacc] Error 1

With a merge from master a different error:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccDataSourceAwsDxConnection_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -count 1 -parallel 20 -run=TestAccDataSourceAwsDxConnection_ -timeout 120m
# github.com/terraform-providers/terraform-provider-aws/aws [github.com/terraform-providers/terraform-provider-aws/aws.test]
aws/data_source_aws_dx_connection.go:40:4: cannot use tagsSchema() (type *"github.com/hashicorp/terraform-plugin-sdk/helper/schema".Schema) as type *"github.com/hashicorp/terraform/helper/schema".Schema in map value
aws/provider.go:190:4: cannot use dataSourceAwsDxConnection() (type *"github.com/hashicorp/terraform/helper/schema".Resource) as type *"github.com/hashicorp/terraform-plugin-sdk/helper/schema".Resource in map value
aws/data_source_aws_dx_connection_test.go:25:3: cannot use testAccProviders (type map[string]"github.com/hashicorp/terraform-plugin-sdk/terraform".ResourceProvider) as type map[string]"github.com/hashicorp/terraform/terraform".ResourceProvider in field value
aws/data_source_aws_dx_connection_test.go:56:3: cannot use testAccProviders (type map[string]"github.com/hashicorp/terraform-plugin-sdk/terraform".ResourceProvider) as type map[string]"github.com/hashicorp/terraform/terraform".ResourceProvider in field value
FAIL    github.com/terraform-providers/terraform-provider-aws/aws [build failed]
FAIL
make: *** [testacc] Error 2

@beezly
Copy link
Contributor

beezly commented Jan 20, 2020

I've rebased this against an up-to-date master and made a few fixes in https://github.com/beezly/terraform-provider-aws/tree/rh007_against_master. It passed the acceptance test when I ran it:

==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./aws/ -v -count 1 -parallel 20 -run=TestAccDataSourceAwsDxConnection_ -timeout 120m === RUN TestAccDataSourceAwsDxConnection_Basic 2020/01/20 16:57:20 [INFO] Building AWS auth structure 2020/01/20 16:57:20 [INFO] Setting AWS metadata API timeout to 100ms 2020/01/20 16:57:22 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id 2020/01/20 16:57:22 [INFO] AWS Auth provider used: "EnvProvider" 2020/01/20 16:57:22 [DEBUG] Trying to get account information via sts:GetCallerIdentity 2020/01/20 16:57:23 [DEBUG] Trying to get account information via sts:GetCallerIdentity Testing Connections in DX Location: ITXD2=== PAUSE TestAccDataSourceAwsDxConnection_Basic === RUN TestAccDataSourceAwsDxConnection_tags 2020/01/20 16:57:24 [INFO] Building AWS auth structure 2020/01/20 16:57:24 [INFO] Setting AWS metadata API timeout to 100ms 2020/01/20 16:57:25 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id 2020/01/20 16:57:25 [INFO] AWS Auth provider used: "EnvProvider" 2020/01/20 16:57:25 [DEBUG] Trying to get account information via sts:GetCallerIdentity 2020/01/20 16:57:25 [DEBUG] Trying to get account information via sts:GetCallerIdentity Testing Connections in DX Location: TEJB1=== PAUSE TestAccDataSourceAwsDxConnection_tags === CONT TestAccDataSourceAwsDxConnection_Basic === CONT TestAccDataSourceAwsDxConnection_tags --- PASS: TestAccDataSourceAwsDxConnection_tags (31.37s) --- PASS: TestAccDataSourceAwsDxConnection_Basic (33.00s) PASS ok github.com/terraform-providers/terraform-provider-aws/aws 36.990s

@beezly
Copy link
Contributor

beezly commented Jan 20, 2020

Having had a try at using this data source, I think it needs a little bit of tweaking. At the moment the only way to identify a dx_connection is using the name field. I think in most cases, the connection ID is going to be the most useful unique piece of information that people will want to search on (certainly in the example that I'm trying, I have been allocated a hosted connection by a service provider into my account and I'm trying to use a data source to interact with the resource... the ID value is the thing that makes most sense and won't change). I'll have a look at extending it to allow you to provide either name or id to identify the resource.

@beezly
Copy link
Contributor

beezly commented Jan 20, 2020

OK - I've done some more work on this and reworked it to use the connection ID to find the dx connection resource. As my code is now diverging from the original quite significantly, I've created a new branch at https://github.com/beezly/terraform-provider-aws/tree/use_connection_id_for_dx_connection with this work in.

Acceptance tests currently pass:

==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./aws/ -v -count 1 -parallel 20 -run=TestAccDataSourceAwsDxConnection_ -timeout 120m === RUN TestAccDataSourceAwsDxConnection_Basic 2020/01/20 18:27:45 [INFO] Building AWS auth structure 2020/01/20 18:27:45 [INFO] Setting AWS metadata API timeout to 100ms 2020/01/20 18:27:47 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id 2020/01/20 18:27:47 [INFO] AWS Auth provider used: "EnvProvider" 2020/01/20 18:27:47 [DEBUG] Trying to get account information via sts:GetCallerIdentity 2020/01/20 18:27:48 [DEBUG] Trying to get account information via sts:GetCallerIdentity Testing Connections in DX Location: EirCL=== PAUSE TestAccDataSourceAwsDxConnection_Basic === CONT TestAccDataSourceAwsDxConnection_Basic --- PASS: TestAccDataSourceAwsDxConnection_Basic (36.93s) PASS ok github.com/terraform-providers/terraform-provider-aws/aws 38.210s

At the moment, my code doesn't support filtering by Tags or by Name, which the original code does attempt to do. However, AWS don't provide an API for filtering on Connections at all as DescrtibeConnectionsInput only provides for ConnectionId.

My gut feeling is that the simple basic functionality of searching by Id should be working first and then more extended functionality (filtering on Name/Tags) could be added on top. @ewbankkit do you have any opinions?

@ewbankkit
Copy link
Contributor

@beezly Agree with keeping things simple to start with, maybe just name and connection_id as arguments.

@beezly
Copy link
Contributor

beezly commented Jan 21, 2020

@ewbankkit thanks - I've got that working on the use_connection_id_for_dx_connection I mentioned above.

I've also added some attributes that the aws_dx_connection data source can have (eg. vlan), but aws_dx_connection resource can't define. The CreateConnection API returns vlan as 0 when you create a "dedicated" aws_dx_connection (which aws_dx_connection does), but if you create a "hosted" (which aws_dx_connection does not do), then the VLAN cannot be specified. It feels like at some point the provider could do with either extending the aws_dx_connection to support "hosted" DirectConnect connections (using the AllocateHostedConnection API) or maybe a new resource type. What do you think?

In my particular use-case, the reason I need the aws_dx_connection data source is because I need to establish what the VLAN ID to use to bring up the BGP connection peering in an aws_dx_hosted_private_virtual_interface resource for a hosted DX connection that has been provided to us by a 3rd party.

@robh007
Copy link
Contributor Author

robh007 commented Jan 22, 2020

Hi @beezly I based the original code from the aws_dx_gateway implementation to use a name which I felt was easier to use rather than an ID. We don't have hosted DX connections so the workflow worked fine because we named the original connections. This was also my first data source I wrote I think & it's been a while since the original commit. Let me know if you want me to do anything.

@beezly
Copy link
Contributor

beezly commented Jan 22, 2020

@robh007 thanks for getting in touch! Your code has been a great place for me to get going with what we needed on my current project. I'm really grateful I didn't have to start from scratch! I've been doing a bit more work on my branch and added support for searching by name back in. I'm just trying to reimplement tag filtering using the new keyvaluetags framework (see #10688).

You do raise a really good point. I think the aws_dx_connection data source could cater for both "hosted" and "dedicated" connections, but I'm not so sure about the resource. The semantics of a hosted connection are a bit different from a dedicated one (eg, they are completely different API calls for the create operation, but the same call for the read operation) and I'm not sure if that warrants a different resource type, say aws_dx_hosted_connection.

@ewbankkit / @bflad - what do you think? Should there be a single aws_dx_connection datasource covering both hosted and dedicated connections, with a view to extending the existing resource to support hosted connections, or is there a preference to have a separate resource for each type?

@ewilde
Copy link
Contributor

ewilde commented Jun 4, 2020

@beezly @ewbankkit is there anything outstanding on this PR that I could help out with? I could really do with this data source to help look up connections by name

@teamterraform
Copy link

Notification of Recent and Upcoming Changes to Contributions

Thank you for this contribution! There have been a few recent development changes that affect this pull request. We apologize for the inconvenience, especially if there have been long review delays up until now. Please note that this is automated message from an unmonitored account. See the FAQ for additional information on the maintainer team and review prioritization.

If you are unable to complete these updates, please leave a comment for the community and maintainers so someone can potentially continue the work. The maintainers will encourage other contributors to use the existing contribution as the base for additional changes as appropriate. Otherwise, contributions that do not receive updated code or comments from the original contributor may be closed in the future so the maintainers can focus on active items.

For the most up to date information about Terraform AWS Provider development, see the Contributing Guide. Additional technical debt changes can be tracked with the technical-debt label on issues.

As part of updating a pull request with these changes, the most current unit testing and linting will run. These may report issues that were not previously reported.

Action Required: Terraform 0.12 Syntax

Reference: #8950
Reference: #14417

Version 3 and later of the Terraform AWS Provider, which all existing contributions would potentially be added, only supports Terraform 0.12 and later. Certain syntax elements of Terraform 0.11 and earlier show deprecation warnings during runs with Terraform 0.12. Documentation and test configurations, such as those including deprecated string interpolations (some_attribute = "${aws_service_thing.example.id}") should be updated to the newer syntax (some_attribute = aws_service_thing.example.id). Contribution testing will automatically fail on older syntax in the near future. Please see the referenced issues for additional information.

Action Required: Terraform Plugin SDK Version 2

Reference: #14551

The Terraform AWS Provider has been upgraded to the latest version of the Terraform Plugin SDK. Generally, most changes to contributions should only involve updating Go import paths in source code files. Please see the referenced issue for additional information.

Action Required: Removal of website/aws.erb File

Reference: #14712

Any changes to the website/aws.erb file are no longer necessary and should be removed from this contribution to prevent merge issues in the near future when the file is removed from the repository. Please see the referenced issue for additional information.

Upcoming Change of Git Branch Naming

Reference: #14292

Development environments will need their upstream Git branch updated from master to main in the near future. Please see the referenced issue for additional information and scheduling.

Upcoming Change of GitHub Organization

Reference: #14715

This repository will be migrating from https://github.com/terraform-providers/terraform-provider-aws to https://github.com/hashicorp/terraform-provider-aws. No practitioner or developer action is anticipated and most GitHub functionality will automatically redirect to the new location. Go import paths including terraform-providers can remain for now. Please see the referenced issue for additional information and scheduling.

Base automatically changed from master to main January 23, 2021 00:56
@breathingdust breathingdust requested a review from a team as a code owner January 23, 2021 00:56
@ewbankkit
Copy link
Contributor

ewbankkit commented Feb 28, 2021

@robh007 There is a newer PR, #17852, that meets the Plugin SDK v2 (and other) requirements listed above and is likely to be merged sooner.
Are you OK with us moving ahead with that PR?

@ewbankkit ewbankkit added waiting-response Maintainers are waiting on response from community or contributor. and removed provider Pertains to the provider itself, rather than any interaction with AWS. labels Feb 28, 2021
@robh007
Copy link
Contributor Author

robh007 commented Mar 1, 2021

@robh007 There is a newer PR, #17852, that meets the Plugin SDK v2 (and other) requirements listed above and is likely to be merged sooner.
Are you OK with us moving ahead with that PR?

@ewbankkit - yeah whatever's easier. I'd just like to see the additional attributes added to that PR as you've mentioned.

@ghost ghost removed waiting-response Maintainers are waiting on response from community or contributor. labels Mar 1, 2021
@ewbankkit
Copy link
Contributor

Closing in favor of #17852.

@ewbankkit ewbankkit closed this Mar 25, 2021
@ghost
Copy link

ghost commented Apr 24, 2021

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked as resolved and limited conversation to collaborators Apr 24, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. new-data-source Introduces a new data source. service/directconnect Issues and PRs that pertain to the directconnect service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants