Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate LakeFormation Service to AWS Go SDKv2 #36362

Merged
merged 15 commits into from
Mar 14, 2024

Conversation

johnsonaj
Copy link
Contributor

Description

Relations

Closes #36179

References

Output from Acceptance Testing

% make testacc TESTARGS="-run=TestAccLakeFormation" PKG=lakeformation

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/lakeformation/... -v -count 1 -parallel 20  -run=TestAccLakeFormation -timeout 360m
--- PASS: TestAccLakeFormationResource_basic (16.64s)
--- PASS: TestAccLakeFormationResourceDataSource_basic (17.17s)
--- PASS: TestAccLakeFormationResource_disappears (17.29s)
--- PASS: TestAccLakeFormationResource_hybridAccessEnabled (17.51s)
--- PASS: TestAccLakeFormationResource_serviceLinkedRole (17.56s)
--- PASS: TestAccLakeFormationResource_updateRoleToRole (27.31s)
--- PASS: TestAccLakeFormationResource_updateSLRToRole (27.70s)
--- PASS: TestAccLakeFormation_serial (1160.88s)
    --- PASS: TestAccLakeFormation_serial/DataLakeSettingsDataSource (23.79s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettingsDataSource/basic (12.72s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettingsDataSource/readOnlyAdmins (11.06s)
    --- PASS: TestAccLakeFormation_serial/PermissionsBasic (289.85s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/lfTagPolicyMultiple (18.91s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/databaseIAMAllowed (41.65s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/dataLocation (18.77s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/disappears (83.31s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/lfTag (20.37s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/lfTagPolicy (21.24s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/basic (20.06s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/database (21.13s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/databaseMultiple (21.50s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/dataCellsFilter (22.91s)
    --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns (139.45s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/implicit (21.49s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardExcludedColumns (23.07s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectOnly (22.55s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectPlus (23.07s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/basic (49.27s)
    --- PASS: TestAccLakeFormation_serial/LFTags (87.71s)
        --- PASS: TestAccLakeFormation_serial/LFTags/basic (13.81s)
        --- PASS: TestAccLakeFormation_serial/LFTags/disappears (11.89s)
        --- PASS: TestAccLakeFormation_serial/LFTags/tagKeyComplex (11.12s)
        --- PASS: TestAccLakeFormation_serial/LFTags/values (23.48s)
        --- PASS: TestAccLakeFormation_serial/LFTags/valuesOverFifty (27.41s)
    --- PASS: TestAccLakeFormation_serial/ResourceLFTags (145.61s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/basic (12.65s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/database (22.53s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/databaseMultipleTags (22.04s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/disappears (12.75s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/hierarchy (27.40s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/table (24.34s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/tableWithColumns (23.90s)
    --- PASS: TestAccLakeFormation_serial/DataLakeSettings (43.13s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/basic (10.59s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/disappears (11.43s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/withoutCatalogId (10.63s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/readOnlyAdmins (10.48s)
    --- PASS: TestAccLakeFormation_serial/DataCellsFilter (43.52s)
        --- PASS: TestAccLakeFormation_serial/DataCellsFilter/basic (16.26s)
        --- PASS: TestAccLakeFormation_serial/DataCellsFilter/columnWildcard (13.12s)
        --- PASS: TestAccLakeFormation_serial/DataCellsFilter/disappears (14.15s)
    --- PASS: TestAccLakeFormation_serial/PermissionsDataSource (171.43s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/dataCellsFilter (18.85s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/database (19.94s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/dataLocation (23.71s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/lfTag (20.56s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/lfTagPolicy (21.25s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/table (23.47s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/tableWithColumns (23.07s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/basic (20.58s)
    --- PASS: TestAccLakeFormation_serial/PermissionsTable (216.40s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/wildcardSelectOnly (22.03s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/basic (18.57s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/multipleRoles (22.82s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/selectOnly (22.94s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/wildcardNoSelect (20.72s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/iamAllowed (43.46s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/implicit (21.42s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/selectPlus (22.42s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/wildcardSelectPlus (22.02s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/lakeformation	1165.042s

Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. service/lakeformation Issues and PRs that pertain to the lakeformation service. client-connections Pertains to the AWS Client and service connections. generators Relates to code generators. labels Mar 13, 2024
@terraform-aws-provider terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Mar 13, 2024
Copy link

Thank you for your contribution! 🚀

A new usage of AWS SDK for Go V1 was detected. Please prefer AWS SDK for Go V2 for all net-new services. If this is an enhancement or bug fix to an existing AWS SDK Go V1 based resource, this comment can be safely ignored.

For additional information refer to the AWS SDK for Go Versions page in the contributor guide.

@johnsonaj johnsonaj added technical-debt Addresses areas of the codebase that need refactoring or redesign. aws-sdk-go-migration Issues that are related to the providers migration to AWS SDK for Go v2. labels Mar 13, 2024
Copy link
Member

@jar-b jar-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎉

% make testacc TESTARGS="-run=TestAccLakeFormation" PKG=lakeformation
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/lakeformation/... -v -count 1 -parallel 20  -run=TestAccLakeFormation -timeout 360m

--- PASS: TestAccLakeFormationResource_basic (17.49s)
--- PASS: TestAccLakeFormationResource_disappears (17.88s)
--- PASS: TestAccLakeFormationResourceDataSource_basic (18.04s)
--- PASS: TestAccLakeFormationResource_serviceLinkedRole (18.17s)
--- PASS: TestAccLakeFormationResource_hybridAccessEnabled (18.28s)
--- PASS: TestAccLakeFormationResource_updateRoleToRole (27.85s)
--- PASS: TestAccLakeFormationResource_updateSLRToRole (28.79s)

=== RUN   TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectPlus
    permissions_test.go:793: Step 1/1 error: Error running apply: exit status 1

        Error: reading Lake Formation permissions: operation error LakeFormation: ListPermissions, https response error StatusCode: 400, RequestID: 40fea2de-35be-46f2-96cf-fc839c53d
161, InvalidInputException: Invalid principal, arn: arn:aws:iam::727561393803:role/tf-acc-test-909699663469884911

          with aws_lakeformation_permissions.test,
          on terraform_plugin_test.tf line 66, in resource "aws_lakeformation_permissions" "test":
          66: resource "aws_lakeformation_permissions" "test" {

--- FAIL: TestAccLakeFormation_serial (1096.20s)
    --- PASS: TestAccLakeFormation_serial/PermissionsBasic (282.60s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/databaseIAMAllowed (43.95s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/databaseMultiple (20.09s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/dataLocation (22.26s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/lfTag (19.16s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/lfTagPolicyMultiple (19.19s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/database (19.78s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/dataCellsFilter (17.77s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/disappears (82.50s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/lfTagPolicy (19.26s)
        --- PASS: TestAccLakeFormation_serial/PermissionsBasic/basic (18.62s)
    --- PASS: TestAccLakeFormation_serial/PermissionsTable (209.43s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/multipleRoles (21.45s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/selectOnly (21.22s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/wildcardSelectOnly (20.90s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/wildcardSelectPlus (21.08s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/basic (21.21s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/iamAllowed (42.72s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/implicit (20.14s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/selectPlus (21.09s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTable/wildcardNoSelect (19.62s)
    --- PASS: TestAccLakeFormation_serial/ResourceLFTags (135.56s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/table (22.26s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/tableWithColumns (22.68s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/basic (11.47s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/database (20.95s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/databaseMultipleTags (20.73s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/disappears (12.56s)
        --- PASS: TestAccLakeFormation_serial/ResourceLFTags/hierarchy (24.91s)
    --- PASS: TestAccLakeFormation_serial/LFTags (79.40s)
        --- PASS: TestAccLakeFormation_serial/LFTags/valuesOverFifty (25.03s)
        --- PASS: TestAccLakeFormation_serial/LFTags/basic (12.81s)
        --- PASS: TestAccLakeFormation_serial/LFTags/disappears (10.88s)
        --- PASS: TestAccLakeFormation_serial/LFTags/tagKeyComplex (10.24s)
        --- PASS: TestAccLakeFormation_serial/LFTags/values (20.44s)
    --- PASS: TestAccLakeFormation_serial/DataLakeSettings (39.21s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/basic (9.58s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/disappears (10.38s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/withoutCatalogId (9.67s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettings/readOnlyAdmins (9.57s)
    --- PASS: TestAccLakeFormation_serial/DataCellsFilter (41.46s)
        --- PASS: TestAccLakeFormation_serial/DataCellsFilter/basic (15.50s)
        --- PASS: TestAccLakeFormation_serial/DataCellsFilter/columnWildcard (12.76s)
        --- PASS: TestAccLakeFormation_serial/DataCellsFilter/disappears (13.20s)
    --- PASS: TestAccLakeFormation_serial/DataLakeSettingsDataSource (20.53s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettingsDataSource/basic (10.17s)
        --- PASS: TestAccLakeFormation_serial/DataLakeSettingsDataSource/readOnlyAdmins (10.36s)
    --- PASS: TestAccLakeFormation_serial/PermissionsDataSource (169.78s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/dataLocation (23.04s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/lfTag (19.74s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/lfTagPolicy (19.85s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/table (22.51s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/tableWithColumns (21.89s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/basic (19.25s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/dataCellsFilter (22.34s)
        --- PASS: TestAccLakeFormation_serial/PermissionsDataSource/database (21.16s)
    --- FAIL: TestAccLakeFormation_serial/PermissionsTableWithColumns (118.24s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/basic (44.28s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/implicit (20.54s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardExcludedColumns (19.25s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectOnly (21.41s)
        --- FAIL: TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectPlus (12.76s)
FAIL
FAIL    github.com/hashicorp/terraform-provider-aws/internal/service/lakeformation      1100.415s

Failure was transient and passed on a second attempt. Non-blocking, but potentially a future enhancement to retry failures related to IAM propagation.

% make testacc TESTARGS="-run=TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectPlus" PKG=lakeformation
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/lakeformation/... -v -count 1 -parallel 20  -run=TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectPlus -timeout 360m
=== RUN   TestAccLakeFormation_serial
=== PAUSE TestAccLakeFormation_serial
=== CONT  TestAccLakeFormation_serial
=== RUN   TestAccLakeFormation_serial/PermissionsTableWithColumns
=== RUN   TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectPlus
--- PASS: TestAccLakeFormation_serial (22.11s)
    --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns (22.11s)
        --- PASS: TestAccLakeFormation_serial/PermissionsTableWithColumns/wildcardSelectPlus (22.11s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/lakeformation      26.315s

@johnsonaj
Copy link
Contributor Author

@jar-b found the error causing that intermittent failure☝🏾

@johnsonaj johnsonaj merged commit fae116d into main Mar 14, 2024
37 checks passed
@johnsonaj johnsonaj deleted the td-lakeformation_migrate_sdkv2 branch March 14, 2024 17:52
@github-actions github-actions bot added this to the v5.41.0 milestone Mar 14, 2024
Copy link

This functionality has been released in v5.41.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Mar 14, 2024
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
aws-sdk-go-migration Issues that are related to the providers migration to AWS SDK for Go v2. client-connections Pertains to the AWS Client and service connections. generators Relates to code generators. service/lakeformation Issues and PRs that pertain to the lakeformation service. size/XL Managed by automation to categorize the size of a PR. technical-debt Addresses areas of the codebase that need refactoring or redesign. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[AWS SDK for Go Migration] Lake Formation Service
2 participants