Lake Formation Data Lake Settings

[gmazelier]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Relates #9700

AWS Lake Formation support: manages the data lake settings for your account.

This is a proposal for the first resource LakeFormation::DataLakeSettings mentioned in #9700, others resources will be provided in upcoming pull requests: LakeFormation::Permissions, LakeFormation::Resource.

Release note for CHANGELOG:

* **New Resource**: `aws_lakeformation_datalake_settings` (#13250)

Output from acceptance testing:

make testacc TEST=./aws TESTARGS='-run=TestAccAWSLakeFormationDataLakeSettings'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSLakeFormationDataLakeSettings -timeout 120m
=== RUN   TestAccAWSLakeFormationDataLakeSettings_basic
=== PAUSE TestAccAWSLakeFormationDataLakeSettings_basic
=== RUN   TestAccAWSLakeFormationDataLakeSettings_withCatalogId
=== PAUSE TestAccAWSLakeFormationDataLakeSettings_withCatalogId
=== CONT  TestAccAWSLakeFormationDataLakeSettings_basic
=== CONT  TestAccAWSLakeFormationDataLakeSettings_withCatalogId
--- PASS: TestAccAWSLakeFormationDataLakeSettings_basic (33.10s)
--- PASS: TestAccAWSLakeFormationDataLakeSettings_withCatalogId (33.50s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	34.906s

[ewbankkit]

Verified acceptance tests:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSLakeFormationDataLakeSettings_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -count 1 -parallel 20 -run=TestAccAWSLakeFormationDataLakeSettings_ -timeout 120m
=== RUN   TestAccAWSLakeFormationDataLakeSettings_basic
=== PAUSE TestAccAWSLakeFormationDataLakeSettings_basic
=== RUN   TestAccAWSLakeFormationDataLakeSettings_withCatalogId
=== PAUSE TestAccAWSLakeFormationDataLakeSettings_withCatalogId
=== CONT  TestAccAWSLakeFormationDataLakeSettings_basic
=== CONT  TestAccAWSLakeFormationDataLakeSettings_withCatalogId
--- PASS: TestAccAWSLakeFormationDataLakeSettings_basic (25.14s)
--- PASS: TestAccAWSLakeFormationDataLakeSettings_withCatalogId (25.20s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	25.239s

[teamterraform]

Notification of Recent and Upcoming Changes to Contributions

Thank you for this contribution! There have been a few recent development changes that affect this pull request. We apologize for the inconvenience, especially if there have been long review delays up until now. Please note that this is automated message from an unmonitored account. See the FAQ for additional information on the maintainer team and review prioritization.

If you are unable to complete these updates, please leave a comment for the community and maintainers so someone can potentially continue the work. The maintainers will encourage other contributors to use the existing contribution as the base for additional changes as appropriate. Otherwise, contributions that do not receive updated code or comments from the original contributor may be closed in the future so the maintainers can focus on active items.

For the most up to date information about Terraform AWS Provider development, see the Contributing Guide. Additional technical debt changes can be tracked with the technical-debt label on issues.

As part of updating a pull request with these changes, the most current unit testing and linting will run. These may report issues that were not previously reported.

Action Required: Terraform 0.12 Syntax

Reference: #8950
Reference: #14417

Version 3 and later of the Terraform AWS Provider, which all existing contributions would potentially be added, only supports Terraform 0.12 and later. Certain syntax elements of Terraform 0.11 and earlier show deprecation warnings during runs with Terraform 0.12. Documentation and test configurations, such as those including deprecated string interpolations (some_attribute = "${aws_service_thing.example.id}") should be updated to the newer syntax (some_attribute = aws_service_thing.example.id). Contribution testing will automatically fail on older syntax in the near future. Please see the referenced issues for additional information.

Action Required: Terraform Plugin SDK Version 2

Reference: #14551

The Terraform AWS Provider has been upgraded to the latest version of the Terraform Plugin SDK. Generally, most changes to contributions should only involve updating Go import paths in source code files. Please see the referenced issue for additional information.

Action Required: Removal of website/aws.erb File

Reference: #14712

Any changes to the website/aws.erb file are no longer necessary and should be removed from this contribution to prevent merge issues in the near future when the file is removed from the repository. Please see the referenced issue for additional information.

Upcoming Change of Git Branch Naming

Reference: #14292

Development environments will need their upstream Git branch updated from master to main in the near future. Please see the referenced issue for additional information and scheduling.

Upcoming Change of GitHub Organization

Reference: #14715

This repository will be migrating from https://github.com/terraform-providers/terraform-provider-aws to https://github.com/hashicorp/terraform-provider-aws. No practitioner or developer action is anticipated and most GitHub functionality will automatically redirect to the new location. Go import paths including terraform-providers can remain for now. Please see the referenced issue for additional information and scheduling.

[YakDriver]

@gmazelier Hello! Are you still interested and available to work on this? We have been unable to get to Lake Formation yet but will be working on it soon.

[gmazelier]

@YakDriver Yes, sure. Planned to work on the potential mentioned issues next week.

[YakDriver]

@gmazelier I want to be upfront about the process so you know what to expect. A design will be created internally and adjusted and eventually approved. It may seem like nothing is happening at that point but work is happening behind the scenes. Once the design is approved, implementation will need to follow the design. Hopefully we can use some or all of your work. We won't know until we have concensus on the design, however.

[gmazelier]

@YakDriver Alright, just let me know if and when I can help.

[YakDriver]

@gmazelier My friend, here's the latest. We have the design approved and are moving forward with implementation. To give you a heads up, I will likely be building on your PR. Make sure that you have checked the "allow edits from maintainers" box. You will get credit for the feature and commits. I'll make any changes needed to bring it into alignment with the design.

[YakDriver]

Personally, I like "datalake" as one word but AWS consistently uses "data lake" as two words so I'm renaming to maintain consistency with AWS.

[gmazelier]

@YakDriver Sure. Nothing to commit right now, I'm waiting for design realignment. Then I will contribute to provider other resources as you mentioned. But I'll you know. Thank you for your work and feedback! (agree with you about data lake wording but let's be consistent with AWS documentation)

[YakDriver]

@ewbankkit @gmazelier Have a look at the latest. If you have any feedback, let me know soon. I'd like to merge today if possible.

Acceptance tests in us-west-2

--- PASS: TestAccAWSLakeFormationDataLakeSettings_serial (34.38s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettings_serial/basic (11.86s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettings_serial/disappears (11.46s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettings_serial/withoutCatalogId (11.07s)
--- PASS: TestAccAWSLakeFormationDataLakeSettingsDataSource_serial (14.70s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettingsDataSource_serial/basic (14.70s)

Acceptance tests in us-gov-west-1

--- PASS: TestAccAWSLakeFormationDataLakeSettings_serial (49.85s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettings_serial/basic (17.66s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettings_serial/disappears (16.37s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettings_serial/withoutCatalogId (15.83s)
--- PASS: TestAccAWSLakeFormationDataLakeSettingsDataSource_serial (18.89s)
    --- PASS: TestAccAWSLakeFormationDataLakeSettingsDataSource_serial/basic (18.89s)

[gmazelier]

LGTM

[YakDriver]

As with any new service, we'll notice things that need adjusting in the first little while. We welcome your help post-release. As for resources and data sources, here is what is planned and not planned. I'll create issues for the items that won't make the first release.

• resource/lakeformation_resource Lake Formation Resource #13267
• data source/lakeformation_resource Lake Formation Permissions #13396
• data source/lakeformation_resources (see ds/lakeformation_resources: New data source #16845)
• resource/lakeformation_data_lake_settings Lake Formation Data Lake Settings #13250
• data source/lakeformation_data_lake_settings Lake Formation Data Lake Settings #13250
• resource/lakeformation_permissions Lake Formation Permissions #13396 (priority 1)
• data source/lakeformation_permissions awsproviderlint: Enforce kms_key_id as ARN attribute in AWSAT001 check #13996 (priority 2)
• data source/lakeformation_effective_permissions (see ds/lakeformation_effective_permissions: New data source #16846)

[ghost]

This has been released in version 3.22.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!