Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

atmos read/write storage interface #865

Merged
merged 32 commits into from
Dec 18, 2024
Merged

atmos read/write storage interface #865

merged 32 commits into from
Dec 18, 2024

Conversation

mcalhoun
Copy link
Member

@mcalhoun mcalhoun commented Dec 17, 2024

what

Add the concept of atmos stores to the app. This PR adds only the read side of stores. The write side will be implemented in a separate PR.

why

Atmos stores allow values to be written to an external store (i.e., AWS SSM Param Store) after apply-time and read from at plan/before apply time.

Stores are pluggable and configurable and multiple may be enabled at the same time (i.e. for prod/non-prod, east/west, etc).

The initial set of stores we are implementing as part of this PR are:

AWS SSM Parameter Store

Summary by CodeRabbit

  • New Features

    • Introduced a new Store interface with methods for storing and retrieving data.
    • Added support for AWS SSM Parameter Store and an in-memory store implementation.
    • New event hooks for Terraform operations, enhancing extensibility.
    • Enhanced CLI configuration with store management capabilities.
    • Added constants for Atmos YAML functions to improve maintainability.
  • Bug Fixes

    • Improved error handling across various functions and methods.
  • Documentation

    • Updated documentation to reflect new functionalities and configurations.
  • Tests

    • Added unit tests for the new store implementations and their methods.

@mcalhoun mcalhoun changed the base branch from main to chore/refactor-checkatmosconfig December 17, 2024 19:55
Copy link

mergify bot commented Dec 17, 2024

💥 This pull request now has conflicts. Could you fix it @mcalhoun? 🙏

@mergify mergify bot added the conflict This PR has conflicts label Dec 17, 2024
@mergify mergify bot added the conflict This PR has conflicts label Dec 18, 2024
@mergify mergify bot removed the conflict This PR has conflicts label Dec 18, 2024
@mcalhoun
Copy link
Member Author

@coderabbitai full review

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
go.mod (3)

19-19: New logging library.

This new library (github.com/charmbracelet/log) can simplify structured logging. Confirm that it is applied consistently across the codebase for a cohesive logging experience.


92-95: Expanded AWS SSO and STS dependencies.

Since SSO and STS logic can introduce complexities in cross-account or session-based operations, confirm that the updated versions do not break any current usage patterns.


132-132: Minimal feedback on go-logfmt/logfmt addition.

No issues here. If you integrate this formatting approach, consider standardizing logs to maintain consistent style across the project.

pkg/store/aws_ssm_param_store.go (1)

62-69: Parameter writing.

Overwriting existing keys simplifies usage. If you anticipate needing a "merge" or "append" approach, consider future extension here.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e7a432a and 8023ddc.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (2)
  • go.mod (4 hunks)
  • pkg/store/aws_ssm_param_store.go (1 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
pkg/store/aws_ssm_param_store.go

1-1: : # github.com/cloudposse/atmos/pkg/store [github.com/cloudposse/atmos/pkg/store.test]
pkg/store/aws_ssm_param_store_test.go:53:17: cannot use true (untyped bool constant) as *bool value in struct literal
pkg/store/aws_ssm_param_store_test.go:109:22: cannot use true (untyped bool constant) as *bool value in struct literal

(typecheck)

🔇 Additional comments (20)
go.mod (5)

10-12: Added AWS SDK dependencies look aligned with PR objectives.

These additions for AWS SDK v2 and its SSM module align with the introduction of the new AWS SSM store functionality. Be mindful to keep them up-to-date to take advantage of any relevant bug fixes or enhancements.


80-81: Updated AWS credentials and IMDS dependencies.

These updates appear necessary for supporting the new store functionality. If possible, confirm that references to the credentials constructor or IMDS usage follow the updated methods and type signatures introduced in these versions.


83-85: Keep an eye on internal config changes.

The internal configsources, endpoints, and INI modules are central to AWS configuration resolution. Verify that existing unit tests cover edge cases like missing or malformed config files.


87-89: Internal S3-related modules updated.

While these changes likely come in indirectly, ensure that the presigned URL usage is tested so that your code references remain compatible with the updated library.


251-251: golang.org/x/exp updated.

This experimental package can be a moving target. Validate that your code is not reliant on APIs that may change or break in future releases.

pkg/store/aws_ssm_param_store.go (15)

1-2: New package declaration looks good.

This package is well-named to target AWS SSM Parameter Store. No immediate concerns noted.

🧰 Tools
🪛 golangci-lint (1.62.2)

1-1: : # github.com/cloudposse/atmos/pkg/store [github.com/cloudposse/atmos/pkg/store.test]
pkg/store/aws_ssm_param_store_test.go:53:17: cannot use true (untyped bool constant) as *bool value in struct literal
pkg/store/aws_ssm_param_store_test.go:109:22: cannot use true (untyped bool constant) as *bool value in struct literal

(typecheck)


3-6: Imports appear valid and minimal.

These imports cover the AWS SDK and standard Go libraries for context and formatting. Good job keeping them concise.


7-11: AWS and SSM references.

The chosen modules from the AWS SDK are exactly what you need to support SSM. Great alignment with your PR objectives.


13-16: SSMStore struct.

The struct definition is clear, focusing only on what’s needed for the SSM client. This keeps the store abstraction straightforward.


18-20: Region specified in SSMStoreOptions.

Mapping the region in the struct is good. This fosters explicit user configuration and reduces mistakes from default region assumptions.


22-24: Store interface satisfaction.

Declaring the interface conformance is neat and ensures that the store pattern is standardized throughout the codebase.


25-29: Interface for SSMClient.

By introducing SSMClient as an interface, it’s easier to mock AWS interactions for testing. This design choice is flexible.


31-32: NewSSMStore constructor.

Straightforward name. The function returns the store interface, which is a good factory pattern.


35-39: Robust AWS config loading.

Appropriately handling errors from config loading ensures the application fails fast if AWS credentials or config are missing.


41-45: Enforcement of a non-empty region.

Requiring a region is a good design choice. Minimizes runtime confusion by failing early if misconfigured.


47-50: SSM client creation.

Constructing the SSM client from the validated config is straightforward and robust. Nice approach.


52-60: Set method enforces string value.

The design ensures type consistency in SSM parameters. However, be sure to note in your docs that only string values are supported.


70-72: Error contextualization.

Appending the key name in the error message helps with debugging. Nicely done.


77-88: Get method with decryption.

Allowing secure parameter decryption is essential for handling secrets. Great that you return an error if retrieval fails instead of ignoring it.


90-91: Returning string data.

Returning the parameter value as a string is consistent with the approach used in the Set method.

@mcalhoun mcalhoun merged commit 05606ad into main Dec 18, 2024
28 of 29 checks passed
@mcalhoun mcalhoun deleted the feature/add-hooks-config branch December 18, 2024 17:36
@mergify mergify bot removed the needs-cloudposse Needs Cloud Posse assistance label Dec 18, 2024
Listener430 pushed a commit that referenced this pull request Dec 18, 2024
@mcalhoun mcalhoun restored the feature/add-hooks-config branch December 18, 2024 19:11
Copy link

These changes were released in v1.131.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-release Do not create a new release (wait for additional code changes)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant