programmatic generate client config starter #43

edwardfoyle · 2023-06-10T01:50:22Z

Issue #, if available:
fixes #40

Description of changes:
Needs tests but it's ready for eyes on the overall approach

Starting point for fetching stack output and reconstructing into objects that can be passed to the construct plugins for parsing to frontend config. Only the output fetching and reconstruction is implemented here.

I refactored the ProjectEnvironmentIdentifier class into a new primitives package. This allows both the reader and writer of the SSM stack pointer to use the same ProjectEnvironmentIdentifier.toOutputStackSSMParameterName when resolving the ssm key name

I also added a zod schema for the metadata object that is stored in the stack. This allows the writer and reader to validate against the schema

The generate package introduces several new classes:

ClientConfigGeneratorFactory produces ClientConfigGenerators given either a stack name or a project environment identifier
DefaultClientConfigGenerator is largely a placeholder right now for when the actual frontend translation logic is implemented
StackMetadataOutputRetrievalStrategy is the counterpart to StackMetadataOutputStorageStrategy that is implemented in backend-engine. It uses the shared schemas from the primitives package to load and validate the stack metadata and output
ProjectEnvironmentStackNameResolver looks up the stack name based on the SSM param for the project environment
PassThroughStackNameResolver returns the given stack name in the case where the stack name is already known (customer is using CDK directly)

Some open questions:

Now that I've implemented this, I've gotten a little more clarity on this issue. I think we might want a BackendIdentificationStrategy in addition to the OutputStorageStrategy. The BackendIdentificationStrategy should perhaps go in the primitives package so that backend-engine can call something like backendIdStrat.setIdentifier(stack) and frontend generation can call backendIdStrat.resolveStackName()
should the entry point to generateClientConfig be a function or an exported class that consumer constructs? How much DI should the consumer do vs the exported function package?

EDIT:
Another idea is instead of a "primitives" package (which could become a dumping ground for "stuff that needs to be shared between different packages"), introduce a "backend-metadata" package that is responsible for reading/writing metadata for a backend. The backend-engine can use this package for writing and the generate package can use it for reading. Essentially the "backend-metadata" package becomes the wrapper around our data layer (which is CFN metadata and outputs)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

packages/backend-engine/src/amplify_stack.ts

packages/backend-engine/API.md

packages/generate/API.md

packages/generate/src/amplify_backend_output.ts

packages/generate/src/client_config_generator.ts

packages/generate/src/output_retrieval_strategy.ts

sobolk · 2023-06-12T22:34:07Z

packages/generate/src/output_retrieval_strategy.ts

+/**
+ * Gets Amplify backend outputs from stack metadata and outputs
+ */
+export class StackMetadataOutputRetrievalStrategy
+  implements OutputRetrievalStrategy


this type and factory should either go to backend-engine or "that long name for package that hosts metadata strategy logic"

packages/generate/package.json

edwardfoyle · 2023-06-14T00:23:15Z

Apart from still needing more tests, this is ready for another review.

What changed:

the primitives package and generate package have been removed. The stuff that was in primitives has been split into putting the types in plugin-types and putting the impl in backend-engine. The stuff that was in generate is now in backend-engine. This was done to prevent premature splitting into a bunch of tiny packages. I have created some README files in backend-engine to explain the current structure and hopefully provide some clarity to a future soul that might have to pull these parts into separate packages
A BackendIdentifier has been introduced that decouples the customer facing backend identification method from the classes that have internal methods hanging off of them. The ProjectEnvironmentIdentifier is now a customer facing type with no internal details on it. Instead, the ProjectEnvironmentIdentifier can be passed into a BackendStackCreator or BackendStackResolver depending on the use case

Amplifiyer · 2023-06-14T12:13:12Z

...es/backend-engine/src/backend-output/get_project_environment_main_stack_ssm_parameter_key.ts

+ * This key is shared between stack creation in \@aws-amplify/backend and stack discovery during client config generation.
+ * It is the "breadcrumb" that enables client config generation to discover the stack outputs for a given project environment
+ */
+export const getProjectEnvironmentMainStackSSMParameterKey = (


I don't think we need to expose SSM in the name of this method as it's just an internal detail. It could just be getMainStackIdentifierKey or something like that.

I'd rather keep this one as-is. It should be super clear this key is for something very specific, ie using SSM to identify a stack name given a project environment. It shouldn't be used for anything else

packages/backend-engine/src/backend-output/stack_metadata_output_retrieval_strategy.ts

packages/backend-engine/src/generate-client-config/stack_name_backend_stack_resolver.ts

...ages/backend-engine/src/generate-client-config/project_environment_backend_stack_resolver.ts

packages/backend/src/project_environment_backend_stack_creator.ts

packages/backend/src/default_stack_factory.ts

packages/backend-engine/package.json

packages/backend-engine/src/backend-output/stack_metadata_output_retrieval_strategy.ts

packages/backend-engine/src/generate-client-config/client_config_generator_factory.ts

packages/backend-engine/src/generate-client-config/stack_name_backend_stack_resolver.ts

packages/backend-engine/src/index.ts

packages/plugin-types/API.md

sobolk · 2023-06-14T18:17:55Z

packages/backend-engine/API.md

+// @public
+export const getProjectEnvironmentMainStackSSMParameterKey: (projectEnvironmentIdentifier: ProjectEnvironmentIdentifier) => string;


After giving it more thoughts I think we should think about duplicating this logic in both packages and perhaps enforce that both implementations stay in sync in other ways.
Reason is - how do we roll major version bump if the identifier convention changes in a breaking way.

If we keep it here I suggest to wrap it in namespace since this is kind of leaking implementation details.

Duplicating is probably okay here as this is something that will likely never change / would be a huge breaking change if we did

…k a singleton

.eslintrc.json

packages/backend-engine/src/backend-output/stack_metadata_output_storage_strategy.ts

sobolk

LGTM.

packages/backend-engine/package.json

packages/backend-engine/src/backend-output/stack_metadata_output_retrieval_strategy.ts

packages/backend-engine/src/client-config-generator/client_config_generator.ts

packages/backend-engine/src/client-config-generator/generate_client_config.ts

...ine/src/client-config-generator/get_project_environment_main_stack_ssm_parameter_key.test.ts

Amplifiyer

Amazing work, thanks @edwardfoyle

edwardfoyle added 3 commits June 9, 2023 13:46

move ProjectEnvironmentIdentifier to primitives package

ff97e5f

output fetching and construct data reconstruction

9abd6ea

rename schema const

d181028

edwardfoyle requested review from sdstolworthy, sobolk, Amplifiyer and danielleadams as code owners June 10, 2023 01:50

update test imports

d377fb0

sobolk reviewed Jun 12, 2023

View reviewed changes

packages/generate/package.json Outdated Show resolved Hide resolved

edwardfoyle added 15 commits June 13, 2023 09:05

Merge remote-tracking branch 'origin/main' into generate

814c77a

rename plugin-types to backend-types

531f09c

getting everything to build

0404853

more refactoring

db2ae4e

more refactoring

fe79b6a

update package json

a1e623c

fix tests

f7a3b72

revert plugin-types rename

3323e74

update api extract

e02dcf0

renaming and self review

59e5273

a bit more cleanup

d1fee64

update api extract

79a56b1

remove unused file

d10e21b

add some comments

56f8ed2

update import and sort lines

00a574a

Amplifiyer reviewed Jun 14, 2023

View reviewed changes

edwardfoyle added 2 commits June 14, 2023 10:20

put all amplify metadata under AWS::Amplify::Output key

11235af

fixing some naming and comments

b23fb30

sobolk reviewed Jun 14, 2023

View reviewed changes

edwardfoyle added 8 commits June 14, 2023 12:05

tests for output retrieval strat

cbc0f6e

renaming BackendStack => MainStack and some other renaming

bf0e849

rename BackendStackCreator => MainStackCreator and make returned stac…

361f61e

…k a singleton

duplicate param key to avoid exporting impl details

a6254c6

update package json and import lint rules

bdba6c5

update lint rule

bf75d22

update api extract

6d4dcd5

turn off lint rule that doesnt play nice with workspaces

66ea0d4

edwardfoyle mentioned this pull request Jun 14, 2023

Use package exports to namespace concerns within packages #45

Closed

edwardfoyle added 2 commits June 14, 2023 15:22

add tests

88ac24c

remove placeholder test

bea561d

sobolk reviewed Jun 14, 2023

View reviewed changes

.eslintrc.json Outdated Show resolved Hide resolved

Amplifiyer reviewed Jun 14, 2023

View reviewed changes

packages/backend-engine/src/backend-output/stack_metadata_output_storage_strategy.ts Show resolved Hide resolved

sobolk previously approved these changes Jun 14, 2023

View reviewed changes

edwardfoyle mentioned this pull request Jun 15, 2023

Rename methods on OutputStorageStrategy #47

Closed

update method name, add shared file for ssm test

4f7862d

edwardfoyle dismissed sobolk’s stale review via 4f7862d June 15, 2023 17:34

update api extract

69cb0f0

Amplifiyer approved these changes Jun 15, 2023

View reviewed changes

sobolk approved these changes Jun 15, 2023

View reviewed changes

edwardfoyle merged commit 0703074 into main Jun 15, 2023

edwardfoyle deleted the generate branch June 15, 2023 17:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

programmatic generate client config starter #43

programmatic generate client config starter #43

edwardfoyle commented Jun 10, 2023 •

edited

Loading

sobolk Jun 12, 2023

edwardfoyle commented Jun 14, 2023

Amplifiyer Jun 14, 2023

edwardfoyle Jun 14, 2023

sobolk Jun 14, 2023

edwardfoyle Jun 14, 2023

sobolk left a comment

Amplifiyer left a comment

		// @public
		export const getProjectEnvironmentMainStackSSMParameterKey: (projectEnvironmentIdentifier: ProjectEnvironmentIdentifier) => string;

programmatic generate client config starter #43

programmatic generate client config starter #43

Conversation

edwardfoyle commented Jun 10, 2023 • edited Loading

sobolk Jun 12, 2023

Choose a reason for hiding this comment

edwardfoyle commented Jun 14, 2023

Amplifiyer Jun 14, 2023

Choose a reason for hiding this comment

edwardfoyle Jun 14, 2023

Choose a reason for hiding this comment

sobolk Jun 14, 2023

Choose a reason for hiding this comment

edwardfoyle Jun 14, 2023

Choose a reason for hiding this comment

sobolk left a comment

Choose a reason for hiding this comment

Amplifiyer left a comment

Choose a reason for hiding this comment

edwardfoyle commented Jun 10, 2023 •

edited

Loading