Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Only publish single aws-pdk package #536

Merged
merged 37 commits into from
Aug 27, 2023
Merged

Conversation

agdimech
Copy link
Contributor

@agdimech agdimech commented Aug 15, 2023

Changes

This PR restructures the PDK as follows:

  • Individual packages are now namespaced as @aws-pdk and are NOT published to npm/pypi/maven.
  • A new aws-pdk package is created which creates a single distributable which IS published to npm/pypi/maven.
  • Dependencies between packages is supported via @aws-pdk/XXX. When packaging into the aws-pdk, these imports are rewritten using a combination of tsconfig paths and tsc-alias.
  • PDKProject has been refactored to pull out PDKRelease as this should now only happen in AwsPdkProject.
  • Package tasks have been removed from individual packages as this now should only occur in aws-pdk.
    -aws_arch and cdk_graph_plugin_diagram have been refactored slightly to ensure exported types make JSII happy.
  • Overall DAG has been simplified and improvements to build times have been made ~8 mins
  • docs has been pulled up to the root level
  • private has been replaces by projenrc
  • PDKPipelineXXProject has been removed from the pipeline package.
  • NXMonorepo ensure NodeResolution is nodenext for TS projects
  • Changed transient deps script to use shallowest transient dep version instead of greatest given we are now bundling everything together in one.
  • Interfaces no longer expose required props when they aren't actually required.
  • All TSProjects under a monorepo use tsconfig module reoslution settings of nodenext and target es2021
  • Added a global pdk command which aliases underlying package manager command 🕺
  • Added an API Explorer powered by SwaggerUI to the CloudscapeReactTsWebsite which allows you to make authenticated API requests
  • Added an Infra package which allows you to create boilerplate CDK code for an API + Website. This means you can get a fully functioning website + API, with all Infra in < 70 LoC 👏

BREAKING CHANGES

  • All imports will need to be changed to aws-pdk/<pkg>
  • All dependencies on @aws-prototyping-sdk/* will need to be changed to a single dependency on aws-pdk
  • Refactored CDKGraph & CdkGraphDiagramPlugin to poly friendly definitions which will break existing consumers in TS (Py and Java never worked).
  • Removed PDKPipeline projen constructs.
  • Renamed nx-monorepo to monorepo
  • TypeSafeApi Python runtime moved to new python-nextgen OpenAPI generator, which means:
    • Properties of models can be referenced as attributes rather than dictionary syntax (eg .my_property rather than ["my_property"]).
    • Models are now serialised and deserialised using the .from_json and .to_json methods.
    • request_parameters and request_array_parameters are also models.
  • TypeSafeApi Python handlers import location changed from <package>.apis.tags.default_api_operation_config to <package>.api.operation_config.
  • TypeSafeApi Java classes for handlers/interceptors in runtime are no longer static subclasses of the Handlers class. The import location has changed from eg <package>.api.Handlers.SayHello to <package>.api.handlers.say_hello.SayHello.

Remaining items

  • Update Github workflow and re-enable doc publishing once written
  • Update documentation including migration guide
  • Document experimental vs stable
  • Bump deps
  • Create Infra project that gens infra for API + Website (Blueprint)
  • Clean up MonorepoTs props to categorize yarn vs pnpm specific props
  • Resolve [BUG] Workspace linking for constructs is not working #365
  • Refactor cloudscape construct to use native AppLayout oppose to NorthStar AppLayout
  • Relax ESLINT/TSC errors in website for dev
  • Namespace scripts
  • Rename NXMonorepo
  • Add powertools to handlers
  • Resolve [BUG] cdk-graph-plugin-diagram has issues in Py/Java #527
  • Refactor projen constructs to remove unneeded props i.e. defaultReleaseBranch
  • Cherry pick @cogwirrel changes

Copy link
Member

@cogwirrel cogwirrel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great so far!

You'll also need to update the import in the addDeps for the tsapi Java/Python infra projects :)

packages/aws-pdk/src/aws_arch/src/index.ts Outdated Show resolved Hide resolved
@agdimech agdimech force-pushed the refactor/uber branch 3 times, most recently from e9d093c to 6b07fa3 Compare August 15, 2023 03:38
Copy link
Contributor

@JeremyJonas JeremyJonas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Great fix :D

cogwirrel and others added 14 commits August 22, 2023 17:14
Due to a limitation in OpenAPI generator I previously generated all of the handler wrappers etc as
static subclasses of a Handlers class, since I didn't have enough control to generate a file per
desired class. With the post processing built for the handler projects, we can now split into
separate files which makes autocomplete for java a bit more intuitive.
… handlers

Generate a lambda function construct for each operation annotated with the @handler trait as part of
the generated infrastructure package to simplify referencing the built artifacts for the handlers.
@agdimech agdimech merged commit c43f3b6 into mainline Aug 27, 2023
3 checks passed
@agdimech agdimech deleted the refactor/uber branch August 27, 2023 22:48
agdimech added a commit that referenced this pull request Sep 8, 2023
…projects (#544)

* feat(pipeline): enable setting CodeBuild properties on FeatureBranch projects

Apply the properties from the PDKPipeline `codeBuildDefaults` property to the CodeBuild project that
creates feature branch pipelines. This enables  configuration such as role policies, filesystems,
and networking. The role policy is useful to enable access to private repos in CodeArtifact.

The feature branch CodeBuild project now defaults to the `STANDARD_7_0` build
image.

* fix: convert link bundled deps to js (#546)

* refactor: Only publish single aws-pdk package (#536)

This commit restructures the PDK as follows:

- Individual packages are now namespaced as @aws-pdk and are NOT published to npm/pypi/maven.
- A new aws-pdk package is created which creates a single distributable which IS published to npm/pypi/maven.
- Dependencies between packages is supported via @aws-pdk/XXX. When packaging into the aws-pdk, these imports are rewritten using a combination of tsconfig paths and tsc-alias.
- PDKProject has been refactored to pull out PDKRelease as this should now only happen in AwsPdkProject.
- Package tasks have been removed from individual packages as this now should only occur in aws-pdk.
- aws_arch and cdk_graph_plugin_diagram have been refactored slightly to ensure exported types make JSII happy.
- Overall DAG has been simplified and improvements to build times have been made ~8 mins
- docs has been pulled up to the root level
- private has been replaces by projenrc
- PDKPipelineXXProject has been removed from the pipeline package.
- Updated Monorepo to ensure NodeResolution is nodenext for TS projects
- Changed transient deps script to use shallowest transient dep version instead of greatest given we are now bundling everything together in one.
- Interfaces no longer expose required props when they aren't actually required.
- All TSProjects under a monorepo use tsconfig module reoslution settings of nodenext and target es2021
- Added a global pdk command which aliases underlying package manager command 🕺
- Added an API Explorer powered by SwaggerUI to the CloudscapeReactTsWebsite which allows you to make authenticated API requests
- Added an Infra package which allows you to create boilerplate CDK code for an API + Website. This means you can get a fully functioning website + API, with all Infra in < 70 LoC 👏

BREAKING CHANGES
- All imports will need to be changed to aws-pdk/<pkg>
- All dependencies on @aws-prototyping-sdk/* will need to be changed to a single dependency on aws-pdk
- Refactored CDKGraph & CdkGraphDiagramPlugin to poly friendly definitions which will break existing consumers in TS (Py and Java never worked).
- Removed PDKPipeline projen constructs.
- Renamed nx-monorepo to monorepo
- TypeSafeApi Python runtime moved to new python-nextgen OpenAPI generator, which means:
  - Properties of models can be referenced as attributes rather than dictionary syntax (eg .my_property rather than ["my_property"]).
  - Models are now serialised and deserialised using the .from_json and .to_json methods.
  - request_parameters and request_array_parameters are also models.
- TypeSafeApi Python handlers import location changed from <package>.apis.tags.default_api_operation_config to <package>.api.operation_config.
- TypeSafeApi Java classes for handlers/interceptors in runtime are no longer static subclasses of the Handlers class. The import location has changed from eg <package>.api.Handlers.SayHello to <package>.api.handlers.say_hello.SayHello.

* feat: Change name to `@aws/pdk` (#550)

Change name to @aws/pdk

* feat(type-safe-api): coerce request parameters into defined types for typescript (#551)

Previously request parameters were always strings no matter the type defined in the model. Now we
add code to the typescript lambda handler wrappers to convert to appropriate types. Supports the
Smithy supported request parameters number/string/bool/date and arrays of those. Responds with a 400
error if the wrong type is passed, and includes headers from any response header interceptors in
this validation response.

BREAKING CHANGE: Removed RequestArrayParameters from input and type signature of
handlers/interceptors. number/bool/date request parameters will be coerced to their type defined in
the model. useIntegerType: true is specified by default for smithy to openapi conversion as this is
recommended and more intuitive behaviour than the default (where integers are still represented as
doubles).

* fix: enhance boilerplate infra & update docs (#552)

- tweak getting started docs
- update infra sample code to apply cognito authrole permissions to call deployed sample API

* fix: add shopping list docs (#554)

* feat(type-safe-api): python and java request parameters (#555)

* feat(type-safe-api): coerce request parameters into defined types for java

Request parameters for java are now of the correct types declared in the model. Note that only
primitive types and single level arrays of primitives are supported (as per Smithy's supported types
for request parameters). OpenAPI allows you to define request parameters as objects with different
encoding strategies but these are not supported by type safe api. Adds validation to make sure that
the request parameters are of the supported types as part of the model build task. Also moves the
Smithy trait injection logic to the model build task such that traits are added to the .api.json
file produced by the model build, rather than as a preprocessing step each time code is generated.

BREAKING CHANGE: Removed RequestArrayParameters from handler inputs and folded them into
RequestParameters. Renamed CorsInterceptor to ResponseHeadersInterceptor. Validated to ensure only
primitives (or arrays of primitives) are used in request paramters (query/path/headers).
RequestParameters coerce string values into their appropriate types for the handler wrappers to
consume, and will respond with a 400 if request parameters are of the wrong type.

* feat(type-safe-api): coerce python request parameters into declared types

Removes request_array_parameters and folds all request parameters into input.request_parameters.
cors_interceptor.py is now response_headers.py for the cors interceptor.

BREAKING CHANGE: Removes request_array_parameters and folds all request parameters into
input.request_parameters. cors_interceptor.py is now response_headers.py for the cors interceptor.

* fix: update monorepo docs (#556)

* docs(type-safe-api): update docs and migration guide (#557)

* docs(type-safe-api): update docs and migration guide

Also enable push to github pages, and add missing webAclOptions property to generated java cdk
construct.

* fix: cleanup sections

---------

Co-authored-by: Adrian Dimech <dimecha@amazon.com>

* docs: resolve banner issue (#558)

* docs: update migration guide (#559)

* docs: update migration guide

* docs(type-safe-api): improve lambda handler docs

---------

Co-authored-by: 🚀 Jack <jacsteve@amazon.com>

* perf(type-safe-api): reduce type-safe-api project build times by over 100 percent (#564)

This change moves the installation of all dependencies for type-safe-api scripts into a central
~/.pdk directory, namespaced by the version and package manager used. This means we save the install
time for every script that runs, once the first script has run.

* fix(cloudscape-react-ts-website): pin Swagger UI (#565)

* fix(type-safe-api): update gradle to version 8 to allow java versions from 8 to 20 (#567)

Gradle 8 is compatible with java 8 to java 20, see
https://docs.gradle.org/current/userguide/compatibility.html

Fixes #566

* feat: refactor pdk to support simplified imports (#568)

---------

Co-authored-by: Adrian Dimech <51220968+agdimech@users.noreply.github.com>
Co-authored-by: Jack Stevenson <jacsteve@amazon.com>
Co-authored-by: Adrian Dimech <dimecha@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] cdk-graph-plugin-diagram has issues in Py/Java [BUG] Workspace linking for constructs is not working
3 participants