Improved deployments

[ofhouse]

This is the groundwork to finally enable multiple deployments, but the goal of this issue is not to enable multiple deployments

Motivation

Deployments are currently handled by using a mix of CLI and Terraform.
Running terraform apply for every deployment is considered as a problematic.
Infrastructure is mostly managed by IT departments (With Terraform) while deployments are managed by developers.
So for larger teams this could lead to responsibility issues.

Idea

• To make the CLI and Terraform independent from each other, the Terraform module will be reduced to global resources (e.g. CloudFront, S3 buckets, SQS queue).
• Resources that are needed for deployments will be removed from Terraform (Lambda, API Gateway).
• Instead the static content deployment Lambda will be extended to handle a complete deployment (Creating Lambdas, uploading static files)

Implementation

• Terraform is used to create the base deployment
• Deployment Lambda uses internally AWS CDK to create a new CloudFormation stack for every Subdeployment (Lambdas, Roles, etc.), this way created resources can be tracked by administrators and simply removed when the whole deployment is removed.
  This has already been tested: playground-cdk-cloudformation
• New global resource dynamoDB is added that holds the routing + image optimizer config for every deployment
• After the SubStack is created successfully (Lambda trigger), fade-out of previous deployment begins (DynamoDB update, CloudFront invalidation)
• After successful deployment of new SubStack, the old SubStack gets removed (S3 assets are set to expire, as not part of the stack)

Further ideas

• Remove API Gateway from the stack
  With the introduction of Function URLs the usage of API Gateway becomes obsolte.
  Advantages:
  • No extra costs for API Gateway (https://twitter.com/jbesw/status/1511812900742172672)
  • Increased timeout 30s (API Gateway) -> 60s (CloudFront)
• Offer multiple CDK constructs for different needs
  • With Function URLs -> Unlimited deployments
  • With API Gateway -> Limited deployments, but access to native features (Lambda authorizer, rate limiting)

Related issues

• [Question] Splitting tf-next build #245
• s3-put fails with error 403 #40
• [0.8.1] Error: AWS Secret Access Key not specified #121
• Use profile credentials for aws s3 upload #126
• Error during deployment regarding concurrency #251 (Concurrent deployments would be possible)

[zatoa]

@ofhouse Hello! I've yet to try this awesome TF module however I'm just wondering about what you're suggesting with API Gateway and the future of it here. I'm designing a multi-tenant app in NextJS which can then be deployed into AWS using this module. The Tenant ID is take from the Domain and each tenant has a separate database

Reading AWS's https://aws.amazon.com/blogs/apn/building-a-multi-tenant-saas-solution-using-aws-serverless-services/#:~:text=API%20Authorization%20and%20Tenant%20Isolation guide on SaaS solutions and multi-tenancy, it looks like they've written up how to achieve good tenant isolation by using an API Gateway Lambda Authoriser to set the context of the tenant and return a dynamic IAM policy based on this.

So for instance I could return a set of temporary credentials from the Lambda Authoriser which the 'NextJS API Lambda' from this module could use to safely talk to a database and ensure good tenant isolation. This would mean that the developers cannot make a mistake and accidentally cross tenant domains in code.

I think this is all only possible to achieve through keeping the API Gateway in place? I haven't tried this yet but I would like to somehow after deploying this module attach this Lambda Authoriser to the newly created API Gateway and achieve this setup. I'm not sure if this is possible without implementing inside the module an API Gateway ID as output so I can then attach something? I think there could be a good benefit to having the option of either using API Gateway or the Function URL as an option.

To sum up what I would like to try achieve:

• Add a Lambda Authoriser to the created API Gateway so the NextJS API Lambda runs with dynamic credentials
• Add a Viewer request lambda@edge to the Cloudfront distribution so that I can preauth for development and not expose anything at all publicly
• Lockdown S3 static bucket to only allow connections from Cloudfront (Not sure if this module does this)
• Lockdown API Gateway so that only connections from Cloudfront are accepted by injecting an API Key?

Thanks for this module , I look forward to using it!

[ofhouse]

@Foralux Thanks for the input!

Regarding the future of API Gateway in this project
API Gateway has a hard limit of 600 instances per account. This technically limits the the total number of concurrent deployments to 600. The current plan is to make the number of deployments indefinitely scalable.
However I think that there are valid use cases (yours & #259) for using native functionality of API Gateway. So it probably makes sense to offer options to swap unlimited deployments for the usage of API Gateway in certain cases.
Since each deployment will use a CDK construct, we could offer different CDK templates that fulfill different purposes.

React 18 and the introduction of streaming SSR responses are another point why I think that function URLs will play an important role in the future.
Since Vercel (as a really big user of Lambda) currently promotes HTTP streaming as the future to decrease TTFB (Time to first byte) further, I think that function URLs is something that was built upon their request.
Because API Gateway cannot be modified to support streams (Since schema validation is synchronous), they build a very light wrapper that converts HTTP requests into Lambda events.
This is just 1 step away from accessing request and response directly inside Lambda, so I think we will see support for streaming responses soon in Lambda through function URLs.

Regarding your plans

• Add a Lambda Authoriser to the created API Gateway so the NextJS API Lambda runs with dynamic credentials

Could be possible when offering different CDK templates as explained above.

• Add a Viewer request lambda@edge to the Cloudfront distribution so that I can preauth for development and not expose anything at all publicly

This is out of scope of the project, but can be archived by modifying the CloudFront distribution which allows to add more Lambda@Edge functions to the flow.

Lockdown S3 static bucket to only allow connections from Cloudfront (Not sure if this module does this)

This is already built into this module, since we already use origin access identity.

Lockdown API Gateway so that only connections from Cloudfront are accepted by injecting an API Key?

Already looked into this (#27), but currently requires to sign each request in in the Lambda@Edge before forwarding it to API Gateway. Decided against it, since the risk of guessing the endpoint is minimal and would increase latency.
However I know that the CloudFront team is looking into expanding origin access identity to more services than S3, so maybe there will be a native solution for this soon.

[ofhouse]

RFC has now landed in the main branch.