Support for AWS Lambda Streaming Responses in @remix-run/architect

[prxg22]

AWS recently announced support for streaming responses in AWS Lambda. It seems that this will require modification of the createRequestHandler function in @remix-run/architect to properly receive the Lambda's responseStream and pass it along to sendRemixResponse, which will then pipe it into the Remix response.

Thank you for considering this feature request.

[prxg22]

It's important to note that the API Gateway won't send responses via stream and that it will be necessary to directly activate the lambda through its URL (or make that URL an origin in Cloudfront). Consequently, createRequestHandler could no longer return the type APIGatewayProxyHandlerV2 nor receive events only from the APIGateway.

Perhaps the best solution would be to create another function that can be used by applications that will use response via stream (createStreamRequestHandler).

[prxg22]

If you're seeking a solution, I've managed to address the issue by making modifications to the remix-architecture server.ts file. I've shared my revised version in this repository. And that's how you use the helper:

import * as build from '@remix-run/dev/server-build'
import { installGlobals } from '@remix-run/node'

import { createStreamRequestHandler } from './utils/remix/aws'
installGlobals()

export const handler = createStreamRequestHandler({
  build,
  mode: process.env.NODE_ENV,
})

It's important to keep in mind that when working with APIGateway, there's a critical aspect to consider: the buffering of stream responses. Essentially, the response is only delivered once the entire stream is loaded. A different approach to achieve stream responses is creating a Lambda function URL.

[prxg22]

Are you accessing your lambda directly via Lambda Url?
You can't stream responses through API Gateway, even if your lambda is configured to RESPONSE_STREAM, the API Gateway will buffer the whole response before send it.

[benjamindulau]

@prxg22 Yes. In fact streaming works. What I discovered is that for some reasons Lambda doesn't send the first chunk immediately into the stream. So I was experiencing things like all data from deferred promises to be streamed at the same moment.

I have exactly the behavior described here https://betterdev.blog/lambda-response-streaming-flush-content/

I'm forced to hack every loader payloads to add a big empty string so Lambda sends the chunk immediately... ugly :D

[owlyowl]

@prxg22 Hi Paulo
https://github.com/prxg22/poc-frontend-infra/blob/main/server/utils/remix/aws.ts I'm not sure how your example would work as it's still using API gateway rather than a direct invoke.

Did you modify it after you wrote the original? It still seems to be dealing with API gateway events

[prxg22]

@prxg22 Hi Paulo
https://github.com/prxg22/poc-frontend-infra/blob/main/server/utils/remix/aws.ts I'm not sure how your example would work as it's still using API gateway rather than a direct invoke.

Did you modify it after you wrote the original? It still seems to be dealing with API gateway events

No,@owlyowl, I'm using function url and CDN to access the lambda. Here's where I create the function url (I'm using pulumi to deploy the application).
You must be misleadead by the readme, ignore it.

[prxg22]

@prxg22 Yes. In fact streaming works. What I discovered is that for some reasons Lambda doesn't send the first chunk immediately into the stream. So I was experiencing things like all data from deferred promises to be streamed at the same moment.

I have exactly the behavior described here https://betterdev.blog/lambda-response-streaming-flush-content/

I'm forced to hack every loader payloads to add a big empty string so Lambda sends the chunk immediately... ugly :D

@benjamindulau it's been a while since the last time I did this POC. I dont remember expericing this behavior, my example streams a list of many items from an API, maybe it always respond with enough size that I could not reproduce the behavior.

[owlyowl]

Thanks Paulo I was a bit confused because your incoming server types are all still api gateway related.

…

On Wed, Dec 6, 2023, 8:58 AM Paulo Ricardo Xavier Giusti < ***@***.***> wrote: @prxg22 <https://github.com/prxg22> Hi Paulo https://github.com/prxg22/poc-frontend-infra/blob/main/server/utils/remix/aws.ts I'm not sure how your example would work as it's still using API gateway rather than a direct invoke. Did you modify it after you wrote the original? It still seems to be dealing with API gateway events No, I'm using function url and CDN to access the lambda. Here's <https://github.com/prxg22/poc-frontend-infra/blob/main/index.ts#L71> where I create the function url (I'm using pulumi to deploy the application) — Reply to this email directly, view it on GitHub <#6037 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOT4CUB53UBH3VZFJUSMDDYH6KGRAVCNFSM6AAAAAAW2PN3BKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONRZGM4DI> . You are receiving this because you commented.Message ID: ***@***.***>

[prxg22]

@owlyowl I see. There's no specific types for function url events, but it follows the API Gateway specification.

[owlyowl]

Thanks so much for replying I really appreciate it! Time to test this out with a few defer suspense calls :)

[owlyowl]

Thanks it does. I managed to run it up. Looks good and streaming works as expected :)

…

On Wed, Dec 6, 2023 at 9:13 AM Paulo Ricardo Xavier Giusti < ***@***.***> wrote: @owlyowl <https://github.com/owlyowl> I see. There's no specific types for function url events, but it follows the API Gateway specification. — Reply to this email directly, view it on GitHub <#6037 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOT4CW27EPNKS5QB6QOQX3YH6MA7AVCNFSM6AAAAAAW2PN3BKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONRZGU2DE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gitaugakwa]

I'm trying to get it to work, could be cause I'm on remix v2. I'm really not too sure how to get it working on aws

[owlyowl]

We found that `defer` is quite broken until React 18.3 so until that resolves it makes SSR not as compelling. API Gateway doesn't support streaming lambdas, you'll need to remove API gateway and make the lambda direct invoke in order to enable streaming. AWS have mentioned streaming behind API Gateway is on the roadmap.

…

On Sun, Dec 31, 2023 at 11:30 PM Gitau Gakwa ***@***.***> wrote: I'm trying to get it to work, could be cause I'm on remix v2. I'm really not too sure how to get it working on aws — Reply to this email directly, view it on GitHub <#6037 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOT4CTWXFDWOROBHAIINTTYMFLFXAVCNFSM6AAAAAAW2PN3BKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TSOBTGEYTA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[godu]

I have some troubles with defer. The first chunk is delayed.
Maybe this issue could help aws/aws-lambda-nodejs-runtime-interface-client#94