Any idea how to disable range requests in Next.js ?

[mderazon]

It seems to be affected by the same issue although it's not using Express AFAIK
Also, is this misbehavior tracked somewhere in a Github issue in Node.js / Express repos ?
Is this even a bug ?

[DanielLarsenNZ]

Hi @mderazon, no idea how to do this in Next.js I am sorry. No issue that I can find in Express repo. I don't think it is a bug because I don't think it is realistic for a streaming HTTP library to be able to calculate compressed content length ahead of time (without sacrificing performance). I think disabling compression is a fair workaround.

[mderazon]

So practically, once this PoP update rollout is completed, anyone using Azure CDN will have to disable compression in origin ?

[DanielLarsenNZ]

Only servers and frameworks that combine compression and range responses, and don't properly calculate the Content-Range header values. My advice is to use compression (only) for scripts, stylesheets, JSON and HTML files; any files that compress well. And use ranges (only) for large media files that need to seek or be pre-fetched in shards. Most media formats (like MPEG) are already compressed.

[mderazon]

Only servers and frameworks that combine compression and range responses, and don't properly calculate the Content-Range header values

Which looks like includes Node.js / Express / Next.js servers with default config and probably more http servers that I don't know of, but even this list alone accounts for a lot of issues.

If this is rolled out, isn't it going to break a lot of things ?

I just tested it by disabling Next.js compression and sent a request through Envoy proxy --> Next.js and the tests are still failing. So looks like Envoy is also not handling it according to the rfc .. And disabling compression on the proxy is not an option (and can't do conditional compression on specific routes)

To me, it seems like this change will break lots of customer's products

envoyproxy/envoy#28828

[DanielLarsenNZ]

Compression is not on by default in Express, but you make a good point. The Envoy implications are troubling... I don't work for Microsoft anymore, so tagging @johndowns who may have an inside view.

[DanielLarsenNZ]

@mderazon can you email me daniel@larsen.nz?

[mderazon]

I just sent you an email. Appreciate your help on this

[joel1st]

Can confirm - we have been effected by Azure FD caching + Next.js default behaviour around compression/range. Next.js provides a setting to disable compression (https://nextjs.org/docs/app/api-reference/next-config-js/compress), but not ranges. Quite a tricky one to diagnose when the PoP migrations are progressively rolled out, so it works for most users but not a subset of PoPs

[mscbpi]

Apps and Servers that make use of compression but cannot compute and set Content-Range header accordingly should ignore Range: header in the request when compression is enabled and answer an HTTP/200 with the entire content in that case.

It makes no sense to respond HTTP/206 Partial Content with wrong headers. I am curious how node.js dev justify it.

And indeed it makes it incompatible with Azure front door caching.

[mderazon]

@mscbpi I don't think it's just node.js, for example we are using Envoy proxy (through Emissary Ingress) which is Golang based and same problem is there as well.

[mscbpi]

Yep but then it limits CDN support at least Front Door and several public examples shows how MSFT is referring the Origin as a non-compliant source of issue (just as this repo describes), being strict about it.

How many users will be able to tweak their origin to comply ?
On the other hand shouldn't apps BE compliant one way or the other ? Concerning OSS how much will it be considered for Envoy or Express or others if the is highlighted mistake (Content-Range not matching actual length when compression is in place, and not complying with RFC) is reported as a bug ?

directus had the same problem with dynamic on-the-fly image compression/transformation (asked in the user request), and combined with Range Requests, wrong response headers in HTTP/206. And trickiness of setting the right values with that kind of process. My (accepted) suggestion has been to ignore Range when transformation is in place and answer an HTTP/200 in that very case. So we can enable caching transformed assets on CDN side.

Our frontend is Angular-based, thus the tweak there was to disable gzip compression, while enabling it on CDN side, so the user gets a cached and (brötli) compressed answer, but not without some adaptation in the code again.

Not every Origin is one's self code or open source, and may not be configurable in that way.

[mscbpi]

Hi @mderazon, no idea how to do this in Next.js I am sorry. No issue that I can find in Express repo. I don't think it is a bug because I don't think it is realistic for a streaming HTTP library to be able to calculate compressed content length ahead of time (without sacrificing performance). I think disabling compression is a fair workaround.

Isn't ignoring Range: when compression is in place a better workaround (what Express should do IMHO?) ?

• Answering HTTP/200 instead of 206 is a compliant answer
• Avoids answering something that is not (HTTP/206 with wrong headers)
• Keeps the Origin full featured (compression still active).
• IMHO it is not regressing as Partial-Content answer for these would not be handled correctly anyway, at best ignoring mistaken headers and accepted as-is, so better response a HTTP/200 for that.

[mderazon]

Our frontend is Angular-based, thus the tweak there was to disable gzip compression, while enabling it on CDN side, so the user gets a cached and (brötli) compressed answer, but not without some adaptation in the code again.

I wanted to disable compression on Next.js as well and let the CDN handle it solely. Problem is that request also passes through Envoy (Ambassador API gateway) and there it's being compressed as well (if not already compressed) and same problem happens there. I can't really disable compression on the API gateway as many other apis are going through there as well and they need compression and I can't do "compression by path", it's either on or off.

Concerning OSS how much will it be considered for Envoy or Express or others if the is highlighted mistake

I haven't been able to find any issue in their Github repos related to this which is weird. But maybe I haven't looked well enough. I am not familiar with the spec well enough to create an issue on these repos either

[mscbpi]

Tried my luck

expressjs/compression#185