CDN purge and version mismatch when self-hosting Next.js

[haapti]

Anyone faced issues with version mismatch when caching Next.js app on CDN?

What we learned so far?

We cant have multiple different versions of HTML documents and RSC payloads on CDN of same Next.js app. Version needs to be same between all HTML documents and RSC payloads, and between origin Next.js server. Next.js version will change on every new build.

Caused issues

• Hard navigation loop: Next.js will automatically reload the application to retrieve new assets when detected. For example, if there is a mismatch in the deploymentId, transitions between pages will perform a hard navigation versus using soft navigation. If RSC payloads are in different version to HTML documents hard navigation is performed on each navigation what is not desired for SPA web app.

• Version mismatch client vs. server: If different versions of Next.js application run on client and server, it causes application errors and other unexpected behavior. For example, imagine your newest deployment modifies the data structure by adding a required field to a user's profile. Older clients wouldn't expect this new field, leading to errors when they submit it via POST to backend Next.js server.

Version

Next.js 14.x with app directory using static and SSG pages.

Terminology

HTML document

Initial HTML payload when user open any page/route first time

RSC Payload

Client either prefetches or fetches the RSC Payload on navigation. For example, when navigating to /page-2 you see RSC fetch /page-2?_rsc=xxx happening on background.

On the client:

• The initial HTML document is used to immediately show a fast non-interactive preview of the route - this is for the initial page load only.
• The React Server Components Payload is used to reconcile the Client and Server Component trees, and update the DOM.
• The JavaScript instructions are used to hydrate Client Components and make the application interactive.

What is the React Server Component Payload (RSC)?

The RSC Payload is a compact binary representation of the rendered React Server Components tree. It's used by React on the client to update the browser's DOM. The RSC Payload contains:

The rendered result of Server Components

Placeholders for where Client Components should be rendered and references to their JavaScript files

Any props passed from a Server Component to a Client Component

References

• RSC and CDN interaction makes next.js inefficient for highload projects #65335 and RSC and CDN cache problems makes the next.js useless for the highload projects #59167 (This can be solved by ignoring _rsc search parameter in cache rules)
• Next.js PR about _rsc search param Add unique search query for RSC requests to be cacable on CDN #50970
• https://nextjs.org/docs/app/building-your-application/rendering/server-components#how-are-server-components-rendered
• How Vercel Cloud uses skew protection https://vercel.com/blog/version-skew-protection
• Cloudflare cache setup on OpenNext project https://github.com/sst/open-next/blob/9d632d1bd5c3a8d7e7bbd13ae9daaca68da9284e/examples/sst/stacks/OpenNextReferenceImplementation.ts#L329-L472
• SST is a framework that makes it easy to build modern full-stack applications on your own infrastructure. They prefer cache invalidation as default setting for Next.js builds https://sst.dev/docs/component/aws/nextjs#invalidation

How to solve the issue?

We very likely need to purge the cache in way or another after every build. This does not feel desired option for hightraffic site.

Options in the air:

• Purge all cache
• Purge HTML documents and RSC payloads using Cache-tag: document
• Purge + “preheat” cache by moving the traffic to purged cache little by little