Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: ADR 59: Incremental static regeneration #5863

Merged
merged 5 commits into from
Feb 8, 2024

Conversation

eatyourgreens
Copy link
Contributor

A quick ADR to describe the consequences of static project page builds, from Engaging Crowds.

Please request review from @zooniverse/frontend team or an individual member of that team.

Linked Issue and/or Talk Post

@coveralls
Copy link

coveralls commented Jan 19, 2024

Coverage Status

Changes unknown
when pulling f7ce73b on eatyourgreens:adr-59
into ** on zooniverse:master**.

@eatyourgreens eatyourgreens mentioned this pull request Jan 19, 2024
3 tasks
@goplayoutside3 goplayoutside3 self-assigned this Jan 22, 2024
When the first Engaging Crowds projects launched, project pages were built dynamically with `getServerSideProps`. Page builds were slow, with pages sometimes taking several seconds to load in the browser ([#2741](https://github.com/zooniverse/front-end-monorepo/issues/2741).)

## Decision
Use [Incremental Static Regeneration](https://nextjs.org/docs/pages/building-your-application/data-fetching/incremental-static-regeneration) (ISR) to build project pages. ISR is a feature of Next.js, which allows page HTML and JSON to be built and cached on demand, then served with `stale-while-revalidate` cache contol headers. While a page is building, visitors will get stale, cached content, resulting in a faster experience.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Serving static content from the CDN is also financially cheaper for Zooniverse, compared with serving HTML and JSON responses from the Next.js origin server.

## Consequences
- Page builds were switched from `getServerSideProps` to `getStaticProps`, with a revalidation time of 60s. Revalidation can be set individually for each page route.
- `getStaticProps` can still be slow, when it runs, so data-fetching from the Panoptes API may need further optimisation ([#3341](https://github.com/zooniverse/front-end-monorepo/issues/3341).)
- Production projects and staging projects each have their own static routes: `/production/:owner/:project` and `/staging/:owner/:project`. Next.js middleware rewrites incoming request URLs to the production and staging routes used by the project app eg. https://fe-project.preview.zooniverse.org/projects/production/nora-dot-eisner/planet-hunters-tess
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In middleware.js, I see .rewrite() when an incoming request URL has a locale param, but there's also rules using .next() instead of redirect:

if (pathname.startsWith('/production')) {
return NextResponse.next()
}
if (pathname.startsWith('/staging')) {
return NextResponse.next()
}
/*
Project pages are served from /projects/staging/[owner]/[project]
and /projects/production/[owner]/[project]
*/
url.pathname = `/${panoptesEnv}${pathname}`
if (url.locale) {
url.href = `${url.origin}/projects/${url.locale}/${panoptesEnv}${pathname}`
}
return NextResponse.rewrite(url)
}

Would you be willing to add a short comment in middleware.js that explains the difference between.next() and redirect() in relation to your point here about rewriting to production and staging routes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Response.next() carries on routing without modifying the response. Here, it’s called when a URL doesn’t need rewriting.
https://nextjs.org/docs/app/api-reference/functions/next-response#next

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the middleware again, I might rearrange it, so that all the responses that should not be rewritten are at the top, followed by the responses that are rewritten.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've rewritten the item about rewrites, rearranged the middleware, and added more comments to middleware.js.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect - thank you!


## Consequences
- Page builds were switched from `getServerSideProps` to `getStaticProps`, with a revalidation time of 60s. Revalidation can be set individually for each page route.
- `getStaticProps` can still be slow, when it runs, so data-fetching from the Panoptes API may need further optimisation ([#3341](https://github.com/zooniverse/front-end-monorepo/issues/3341).)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a cold-start issue when the function first runs? Similar to vercel/next.js#12447.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about that, as the slowness persists after the first call. Here's a screenshot of the network tab for Deciphering Secrets, showing slow requests for JSON page props (marked with a turtle icon in Firefox dev tools.)

https://www.zooniverse.org/projects/profdrrogerlouismartinez-davila/deciphering-secrets-unlocking-the-manuscripts-of-medieval-spain/classify/workflow/19276?demo=true&env=production

Network requests for workflow and subject set page props on Deciphering Secrets. Several prefetch requests for JSON props are slow, taking ~0.5s to download.

Next.js prefetching probably doesn't help here, in that page props are fetched whenever you hover over a link. getStaticProps will run for those links if the cache is stale.

@eatyourgreens eatyourgreens changed the title ADR 59: Incremental static regeneration docs: ADR 59: Incremental static regeneration Feb 2, 2024
@goplayoutside3 goplayoutside3 merged commit 0f56899 into zooniverse:master Feb 8, 2024
7 checks passed
@eatyourgreens eatyourgreens deleted the adr-59 branch February 8, 2024 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing ADRs
3 participants