feat(gatsby): enable granular chunks #22253

wardpeet · 2020-03-13T15:50:48Z

Description

Enable granular chunking for Gatsby. This PR helps us improve our webpages in 2 ways:

Less duplications (less javascript)

By bundling a dependency that is at least used in 2 pages we can bundle them together so we don't have to download duplicate libraries over and over again. This won't benefit first-page load but it improves page navigation as you'll need less javascript for the next route.

Big libraries over 160kb are moved to a separate library all together to improve js parsing & execution costs.

Our commons chunk is used to put in code that is used over all our pages (commons). So we don't bloat any pages more than we need

Caching

We've added a framework bundle which contains react, react-dom as this is a chunk that will hardly ever change. Commons & shared libraries can change when new pages are created which is more likely to happen than react being upgraded. We might want to move other modules into it, like @reach/router.

Overall caching is improved as chunks will change less often.

site	kb diff	% diff
https://www.gatsbyjs.org/	-680kb	-22,6%
https://www.thirdandgrove.com/	-390kb	-25,5%
https://ghost.org/	-1110kb	-34,79%
https://reactjs.org/	-80,58kb	-8,15%
https://gatsby-starter-blog-demo.netlify.com/	+0,38kb	-0%

These are the links for gatsbyjs.org & reactjs.org as they are opensource sites.

gatsbyjs.org before
gatsbyjs.org after
reactjs.org before
reactjs.org after

What's next?

MDX is still a black box as it's now added to app.js and should be part of the shared libs, which is not the case. Firstly I'll be looking at our polyfill strategy to reduce our bundles and afterwards, I'll check out how to optimize mdx.

Big shoutout to @developit, @housseindjirdeh and the nextjs team

Documentation

I might need to update https://www.gatsbyjs.org/docs/production-app/ to resemble the correct information.

Related Issues

pieh

Looking awesome!

packages/gatsby/src/utils/webpack.config.js

developit · 2020-03-13T20:59:28Z

packages/gatsby/src/utils/webpack-utils.js

-            if (/@babel(?:\/|\\{1,2})runtime|core-js/.test(modulePath)) {
+            // TODO this needs rework, this is buggy as hell
+            if (
+              /@babel(?:\/|\\{1,2})runtime|core-js|react|react-dom|scheduler|prop-types/.test(


this regex probably needs directory boundaries to avoid /react/ matching react-foo.

laurieontech · 2020-03-20T13:49:36Z

Yes, I'd say https://www.gatsbyjs.org/docs/production-app/ should be updated as part of this.
Also wondering if other pages like creating a custom webpack config need a section on this. Essentially I'd like there to be at least one write up on how users can take advantage of chunking themselves.

Improved webpack bundling by optimizing caching and less overall javascript

laurieontech

Thanks Ward! Left some suggestions below but looks really good.

docs/docs/production-app.md

laurieontech · 2020-03-23T12:56:59Z

docs/docs/production-app.md

+        commons: {},
+        // When a module is used more than once we create a shared bundle to save user's bandwidth
+        shared: {},
+        // All css is bundled into one stylesheet


Suggested change

// All css is bundled into one stylesheet

// All CSS is bundled into one stylesheet

laurieontech · 2020-03-23T12:57:24Z

docs/docs/production-app.md

+      },
+      // Keep maximum initial requests to 25
+      maxInitialRequests: 25,
+      // A chunk should be at least 20kb to be considered into splitChunks


Suggested change

// A chunk should be at least 20kb to be considered into splitChunks

// A chunk should be at least 20kb before using splitChunks

laurieontech · 2020-03-23T12:57:33Z

docs/docs/production-app.md

+    minimizers: [
+      // Minify javascript using Terser (https://terser.org/)
+      plugins.minifyJs(),
+      // Minify css by using cssnano (https://cssnano.co/)


Suggested change

// Minify css by using cssnano (https://cssnano.co/)

// Minify CSS by using cssnano (https://cssnano.co/)

laurieontech · 2020-03-23T12:58:10Z

docs/docs/production-app.md

@@ -62,6 +93,14 @@ This bundle is produced from [production-app.js](https://github.com/gatsbyjs/gat

 This contains the small [webpack-runtime](https://webpack.js.org/concepts/manifest/#runtime) as a separate bundle (configured in `optimization` section). In practice, the app and webpack-runtime are always needed together.

+##### framework-[contenthash].js
+
+The framework bundle contains the react framework. We've noticed that React hardly gets upgraded to a newer version. Creating a separate bundle improves users' browser cache hit rate as this bundle is likely not going to be updated often.


Suggested change

The framework bundle contains the react framework. We've noticed that React hardly gets upgraded to a newer version. Creating a separate bundle improves users' browser cache hit rate as this bundle is likely not going to be updated often.

The framework bundle contains the React framework. Based on user behavior, React hardly gets upgraded to a newer version. Creating a separate bundle improves users' browser cache hit rate as this bundle is likely not going to be updated often.

laurieontech · 2020-03-23T12:59:07Z

docs/docs/production-app.md

+
+##### commons-[contenthash].js
+
+Libraries used on every gatsby page are bundled into the commons javascript file. By bundling these all together, we make sure your user only needs to download this bundle once.


Suggested change

Libraries used on every gatsby page are bundled into the commons javascript file. By bundling these all together, we make sure your user only needs to download this bundle once.

Libraries used on every Gatsby page are bundled into the commons javascript file. By bundling these together, you can make sure your users only need to download this bundle once.

docs/docs/production-app.md

laurieontech

Thanks Ward! Would it be possible to add a paragraph that just primes the user on chunking as a concept?

wardpeet · 2020-03-23T15:13:24Z

Thanks Ward! Would it be possible to add a paragraph that just primes the user on chunking as a concept?

To the app-production.md file? Not 100% sure what you want me to explain, chunks & modules are webpack terms. Technically a module contains chunks & chunks are pieces of code (a file or bundle of files)

laurieontech · 2020-03-23T15:23:57Z

This section has an intro paragraph. Perhaps mention that we make use of chunking with a link to the applicable webpack docs on the topic?

laurieontech

Thanks Ward! Think that helps a lot. I made some suggestions since we're trying to move away from "we", but overall think it looks good.

docs/docs/production-app.md

Co-Authored-By: LB <laurie@gatsbyjs.com>

laurieontech

Thanks for adding the right docs info!

LekoArts

Added just one question :)

docs/docs/production-app.md

KyleAMathews

Looks awesome! Excited to get this in!

wardpeet · 2020-03-26T07:25:45Z

Published in gatsby@2.20.7

fk · 2020-04-21T12:16:05Z

(very late to the party, but 💜 🙏 🎉)

wardpeet requested a review from a team as a code owner March 13, 2020 15:50

pieh reviewed Mar 13, 2020

View reviewed changes

packages/gatsby/src/utils/webpack.config.js Show resolved Hide resolved

packages/gatsby/src/utils/webpack.config.js Outdated Show resolved Hide resolved

packages/gatsby/src/utils/webpack.config.js Show resolved Hide resolved

blainekasten reviewed Mar 13, 2020

View reviewed changes

packages/gatsby/src/utils/webpack.config.js Show resolved Hide resolved

developit reviewed Mar 13, 2020

View reviewed changes

wardpeet force-pushed the wardpeet/ch298/granular-code-splitting branch from 1635e9c to d918a05 Compare March 20, 2020 13:45

freiksenet requested review from a team as code owners March 20, 2020 13:45

wardpeet added 5 commits March 20, 2020 18:33

feat(gatsby): enable granular chunks

f06e0b8

Improved webpack bundling by optimizing caching and less overall javascript

update config

ad8c5b1

fix dependency test

9bbadb3

fix runtime

5750401

update docs

946f6d3

wardpeet force-pushed the wardpeet/ch298/granular-code-splitting branch from d918a05 to 946f6d3 Compare March 20, 2020 17:38

wardpeet added 3 commits March 20, 2020 18:42

fix typo

3b76d5f

fix tests

98adf10

Merge branch 'master' into wardpeet/ch298/granular-code-splitting

6b86443

laurieontech reviewed Mar 23, 2020

View reviewed changes

fix review comments

f3f71c4

laurieontech reviewed Mar 23, 2020

View reviewed changes

docs/docs/production-app.md Outdated Show resolved Hide resolved

laurieontech previously approved these changes Mar 23, 2020

View reviewed changes

add small section on granular chunks

2b26f06

wardpeet dismissed laurieontech’s stale review via 2b26f06 March 24, 2020 14:28

wardpeet force-pushed the wardpeet/ch298/granular-code-splitting branch from 2cf2fab to 2b26f06 Compare March 24, 2020 14:28

laurieontech reviewed Mar 24, 2020

View reviewed changes

docs/docs/production-app.md Outdated Show resolved Hide resolved

Update docs/docs/production-app.md

241b52b

Co-Authored-By: LB <laurie@gatsbyjs.com>

laurieontech approved these changes Mar 24, 2020

View reviewed changes

LekoArts reviewed Mar 25, 2020

View reviewed changes

docs/docs/production-app.md Show resolved Hide resolved

KyleAMathews approved these changes Mar 25, 2020

View reviewed changes

wardpeet merged commit 0f02ea7 into gatsbyjs:master Mar 26, 2020

wardpeet deleted the wardpeet/ch298/granular-code-splitting branch March 26, 2020 07:10

LekoArts mentioned this pull request Apr 1, 2020

[gatsby-theme-minimal-blog]: All code blocks missing in RSS feed LekoArts/gatsby-themes#299

Closed

moonmeister mentioned this pull request Apr 7, 2020

Proposal: Implement optimizations for JSX, Hooks and Code Splitting #19943

Closed

juanferreras mentioned this pull request May 22, 2020

Worse performance results with Lighthouse v6 (?) #24332

Closed

herecydev mentioned this pull request Jun 8, 2020

Treeshaking and Codesplitting Default Behaviour #24866

Closed

3 tasks

lucasecdb mentioned this pull request Nov 7, 2020

Implement granular chunks lucasecdb/casterly#325

Merged

fengzilong mentioned this pull request Oct 18, 2021

[RFC] 更细粒度的 splitChunks 策略 raxjs/rax-app#858

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gatsby): enable granular chunks #22253

feat(gatsby): enable granular chunks #22253

wardpeet commented Mar 13, 2020 •

edited

Loading

pieh left a comment

developit Mar 13, 2020

laurieontech commented Mar 20, 2020

laurieontech left a comment

laurieontech Mar 23, 2020

laurieontech Mar 23, 2020

laurieontech Mar 23, 2020

laurieontech Mar 23, 2020

laurieontech Mar 23, 2020

laurieontech left a comment

wardpeet commented Mar 23, 2020

laurieontech commented Mar 23, 2020

laurieontech left a comment

laurieontech left a comment

LekoArts left a comment

KyleAMathews left a comment

wardpeet commented Mar 26, 2020

fk commented Apr 21, 2020

	// All css is bundled into one stylesheet
	// All CSS is bundled into one stylesheet

	// A chunk should be at least 20kb to be considered into splitChunks
	// A chunk should be at least 20kb before using splitChunks

	// Minify css by using cssnano (https://cssnano.co/)
	// Minify CSS by using cssnano (https://cssnano.co/)

	The framework bundle contains the react framework. We've noticed that React hardly gets upgraded to a newer version. Creating a separate bundle improves users' browser cache hit rate as this bundle is likely not going to be updated often.
	The framework bundle contains the React framework. Based on user behavior, React hardly gets upgraded to a newer version. Creating a separate bundle improves users' browser cache hit rate as this bundle is likely not going to be updated often.


		##### commons-[contenthash].js

		Libraries used on every gatsby page are bundled into the commons javascript file. By bundling these all together, we make sure your user only needs to download this bundle once.

feat(gatsby): enable granular chunks #22253

feat(gatsby): enable granular chunks #22253

Conversation

wardpeet commented Mar 13, 2020 • edited Loading

Description

Less duplications (less javascript)

Caching

What's next?

Documentation

Related Issues

pieh left a comment

Choose a reason for hiding this comment

developit Mar 13, 2020

Choose a reason for hiding this comment

laurieontech commented Mar 20, 2020

laurieontech left a comment

Choose a reason for hiding this comment

laurieontech Mar 23, 2020

Choose a reason for hiding this comment

laurieontech Mar 23, 2020

Choose a reason for hiding this comment

laurieontech Mar 23, 2020

Choose a reason for hiding this comment

laurieontech Mar 23, 2020

Choose a reason for hiding this comment

laurieontech Mar 23, 2020

Choose a reason for hiding this comment

laurieontech left a comment

Choose a reason for hiding this comment

wardpeet commented Mar 23, 2020

laurieontech commented Mar 23, 2020

laurieontech left a comment

Choose a reason for hiding this comment

laurieontech left a comment

Choose a reason for hiding this comment

LekoArts left a comment

Choose a reason for hiding this comment

KyleAMathews left a comment

Choose a reason for hiding this comment

wardpeet commented Mar 26, 2020

fk commented Apr 21, 2020

wardpeet commented Mar 13, 2020 •

edited

Loading