Add Offline and Manifest plugin

[KyleAMathews]

With these two new plugins, most Gatsby sites can very easily become full-blown Progressive Web Apps! This includes working offline and (on Android) being able to “install” the website to your homescreen. Nifty! Once I got things working on my blog, I was able to quickly "offline-ize" three other websites very quickly.

This is the Lighthouse result I recorded while testing my blog.

These plugins are also a nice illustration of the versatility of Gatsby's plugin system and its ability to extract complex intertwining (complected) behaviors into a modular, reusable plugin. I'll walk through how the offline plugin works to illustrate this.

The offline plugin converts Gatsby to use the AppShell pattern. For SPA apps, in short this pattern works by caching an HTML page which acts as a "shell" for the app. Since it'd be tedious to ask every user of the offline plugin to add a special page, the plugin simply implements the createPages API and adds one itself.

exports.createPages = () => [
  {
    path: `/offline-plugin-app-shell-fallback/`,
    component: path.resolve(`${__dirname}/app-shell.js`),
  },
]

The plugin also ships with the component seen there.

Next the plugin uses postBuild to create the service worker w/ precaching and offline support enabled:

exports.postBuild = () => new Promise((resolve, reject) => {
  const rootDir = `public`

  const options = {
    staticFileGlobs: [
      `${rootDir}/**/*.{js,woff2}`,
      `${rootDir}/index.html`,
      `${rootDir}/manifest.json`,
      `${rootDir}/offline-plugin-app-shell-fallback/index.html`,
    ],
    stripPrefix: rootDir,
    navigateFallback: `/offline-plugin-app-shell-fallback/index.html`,
    cacheId: `gatsby-plugin-offline`,
    dontCacheBustUrlsMatching: /(.*.woff2|.*.js)/,
    runtimeCaching: [
      {
        urlPattern: /.*.png/,
        handler: `fastest`,
      },
      {
        urlPattern: /.*.jpg/,
        handler: `fastest`,
      },
      {
        urlPattern: /.*.jpeg/,
        handler: `fastest`,
      },
    ],
    skipWaiting: false,
  }

  precache.write(`public/sw.js`, options, err => {
    if (err) {
      reject(err)
    } else {
      resolve()
    }
  })
})

And finally, it uses the modifyPostBodyComponents API to add the snippet of Javascript to the end of the body necessary to load the service worker.

exports.modifyPostBodyComponents = () => [
  (
    <script
    dangerouslySetInnerHTML={{
        __html: (
          `
        if ('serviceWorker' in navigator) {
          // Delay registration until after the page has loaded, to ensure that
          // our precaching requests don't degrade the first visit experience.
          // See https://developers.google.com/web/fundamentals/instant-and-offline/service-worker/registration
          window.addEventListener('load', function() {
            navigator.serviceWorker.register('/sw.js');
          })
        }
      `
        ),
      }}
  />
  ),
]

Still need to add READMEs to the plugins before I merge this.

[gatsbybot]

Deploy preview ready!

Built with commit e74abe6

https://deploy-preview-687--gatsbyjs.netlify.com

[gatsbybot]

Deploy preview ready!

Built with commit e74abe6

https://deploy-preview-687--gatsbygram.netlify.com

[jeffposnick]

I'm excited for this! I left a few comments—sorry that the way sw-precache uses RegExps isn't as clear as it could be.

[jeffposnick]

Looks like there's an extra trailing \" escape sequence.

[jeffposnick]

urlPattern will trigger on partial URL matches, so there's no need to put in the leading .* wildcard.

If you wanted to combine all of the RegExps, you could do /\.(?:png|jpg|jpeg)$/

If you want to keep them separate, then I'd just tweak each of them a bit to escape that . character, and enforce that the match happens at the end of the URL string. So, e.g., /\.png$/.

[jeffposnick]

sw-precache's write() method returns a promise, so you should just be able to return it and not mess around with the callback.

[KyleAMathews]

ah nice :-)

[jeffposnick]

It's probably safer to include a RegExp here that matches the specific hash format you're using in your versioned filenames. So if you're using an 8 character hash surrounded by . characters, leading to filenames like 'app.123456ab.js, something like /.\w{8}./` would make sense.

You don't need to match the entire URL, so there's no need for the leading .* wildcard.

[KyleAMathews]

@jeffposnick thanks for the feedback! And of course doing a lot of heavy-lifting on building/maintaining sw-precache. It's super easy to use and very flexible.

[KyleAMathews]

Notes on implementation and ideas for future.

• For taking Gatsby offline, I went with Option 2 of the three described by @jeffposnick in https://jeffy.info/2017/01/24/offline-first-for-your-templated-site-part-2.html This option, AppShell, was the easiest/least invasive/cheapest option. It avoids the overhead of caching all html pages but is still really fast to load generally. The AppShell technique is we cache an HTML version of the "shell" for the site (which for most Gatsby sites is the header & perhaps footer) which loads first and renders and then pulls in the javascript for the site, boots that, and then renders in the client the actual page. On my Google Pixel, pages load almost instantly. On my slowest phone (Moto X 2014) and slowest site with the most javascript to parse/eval (https://advisor-otter-67205.netlify.com/guides/api-file-image-uploads/ w/ ~275 gzipped js) the main content is still loaded around ~1.5 seconds after the shell. Option 3, stream rendering HTML directly from the service worker is very exciting as the time to first byte would be significantly sooner and we'd avoid the somewhat odd double render of first the shell and then the body. See the above article and @jeffposnick's comment gatsbygram.gatsbyjs.org SW issues #670 (comment) Caching all html files would work very well for small sites or where it'd be ok for users to download megabytes of data.
• We set skipWaiting to false as otherwise the new service worker could delete older versions of chunks which code in memory would expect to be there causing things to break if they're requested. I think it makes the most sense to see different builds of a Gatsby site as separate versions of an app.
• By default sw-precache cache busts files when loading a new service worker to be sure it gets the latest version of a file. We don't need to do this as Gatsby's webpack is setup to create file names based on hash of contents so files are guaranteed to be unique. Which makes updates cheap for us as when a user visits the site there'll only be a handful of files they need to re-download.
• TODO allow overriding staticFileGlobs & other options.
• TODO add lazy caching option — don't precache anything other than basics and then borrow next.js pattern to precache files based on page links. E.g. if page 1 links to pages 2, 3, 4 — the client would instruct its SW to cache the components & data necessary for those pages. This is perhaps the ideal default and absolutely necessary for very large sites (1000+ pages).

[franzejr]

Was the README updated?

[bradleyess]

Looking to get the sw.js working in my gatsby build but getting this

Error during service worker registration: DOMException: Failed to register a ServiceWorker: The script has an unsupported MIME type ('text/js').