RFC: Slice API

[imjoshin]

Summary

Gatsby is adding a new API that we are calling “Slices”. By using a new <Slice /> React component in combination with a src/slices directory or createSlice API for common UI features, Gatsby will be able to build and deploy individual pieces of your site that had content changes, not entire pages.

Motivation

Gatsby already supports Incremental Builds for speeding up content changes for individual (or a subset of) pages, but what if there’s a common component across all pages? Today, when a change to the content of that component happens, every page using that component needs to be rebuilt.

The goal of this feature is to provide new ways to speed up builds. While there is a lot going on behind the scenes to make Slices work, we want to provide an easy-to-use API for Gatsby sites to take advantage of this.

Design

We’ll be adding a new concept to Gatsby called “Slices”. This will come with a <Slice /> React component to be used with either the filesystem (similar to pages) and an optional slices parameter on the page config, or the createSlice API and an optional slices parameter to createPage .

Note: Nested Slices will not be supported.

Using the Filesystem

Simple

1. Create a footer.js in src/slices

2. Add a <Slice /> component on your site, providing an alias string prop, where alias is either name of the file (in our case, footer). Any additional props passed will be handed down to the underlying component.

   <Header className="my-header" />
   {children}
   <Slice alias="footer" />

With Changing Header

1. Create aheader-{MarkdownRemark.frontmatter__header}.js and footer.js in src/slices

   1. {variant} can be accessed on your component’s props.params just like pages

2. Pass a map of slices ( alias -> id ) to through your page’s config function (from DSG) to apply “overrides” of sorts for all slices using a given alias on the page.

   export async function config() {
     return ({ params }) => {
       return {
         // This map will tell any 'header' slice on this page
         // to alias to the filesystem routes with variables
         slices: {
           header: `header-${params.frontmatter__header}`,
           // no alias/override for footer
         }
       }
     }
   }

3. Add a <Slice /> component on your site, providing an alias string prop, where alias is either the exact id when creating the slice, or the alias key in the map provided to createPage. Any additional props passed will be handed down to the underlying component.

   /* This will find the slice with id "header-${headerVariant}" */
   <Slice alias="header" color="#FF0000" />
   {children}
   /* 
     The footer wasn't given an alias in `config`
     so just grab the slice with id "footer" 
   */
   <Slice alias="footer" />

Using the API

Implementing a slice will take the following flow:

1. Call createSlice in gatsby-node's createPages - passing an id, component, and optional context

   exports.createPages = async ({ actions }) => {
     ...
   
     actions.createSlice({
       // Give each header variant a unique ID
       id: `header-${headerVariant}`,
       component: require.resolve(`./src/components/header.js`),
       context: {
         someVar: 123,
       },
     })
   
     actions.createSlice({
       // In this example, there are no variants for footers
       // so just use a simple ID (but still unique)
       id: `footer`,
       component: require.resolve(`./src/components/footer.js`),
     })
   
     ...
   }

2. (optional) Pass a map of slices ( alias -> id ) to createPage to apply “overrides” of sorts for all slices using a given alias on the page. Any keys not given will default to the id given to createSlice.

   createPage({
     path,
     component: require.resolve(`./src/templates/my-component.js`),
     
     // This map will tell any 'header' slice on this page
     // to alias to the IDs set up in the `createSlice`
     slices: {
       header: `header-${headerVariant}`,
       // no alias/override for footer
     }
   })

3. Add a <Slice /> component on your site, providing an alias string prop, where alias is either the exact id when creating the slice, or the alias key in the map provided to createPage. Any additional props passed will be handed down to the underlying component.

   /* This will find the slice with id "header-${headerVariant}" */
   <Slice alias="header" color="#FF0000" />
   {children}
   /* 
     The footer wasn't given an alias in `createPages`
     so just grab the slice with id "footer" 
   */
   <Slice alias="footer" />

Once these slices are created, any time Gatsby encounters a <Slice /> component, it will be replaced with the component set up in createSlice. This will enable Gatsby to build the individual slices and deploy them separately from the rest of the page they live in.

Prior Art

• Jekyll’s include: https://jekyllrb.com/docs/includes/
• Hugo’s partial: https://gohugo.io/templates/partials/

Examples

Navigation Bar (Simple)

In a very simple example, let’s take a site that’s mostly handled with filesystem routes, but with a navigation bar who’s banner content comes from our CMS. Right now, whenever that banner content changes, every page needs to rebuild to reflect the changes. Instead, we just want to create a single slice for the navigation bar.

Using the Filesystem

src/templates/blog-post.js

import React from "react";
import { Slice } from "gatsby";

const DefaultPageLayout = ({ navbarClassname = '', children }) => {
  return <>
    {/* The old implementation looked like this: */}
    {/* <Navbar className={navbarClassname} */}

    {/* Now, just change it to the slice! */}
    {/* The "alias" prop is the same as the id passed in gatsby-node.js */}
    <Slice alias="navbar" className={navbarClassname} />
    <div>
      {children}
    </div>
  </>
}

export default DefaultPageLayout 

src/slices/navbar.js

import React from "react";
import { Link, useStaticQuery, graphql } from "gatsby";

export default function Navbar({ className = '' }) {
  const { content } = useStaticQuery(graphql`
    query BannerContent {
      content: BannerContent(name: { eq: "Banner" }) {
        body {
          childMarkdownRemark {
            html
          }
        }
      }
    }
  `)

  return (
    /* We can also use the `className` passed from our DefaultPageLayout component */
    <div className={className}>
      <div
        className="banner"
        dangerouslySetInnerHTML={{
          __html: content?.body?.childMarkdownRemark?.html,
        }}
      />
      <Link to="/">Home</Link>
      <Link to="/store">Store</Link>
      <Link to="/about">About</Link>
    </div>
  );
}

export default Navbar

Using the API

gatsby-node.js

exports.createPages = ({ actions }) => {
  actions.createSlice({
    id: `navbar`,
    component: require.resolve(`./src/templates/navbar.js`),
  });
}

src/templates/blog-post.js

import React from "react";
import { Slice } from "gatsby";

const DefaultPageLayout = ({ navbarClassname = '', children }) => {
  return <>
    {/* The old implementation looked like this: */}
    {/* <Navbar className={navbarClassname} */}

    {/* Now, just change it to the slice! */}
    {/* The "alias" prop is the same as the id passed in gatsby-node.js */}
    <Slice alias="navbar" className={navbarClassname} />
    <div>
      {children}
    </div>
  </>
}

export default DefaultPageLayout 

src/templates/navbar.js

import React from "react";
import { Link, useStaticQuery, graphql } from "gatsby";

export default function Navbar({ className = '' }) {
  const { content } = useStaticQuery(graphql`
    query BannerContent {
      content: BannerContent(name: { eq: "Banner" }) {
        body {
          childMarkdownRemark {
            html
          }
        }
      }
    }
  `)

  return (
    /* We can also use the `className` passed from our DefaultPageLayout component */
    <div className={className}>
      <div
        className="banner"
        dangerouslySetInnerHTML={{
          __html: content?.body?.childMarkdownRemark?.html,
        }}
      />
      <Link to="/">Home</Link>
      <Link to="/store">Store</Link>
      <Link to="/about">About</Link>
    </div>
  );
}

export default Navbar

Now we have a site where the navbar is pulling in banner content from the CMS, but whenever the content is changed outside of Gatsby, we’ll only rebuild and deploy the navigation bar once. The pages no longer need to be rebuilt!

Blog Authors (Complex)

Here we have a blog site where markdown files control our blog content and another source is providing information about authors. We want to create a slice for every author.

gatsby-node.js

exports.createPages = async ({ actions }) => { 
  // Query for all authors from some source
  const authorResult = await graphql(
    `
      {
        allAuthor {
          edges {
            node {
              id
            }
          }
        }
      }
    `
  )

  // Create slices for each author
  authorResult.data.allAuthor.edges.forEach(({ node }) => {
    actions.createSlice({
      // Give each author a unique ID
      id: `author-${node.id}`,
      component: require.resolve(`./src/templates/author-details.js`),
      context: {
        id: node.id,
      },
    })
  })
  
  // Query for all blog posts
  const blogResult = await graphql(
    `
      {
        allMarkdownRemark(limit: 1000) {
          edges {
            node {
              frontmatter {
                path
                authorId
              }
            }
          }
        }
      }
    `
  )

  // Create blog pages for each blog post
  blogResult.data.allMarkdownRemark.edges.forEach(({ node }) => {
    createPage({
      path: node.frontmatter.path,
      component: require.resolve(`./src/templates/blog-post.js`),
      context: {
        pagePath: node.frontmatter.path,
      },
      
      // Set 'author' slice alias for this page to use the author-${authorId} slice
      slices: {
        author: `author-${node.frontmatter.authorId}`
      }
    })
  })
}

src/templates/blog-post.js

import React from "react";
import { Slice } from "gatsby";
import { Nav } from "../components/nav";

const BlogPost = ({data}) => {
  return <>
    <Nav />
    /* 
      Set the 'alias' prop as author, which will be filled by the
      correct slice from `createPage`. We can also set other props!
    */
    <Slice alias="author" className="blog-author" />
    <div>
      <section
        dangerouslySetInnerHTML={{ __html: data.markdownRemark.html }}
        itemProp="articleBody"
      />
    </div>
  </>
}

export const pageQuery = graphql`
  query BlogPostBySlug(
    $id: String!
  ) {
    markdownRemark(id: { eq: $id }) {
      html
    }
  }
`

export default BlogPost

src/templates/author-details.js

import React from "react";
import { graphql } from "gatsby";

// Here we can define a slice query using slice context (author id) to get details about author
export const sliceQuery = graphql`
  query (id: String!) {
    author(id: { eq: $ID }) {
      name
      twitter
      email
    }
  }
`

export default function AuthorDetails({ data, className = '' }) {
  const { author } = data

  return (
    /* We can also use the `className` passed from our BlogPost component */
    <div className={className}>
      <div>Written by {author.name}</div>
      <div>Twitter: {author.twitter}</div>
      <div>Email: {author.email}</div>
    </div>
  );
}

export default AuthorDetails

Now, if our CMS issues a change to an author’s twitter handle, Gatsby doesn’t have to rebuild every post that author has created - just the small AuthorDetail component. This will then be updated on the blogs that author has written.

Alternatives

• Don’t include separate context for Slices, and each Slice gets handed the PageContext of the page it’s rendered on
  • It would simplify the API and Filesystem routes a bit, but would also mean passing all Slice context to all pages.
• Generate a MyComponentSlice that is exported from "gatsby/slices" as opposed to <Slice alias="my-component" />

Adoption Strategy

• We will be releasing a canary of this feature before the Gatsby 5 Beta. Follow this discussion in order to be notified when canaries become available.
• While the feature will work everywhere you build a Gatsby site, you’ll see additional build speed benefits on Gatsby Cloud on all tiers at least until the feature is marked as stable.
• This will also be a part of the Gatsby 5 Beta, as well as the full Gatsby 5 launch. A blog post will be posted that is dedicated to describing this feature, its benefits, and how to use it.

Considerations

• Using the Slice component, your IDE of choice will not know exactly which component is used under the hood. We will be able to generate a union type for all slices, so it will take 2 clicks to go from the component utilizing the slice to the true component being rendered.
• Keeping track of what components are acting as a Slice and which are not during development could complicate the developer experience.

How can you help?

A few questions we’re looking to get answered:

• What do you think of the name “Slice”?
• What do you think of the createSlice API design?
  • Does the per-page alias design (by passing slices to createPage) make sense?
• In what scenarios would you expect to use slices?
  • How many would you create in your use case?
  • Do you see any case where nested slices would be necessary? (we don’t have plans to support them now)
• Do you have any other alternative ideas?
• Are there other considerations we missed in the list above?

We’ll also be publishing a canary out before the Gatsby 5 Beta. Trying that on your sites would be a great source of feedback for us!

Thank you for your thoughts, and thanks for using Gatsby!

[me4502]

There are two common performance/size issues which sites that generate pages on a template via a CMS face, that I wonder if these would be able to additionally solve.

Firstly, if the CMS provides a lot of building blocks that a page can conditionally add or not, these all enter the bundle. This can be resolved by lazy loading, but I wonder if these could be bundled separately as a "Slice bundle" out of the box.

Secondly, any static queries that are used within any component on any page using that template (due to bundling etc) are then included on every page using that template. This increases the amount of data that is downloaded when loading the page. If the above is done where slices are separated from a bundle perspective, would it also be possible to only included static queries that slices pull in on pages that actually use the slice?

It's relatively common for sites to use a "Page" template type with a bunch of pre-setup components that can be added via a CMS, many of which might make use of static queries, so this can quite quickly become a large issue.

[pieh]

Slices don't really target frontend performance:
a) they target build perf (so we don't re-render same component on every page if it's shared, instead render it once and stitch it back where needed)
b) they provide something in between Page and Static queries - you can have slice queries (similar to page queries) where you can use slice context as paramaters ( example use case here is to have Slice per language for your shared ui elements ) and have clearer "boundaries" - your page doesn't need to query for navigation items for given locale, your main navigation slice should (which currently is not exactly possible - it's either parameterized page query or global static query with further filtering in JS and overfetching). I also just realized this (slice queries) was not yet included in RFC - I will update it to include example of it. (RFC now contains example of slice query in the "Blog Authors (Complex)" example)

With that said, it's actually possible to "abuse" slices to do what you described (tho there are some additions needed to RFC to support it). How I imagine setup with slices for this:

1. Generate a slice per each component that maps to a block type in CMS. For simplicity let's say we have "heading", "paragraph" and "image" as an example blocks. We would name those slices block-heading-impl (and other types similarly)
2. In your "block -> component" mapper, instead of rendering already imported component (either "statitically" imported or with dynamic import, doesn't matter) you would remove all imports and in places where you render block components you would have <Slice alias="block-heading" allowEmpty> - notice that this alias doesn't match to previously created slice (it's missing -impl) - this is intentional, also allowEmpty is new thing being introduced that tell gatsby not to crash if it can't find matching slice and instead render null
3. Most important - when creating pages, we need to know what block types will be needed to render a page (so you might need to query at least for used block types when creating pages). Then, with list of used blocks we set slice alias map on a page that set actual implementation for used blocks:

createPage({
  path: <whatever>,
  component: <whatever>,
  context: <whatever>,
  // hardcoded to ilustrate, but should be generated
  // let's say this page only uses "heading" and "image", but not "paragraph" block
  slices: {
    "block-heading": "block-heading-impl",
    "block-image": "block-image-impl",
    // we are omitting "paragraph" block here which means it will not be loaded in frontend
  }
})

With regards to static queries, slices will introduce boundary for those, so if you will have static queries in the block components - we will only load static queries in frontend if we also load a slice. With above example of omitting slice - if "paragraph" component would have static query, the created page wouldn't load it, because paragraph wasn't loaded

[me4502]

Thanks, definitely understood it wasn't the intended purpose of the feature, but sounds like it'll definitely be usable for this. Thanks :)

[tujoworker]

This Festure looks very promising ✨ I assume that src/slices will be supported from within Gatsby Plugins – just as src/pages are.

[pieh]

It's early and we didn't validate it, but initially at least I don't see reasons to not support this. We already support page creation from plugin's src/pages (Themes) and I think in every place we support page creation we would support slices creation as well.

[rogermparent]

I would love to see Slices be a granular Plugin API that covers where Themes are currently lacking! I have some questions, though:

1. Would plugin slices be able to be overridden by consuming projects, or even other plugins/themes?
2. If an overridden slice has a query, would it suffer from Shadowing a component with a query throws an error #18451?

[angeloashmore]

This sounds like a useful feature. I don't have any comments on the implementation details, but I wanted to throw in a quick comment about the "Slices" name.

Prismic CMS uses the term "Slices" to represent page template fragments that can be combined to build a page. This is essentially what @me4502 described in her comment.

More information about Prismic Slices can be found here: What is a Slice? - Prismic

While I understand Prismic's Slices feature and Gatsby's proposed Slices feature solve different problems (at least in their intention), overloading the term "Slices" could be confusing for existing Gatsby + Prismic users.

In addition to the name confusion for Prismic developers, Prismic recommends using the src/slices directory to hold React components that are rendered for each Prismic Slice from the CMS. If Gatsby takes over the src/slices directory, existing apps will need to rename their src/slices directory and update component paths throughout the app's code.

If the name is truly not set in stone, would you consider a rename? Perhaps to something that more directly implies its purpose, such as "Partial."

• Concept Name: Partials
• Concept Description: Reusable partial page pieces that can be built independently from pages that use them.
• Directory: src/partials

[imjoshin]

Thank you for the response. One reason we didn't go with the name "Fragment" was due to avoid an overloaded term, i.e. React.Fragment. This is good information and we'll definitely take this into consideration. 💜

[pieh]

@angeloashmore I was looking at Prismic Slices and it actually looks like proposed Gatsby Slices and exsiting Prismic Slices are actually complimentary and not conflicting (as it looked like from first glance). This is how I see it:

From reading docs on Prismic Slices it seems like this is Page Builder pattern that looks to cover also rendering in frontend (at least Slice/block component authoring) and not just CMS content/data/authoring experience part (Slice Templates and Slice Machine). With exploration on Page Builder scenario in #36339 (reply in thread) it seems that Gatsby Slices can/will be great fit for Page Builders as they will allow to load only needed slice components code.

When looking at "example" project that uses Prismic Slices / SliceMachine on Next.js site - the shape of the Prismic Slice Template matches what we would expect from Gatsby Slice Template (well basically it exports component as default and get some props) - here's example I was looking at https://github.com/prismicio/slice-machine/blob/master/e2e-projects/next/slices/ecommerce/CategoryPreviewWithImageBackgrounds/index.js

It seems to me that this is all actually compatible and you would WANT to use same slices directory, because then you get:

• default setup for Prismic, so your slice components (and schema / model) in place that will be common to Prismic projects
• default setup for Gatsby Slices that will just register those slices
• user will just have to implement ~SlicesRenderer (something that loops over slices data and is using correct component to render each slice type) that would use Gatsby Slices (instead of importing all possible component). And that will make Page Builder sites more performant frontend wise due to not loading js for slice templates that are not being shown

So all-in-all (to me) it looks like directory (/src/slices) being shared in this scenario is not a problem, in fact it's lucky coincidence because those 2 are complementary and you would actually always want to use Gatsby Slices on Gatsby site if you are making use of Prismic Slices for best result.

Did I get something wrong?

[jordanpsleeper]

Please have folder location be a configuration where you set the path and not default. Or at least be overridable.

[sandren]

What do you think of the name “Slice”?

I think the name "Slice" is great. It's really effective in communicating the motivation and purpose of the API since it's literally slicing your data layer orthogonally to pages/templates. 🙂

If the name is truly not set in stone, would you consider a rename? Perhaps to something that more directly implies its purpose, such as "Partial."

The proposed name and convention overlapping with an existing Prismic API is unfortunate, but I think that renaming the Gatsby Slice API to "Parts" or "Partials" could accidentally cause people to confuse it as a best practice when really it's more of an escape hatch. The Prismic API is already pretty unstable so they might end up renaming their slices to something more straightforward like sections anyways.

[angeloashmore]

Thanks for your thoughts - I definitely see where you're coming from.

Full disclosure: I work at Prismic. Prismic's "Slices" won't be renamed in the foreseeable future since it is part of our core branding.

[imjoshin]

We like the name Slices, too, but are open to any and all suggestions. As @angeloashmore phrased it, it is not "set in stone".

I think that renaming the Gatsby Slice API to "Parts" or "Partials" could accidentally cause people to confuse it as a best practice when really it's more of an escape hatch.

I wouldn't call it an escape hatch, but moreso a build optimization. 😄 We had considered Partial, but didn't feel as nice as Slice. Do you have any other suggestions?

[coreyward]

This naming is also used by a number of Sanity developers to refer to modules/blocks/LEGOs/plugs.

[ashhitch]

I like the name 🍕 Slices 🍕 (Who doesn't want to be reminded of pizza while they code??)
On a more serious note, it is a really approachable name that does what it says on the tin.

On large sites not having to rebuild every page just because a link in the footer changed its going to be a big win.

We also do a lot with cross-linking... e.g you might have a page built up of components with one of them being a featured list of guides I can see this approach working well for that.

I can see being able to build specific components out with createPages could be hand so a nice addition to the API.

[imjoshin]

Who doesn't want to be reminded of pizza while they code??

💯

you might have a page built up of components with one of them being a featured list of guides I can see this approach working well for that.

Perfect use case!

I can see being able to build specific components out with createPages could be hand so a nice addition to the API.

Great! Thank you for the feedback. 😄

[bartdag]

Hi! This is a neat idea and we can't wait to test it!

What do you think of the name “Slice”?

I find the usage of the word Slice unfortunate in this context. Slice often means cutting a list or getting a subset of a collection. See javascript slice and python slice

I wonder if "fragment" would be more appropriate. See

• WordPress cache fragment
• WooCommerce fragments
• Django Template Fragment Caching

Another word that comes to mind is partials.

Are there other considerations we missed in the list above?

Just to make sure I understood correctly, the goal of this RFC is to optimize the scenario where the same content changes on many pages. If we change the code of the slice, it would still trigger a full build right?

[pieh]

If we change the code of the slice, it would still trigger a full build right?

With slices (and the stitching it back up to get full atomic page html), changing the code will mean only that slice (or page template) will be re-rendered using full react machinery, but pages using this slices will do "simple" string replacement (instead of doing full render for all pages). This means, that all page html files will be updated, but actual react render will be limited to only page and/or slice template that changed

[nrandell]

Has something changed? I just tried to use slices in 4.24.3 and I can't seem to find the Slice component. Or are things being changed as V5 is getting close?

[imjoshin]

Sorry for the late response, @nrandell . Slices will be released in Gatsby 5, which is in beta now!

[imjoshin]

Hey everyone! Thank you to all who have tested the Gatsby Slice API. We're excited to announce that Gatsby 5 will be out soon and launch with the Gatsby Slice API.

You can try out the beta now!

Here are a few docs to get you started upon launch:

• Reference guide: https://v5.gatsbyjs.com/docs/reference/built-in-components/gatsby-slice/
• How-to: https://v5.gatsbyjs.com/docs/how-to/performance/using-slices/

Note: The initial launch of this feature will not include the Filesystem Routes path of using Slices.

Feel free to hop into our Discord if you have any questions. Thank you! 💜

[angeloashmore]

Note: The initial launch of this feature will not include the Filesystem Routes path of using Slices.

Does that mean the src/slices directory will not have special meaning until a future release? From the docs, it looks like createSlice is currently the only way to register Slices.

[imjoshin]

Does that mean the src/slices directory will not have special meaning until a future release? From the docs, it looks like createSlice is currently the only way to register Slices.

Correct, we will only be support creating Slices through createSlice. However, if you do not need to swap slices via aliases, you can use those slices in FS-route pages.

[LekoArts]

This is released in Gatsby 5 now 🎉

https://www.gatsbyjs.com/docs/reference/release-notes/v5.0/#slice-api

[tsdexter]

@LekoArts how can I create the "slices map" for pages created with the filesystem routing instead of the createPage function?

createPage({
  path,
  component: require.resolve(`./src/templates/my-component.js`),
  slices: {
    header: `header-${headerVariant}`,  // HOW TO DO THIS WITH A FILESYSTEM ROUTE?
  }
})