RFC: Gatsby Functions

[sidharthachatterjee]

Summary

Gatsby Functions are serverless functions hosted on Gatsby Cloud that are deployed along with your Gatsby site. Creating a new function is as simple as creating a file in the src/functions directory of your Gatsby site.

Gatsby Functions are available in your local development environment automatically and running a production build will now contain not just your static assets, but also your functions compiled with their dependencies into a single file, ready to upload to Gatsby Cloud.

Basic example

Let’s look at a quick example of a Gatsby Function. Create a folder in your Gatsby site’s src directory called api and create a file called time.js.

Now, type following code in time.js

export default (req, res) => {
  res.json({ time: Date.now() })
}

And that’s it! You just wrote your first Gatsby Function. This will be available on /api/time in your local development environment and in production when you push to Gatsby Cloud.

Detailed design

Gatsby Functions are native to Gatsby Cloud. Functions are auto discovered from your project’s src/api directory.

Every Gatsby Function is an HTTP function and is deployed to an HTTP endpoint. To author a Gatsby Function, a user can write an express compatible handler as a default export in a file.

Dependencies for Functions can be included in the project’s package.json and they will be bundled on build.

Hence, each function receives two parameters, request and response

Both of these will broadly conform to the Node HTTP spec including request.body, request.headers and response.json, response.status, response.send.

Configuration for Functions may be specified in a named export called config in the function’s file. All configuration is completely optional. Options include HTTP method and timeout, CORS domain whitelist.

export default (req, res) => {
  res.json({ time: Date.now() })
}

export const config = {
  method: `GET`
}

During gatsby develop, all functions in a project are automatically available on localhost:8000

When running gatsby build, the functions for a project will be built and saved into the .cache directory. This directory will contain a single built file for each function. This file will be a transpiled and a bundled output file including all of the function’s dependencies.

When a user runs gatsby serve, functions for a project will also be hosted on the express server on localhost:9000. This should make local development simple and straightforward. Logs for Gatsby Functions will be included inline with other logs for the develop or serve process.

TypeScript is supported out of the box for Functions.

Since all Gatsby Functions get deployed to HTTP endpoints, consuming them is like calling any other API. For example, calling our time function that we created above is as simple as calling it via fetch.

Users can assume that functions will be hosted on the same domain as the static assets and call them via relative urls.

fetch('/api/time')
  .then(response => response.json())
  .then(data => console.log(data.time));

Motivation

30% of our Gatsby users (from a sample size of a 175 on this poll) said they use serverless functions often when building a Gatsby site. These are typically used to access some dynamic data at runtime or as an endpoint to POST some data to a third party service or database.

Currently these users use services like Netlify Functions, Vercel, AWS Lambda and many more. These services are great but their development experience requires users to step out of Gatsby to write a serverless function. They also require users to run other CLI tools as part of their development environment.

Gatsby Functions aims to make this much simpler by enabling our users to create serverless functions right in their Gatsby repository, without needing to setup any other service.

Drawbacks

One possible drawback is that Gatsby Functions will only support Gatsby Cloud on initial release. However, we will work with the community to enable support on Netlify, Vercel and other existing platforms via plugins like gatsby-plugin-netlify and gatsby-plugin-vercel.

Alternatives

An alternative strategy would include designing Gatsby Functions to support multiple platforms out of the box. We are currently not considering this because:

1. Several platforms have their own accepted function signatures and configuration file formats that may vary considerably from one to another. Supporting all of these would add significant complexity and increase maintenance burden.

2. Platforms have their own limitations. An example of this is the lack of multipart form data support on Netlify. We would prefer not to limit functionality in our implementation of Functions due to a limitation of a supported platform.

Adoption strategy

This is not a breaking change at the moment. We expect users to adopt this for their next Gatsby project when they find the need to use a serverless function. Codemods for existing projects can be complex so we do not intend to provide these at release.

However, given that our API function signature intends to converge as much as possible with the majority of the ecosystem, we expect incremental adoption to not be too difficult and definitely worth the simplified development experience.

How we teach this

Gatsby Functions should be documented with the rest of the Gatsby documentation at gatsbyjs.com. We should also create a few quick start videos on how to create Gatsby Functions. These can live on egghead.io with the rest of our videos.

We should also create a new official starter that includes a Gatsby Function out of the box so users can create their new sites using this starter and edit the included Gatsby Function to get started quickly.

We should also create several examples on Gatsby Functions that perform common tasks including authentication, a POST handler for uploading an image, accepting a form response and writing it to a database and so on.

Unresolved questions

1. How important is being able to configure a different directory location for Gatsby Functions?
2. Should we consider support for JSX and React out of the box? Anything else we ought to support out of the box?
3. Do you want to configure custom paths for your API endpoints?
4. Do we want to namespace functions within /functions?

Please let us know what you think of this API, what are you missing? What use cases do you want us to cover? Looking forward to hearing more!

[OlliV]

Sounds like a great idea! 🎉 Have you considered about streaming responses and/or websockets?

[sidharthachatterjee]

We're open to considering it. Can you describe the sort of use cases you intend to support with these?

[dwwoelfel]

Note that you won't be able to support streaming if your underlying implementation uses AWS lambda. You'll also be limited to a 2mb response size.

Have you considered building this on top of Cloudflare workers or Fastly's compute@edge or on top of your own implementation of the underlying technologies (Fastly uses lucet, not sure if cloudflare has open-sourced theirs, but it uses v8 isolates last I checked)?

[sidharthachatterjee]

@dwwoelfel Excellent point! We’re currently prototyping and aren’t clear on what Functions will be hosted yet. We’re definitely taking these limitations into account though.

[1988LW2020]

Do you have an update on this? It would seem you are using Fastly but wanted to double-check?

[jsumnersmith]

Do you have an update on this? It would seem you are using Fastly but wanted to double-check?

Responded below but also noting that we are serving Function through our Hosting offering which is indeed powered by Fastly. They don't yet leverage compute at the edge, but that's certainly a mid-term goal on our roadmap!

[m-abdelwahab]

Will this feature be paid like Netlify? Where you pay if you exceed a certain number of calls per month?

[1988LW2020]

Do you guys know if functions is run on Compute@edge or Lambda or another service?

[jsumnersmith]

Hey @sw-yx. We just launched our initial beta! You can learn more here: https://www.gatsbyjs.com/docs/how-to/functions/

As far as the pricing and inclusion, Functions are included in Hosting plans, and we've added our initial limits on the pricing page! https://www.gatsbyjs.com/pricing/

[jsumnersmith]

Do you guys know if functions is run on Compute@edge or Lambda or another service?

We serve all Functions routing through Fastly, but we're not yet leveraging the Compute@edge.

[swyxio]

welp i missed that announcement, thank you

[jsumnersmith]

No problem! It went out to the Functions mailing list, so you may just not have signed up for it!

[benborgers]

I might be wrong, but doesn't Gatsby Cloud build the site and then host it on a provider of your choice? Would the site still be deployed on Netlify, Vercel, Firebase, etc but the serverless functions be hosted by Gatsby Cloud?

[jsumnersmith]

Hi @ben-rogerson, great catch. As we mentioned back in August, we're working on rolling out a default hosting solution for Gatsby sites and apps and so the serverless functions would serve those sites.

[jsumnersmith]

Just re-read this and realized that I was a bit unclear about the "we mentioned" aspect. Here's our blog post from mid-August where we talk a bit about a Gatbsy-native hosting solution: https://www.gatsbyjs.com/blog/announcing-unified-gatsby

[schneidmaster]

Excited to see this. One thing I didn't see mentioned above -- Netlify functions (and probably other clouds' equivalents?) have environment variable support where the set of environment variables that you configure to be available in the build are also available within Netlify functions via process.env.SOME_VARIABLE. It's important to have those available because a lot of the use case of server-side functions is to prevent client-side access to sensitive API tokens etc. Do y'all have any plans around that yet?

[sidharthachatterjee]

Yup. Env vars are a very common use case and we’ll definitely be supporting this.

[satyarohith]

If anyone is excited to try this out, I built a small repo that deploys functions inside src/functions to /functions/* endpoint on push.

[geelen]

Interesting! Have you considered Frontend Application Bundles (FABs) as a compile target? They've already got deployment adapters for Cloudflare Workers, AWS Lambda (& Lambda@Edge) and self-hosted NodeJS, and have a native Fetch-based API for server-side logic.

[sidharthachatterjee]

Hey Glen! Nice to see you around. Interesting recommendation. We aren’t clear yet on whether we’ll support a compile target like FAB. We’re likely to ship an initial release that sticks to the simpler express middleware spec which a couple of platforms support. I’ll take a look at FAB though! Thank you!

[phil-lgr]

First Class TypeScript support would be 👌🏻

Also, the ability to run the lastest NodeJS release

[muescha]

i would vote for (3)custom paths and (1)custom src path, if a website has already an page with this generic name MyFanyApp.com/functions

[jsumnersmith]

👋 Howdy to everyone who commented along or was following our original conversations around Functions. We've added a Beta for our initial version and you can check it out and get started here: https://www.gatsbyjs.com/docs/how-to/functions/.

Functions are included as part of the Hosting plans on Gatsby Cloud (including the Free tier), and you read more about the limits on our pricing page.

Happy building, and let us know what you think! There's an umbrella issue where you can add any feedback: #30735

[LekoArts]

Hello everyone!

With Gatsby v3.7 (https://www.gatsbyjs.com/docs/reference/release-notes/v3.7) Gatsby Functions are available to everyone now. You can also use them on Gatsby Cloud now: https://www.gatsbyjs.com/blog/announcing-launch-of-functions-on-gatsby-cloud

If you have questions or feature requests regarding Gatsby Functions, please use GitHub discussions: https://github.com/gatsbyjs/gatsby/discussions

If you have found a bug, please report an issue: https://github.com/gatsbyjs/gatsby/issues/new?assignees=&labels=type%3A+bug&template=bug_report.md