[feat] Improve experience of introducing new APIs to Gatsby Core

[DSchau]

Summary

When we have a need to introduce a new API, e.g. see #16027, we also unfortunately tend to break support/introduce some churn for older versions of Gatsby in the process.

For example, given the above API validatePluginOptions -- if we added that API to a plugin, and that plugin was updated but the gatsby version was kept constant (very common!) the following error would appear:

The plugin "gatsby-source-contentful@2.1.5" is exporting a variable named "validatePluginOptions" which isn't an API.

See #15680 for a real world example of the issue.

Basic example

This is a straw man (aka a proposal that potentially has flaws, and that I'd love to iterate to something better) proposal.

1. We create (at build/publish time) a list of known APIs in api-node-docs.js as JSON (e.g. gatsby/node-apis.json, gatsby/browser-apis.json, etc. or perhaps one big api.json file with keys for node, browser, etc.)
   • We'll probably want to use Babel or something to extract all the exports
   • Ideally, would also include the version introduced and then some metadata as to whether it should be "breaking" to older versions or not (e.g. some are crucial for functionality, others can/should be a noop as they're an enhancement, not crucial)
2. We build into bootstrap a request to these files (with Unpkg or equivalent), to know about the latest APIs
3. If a breaking API, break with a message like the below
4. If not breaking, warn with a similar message

The structure could look something like this:

{
  "createPages": {
    "version": "0.0.0",
    "breaking": true
  },
  "validatePluginOptions": {
    "version": "2.2.0",
    "breaking": false
  }
}

Breaking Message

The plugin {PLUGIN_NAME}@{PLUGIN_VERSION} is trying to use the API: {API_NAME} which is not available in gatsby@{GATSBY_VERSION}

To fix, upgrade to gatsby@{GATSBY_API_VERSION} with:

npm install gatsby@^{GATSBY_API_VERSION}

Note: if the network call fails, we need to fail quickly, and then tweak the message to be less useful and probably presumed breaking.

Motivation

We need to introduce new APIs. We can't break people's builds when they (very likely!) upgrade plugin versions, while keeping Gatsby versions constant.

We should encourage to upgrade, but not break builds whenever possible. We want to continue to iterate and build in useful functionality, but we also don't want to cultivate an environment of churn and frustration.

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contributefor more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[gatsbot]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!