[docs] Improve documentation for plugin authors

[jlengstorf]

Currently, the documentation for plugin authors is pretty sparse. On the contributing page, there's a single paragraph, and the plugins section of the docs is super high-level.

What I'm proposing in the short term

The docs should be updated to:

• Explain naming conventions for plugins (e.g. when should I use gatsby-plugin-* vs gatsby-transformer-*?)
• Explicitly explain what files can be contained in a plugin, and which APIs are available to plugins
• Create a boilerplate to create a consistent Gatsby plugin README experience

What I'm proposing in the medium term

• Write tutorials for each of the plugin types to show real examples of using the plugin APIs
• Add video tutorials to complement the written versions
• Find a way to work plugin documentation into the new plugins page being created in Plugin Library build fixes #3906

What I'm proposing in the long term

• Add a CLI tool that asks a few questions about the plugin you want to create and sets you up with the proper boilerplate
• Add plugin validation to ensure that a plugin is compatible with your Gatsby install (e.g. no conflicts, versions are compatible)

Did I miss anything here? I can get started on the short-term updates right away — would love to get any feedback from other folks who've written plugins:

• where did you get stuck?
• what questions did you have trouble finding answers to?
• how could Gatsby's docs make the plugin authoring experience more friendly?

This may need to be pulled into a project — "Improve the Plugin Discovery and Authoring Experience" — or maybe we can solve it all in a few issues.

What do you think?

[KyleAMathews]

This looks fantastic! Let me know how I can help!

[markmichon]

Looks good! I was thinking about building a "gatsby-plugin-starter" repo, but haven't had the time. I like the idea of a CLI to handle that too.

[jlengstorf]

@KyleAMathews I'll probably need your help double-checking the API/naming conventions write-ups to make sure I didn't miss anything.

@markmichon A starter repo would be fantastic, but I wonder if we'd need multiple versions? For example, a plugin to add third-party code (e.g. Google Analytics) is really different from a transformer or a data source plugin. 😕 (That's why I put this in the long term category. 😂)

[jlengstorf]

I've got a PR started at #4272 to tackle the short-term changes. Any feedback is welcome as I work toward getting this done.

[jlengstorf]

@KyleAMathews can a plugin provide a gatsby-config.js? If so, do docs exist on how that works (e.g. loading order, is that available site-wide or scoped to the plugin)?

[jlengstorf]

When/how is index.js used? In the vast majority of plugins it's a no-op, but in a few places it appears to do things (e.g. https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-contentful/index.js). I'm not super clear on how this is being handled under the hood, so I could use some guidance and/or a link to the source where this gets picked up.

[jlengstorf]

If I'm in the right place in the code, it looks like index.js isn't used in plugins, which is confusing given that gatsby-source-contentful exports one.

https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/bootstrap/load-plugins/validate.js#L94-L105

This also appears to answer my previous question about gatsby-config.js — it's not possible to add or modify the config from a plugin. (Please confirm this.) 😅

[pieh]

index.js is used when plugin exports something that user (or other plugin) might want to import in their code via require/import - it doesn't use any of gatsby apis and isn't run by gatsby itself

in your gatsby-source-contentful this was added to reuse graphql types in plugin that is currently under development ( https://github.com/gatsbyjs/gatsby/pull/4205/files#diff-7629f8c5bcb62e07dc5b843af493a8e3R80 ) that want to reuse some of already defined types

gatsby-source-filesystem will export utility function that will create File nodes from remote urls (that will be used by other plugins - for example gatsby-source-wordpress

this is not gatsby specific - just normal module export

[jlengstorf]

#4272 is ready for review. Only thing left to add is the boilerplate README, which might actually make more sense in a starter repo. It feels like it wouldn't fit on the docs page right now; would love some feedback on this.

[pieh]

Write tutorials for each of the plugin types to show real examples of using the plugin APIs

In #2586 in comments I tried to explain base concepts of data sourcing and how it ties into gatsby graphql layer ( #2586 (comment) ) and provided examples that use demo REST api ( https://jsonplaceholder.typicode.com/ - #2586 (comment) and #2586 (comment) ).

I think some of it (mostly examples as I'm not good writer) might be used to update Create a source plugin page with real live (ish) use of related APIs. What is missing there and I feel is important is fetching assets (for example images) from remote urls. Demo REST api I used there do have endpoint which point to images ( https://jsonplaceholder.typicode.com/photos ) and I can adjust examples to use /photos and /albums endpoints instead of /todos for that. Alternatively we can prepare static .json files that will mimic REST api modeled to represent something that will show capabilities of gatsby data store -> graphql layer, but I think using well known demo REST api is good idea as this removes additional barrier of getting familiar with new REST api to follow this guide.

[KyleAMathews]

can a plugin provide a gatsby-config.js? If so, do docs exist on how that works (e.g. loading order, is that available site-wide or scoped to the plugin)?

No — plugins can have sub-plugins e.g. all the gatsby-remark-* plugins.

I thought about doing this but decided this would add a lot of complexity around issues like you mentioned.

We'll probably need this for themes though so will need to address this when we start work on them.

[KyleAMathews]

When/how is index.js used? In the vast majority of plugins it's a no-op, but in a few places it appears to do things (e.g. https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-contentful/index.js). I'm not super clear on how this is being handled under the hood, so I could use some guidance and/or a link to the source where this gets picked up.

It's not. Node module resolution requires a directory to have an index.js or something in the main field of a package.json. Since Gatsby requires each JavaScript environment (node, ssr, browser) to have a seperate file (so the gatsby-* files), we decided to eschew index.js but to keep Node happy, we add blank ones there.

Plugins can have something in index.js like gatsby-source-contentful if you want to export helper functions for other plugins to use e.g. gatsby-source-filesystem does the same thing https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-filesystem/src/index.js

[KyleAMathews]

oh, @pieh already answered these haha 😅

In the mouth of two or three witnesses...

[shannonbux]

Hi @jlengstorf just revisiting this. I was thinking of closing this issue and creating new issues that mirror the checklist you have above in [EPIC] v2.0 plugin library and [EPIC] v3.0 plugin library like so.

• Create a boilerplate to create a consistent Gatsby plugin README experience

• Could we check this off with the READme template in this PR that just got merged? templates.md: https://github.com/gatsbyjs/gatsby/pull/6489/files

[EPIC] v2.0 plugin library

• Create a boilerplate to create a consistent Gatsby plugin README experience
• Write tutorials for each of the plugin types to show real examples of using the plugin APIs
• Add plugin validation to ensure that a plugin is compatible with your Gatsby install (e.g. no conflicts, versions are compatible)

[EPIC] v3.0 plugin library

• Add video tutorials to complement the written versions
• Find a way to work plugin documentation into the new plugins page being created in Plugin Library build fixes #3906

And could this one get added to @ChristopherBiscardi's project?

• Add a CLI tool that asks a few questions about the plugin you want to create and sets you up with the proper boilerplate

Let me know if you'd like the Peril issues to belong to a different epic instead, or any feedback you have on this proposed change.

[jlengstorf]

@shannonbux The CLI stuff is captured in #6729, so I don't think that needs to be added to anyone's work.

The README template looks great. I think you can close that out.

I think the other organization you're suggesting looks good, too. Feel free to move things around and close this out. Thanks!

[shannonbux]

Ok, all those suggestions are implemented now, @jlengstorf! Thanks!