Improve tutorial "create a source plugin"

[muescha]

DRAFT ISSUE

Summary

while i go through the tutorial i had this problems and i have some suggestions...

1. Clone this Code

Where

in https://www.gatsbyjs.org/tutorial/source-plugin-tutorial

If you want to follow along with this tutorial, you can find the codebase inside [the examples folder of the Gatsby repository]https://github.com/gatsbyjs/gatsby/tree/master/examples/creating-source-plugins. Once you clone this code, make sure to delete the source-plugin and example-site folders.

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 28 in 2630758

This tutorial builds off of an existing Gatsby site and some data. If you want to follow along with this tutorial, you can find the codebase inside [the examples folder of the Gatsby repository](https://github.com/gatsbyjs/gatsby/tree/master/examples/creating-source-plugins). Once you clone this code, make sure to delete the `source-plugin` and `example-site` folders. Otherwise, the tutorial steps will already be completed.

What

"Once you clone this code"

i miss a hint or a command how to clone a subdirectory. as new user i do not have a clue how to do this.

maybe he end up with full cloning the gatsby repo and this takes a long time and big space on hard drive

Suggestion

• add an easy command for this step
• maybe add a git sparse checkout command for this?

2. Path and new Tab

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 65 in 2630758

Create a new Gatsby site with the `gatsby new` command, based on the hello world starter.

What

as a new user i am not shure where to put the next commmands

Suggestion

• add: "leave the api server running (you can restart it to restore the api database) and open a new terminal (tab) to create a new Gatsby site."

• and add: " move to the root folder of the example project" (because we was before in the api folder)

3. Copy Diffs is hard

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 302 in 2630758

+ })

What

There is a diff code block, which i should copy and past in to my code

but with the diff tag it is not that easy to copy because it includes the + :(

Suggestion

• A

add a "normal" code block above for easy copy and paste and leave the current code block to see where i should post it

• B

change the code block in a normal code block and mark the changed files with highlight

4. Gatsby clean is needed

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 506 in 2630758

remoteImage # returns an ID like "ecd83d94-7111-5386-bd3f-0066248b6fa9"

What

if i run gatsby develop before adding the onCreateNode it not get called with type=Post, because the Post nodes already created and cached, so the onCreateNode is never called with type=Post

i had to do a gatsby clean to see the remoteImage...

Suggestion

• add an info to run gatsby clean if it not show up
• explain why gatsby clean is needed, so that in additional problems this solution comes in mind

5. gatsby-source-filesystem

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 437 in 2630758

Start by installing `gatsby-source-filesystem` in the `source-plugin` project:

What

normally is the way for plugins: after installing it you to have to gatsby know it you have to put it in the config

also when i see the note here:

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 20 in 2630758

_**NOTE:** if your data is local i.e. on your file system and part of your site's repo, then you generally don't want to create a new source plugin. Instead you want to use [gatsby-source-filesystem](/packages/gatsby-source-filesystem/) which handles reading and watching files for you. You can then use [transformer plugins](/plugins/?=gatsby-transformer) like [gatsby-transformer-yaml](/packages/gatsby-transformer-yaml/) to make queryable data from files._

i was wondering why i should install this plugin

Suggestion

• explain that we only need the createRemoteFileNode function
• thats why we don't need to include it into the gatsby-config.js, because we not using it sourcing from file system

6. Fetch timeout

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 502 in 2630758

query {

What

for debug i added:

  console.log(JSON.stringify(node.internal,null,2))
  if (node.internal.type === POST_NODE_TYPE) {
	console.log( `add ${node.imgUrl} ...`)

and

    if (fileNode) {
      console.log(`added as ${fileNode.id}`)

and get this error:

add https://images.unsplash.com/photo-1534432586043-ead5b99229fb ...
add https://images.unsplash.com/photo-1530041539828-114de669390e ...
add https://images.unsplash.com/photo-1541364983171-a8ba01e95cfc ...
added as 8d0c7df7-c236-51a2-87a9-035bde315431

 ERROR #11321  PLUGIN

"gatsby-starter-plugin" threw an error while running the onCreateNode lifecycle:

failed to process https://images.unsplash.com/photo-1534432586043-ead5b99229fb
TimeoutError: Timeout awaiting 'request' for 30000ms


 ERROR #11321  PLUGIN

"gatsby-starter-plugin" threw an error while running the onCreateNode lifecycle:

failed to process https://images.unsplash.com/photo-1541364983171-a8ba01e95cfc
TimeoutError: Timeout awaiting 'request' for 30000ms

the image https://images.unsplash.com/photo-1530041539828-114de669390e takes 1,8min to load

Question

• also i see that the original node not gets dirty and i have to gatsby clean to have refetched the images again

• can the nodes declared as "dirty" directly if it enters the the if type=Post statement? then i not need always do a gatsby clean while testing

Suggestion

the images are dynamically-resizable and adding ?w=0.5 to the url fixed it for me. the qualitiy of the images are not that important in this tutorial and that fix the timeout for me.

a drawback is that this suggest to load a lower quality, but thats not exactly how gatsby works. the images should be downscaled by transformer.

maybe it would be better to use an other remote file source for the images with better speed.

    const fileNode = await createRemoteFileNode({
      // the url of the remote image to generate a node for
      url: node.imgUrl+`?w=0.5`,
      parentNodeId: node.id,
      createNode,
      createNodeId,
      getCache,
    })

{
  "contentDigest": "bf4301eabd6cddd61ba002d1c389e433",
  "type": "File",
  "mediaType": "image/jpeg",
  "description": "File \"https://images.unsplash.com/photo-1530041539828-114de669390e?w=0.5\"",
  "counter": 24,
  "owner": "gatsby-source-filesystem"
}
added as 15a03e7a-733c-5656-9a2d-6c3b838690eb

7. gatsby clean needed

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 617 in 2630758

Now running the site will allow you to query authors and remoteImages from the post nodes!

what

here i need to do a gatsby clean to get the changes done - in my logs i see the items are using the cached

Suggestion

• add here a line / hint for gatsby clean

8. replace instead of create new

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 617 in 2630758

Now running the site will allow you to query authors and remoteImages from the post nodes!

What

the file example-site/src/pages/index.js here exists from starter:

import React from "react"

export default function Home() {
  return <div>Hello world!</div>
}

Suggestion

• change wording from new file to open file and replace

9. gatsby-image is not installed

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 617 in 2630758

Now running the site will allow you to query authors and remoteImages from the post nodes!

What

 ERROR #98124  WEBPACK

Generating development JavaScript bundle failed

Can't resolve 'gatsby-image' in '/Users/muescha/Work/gatsby/github/pr-23363/creating-source-plugins/example-site/src/pages'

If you're trying to use a package make sure that 'gatsby-image' is installed. If you're trying to use a local file make sure that the path is correct.

File: src/pages/index.js

Why

Add a file at `example-site/src/pages/index.js` and copy the following code into it:

This conversation was marked as resolved by gillkyle
Ensure you have `gatsby-image` installed in the site by running `npm install gatsby-image`. It provides a component that can take the optimized image data and render it.

```javascript:title=example-site/src/pages/index.js
import React from "react"
import { graphql } from "gatsby"
import Img from "gatsby-image"

i overlooked the hint because it is between the command to add the text to a file and the content i have to copy. so you jump over the lines when following the commands step by step

Suggestion

rewording:

To provide a component that can take the optimized image data and render it install `gatsby-image` in the `example-site` (not the plugin):

```shell:title=example-site
npm install gatsby-image
```

Then add a file at `example-site/src/pages/index.js` and copy the following code into it:

```javascript:title=example-site/src/pages/index.js
import React from "react"
import { graphql } from "gatsby"
import Img from "gatsby-image"

10. check the result

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 709 in 2630758

This code uses a [page query](/docs/page-query/) to fetch all posts and provide them to the component in the `data` prop at build time. The JSX code loops through the posts so they can be rendered to the DOM.

What

i miss and instruction to check the new result

Suggestion

• add a text (and an screenshot) :

to see the result open your browser at `http://localhost:8000/` and you see this overview:

10. to much code here

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Lines 739 to 742 in 2630758

	const { createNode, touchNode, deleteNode } = actions
	// highlight-start
	console.log(pluginOptions.previewMode) // true
	// highlight-end

What

there is code which is not added at this point in the tutorial

Suggestion

-  const { createNode, touchNode, deleteNode } = actions
+  const { createNode } = actions

11. put removed code from 10 back

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 769 in 2630758

touchNode({ nodeId: node.id })

What

only in this step we are adding this actions

Suggestion

add a hightlight-line and inform in text that this new helpers are added, leave the console.log from previous step:

-  const { createNode, touchNode, deleteNode } = actions
+  const { createNode, touchNode, deleteNode } = actions // highlight-line
+  console.log(pluginOptions.previewMode) // true

12. DRY

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 887 in 2630758

const nodeId = createNodeId(`${POST_NODE_TYPE}-${post.id}`)

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 891 in 2630758

node: getNode(nodeId),

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 901 in 2630758

id: createNodeId(`${POST_NODE_TYPE}-${post.id}`),

What

should reuse the nodeId from 887

Suggestion

• reuse nodeid from 887:

-              id: createNodeId(`${POST_NODE_TYPE}-${post.id}`),
+              id: nodeId,

• or inline it in 891

-              node: getNode(nodeId),
+              node: getNode(createNodeId(`${POST_NODE_TYPE}-${post.id}`)),

13. getNode is not defined

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 887 in 2630758

const nodeId = createNodeId(`${POST_NODE_TYPE}-${post.id}`)

What

i tried the deletePost mutation and get this error:

ReferenceError: getNode is not defined
    at /Users/muescha/Work/gatsby/github/pr-23363/creating-source-plugins/source-plugin/gatsby-node.js:150:21
    at Array.forEach (<anonymous>)

Suggestion

• add getNode to this line (in many examples)

         getNodesByType,
+        getNode,

14. more implementations in example

Where

gatsby/docs/tutorial/source-plugin-tutorial.md

Line 976 in 2630758

- [Example repository](https://github.com/gatsbyjs/gatsby/tree/master/examples/creating-source-plugins) with all of this code implemented

What

it says just all code from tutorial is implemented, but there is a little bit more implemented.

• helper for creating nodes
• cache

Suggestion

• add a little info that there is much more interesting stuff in the example code

Motivation

the tutorial should be new user friendly.

Steps to resolve this issue

reword the phase

Draft the doc

• rewrite the doc, following the format listed in these resources:
  • Overview on contributing to documentation

Open a pull request

• Open a pull request with your work including the words "closes #[this issue's number]" in the pull request description

Related Issue

• docs: new source plugin tutorial #23363 docs: new source plugin tutorial

[muescha]

i will create later a summary of all ideas for this tutorial. i close this until then