[gatsby-source-graphcms] Add GraphCMS Plugin #1867

Redmega · 2017-08-21T03:21:46Z

Adds the gatsby-source-graphcms plugin to fetch all data from a GraphCMS backend.

Need to experiment with more complex data types and relationships to see if they're supported as is or if changes are needed.

As per @jorishermans in Discord:

Let the user decide the mapping in the plugin config in gatsby-config.js
[...] I implemented a mapping feature in gatsby-source-mongodb
it would be nice if the mapping of mediatypes is almost the same in all the other plugins

TODO:

Fix relationships/embedded fields
Implement mapping feature for transformation plugins, like the MongoDB plugin
Write tests
Create demo site

We can use the Starwars API for tests and the demo site.

As per @mlukaszczyk from @GraphCMS, Vinylbase is the "official" demo project for GraphCMS, and they'd be delighted to have a Gatsby clone of it using the same API (https://api.graphcms.com/simple/v1/vinylbase). Spike source for reference: https://github.com/GraphCMS/example_04_static_site_generation

Issues:

1. Fields which have sub selections do not work

This one pertains to the introspection metaquery method. I'll work on that later.

Example:

Artist {
  picture {
    url
...

The metaQuery currently used is not capable of finding url from the above query, which will cause the query that fetches all data to fail. It will have to be modified. It looks like it will be a bit more difficult to find that url and add it to the query, as it might require making a __type query for each field of a Type that has subfields and then modifying the final query before firing it.

Errors can be seen here: https://github.com/Redmega/example_05_static_site_generation_with_gatsby

2. Embedded fields aren't found by GraphQL

Example:

query getAllArtists {
  allArtists {
    name
    id
    records {
      id
      title
      tracks {
        id
        title
        length
      }
    }
  }
}

This query fed into the graphcms source plugin produces artists. name, id, records, id, and titles - but no tracks, or the subfields within. They exist in the redux store json, but graphql can't seem to identify them. More investigation is needed.

gatsbybot · 2017-08-21T03:34:51Z

Deploy preview ready!

Built with commit 7f6851c

https://deploy-preview-1867--using-drupal.netlify.com

gatsbybot · 2017-08-21T03:36:42Z

Deploy preview ready!

Built with commit d6de985

https://deploy-preview-1867--gatsbygram.netlify.com

KyleAMathews

This is looking fantastic!!

Excellent work!

Really love how simple the code you've come up with.

Tests would be great + a simple example site is super helpful for manual testing in the future + as a base for people to try out the plugin.

KyleAMathews · 2017-08-22T14:35:07Z

packages/gatsby-source-graphcms/README.md

+plugins: [
+  /*
+   * Gatsby's data processing layer begins with “source”
+   * plugins. Here the site sources its data from a graphql endpoint


"sources its data from the GraphCMS endpoint."

KyleAMathews · 2017-08-22T14:40:51Z

packages/gatsby-source-graphcms/README.md

+  }
+],
+```
+I suggest either using a `.env` file or setting environment variables to access ala `process.env.GATSBY_GRAPHQL_ENDPOINT`, `process.env.GATSBY_GRAPHQL_TOKEN` so theres no endpoint and token on git ;)


Linking to https://www.gatsbyjs.org/docs/environment-variables/ would be better.

That page needs updated to include a reference to using process.env for plugins.

Also you don't want to use "I" when writing docs as this is a community plugin so generally either just use command form "Use either a .env file or set environment variables directly…" or use "We". But "we" is rarely what you want.

Is there a special way to use process.env for plugins? Does the gatsby-config not have access to webpack's definitions at execution? I haven't tried it out yet, admittedly, so I will when I get home tonight.

KyleAMathews · 2017-08-22T14:43:49Z

packages/gatsby-source-graphcms/src/util.js

+
+
+// If type ends in a non-vowel, we need to append es. Else s.
+export const formatTypeName = t => `all${t}${/s$/.test(t) ? `es` : `s`}`


perhaps should use this instead? https://www.npmjs.com/package/pluralize

Or is this what GraphCMS says is their process?

I'll use pluralize. Forgot about all the special edge cases!

Well… what does GraphCMS do? That's what matters. Unless they use pluralize as well you'll just generate a bunch of invalid queries. Can't you get directly from the available types the connection name?

According to @GraphCMS

[...] the process of pluralizing names is on their side. They use this library to do this: https://github.com/atteo/evo-inflector

~~I'll implement evo-inflector tonight.~~ Looks like that's only on Java with Maven. ~~I'll implement pluralize instead as it seems to be using the same plural rules.~~

Can't you get directly from the available types the connection name?

Unfortunately no, that isn't in the data returned by the __type query. It is however returned by the __schema query, so maybe something can be done with that...

KyleAMathews · 2017-08-22T21:43:49Z

Deploy preview failed.

Built with commit 4ded842

https://app.netlify.com/sites/using-glamor/deploys/599c9aa06f4c502da66fe33b

Redmega · 2017-09-28T02:23:28Z

For some reason when using the custom query to fetch data, the nodes are not being created properly. It is missing certain fields. I haven't had the chance to test it with different data, I'll do more tests tomorrow.

Thebigbignooby · 2017-09-28T14:23:44Z

looking forward to using this !

Thebigbignooby · 2017-10-10T07:54:53Z

what's the status on this ?

Redmega · 2017-10-10T15:25:06Z

@Thebigbignooby Been busy with work! Sorry!

Trying to get this out as simply as possible, but passing the query through directly causes some issues with the createNode function. It's not populating nested data more than one or two levels deep. I'll add more context to the main post tonight.

rafacm · 2017-10-13T12:58:32Z

I have created PR #2445 since I did not know how to add to this existing one. That PR contains a working example for this plugin in examples/using-graphcms.

/cc @Redmega @KyleAMathews @Thebigbignooby

tlvenn · 2017-10-24T02:16:38Z

Question, with the recent addition of schema stitching feature, could this be leveraged as a way to more easily plug a graphql endpoint as a data source in Gatsby ?

@KyleAMathews what do you think ?

rafacm · 2017-11-03T11:01:53Z

with the recent addition of schema stitching feature, could this be leveraged
as a way to more easily plug a graphql endpoint as a data source in Gatsby ?

@tlvenn: can you elaborate?

Redmega · 2017-11-03T15:18:54Z

Schema stitching is a way to stitch schema from multiple sources together into a single query: ardatan/graphql-tools#382

Redmega · 2017-11-09T16:45:28Z

Continued in #2445

Redmega added 2 commits August 20, 2017 22:48

feat: add graphcms plugin

be5546b

refactor: extract method from sourceNodes

78054b1

docs: update example config to use right name

185d367

KyleAMathews suggested changes Aug 22, 2017

View reviewed changes

Angel Piscola added 3 commits August 22, 2017 14:44

Merge branch 'master' of git://github.com/gatsbyjs/gatsby

7f6851c

fix: allow public apis to be used sans token

2b423ee

docs: update README.md

4ded842

Redmega changed the title ~~Add GraphCMS Plugin~~ [gatsby-source-graphcms] Add GraphCMS Plugin Aug 24, 2017

feat: add custom query execution

3a4d149

Merge branch 'master' into master

7455d2a

Merge branch 'master' into master

d6de985

rafacm mentioned this pull request Oct 13, 2017

GraphCMS source plugin and usage example #2445

Closed

Redmega closed this Nov 9, 2017

snyk-bot mentioned this pull request Feb 19, 2022

[Snyk] Fix for 17 vulnerabilities 0xSebin/gatsby#4

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[gatsby-source-graphcms] Add GraphCMS Plugin #1867

[gatsby-source-graphcms] Add GraphCMS Plugin #1867

Redmega commented Aug 21, 2017 •

edited

Loading

gatsbybot commented Aug 21, 2017 •

edited

Loading

gatsbybot commented Aug 21, 2017 •

edited

Loading

KyleAMathews left a comment

KyleAMathews Aug 22, 2017

KyleAMathews Aug 22, 2017

Redmega Aug 22, 2017

KyleAMathews Aug 22, 2017

Redmega Aug 22, 2017

KyleAMathews Aug 22, 2017

Redmega Aug 22, 2017 •

edited

Loading

KyleAMathews commented Aug 22, 2017

Redmega commented Sep 28, 2017

Thebigbignooby commented Sep 28, 2017

Thebigbignooby commented Oct 10, 2017

Redmega commented Oct 10, 2017

rafacm commented Oct 13, 2017 •

edited

Loading

tlvenn commented Oct 24, 2017

rafacm commented Nov 3, 2017 •

edited

Loading

Redmega commented Nov 3, 2017

Redmega commented Nov 9, 2017



		// If type ends in a non-vowel, we need to append es. Else s.
		export const formatTypeName = t => `all${t}${/s$/.test(t) ? `es` : `s`}`

[gatsby-source-graphcms] Add GraphCMS Plugin #1867

[gatsby-source-graphcms] Add GraphCMS Plugin #1867

Conversation

Redmega commented Aug 21, 2017 • edited Loading

TODO:

Issues:

1. Fields which have sub selections do not work

This one pertains to the introspection metaquery method. I'll work on that later.

2. Embedded fields aren't found by GraphQL

gatsbybot commented Aug 21, 2017 • edited Loading

gatsbybot commented Aug 21, 2017 • edited Loading

KyleAMathews left a comment

Choose a reason for hiding this comment

KyleAMathews Aug 22, 2017

Choose a reason for hiding this comment

KyleAMathews Aug 22, 2017

Choose a reason for hiding this comment

Redmega Aug 22, 2017

Choose a reason for hiding this comment

KyleAMathews Aug 22, 2017

Choose a reason for hiding this comment

Redmega Aug 22, 2017

Choose a reason for hiding this comment

KyleAMathews Aug 22, 2017

Choose a reason for hiding this comment

Redmega Aug 22, 2017 • edited Loading

Choose a reason for hiding this comment

KyleAMathews commented Aug 22, 2017

Redmega commented Sep 28, 2017

Thebigbignooby commented Sep 28, 2017

Thebigbignooby commented Oct 10, 2017

Redmega commented Oct 10, 2017

rafacm commented Oct 13, 2017 • edited Loading

tlvenn commented Oct 24, 2017

rafacm commented Nov 3, 2017 • edited Loading

Redmega commented Nov 3, 2017

Redmega commented Nov 9, 2017

Redmega commented Aug 21, 2017 •

edited

Loading

gatsbybot commented Aug 21, 2017 •

edited

Loading

gatsbybot commented Aug 21, 2017 •

edited

Loading

Redmega Aug 22, 2017 •

edited

Loading

rafacm commented Oct 13, 2017 •

edited

Loading

rafacm commented Nov 3, 2017 •

edited

Loading