-
Notifications
You must be signed in to change notification settings - Fork 10.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CTF-v6: ☂️ Umbrella issue for next major version of gatsby-source-contentful #31385
Comments
Breaking changes: Migration / steps to migrateThe new generated schema is pretty much aligned with the schema the Contentful GraphQL API generates for your space. This means you have to change your queries a little: GeneralRemove your workaroundsIn case you froze your schema via New content type naming patternThe old naming pattern for generated node types based on your Contentful content model was pretty simple. This gave us shorter queries, but came with restrictions for content type names. Content type names like
This is close to the behaviour of the Contentful GraphQL API:
Instead of doing this on a collision, we always prefix for consistency. export const pageQuery = graphql`
query {
- allContentfulProduct {
+ allContentfulContentTypeProduct {
...
}
}
` Metadata / Data from
|
Old Path | New Path | Comment |
---|---|---|
node_locale | sys.locale | this does not exists in Contentful's sys but we need it (for now) as Contentful GraphQL handles locales different |
contentful_id | sys.id | |
spaceId | sys.spaceId | |
createdAt | sys.firstPublishedAt | |
updatedAt | sys.publishedAt | |
revision | sys.publishedVersion | |
sys.environmentId | This is new | |
sys.contentType | sys.contentType | This is now a real reference to the Content Type node. If you used thsi field, you will probably need to change the__typename subfield to name |
Sorting
allContentfulBoolean(sort: { fields: contentful_id }) {}
becomes
allContentfulBoolean(sort: { fields: sys___id }) {}
Filtering
contentfulDate(contentful_id: { eq: "38akBjGb3T1t4AjB87wQjo" }) {}
becomes
contentfulDate(sys: { id: { eq: "38akBjGb3T1t4AjB87wQjo" } }) {}
Fields
Text fields
- the raw field value is now stored in
FIELD_NAME { raw }
instead ofFIELD_NAME { FIELD_NAME }
JSON fields
- You can no more queries sub-fields, which ensures your queries won't break on changed content. This means you now have to check if the results have a value on your own.
- Spaces within keys of JSON fields are now kept.
<p data-cy-value-born-at>Born at: {actor.Born_At}</p>
becomes
<p data-cy-value-born-at>Born at: {actor["Born At"]}</p>
Rich Text field
The Rich Text fields schema got changed to match the schema of the Contentful GraphQL API. You will need to adjust your code for rendering Rich Text as well.
Also:
- You can now query all Contentful content types in the
links
subfield, not matter if they are linked or not. - The entry id can now be queried directly as child-field of
links
instead of fields within each referenced content type. Simplifies your queries!
Old schema
type ContentfulRichTextRichText {
raw: String
references: [ContentfulAssetContentfulContentReferenceContentfulLocationContentfulTextUnion] @link(by: "id", from: "references___NODE")
}
union ContentfulAssetContentfulContentReferenceContentfulLocationContentfulTextUnion = ContentfulAsset | ContentfulContentReference | ContentfulLocation | ContentfulText
New schema
type ContentfulNodeTypeRichText @dontInfer {
json: JSON
links: ContentfulNodeTypeRichTextLinks
}
type ContentfulNodeTypeRichTextLinks {
assets: ContentfulNodeTypeRichTextAssets
entries: ContentfulNodeTypeRichTextEntries
}
type ContentfulNodeTypeRichTextAssets {
block: [ContentfulAsset]!
hyperlink: [ContentfulAsset]!
}
type ContentfulNodeTypeRichTextEntries {
inline: [ContentfulEntry]!
block: [ContentfulEntry]!
hyperlink: [ContentfulEntry]!
}
Old example query
query pageQuery {
allContentfulRichText {
nodes {
richText {
raw
references {
... on ContentfulAsset {
url
}
... on ContentfulText {
id
}
}
}
}
}
}
New example query
query pageQuery {
allContentfulContentTypeRichText {
nodes {
richText {
json
links {
assets {
block {
url
}
}
entries {
block {
... on ContentfulContentTypeText {
id
}
}
}
}
}
}
}
}
NOTE:
raw
now isjson
references
now islinks
, with assets and entries separated and increased granularity. Now you can provide different data for block or inline references in your linked rich text entities.
Rendering changes
Instead of passing your option object into renderRichText()
you now pass a option factory:
-const options = {
+const makeOptions = ({ assetBlockMap, entryBlockMap, entryInlineMap }) => ({
[BLOCKS.EMBEDDED_ENTRY]: node => {
- const entry = node?.data?.target
+ const entry = entryBlockMap.get(node?.data?.target?.sys.id)
}
}
-<div>{renderRichText(richText, options)}</div>
+<div>{renderRichText(richText, makeOptions)}</div>
Assets
Assets got simplified a lot!
Old schema
type ContentfulAsset implements ContentfulInternalReference & Node @dontInfer {
file: ContentfulAssetFile
title: String
description: String
sys: ContentfulInternalSys
}
type ContentfulAssetFile {
url: String
details: ContentfulAssetFileDetails
fileName: String
contentType: String
}
type ContentfulAssetFileDetails {
size: Int
image: ContentfulAssetFileDetailsImage
}
type ContentfulAssetFileDetailsImage {
width: Int
height: Int
}
New schema
type ContentfulAsset implements ContentfulInternalReference & Node @dontInfer {
sys: ContentfulInternalSys
title: String
description: String
mimeType: String
fileName: String
url: String
size: Int
width: Int
height: Int
}
Breaking changes: Schema updatesThis section should help you to understand the changes we've done to the schema generation. This compares the old and new schema of our Contentful e2e test suite which should cover all possible data variants in Contentful.
As you can see, the new schema is about 50% smaller. Output of
|
Explain once |
More information will follow as soon I am done with the core PR. (#30855) |
Any chance you can implement to include entity.sys.firstPublishedAt in the GraphQL responses? |
@a4GevatterTod I can only add information that is exposed via the Contentful Sync API. As you can see above, we now have sys.firstPublishedAt and sys.publishedAt available. |
Hi @axe312ger, So this 'should' include firstPublishedAt. |
@a4GevatterTod Alright. I'll check this week if it is available. Otherwise check if there is a chance to get it exposed in the sync api as well. |
@axe312ger This looks great! I have a question: My team is using Contentful's REST API to build our gatsby site, but we're interesting in starting to use the source plugin, but from what I've played around with it it seems that the sync API basically downloads the entire contentful space/env data and makes it available no matter whether your using or not, and as you can imagine that takes a while. Is there a way to only pull certain content types? Otherwise we have to transition all our templates at once, because having a mix means that our build times would be insane (for context we build ~10k pages). I'm also wondering how Gatsby builds would benefit from the Sync API local caching of data? We build our site using a Docker container but that gets propped up for every deploy, which means there wouldn't be any "cached" data to work off of, so we'd have to do a full Sync every time. I'm assuming there's way to take advantage of the caching? Thank you!! If this is not the right forum let me know where I should post this :) |
It might be possible, but would require multiple rewrites: https://www.contentful.com/developers/docs/references/content-delivery-api/#/reference/synchronization/initial-synchronization-of-entries-of-a-specific-content-type Looks like we can only sync one single content type, that means if we add that feature, we'd need to run a sync for every given content type. Plus another one for the assets. Additionally the plugin downloads 1000 entities per call sync since the last version in Gatsby v2 (Before it was 100). That means even 10k of pages will be downloaded now 10 times faster as before.
The plugin uses the Gatsby core caching mechanism. As long you share your |
@axe312ger Thanks for the response, appreciate it! I've played with it a bit locally and yes, if we persisted our |
Hi @axe312ger I have a problem with the current gatsby-source-contentful module and wondering if the v6 upgrade will help. The project I work on has rich text field in one entry that embeds another entry that also has a rich text field. And sometimes it creates a circular reference with the other rich text field embedding the first entry. Before the v4 upgrade I was using a custom fix to render the JSON from rich text field that works with the circular references. But I have been struggling to get it working since, with the rich text field change to use raw and references. Is the v6 upgrade going to fix this issue? If not how can I make the plugin work with the loop in rich text embedded entries? Thanks. |
To keep everybody in the loop:
|
@stevenyang1011 sorry for overlooking your comment. Your issue should be fixed in the current version already. Could you please open a separate ticket to give me more details? I will then investigate whats going on :) |
Thanks for getting back to me, I've posted the question here https://github.com/gatsbyjs/gatsby/discussions/31932 |
Status update: We will soon be able to merge #31395, which then should lead to the first beta release! |
Awesome @axe312ger, I think this will be definitely a major difference for people who are using gatsby + contentful. As soon as we have a beta version I will start the migration of medium size ( 1000-2k) pages website. Thaaanks :) |
@lucashfreitas You are welcome. Looking forward to the first migrations and see how it will perform in your projects. |
Hey @axe312ger - the schema generation task (#30855) will be shipped alongside the gatsby v4 upgrade? |
@lucashfreitas unfortunately, no :( But I still hope for this year :) |
Cool - please let me know if any contribution from the community is welcome or you need help with anything - I am familiar with this plugin and have been developing my own source plugin, but I stopped as we will have a new version soon fixing one of the main issues with contentful/gatsby IMO which is the graphql schema generation, not reflectin the content types structures in the cms backend. Thank you very much for all the work and for developing this plugin :) Have a nice week ahead! |
@lucashfreitas thank you so much. I'll keep everyone updated in this thread as soon I have news. I'll also start rebasing the PRs soon on the latest changes we did to the plugin for Gatsby v4 :) |
hi @axe312ger! My team is SUPER excited for this version to land so that we can start using it - any ideas when you'd expect there to be at least a beta branch? |
@juanbiberretta This is great! We currently testing the new version with some projects we got access to. As soon we are happy with the results, we will make a test version available to everyone! |
A small update: The code mods are taking shape, see progress in #35113 I also discovered a few issues and things that are still to do. See |
Hello @axe312ger, do we have any timeline on when this will be released? |
Will this be compatible with Gatsby v4 only or will it also work with v3? |
@WillsB3 This will be probably Gatsby v5 only. I am currently waiting for changes in the API before this will be continued. I'll update you as soon something happens! |
@axe312ger Now that Gatsby v5 is out, is there an estimate for release of the next version? We would really like to not worry about the graphql schema anymore 😭 |
Hello guys, Sharing a personal experience here as it might shed light on anyone else facing problems with the current plugin for gatsby/contentful. We had several projects in gatsby + contentful and all had the same problem: building frequently breaking, a lot of workarounds because contentful doesn't infer the schema from the graphql, etc. The limitation seems simple but it started to really impact our customers and we had to spend a lot of time building workarounds due to the fact the plugin doesn't read/infer the schema type from contentful and only existent fields, so quite frequently someone would remove a field and the build would break complaining that the field was not existent. So we were very excited about this new version of this plugin to be released so we told our clients to wait so that as soon as this upgrade would come we could upgrade the plugin and solve the frequent issues/limitations of this plugin. After a year of pressure from our customers, we have finally decided to migrate the websites to Note that I was using an old version of gatsby 3x and am not sure how the v4 and v5 improved the build issues, etc, but just sharing this. Again, thanks to @axe312ger for putting a lot of hard work into this and keeping us up to date. That was our experience. Hope that helps |
@lucashfreitas great you solved your issues! I would love to integrate directly with Contentfuls GraphQL API, but it lacks the feature of delta updates and subscriptions. Without these, the preview feature, rebuilds and more would take ages. Therefore, this rewrite brings it as close as possible to the Contentful GraphQL API Schema (100% is not possible, it collides with the one of Gatsby) & it resolves issues like breaking schemas when fields get removed. How big are your projects? I guess below 100k entities in Contentful? |
Can I ask if you guys know if the plugin is supposed to cover this usecase? Accessing content from other spaces from fields that reference content models from another space? |
We finally released a public alpha :) Please try it out and give feedback in the new discussion -> #38585 |
Discussion for the upcoming major release: #38585
This umbrella issue collects all information about the upcoming new version of the gatsby-source-contentful plugin.
The core goal of this version is to align the GraphQL schema of the plugin to Contentfuls GraphQL API. These breaking changes allow us to introduce further features and performance improvements as well.
The new schema will be generated based on your content types instead of the existing content. This gives huge benefits in reliability and other things, but requires several rewrites of code, especially on GraphQL query level.
Branching & prerelease
feat/contentful-schema-generation
branch.CTF-v6:
in the titleBreaking changes
Goals
There are conflicting field types in your data.
warning and allows the JSON data structure to be changed in Contentful without breaking the Gatsby GraphQL Queryreferences
is now calledlinks
and will always be available. This allows querying__typename
and data fromsys
for all links at once as well.sys
propertyContentfulEntry
interface. This should change depending on the field validation. (Either single type, union or the interface) (CTF-next: use union for reference fields with validation #31395)images-contentful) to match new asset structure
introduce new metadata/tags feature aka the newwill be released earlier via feat(contentful): add support for tags #31746metadata.tags
propertyNext action items
sys.type
is the entity type (Entry, Asset), so it should not becomesys.contentType
linkedFrom
like Contentful doesUpdate docs
use -1 for max
Changes in related plugins
Potential performance improvements
JSON.stringify()
and rich text renderer no more needs toJSON.parse()
(CTF-next: Generate GraphQL schema based on Contentful Content Types #30855)a. make it configurable via directives and/or plugin config or
b. move markdown in extra field
fieldNameMarkdown
, we still might need to make it configurableExamples and test projects
examples/using-contentful
(CTF-next: migrate using-contentful #31270)updateDeprecated featurebenchmarks/contenful
figure out what to do inDeprecated featurepackages/gatsby-recipes/src
updateDeprecated featureexamples/recipe-sourcing-contentful
The text was updated successfully, but these errors were encountered: