Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add fallback for createContentDigest #13584

Merged
merged 74 commits into from
Apr 24, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
353b60d
add createContentDigest helper for gatsby-transformer-remark
samscha Oct 10, 2018
9c0e187
add createContentDigest helper for gatsby-source-filesystem
samscha Oct 10, 2018
d3b3931
Added createContentDigest in test
samscha Oct 10, 2018
4c13027
Added missing createContentDigest param
samscha Oct 10, 2018
3bfd43a
update createContentDigest helper tests for gatsby-transformer-remark
samscha Oct 15, 2018
46e277a
update createContentDigest helper tests for gatsby-source-filesystem
samscha Oct 15, 2018
d8e9689
update createContentDigest helper tests for gatsby-transformer-csv
samscha Oct 15, 2018
096f1c8
add createContentDigest helper for internal-plugins/internal-data-bridge
samscha Oct 15, 2018
9d28bec
add createContentDigest helper for gatsby-source-hacker-news
samscha Oct 15, 2018
f379bb6
add createContentDigest helper for gatsby-transformer-excel
samscha Oct 15, 2018
2320a48
add createContentDigest helper for gatsby-source-mongodb
samscha Oct 17, 2018
0a1e9c2
add createContentDigest helper for gatsby-source-lever
samscha Oct 17, 2018
a5e2cd4
add createContentDigest helper for internal-plugins/query-runner
samscha Oct 17, 2018
917b140
Updated gatsby dep
samscha Oct 17, 2018
e936637
add createContentDigest helper for gatsby-source-contentful
samscha Oct 17, 2018
524ca7b
add createContentDigest helper for gatsby-transformer-hjson
samscha Oct 17, 2018
9db133e
add createContentDigest helper for gatsby-transformer-sqip
samscha Oct 17, 2018
90e40f7
add createContentDigest helper for gatsby-source-npm-package-search
samscha Oct 17, 2018
66674f1
add createContentDigest helper for gatsby-transformer-toml
samscha Oct 17, 2018
002c8c3
update createContentDigest helper for gatsby-source-mongodb
samscha Oct 18, 2018
4b79ae9
update createContentDigest helper for gatsby-source-lever
samscha Oct 18, 2018
aa9c1ff
update createContentDigest helper for internal-plugins/query-runner
samscha Oct 18, 2018
4a9c152
update createContentDigest helper for gatsby-source-npm-package-search
samscha Oct 18, 2018
58ecb37
update createContentDigest helper for gatsby-source-contentful*
samscha Oct 18, 2018
cdb0a8a
update createContentDigest helper for gatsby-transformer-sqip*
samscha Oct 18, 2018
c0f681c
update createContentDigest helper for gatsby-transformer-sqip
samscha Oct 18, 2018
a493701
Merge branch 'master' into topics/createContentDigest-plugins
m-allanson Dec 14, 2018
ab255b1
Merge remote-tracking branch 'origin/master' into topics/createConten…
pieh Dec 31, 2018
26fa18a
use caret version selector
pieh Dec 31, 2018
10d995a
no need to import it - it's passed to `sourceNodes`
pieh Dec 31, 2018
2af8461
merge master into createContestDigest-plugins
samscha Mar 2, 2019
595e5ce
merge again
samscha Mar 2, 2019
93ee035
update to pass tests
samscha Mar 2, 2019
625e16f
Merge branch 'master' of https://github.com/samscha/gatsby into topic…
samscha Mar 7, 2019
6fd3850
update hash to use createContentDigest
samscha Mar 7, 2019
ce82633
Merge branch 'master' of https://github.com/samscha/gatsby into topic…
samscha Mar 8, 2019
4360e58
move createContentDigest to import
samscha Mar 8, 2019
f5a5583
move createContentDigest to import for gatsby-source-contentful
samscha Mar 8, 2019
942a174
remove createContentDigest import in gatsby-source-filesystem test
samscha Mar 8, 2019
58a5ef1
rereplace createContentDigest with crypto in gatsby-source-filesystem
samscha Mar 8, 2019
42f7d4e
revert createContentDigest to crypto for create-file-node in gatsby-s…
samscha Mar 8, 2019
2320c79
revert createContentDigest for gatsby-plugin-sharp
samscha Mar 8, 2019
9153109
update crypto misspell
samscha Mar 8, 2019
2e7b305
revert gatsby-remark-contentful and gatsby-source-contentful
samscha Mar 8, 2019
bd9e193
fix old code
samscha Mar 8, 2019
e7e2625
fix old code v2
samscha Mar 8, 2019
0752777
update gatsby pkg
samscha Mar 8, 2019
c3d6853
revert gatsby-transformer-remark
samscha Mar 8, 2019
d54c261
add export for gatsby utils
samscha Mar 8, 2019
e90f381
update gatsby-source-filesystem
samscha Mar 9, 2019
1fe7b89
revert gatsby-source-graphql
samscha Mar 9, 2019
84558cd
revert gatsby-source-graphql
samscha Mar 9, 2019
335c0c6
revert gatsby-source-hacker-news
samscha Mar 9, 2019
9f709e0
revert gatsby-source-lever
samscha Mar 9, 2019
c2b4c1a
revert gatsby-source-medium
samscha Mar 9, 2019
5537c77
revert gatsby-source-mongodb
samscha Mar 9, 2019
da57567
revert gatsby-source-mongodb mapping
samscha Mar 9, 2019
74222a5
revert gatsby-source-npm-package-search
samscha Mar 9, 2019
41b2deb
revert gatsby-source-wikipedia
samscha Mar 9, 2019
c105a3f
revert gatsby-source-wordpress
samscha Mar 9, 2019
90b54c5
revert gatsby-transformer-csv
samscha Mar 9, 2019
cf92d0e
revert gatsby-transformer-documentationjs
samscha Mar 9, 2019
18c86b1
revert gatsby-transformer-excel
samscha Mar 9, 2019
da90859
revert gatsby-transformer-hjson
samscha Mar 9, 2019
e005647
revert other packages
samscha Mar 9, 2019
2e5846c
revert some pkgs
samscha Mar 10, 2019
66136a0
add gatsby
samscha Mar 10, 2019
5a313e4
Merge branch 'master' of https://github.com/samscha/gatsby into topic…
samscha Mar 10, 2019
5e908e6
Merge remote-tracking branch 'upstream/master' into pr/8992
wardpeet Apr 23, 2019
fd1217d
move gatsby to peerdeps
wardpeet Apr 23, 2019
d263b80
Merge branch 'master' of https://github.com/samscha/gatsby into topic…
samscha Apr 24, 2019
23e2a4b
add fallbacks
samscha Apr 24, 2019
c702ce4
Remove fallback from gatsby-transformer-remark
sidharthachatterjee Apr 24, 2019
23d8b0d
Remove peer dependency bump
sidharthachatterjee Apr 24, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions packages/gatsby-source-filesystem/src/create-file-node.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ const mime = require(`mime`)
const prettyBytes = require(`pretty-bytes`)

const md5File = require(`bluebird`).promisify(require(`md5-file`))
const crypto = require(`crypto`)
const { createContentDigest } = require(`./fallback`)

exports.createFileNode = async (
pathToFile,
Expand All @@ -27,12 +27,10 @@ exports.createFileNode = async (
const stats = await fs.stat(slashedFile.absolutePath)
let internal
if (stats.isDirectory()) {
const contentDigest = crypto
.createHash(`md5`)
.update(
JSON.stringify({ stats: stats, absolutePath: slashedFile.absolutePath })
)
.digest(`hex`)
const contentDigest = createContentDigest({
stats: stats,
absolutePath: slashedFile.absolutePath,
})
internal = {
contentDigest,
type: `Directory`,
Expand Down
22 changes: 2 additions & 20 deletions packages/gatsby-source-filesystem/src/create-remote-file-node.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const fs = require(`fs-extra`)
const got = require(`got`)
const crypto = require(`crypto`)
const { createContentDigest } = require(`./fallback`)
const path = require(`path`)
const { isWebUri } = require(`valid-url`)
const Queue = require(`better-queue`)
Expand Down Expand Up @@ -53,24 +53,6 @@ const bar = new ProgressBar(
* @param {Auth} [options.auth]
*/

/*********
* utils *
*********/

/**
* createHash
* --
*
* Create an md5 hash of the given str
* @param {Stringq} str
* @return {String}
*/
const createHash = str =>
crypto
.createHash(`md5`)
.update(str)
.digest(`hex`)

const CACHE_DIR = `.cache`
const FS_PLUGIN_DIR = `gatsby-source-filesystem`

Expand Down Expand Up @@ -215,7 +197,7 @@ async function processRemoteNode({
}

// Create the temp and permanent file names for the url.
const digest = createHash(url)
const digest = createContentDigest(url)
if (!name) {
name = getRemoteFileName(url)
}
Expand Down
15 changes: 15 additions & 0 deletions packages/gatsby-source-filesystem/src/fallback.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// TODO: remove in gatsby v3
exports.createContentDigest = input => {
try {
const { createContentDigest } = require(`gatsby/utils`)

return createContentDigest(input)
} catch (e) {
const content = typeof input !== `string` ? JSON.stringify(input) : input

return require(`crypto`)
.createHash(`md5`)
.update(content)
.digest(`hex`)
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
const { graphql } = require(`gatsby/graphql`)
const { onCreateNode } = require(`../gatsby-node`)
const extendNodeType = require(`../extend-node-type`)
const { createContentDigest } = require(`gatsby/utils`)

// given a set of nodes and a query, return the result of the query
async function queryResult(
Expand Down Expand Up @@ -104,6 +105,7 @@ const bootstrapTest = (
loadNodeContent,
actions,
createNodeId,
createContentDigest,
},
{ ...additionalParameters, ...pluginOptions }
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ const _ = require(`lodash`)
const onCreateNode = require(`../on-node-create`)
const { graphql } = require(`gatsby/graphql`)

const { createContentDigest } = require(`gatsby/utils`)

let node
let actions
let createNodeId
Expand Down Expand Up @@ -38,6 +40,7 @@ Where oh where is my little pony?
loadNodeContent,
actions,
createNodeId,
createContentDigest,
}).then(() => {
expect(actions.createNode.mock.calls).toMatchSnapshot()
expect(
Expand Down Expand Up @@ -75,6 +78,7 @@ Sed bibendum sem iaculis, pellentesque leo sed, imperdiet ante. Sed consequat ma
loadNodeContent,
actions,
createNodeId,
createContentDigest,
},
{ excerpt_separator: `<!-- end -->` }
).then(() => {
Expand Down Expand Up @@ -106,6 +110,7 @@ yadda yadda
actions,
createNodeId,
loadNodeContent,
createContentDigest,
})

expect(parsed.frontmatter.date).toEqual(new Date(date).toJSON())
Expand Down Expand Up @@ -207,6 +212,7 @@ In quis lectus sed eros efficitur luctus. Morbi tempor, nisl eget feugiat tincid
loadNodeContent,
actions,
createNodeId,
createContentDigest,
},
{ excerpt_separator: `<!-- end -->` }
)
Expand Down Expand Up @@ -261,6 +267,7 @@ Sed bibendum sem iaculis, pellentesque leo sed, imperdiet ante. Sed consequat ma
loadNodeContent,
actions,
createNodeId,
createContentDigest,
})
})
})
Expand Down
15 changes: 9 additions & 6 deletions packages/gatsby-transformer-remark/src/on-node-create.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,15 @@
const grayMatter = require(`gray-matter`)
const crypto = require(`crypto`)
const _ = require(`lodash`)

module.exports = async function onCreateNode(
{ node, loadNodeContent, actions, createNodeId, reporter },
{
node,
loadNodeContent,
actions,
createNodeId,
reporter,
createContentDigest,
},
pluginOptions
) {
const { createNode, createParentChildLink } = actions
Expand Down Expand Up @@ -53,10 +59,7 @@ module.exports = async function onCreateNode(
markdownNode.fileAbsolutePath = node.absolutePath
}

markdownNode.internal.contentDigest = crypto
.createHash(`md5`)
.update(JSON.stringify(markdownNode))
.digest(`hex`)
markdownNode.internal.contentDigest = createContentDigest(markdownNode)

createNode(markdownNode)
createParentChildLink({ parent: node, child: markdownNode })
Expand Down
3 changes: 2 additions & 1 deletion packages/gatsby/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,8 @@
"cache-dir",
"dist",
"graphql.js",
"index.d.ts"
"index.d.ts",
"utils.js"
],
"homepage": "https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby#readme",
"keywords": [
Expand Down
1 change: 1 addition & 0 deletions packages/gatsby/utils.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
exports.createContentDigest = require(`./dist/utils/create-content-digest`)