createRemoteFileNode doesn't use inferred fileType to check existence

[thomascorthouts]

Description

Sometimes we need to use APIs of which not all asset URLs contain the extension of the file type, this is something that Gatsby is capable of handling because the createRemoteFileNode will call the file-type npm package to infer file types when the extension is not in the URL, nor in the createRemoteFileNode options. This covers a lot of use-cases, but far from all.

When we need to use an images endpoint for example, where some of the image file extensions are not part of the asset URLs. On the first gatsby build or gatsby develop in a new environment this works fine, but in consecutive runs of the gatsby develop command, the system will try to look in its local cache whether the file node is still there. This step will fail given that the inferred file extension is not used to check whether said node exists, resulting in errors like this one:

Unexpected error value: "failed to process https://fakeimg.pl/300x300/?text=2
Error: ENOENT: no such file or directory, stat '.../.cache/gatsby-source-filesystem/7a1221ea4c61551f928908dd873a3738/300x300'"

But when you would go look, it could be that said file does exist as 300x300.png or 300x300.jpg implying that Gatsby doesn't use the inferred extension to reuse the file node it created.
Potential solutions could be to use a wildcard for the extension when the checking for a fileNode fails or using the inferred file extension in looking for a locally created fileNode by the createRemoteFileNode method.

Steps to reproduce

A minimal reproduction was created by @chpio already in #21901: https://github.com/chpio/gatsby-remote-wo-ext

Expected result

On consecutive runs of gatsby develop, without cleaning the cache in between, the local file nodes should be used to speed up the development process.

Actual result

The Gatsby development setup crashed, looking for a file that does exist, but it cannot find because it doesn't look for the filed with extension.

Environment

  System:
    OS: macOS 10.15.4
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 12.14.1 - ~/.nvm/versions/node/v12.14.1/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.13.4 - ~/.nvm/versions/node/v12.14.1/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 81.0.4044.138
    Safari: 13.1
  npmPackages:
    gatsby: ^2.21.9 => 2.21.9 
    gatsby-image: ^2.4.0 => 2.4.0 
    gatsby-plugin-canonical-urls: ^2.3.0 => 2.3.0 
    gatsby-plugin-portal: ^1.0.7 => 1.0.7 
    gatsby-plugin-purgecss: ^4.0.1 => 4.0.1 
    gatsby-plugin-react-helmet: ^3.3.0 => 3.3.0 
    gatsby-plugin-react-svg: ^2.1.2 => 2.1.2 
    gatsby-plugin-sass: ^2.3.0 => 2.3.0 
    gatsby-plugin-sentry: ^1.0.1 => 1.0.1 
    gatsby-plugin-sharp: ^2.6.0 => 2.6.0 
    gatsby-source-apiserver: ^2.1.4 => 2.1.4 
    gatsby-source-filesystem: ^2.3.0 => 2.3.0 
    gatsby-source-graphql: ^2.5.0 => 2.5.0 
    gatsby-source-greenhouse-job-board: ^1.0.3 => 1.0.3 
    gatsby-transformer-sharp: ^2.5.0 => 2.5.0 
    gatsby-transformer-yaml: ^2.4.0 => 2.4.0 
  npmGlobalPackages:
    gatsby-cli: 2.12.5

[pieh]

Thanks for report! I can reproduce this locally, and investigating now

[pieh]

Hmm I actually can't reproduce it anymore. I was getting different error because https://fakeimg.pl/300x300/ url can't be resolved now (no DNS entry anymore? 🤔) I tried it few days ago and then could reproduce, but couldn't investigate yet and now reproduction doesn't work anymore :/

[thomascorthouts]

Interesting 🤔 the website does seem to have just dissappeared. I will create a new example later

[thomascorthouts]

@pieh I forked the repository to use placeholder.com instead: https://github.com/thomascorthouts/gatsby-remote-wo-ext

Confirmed I get the same error on the second develop without cleaning cache:

 ERROR #85901  GRAPHQL

There was an error in your GraphQL query:

Unexpected error value: "failed to process https://via.placeholder.com/150
Error: ENOENT: no such file or directory, stat '/***/gatsby-remote-wo-ext/.cache/gatsby-source-filesystem/862db6c25dd320d02cce6612464221d4/150'"

[pieh]

Thanks! Opened pull request with proposed fix - #24613