Regression: self-signed certificates no longer work

[aaronjensen]

Description

I'm having a hard time sorting out what package version where causes the issue, but when upgrading from gatsby 2.9.8 to 2.9.9 I now get this error when starting gatsby with a self-signed certificate:

error UNHANDLED REJECTION


  FetchError: request to https://localhost:8002/___graphql failed, reason: self signed certificate

  - index.js:1455 ClientRequest.<anonymous>
    [hello-epics]/[node-fetch]/lib/index.js:1455:11

  - destroy.js:82 emitErrorNT
    internal/streams/destroy.js:82:8

  - destroy.js:50 emitErrorAndCloseNT
    internal/streams/destroy.js:50:3

  - next_tick.js:63 process._tickCallback
    internal/process/next_tick.js:63:19

My guess is it has to do with the upgrade of node-fetch in gatsby-cli

Steps to reproduce

Create and trust ssl certificates, then run gatsby using them. I've also tried with the auto generated certificates and I get a different error.

gatsby develop --port 8002 --https --key-file ../config/localhost/key.pem --cert-file ../config/localhost/cert.pem

Expected result

It should start gatsby

Actual result

error UNHANDLED REJECTION


  FetchError: request to https://localhost:8002/___graphql failed, reason: self signed certificate

  - index.js:1455 ClientRequest.<anonymous>
    [hello-epics]/[node-fetch]/lib/index.js:1455:11

  - destroy.js:82 emitErrorNT
    internal/streams/destroy.js:82:8

  - destroy.js:50 emitErrorAndCloseNT
    internal/streams/destroy.js:50:3

  - next_tick.js:63 process._tickCallback
    internal/process/next_tick.js:63:19

Environment

Run gatsby info --clipboard in your project directory and paste the output here.

  System:
    OS: macOS 10.14.5
    CPU: (8) x64 Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
    Shell: 5.6.2 - /usr/local/bin/zsh
  Binaries:
    Node: 10.15.3 - /var/folders/p4/0ptjct956432wm2q1dd048g40000gn/T/yarn--1561063929532-0.7049991999729923/node
    Yarn: 1.16.0 - /var/folders/p4/0ptjct956432wm2q1dd048g40000gn/T/yarn--1561063929532-0.7049991999729923/yarn
    npm: 6.4.1 - ~/.asdf/shims/npm
  Languages:
    Python: 2.7.15 - /usr/local/bin/python
  Browsers:
    Chrome: 75.0.3770.100
    Firefox: 67.0.3

[aaronjensen]

Is there anyone to ping to get visibility on this? As far as i can tell, self signed certificates no longer work, which means that we (and likely others) are blocked from upgrading Gatsby. If there's anything else I can do to help, please let me know.

Thanks!

[sidharthachatterjee]

Potentially related to #15043

[Undistraction]

We are seeing the same issue as of 2.9.9. Using self-signed certs now causes builds to fail.

The workaround in #15043 works, but isn't a long-term solution.

The workaround is to set this env before running start:

NODE_TLS_REJECT_UNAUTHORIZED=0 gatsby start --https --cert-file ./certificates/dev.crt --key-file ./certificates/dev.key

[pieh]

Seems like this is coming from https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-cli/src/reporter/reporters/ink/components/develop.js#L46-L56

Potential code change that would fix crashes would be using https.Agent which doesn't reject unauthorized there - here's snippet I found in node-fetch/node-fetch#19

const https = require("https");
const agent = new https.Agent({
  rejectUnauthorized: false
})
fetch(myUrl, { agent })

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contributefor more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[Undistraction]

Still broken.

[gatsbot]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!

[Undistraction]

Be gone gatsbot. Still broken.

[Js-Brecht]

I guess I should have posted my comment here 😕. Forgot about this thread.

#16726 (comment)

[ghost]

Hi Everyone, I am still seeing this issue after installing a latest gatsby project today. What is the solution for this issue?

[Js-Brecht]

One issue here is that node is not trusting self-signed certificates. A workaround, for now, is to disable TLS rejection:

NODE_TLS_REJECT_UNAUTHORIZED=0 gatsby develop --https --key /path/to/key -cert /path/to/cert

If you have a cert signed by a certificate authority, like the automated process uses, then a better, but more complicated, workaround is to add the certificate authority to the trusted roots chain:

NODE_EXTRA_CA_CERTS=[your CA certificate file path] gatsby develop --https

The automated process does write your CA cert to disk, in different locations depending on your OS. If you tell me your OS, I can tell you where it will put it, and then you can use that path.

The current package that manages certificates, devcert-san, is buggy, and the upgraded version has plenty of new bugs, too; and there doesn't seem to be anybody maintaining it. It also doesn't provide access to the certificate authority's cert. I've done some checking around for different packages that will fix this, but they all seem to have issues, and very little maintenance. I'm planning on building a small module that should fix the problem; just haven't had the time to come back to it yet.

[ghost]

Thanks @Js-Brecht for the detailed response. I am using MAC OS. For now, i have used the following command NODE_TLS_REJECT_UNAUTHORIZED=0 gatsby develop --https --port 3000
to move forward with this issue, but i see this warning.

(node:3596) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.

I am not sure whether this is a long term solution. So , i will go with your second suggestion to use the certificate. Can you please tell me the path to place the cert?

thanks
Aravind

[Js-Brecht]

The cert should be located at ~/.config/devcert/devcert-ca-root.crt.

[joebentaylor1995]

Having this issue too...

[JustFly1984]

We are having this issue too. Please update docs and default build script

[JustFly1984]

So issue today became critical. Chrome stopped allowing NODE_TLS_REJECT_UNAUTHORIZED=0 for me.

We have found solution:

navigate to chrome://flags in chrome, and switch Allow invalid certificates for resources loaded from localhost. to Enable

[Js-Brecht]

I just started working on a fix yesterday that should resolve this and a couple other open tickets.

For now, what OS are you using, @JustFly1984?

[JustFly1984]

@Js-Brecht MacOS Catalina

[Js-Brecht]

Since it is giving you the error "ERR_CERT_REVOKED", I'd like to see if creating a new certificate will fix the issue.

Remove the certificate devcert from your keychain, and then delete the directory ~/.config/devcert/. Then run gatsby develop --https to recreate the certificates; it should automatically add the root CA as a trusted root certificate to your keychain. It will also likely fail to start the server because node itself doesn't trust the certificate.

After the certificates have been installed and trusted, you should be able to run this to get it to work correctly:

NODE_EXTRA_CA_CERTS=~/.config/devcert/devcert-ca-root.crt gatsby develop --https

Alternatively, using openssl, you can create your own certificate authority (add it to your keychain, trusted for SSL), and your own server cert/key signed by that CA. Then use this:

NODE_EXTRA_CA_CERTS=/path/to/ca.crt gatsby develop --https --key /path/to/cert.key --cert /path/to/cert.crt

Please let me know if that works out. I know it works on Windows & Linux (except chrome on Linux uses its own nssdb for certificates, so using the system store doesn't work for it), but I'm not sure about MacOS, as I haven't tried it yet.

[in-in]

except chrome on Linux uses its own nssdb for certificates, so using the system store doesn't work for it

I created a certificate using mkcert and it works on my system (Linux Mint) with Chromium and Firefox

develop": "NODE_TLS_REJECT_UNAUTHORIZED=0 gatsby develop --https --key-file $KEY_FILE_LOCALHOST --cert-file $CERT_FILE_LOCALHOST ",

[Js-Brecht]

If you want full trust, so you don't have to deal with errors/warnings in your browser or in node, try this:

The following works flawlessly for me using self signed certificates.

• In my site directory, I've created a folder called ".certs".
• When using openssl, you can write this configuration to file. I'll call it ".certs/selfsigned.cnf":

[req]
req_extensions = v3_req
distinguished_name = distinguished_name
prompt = no

[distinguished_name]
commonName = localhost

[v3_req]
keyUsage = digitalSignature, keyEncipherment, nonRepudiation
subjectAltName = @alt_names

[alt_names]
DNS.1 = localhost
IP.1 = 127.0.0.1

• Then run openssl like this to create a self-signed certificate

openssl req -x509 -newkey rsa:2048 -nodes -keyout ./.certs/key.pem -days 365 -out ./.certs/cert.pem -config ./.certs/selfsigned.cnf -extensions "v3_req"

• Add it to your keychain with SSL trust.
• Then run one of the following
  • I indicated invalid command line arguments in my last post. You can use this, or you can use the same arguments as @in-in.
  • Shouldn't need to use NODE_TLS_REJECT_UNAUTHORIZED running it this way.

# If you run it this way, then you only need to do the export one time in your current session.
# After that, you just use the "gatsby develop" line.
export NODE_EXTRA_CA_CERTS=$PWD/.certs/cert.pem
gatsby develop --https -c ./.certs/cert.pem -k ./.certs/key.pem

# Or, in one line
NODE_EXTRA_CA_CERTS=$PWD/.certs/cert.pem gatsby develop --https -c ./.certs/cert.pem -k ./.certs/key.pem

I get no errors in Node, and the certificate is trusted in Chrome, so no error there either. For Chrome to trust it, the certificate you created has to be in the trusted root certificates store, whichever one it uses in your OS.

A couple notes

Browsers don't use the commonName to verify certificates anymore, so you have to include the commonName in the subjectAltName as well.

Using self signed certificates in Firefox require the IP address in the subjectAltName, too, otherwise they won't be trusted (took a bit of trial & error to figure that one). Firefox also uses its own trust store, so you have to go into its settings and add the cert into its trusted root certificates.

---

I'd like to hear from people if this doesn't work as expected. This is going to be similar to what the fix does automatically, just all within node.

[Js-Brecht]

I created a certificate using mkcert and it works on my system (Linux Mint) with Chromium and Firefox

@in-in I've never used mkcert before, but it looks pretty cool; looks like its pretty reliable in setting the trust for its CA for you. I can only speak to how I got the trust working correctly in Ubuntu (which shouldn't differ that much, in this regard, from Mint). It took adding the certificate to ~/.pki/nssdb, which I guess is not really specifically Chrome's nssdb (more like the current user's trust store). The point is that it wasn't using the system trust store.

It looks like mkcert stores its CA at $HOME/Library/Application Support/mkcert on MacOS, so you could use the same gatsby develop command, but point NODE_EXTRA_CA_CERTS at mkcert's CA. Assuming node can read the file, that should work without having to allow unauthorized certificates.

[Js-Brecht]

Here's something right out of their docs, which I assume should work on linux and mac:

export NODE_EXTRA_CA_CERTS="$(mkcert -CAROOT)/rootCA.pem"

[Js-Brecht]

I have submitted a PR that should fix the problem, but it's dependent on an upstream PR. Check out #18703. Could be a little while before it gets merged, though, depending on how long we wait for devcert to merge the PR there

[joeldbirch]

I wish I understood enough of this thread to get any of the workarounds working 😔

[Js-Brecht]

#18703 is really, really close to being merged. Just waiting on one more review