Update to openssl1.1.1a #25381

sam-github · 2019-01-07T22:34:10Z

Most of this work comes from @shigeki, who got openssl 1.1.1 building and running across all Node's platforms.

Last time I tested this branch on ci it passed, and it passes locally.

I've done a fair amount of ABI testing, as well. It looks pretty compatible to me (as OpenSSL intended).

See #18770 (comment) (and around) for more information.

@nodejs/crypto , particularly @rvagg @shigeki

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

nodejs-github-bot · 2019-01-07T22:34:12Z

@sam-github build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/2168/pipeline

shigeki · 2019-01-08T06:25:41Z

@sam-github Thanks for the PR. I could not imagine we can fix an async cipher error and that's a great work. For I'm in a business trip during this week, I'm going to take a detail look at this in later.

sam-github · 2019-01-08T16:55:14Z

@shigeki Enjoy your trip, I'll make sure you have time to review.

ci: https://ci.nodejs.org/job/node-test-commit-linux-containered/9865/

Build and test against openssl 1.1.0 was the failure above. I'll look into it.

sam-github · 2019-01-15T16:55:50Z

The last build passed on everything except ARM.

re-ci: https://ci.nodejs.org/job/node-test-commit/25052/

This commit updates option ciphers from 'RC4' to 'missing' in test/parallel/test-tls-handshake-error.js. The motivation for this change is that this test is verifying that a 'no ciphers match' error be thrown, but 'RC4' might be among the ciphers supported by the OpenSSL version when dynamically linking. I ran into this specific issue when dynamically linking against OpenSSL 1.1.1 on RHEL8 using nodejs#25381.

sam-github · 2019-01-17T15:34:16Z

So, been more than a week, green in CI, one approval, I guess I could just land this... but it is a significant enough update I'd like a few more reviews! @nodejs/crypto @shigeki @rvagg @bnoordhuis @indutny

1.1.1 adds some crypto algs and minor features (like zero-length PKCS8 passphrases, @tniessen ), so I'm labelling semver-minor.

sam-github · 2019-01-17T15:37:56Z

Updated against node/master, squashed fixups, started a fresh CI.

ci: https://ci.nodejs.org/job/node-test-pull-request/20184/

This commit updates option ciphers from 'RC4' to 'no-such-cipher' in test/parallel/test-tls-handshake-error.js. The motivation for this change is that this test is verifying that a 'no ciphers match' error be thrown, but 'RC4' might be among the ciphers supported by the OpenSSL version when dynamically linking. I ran into this specific issue when dynamically linking against OpenSSL 1.1.1 on RHEL8 using #25381. PR-URL: #25534 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

shigeki

@sam-github Sorry for my late response. I checked your and my change commits and they are fine but some of my commits are missing commit descriptions.

I made a branch to fill the commit descriptions in https://github.com/shigeki/node/commits/PR25381.
Please rebase this to add the commit descriptions from 8e05aa0, fd5d8cb, 0dd90f3 and 01fcc96.
fd5d8cb is also fixed so as to fix within 50 chars for commit title.

Thanks for making this PR.

sam-github · 2019-01-21T17:49:10Z

@shigeki Thanks for reviewing, I'll go through and rebase and fixup the messages.

sam-github · 2019-01-21T22:03:22Z

Added commit bodies from @shigeki, rebased, re-ci.

ci: https://ci.nodejs.org/job/node-test-pull-request/20248/

sam-github · 2019-01-22T16:20:32Z

git node land claims commits were pushed after last CI, which I don't think is true, but re-running just in case.

ci: https://ci.nodejs.org/job/node-test-pull-request/20261/

This updates all sources in deps/openssl/openssl with openssl-1.1.1a.

Some of defines and cppflags in the build config of OpenSSL-1.1.1 were moved to new attributes. Gyp and gypi file generations are needed to be fixed to include them.

Because llvm on MacOS does not support AVX-512, asm files need to be limited to AVX-2 support even when they are generated on Linux. fake_gcc.pl returns the fake llvm banner version for MacOS as if the assembler supports upto AVX-2. For Windows, makefiles for nmake were updated in OpenSSL-1.1.1 and they are rewritten into GNU makefile format by hand.

This is a floating patch against OpenSSL-1.1.1 to generate asm files with Makefile rules.

OpenSSL-1.1.1 has new support of AVX-512 but AVX-2 asm files still need to be generated for the older assembler support to keep backward compatibilities.

This is a floating patch against OpenSSL-1.1.1 to generate asm files with Makefile rules. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

Some of defines and cppflags in the build config of OpenSSL-1.1.1 were moved to new attributes. Gyp and gypi file generations are needed to be fixed to include them. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

Because llvm on MacOS does not support AVX-512, asm files need to be limited to AVX-2 support even when they are generated on Linux. fake_gcc.pl returns the fake llvm banner version for MacOS as if the assembler supports upto AVX-2. For Windows, makefiles for nmake were updated in OpenSSL-1.1.1 and they are rewritten into GNU makefile format by hand. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

OpenSSL-1.1.1 has new support of AVX-512 but AVX-2 asm files still need to be generated for the older assembler support to keep backward compatibilities. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

Add new requirements of assembler version for AVX-512 support in OpenSSL-1.1.1. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

AIX has own assembler not GNU as that does not support --noexecstack. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

`SSL_CB_HANDSHAKE_START` and `SSL_CB_HANDSHAKE_DONE` are called sending HelloRequest in OpenSSL-1.1.1. We need to check whether this is in a renegotiation state or not. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

This gets better coverage of the codes, and is more explicit. It also works around ordering differences in the errors produced by openssl. The approach was tested with 1.1.0 and 1.1.1, as well as TLSv1.2 vs TLSv1.3. OpenSSL 1.1.0 is relevant when node is built against a shared openssl. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

`cd deps/openssl/config; make` updates all archs dependant files. Backport-PR-URL: #26270 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org>

There is a description of how to use s_client for testing of renegotiation limits in the `tls` module documentation. The information is somewhat out of scope, but it also may be somewhat problematic due to changes/peculiarities (bugs?) in recent s_client. Remove the text. Refs: #25381 (comment) PR-URL: #25678 Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

This is a floating patch against OpenSSL-1.1.1 to generate asm files with Makefile rules. PR-URL: nodejs#25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Backport-PR-URL: nodejs#25688

There is a description of how to use s_client for testing of renegotiation limits in the `tls` module documentation. The information is somewhat out of scope, but it also may be somewhat problematic due to changes/peculiarities (bugs?) in recent s_client. Remove the text. Refs: #25381 (comment) PR-URL: #25678 Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

This is a floating patch against OpenSSL-1.1.1 to generate asm files with Makefile rules. PR-URL: nodejs#25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Backport-PR-URL: nodejs#25688

This is a floating patch against OpenSSL-1.1.1 to generate asm files with Makefile rules. Backport-PR-URL: #27419 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Backport-PR-URL: #25688

There is a description of how to use s_client for testing of renegotiation limits in the `tls` module documentation. The information is somewhat out of scope, but it also may be somewhat problematic due to changes/peculiarities (bugs?) in recent s_client. Remove the text. Refs: #25381 (comment) PR-URL: #25678 Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

This is a floating patch against OpenSSL-1.1.1 to generate asm files with Makefile rules. Backport-PR-URL: #27419 PR-URL: #25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> Backport-PR-URL: #25688

Notable changes: - deps: - icu 63.1 bump (CLDR 34) (Steven R. Loomis) [#23715](#23715) - upgrade npm to 6.9.0 (Kat Marchán) [#26244](#26244) - upgrade openssl sources to 1.1.1a (Sam Roberts) [#25381](#25381) - upgrade to libuv 1.24.1 (cjihrig) [#25078](#25078) - events: add once method to use promises with EventEmitter (Matteo Collina) [#26078](#26078) - n-api: mark thread-safe function as stable (Gabriel Schulhof) [#25556](#25556) - repl: support top-level for-await-of (Shelley Vohr) [#23841](#23841) - zlib: - add brotli support (Anna Henningsen) [#24938](#24938) PR-URL: #27514

Ported from OpenSUSE:nodejs8-8.17.0-lp152.147.1:openssl_1_1_1.patch Original commit message: Backport OpenSSL 1.1.1 support, mostly be disabling TLS 1.3 Upstream commits: commit 8dd8033 Author: Shigeki Ohtsu <ohtsu@ohtsu.org> Date: Wed Sep 12 17:34:24 2018 +0900 tls: workaround handshakedone in renegotiation `SSL_CB_HANDSHAKE_START` and `SSL_CB_HANDSHAKE_DONE` are called sending HelloRequest in OpenSSL-1.1.1. We need to check whether this is in a renegotiation state or not. Backport-PR-URL: nodejs#26270 PR-URL: nodejs#25381 Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Shigeki Ohtsu <ohtsu@ohtsu.org> commit 161dca7 Author: Sam Roberts <vieuxtech@gmail.com> Date: Wed Nov 28 14:11:18 2018 -0800 tls: re-define max supported version as 1.2 Several secureProtocol strings allow any supported TLS version as the maximum, but our maximum supported protocol version is TLSv1.2 even if someone configures a build against an OpenSSL that supports TLSv1.3. Fixes: nodejs#24658 PR-URL: nodejs#25024 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Partial port, remain compatible with 1.0.2: commit 970ce14 Author: Shigeki Ohtsu <ohtsu@ohtsu.org> Date: Wed Mar 14 14:26:55 2018 +0900 crypto: remove deperecated methods of TLS version All version-specific methods were deprecated in OpenSSL 1.1.0 and min/max versions explicitly need to be set. This still keeps comptatible with JS and OpenSSL-1.0.2 APIs for now. crypto, constants: add constant of OpenSSL-1.1.0 Several constants for OpenSSL-1.1.0 engine were removed and renamed in OpenSSL-1.1.0. This added one renamed constant in order to have a compatible feature with that of OpenSSL-1.0.2. Other missed or new constants in OpenSSL-1.1.0 are not yet added. crypto,tls,constants: remove OpenSSL1.0.2 support This is semver-majar change so that we need not to have compatibilities with older versions. Fixes: nodejs#4270 PR-URL: nodejs#19794 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Signed-off-by: Su Baocheng <baocheng.su@siemens.com>

nodejs-github-bot added build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. openssl Issues and PRs related to the OpenSSL dependency. labels Jan 7, 2019

sam-github mentioned this pull request Jan 7, 2019

OpenSSL 1.1.1 Support TODO (pls help) #18770

Closed

danbev approved these changes Jan 8, 2019

View reviewed changes

sam-github force-pushed the update_openssl1.1.1a branch from 8f1b9bd to e70a5e1 Compare January 10, 2019 21:01

This was referenced Jan 16, 2019

CI failures: 20190110 - 20190116 nodejs/reliability#20

Closed

Investigate flaky known_issues/test-vm-timeout-escape-queuemicrotask #25529

Closed

danbev mentioned this pull request Jan 16, 2019

test: change ciphers from 'RC4' to 'no-such-cipher' #25534

Closed

2 tasks

sam-github force-pushed the update_openssl1.1.1a branch from d4bfbfb to fda3b55 Compare January 17, 2019 15:35

sam-github added the semver-minor PRs that contain new features and should be released in the next minor version. label Jan 17, 2019

officiallymarky mentioned this pull request Jan 19, 2019

ed25519 keys not working mscdex/ssh2#352

Closed

shigeki approved these changes Jan 21, 2019

View reviewed changes

sam-github force-pushed the update_openssl1.1.1a branch from fda3b55 to 572326a Compare January 21, 2019 22:02

sam-github and others added 5 commits January 22, 2019 11:36

deps: upgrade openssl sources to 1.1.1a

d0151fd

This updates all sources in deps/openssl/openssl with openssl-1.1.1a.

deps: fix gyp/gypi for openssl-1.1.1

00a24e1

Some of defines and cppflags in the build config of OpenSSL-1.1.1 were moved to new attributes. Gyp and gypi file generations are needed to be fixed to include them.

deps: add s390 asm rules for OpenSSL-1.1.1

4e75c01

This is a floating patch against OpenSSL-1.1.1 to generate asm files with Makefile rules.

deps: add only avx2 configs for OpenSSL-1.1.1

7c0b4df

OpenSSL-1.1.1 has new support of AVX-512 but AVX-2 asm files still need to be generated for the older assembler support to keep backward compatibilities.

BethGriggs mentioned this pull request May 1, 2019

v10.16.0 proposal #27514

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to openssl1.1.1a #25381

Update to openssl1.1.1a #25381

sam-github commented Jan 7, 2019 •

edited

Loading

nodejs-github-bot commented Jan 7, 2019

shigeki commented Jan 8, 2019

sam-github commented Jan 8, 2019

sam-github commented Jan 15, 2019

sam-github commented Jan 17, 2019

sam-github commented Jan 17, 2019

shigeki left a comment

sam-github commented Jan 21, 2019

sam-github commented Jan 21, 2019

sam-github commented Jan 22, 2019

Update to openssl1.1.1a #25381

Update to openssl1.1.1a #25381

Conversation

sam-github commented Jan 7, 2019 • edited Loading

Checklist

nodejs-github-bot commented Jan 7, 2019

shigeki commented Jan 8, 2019

sam-github commented Jan 8, 2019

sam-github commented Jan 15, 2019

sam-github commented Jan 17, 2019

sam-github commented Jan 17, 2019

shigeki left a comment

Choose a reason for hiding this comment

sam-github commented Jan 21, 2019

sam-github commented Jan 21, 2019

sam-github commented Jan 22, 2019

sam-github commented Jan 7, 2019 •

edited

Loading