-
Notifications
You must be signed in to change notification settings - Fork 30.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add automation for updating OpenSSL
dependency
#45605
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
name: OpenSSL update | ||
on: | ||
schedule: | ||
# Run once a week at 00:05 AM UTC on Sunday. | ||
- cron: 5 0 * * 0 | ||
|
||
workflow_dispatch: | ||
|
||
permissions: | ||
contents: read | ||
|
||
jobs: | ||
openssl-update: | ||
if: github.repository == 'nodejs/node' | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
with: | ||
persist-credentials: false | ||
- name: Check if update branch already exists | ||
run: | | ||
BRANCH_EXISTS=$(git ls-remote --heads origin actions/tools-update-openssl) | ||
echo "BRANCH_EXISTS=$BRANCH_EXISTS" >> $GITHUB_ENV | ||
- name: Check and download new OpenSSL version | ||
# Only run rest of the workflow if the update branch does not yet exist | ||
if: ${{ env.BRANCH_EXISTS == '' }} | ||
run: | | ||
NEW_VERSION=$(gh api repos/quictls/openssl/releases -q '.[].tag_name|select(contains("openssl-3"))|ltrimstr("openssl-")' | head -n1) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this also blocks the automatic backport to v14.x. We use the original repo on v14.x There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Currently there is no automation via Actions for dependency updates for any version of Node.js outside of the main branch. |
||
NEW_VERSION_NO_RELEASE_1=$(case $NEW_VERSION in *quic1) echo ${NEW_VERSION%1};; *) echo $NEW_VERSION;; esac) | ||
VERSION_H="./deps/openssl/config/archs/linux-x86_64/asm/include/openssl/opensslv.h" | ||
CURRENT_VERSION=$(grep "OPENSSL_FULL_VERSION_STR" $VERSION_H | sed -n "s/^.*VERSION_STR \"\(.*\)\"/\1/p") | ||
if [ "$NEW_VERSION_NO_RELEASE_1" != "$CURRENT_VERSION" ]; then | ||
echo "NEW_VERSION=$NEW_VERSION" >> $GITHUB_ENV | ||
RafaelGSS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
echo "HAS_UPDATE=true" >> $GITHUB_ENV | ||
./tools/dep_updaters/update-openssl.sh download "$NEW_VERSION" | ||
fi | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }} | ||
- name: Create PR with first commit | ||
if: env.HAS_UPDATE | ||
uses: gr2m/create-or-update-pull-request-action@v1 | ||
# Creates a PR with the new OpenSSL source code committed | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }} | ||
with: | ||
author: Node.js GitHub Bot <github-bot@iojs.org> | ||
body: This is an automated update of OpenSSL to ${{ env.NEW_VERSION }}. | ||
branch: actions/tools-update-openssl # Custom branch *just* for this Action. | ||
commit-message: 'deps: upgrade openssl sources to quictls/openssl-${{ env.NEW_VERSION }}' | ||
labels: dependencies | ||
title: 'deps: update OpenSSL to ${{ env.NEW_VERSION }}' | ||
path: deps/openssl | ||
- name: Regenerate platform specific files | ||
if: env.HAS_UPDATE | ||
run: | | ||
sudo apt install -y nasm libtext-template-perl | ||
./tools/dep_updaters/update-openssl.sh regenerate | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }} | ||
- name: Add second commit | ||
# Adds a second commit to the PR with the generated platform-dependent files | ||
if: env.HAS_UPDATE | ||
uses: gr2m/create-or-update-pull-request-action@v1 | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }} | ||
with: | ||
author: Node.js GitHub Bot <github-bot@iojs.org> | ||
branch: actions/tools-update-openssl # Custom branch *just* for this Action. | ||
commit-message: 'deps: update archs files for openssl-${{ env.NEW_VERSION }}' | ||
path: deps/openssl |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -61,6 +61,45 @@ been created with the changes), do the following: | |
4. Create a commit for the update and in the commit message include the | ||
important/relevant items from the changelog. | ||
|
||
## OpenSSL | ||
|
||
The `update-openssl.sh` script automates the steps described in | ||
[`maintaining-openssl.md`][]. The main difference is that the script downloads | ||
the release tarball from GitHub, instead of cloning the repo and using that as | ||
the source code. This is useful since the release tarball does not include | ||
development-specific files and directories (e.g the `.github` folder). | ||
|
||
The script has to be run in two steps. The first one (using the `download` | ||
sub-command) replaces the OpenSSL source code with the new version. The second | ||
one (using the `regenerate` sub-command) regenerates the platform-specific | ||
files. This makes it easier to create two separate git commits, making the git | ||
history more descriptive. | ||
|
||
For example, in order to update to version `3.0.7+quic1`, the following commands | ||
should be run: | ||
|
||
```bash | ||
./tools/dep_updaters/update-openssl.sh download 3.0.7+quic1 | ||
git add -A deps/openssl/openssl | ||
git commit -m "deps: upgrade openssl sources to quictls/openssl-3.0.7+quic1" | ||
|
||
./tools/dep_updaters/update-openssl.sh regenerate 3.0.7+quic1 | ||
git add -A deps/openssl/config/archs deps/openssl/openssl | ||
git commit -m "deps: update archs files for openssl" | ||
``` | ||
|
||
Once the script has run (either manually, or by CI in which case a PR will have | ||
been created with the changes), do the following: | ||
|
||
1. Check the `CHANGES.md` file in the [repo](https://github.com/quictls/openssl) | ||
for things that might require changes in Node.js. | ||
2. Check the diffs to ensure the changes are right. Even if there are no changes | ||
in the source, `buildinf.h` files will be updated because they have timestamp | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not related to this PR, but we should add a task somwhere to see if we can address this. Having it be more repeatable would be good. |
||
data in them. | ||
3. Check that Node.js compiles without errors and the tests pass. | ||
4. Create a commit for the update and in the commit message include the | ||
important/relevant items from the changelog. | ||
|
||
## postject | ||
|
||
The `update-postject.sh` script downloads postject from the [npm package](http://npmjs.com/package/postject) | ||
|
@@ -80,3 +119,5 @@ been created with the changes), do the following: | |
2. Check that Node.js compiles without errors and the tests pass. | ||
3. Create a commit for the update and in the commit message include the | ||
important/relevant items from the changelog. | ||
|
||
[`maintaining-openssl.md`]: https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-openssl.md |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
#!/bin/sh | ||
set -e | ||
# Shell script to update OpenSSL in the source tree to a specific version | ||
# Based on https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-openssl.md | ||
|
||
cleanup() { | ||
EXIT_CODE=$? | ||
[ -d "$WORKSPACE" ] && rm -rf "$WORKSPACE" | ||
exit $EXIT_CODE | ||
} | ||
|
||
download() { | ||
if [ -z "$1" ]; then | ||
echo "Error: please provide an OpenSSL version to update to" | ||
echo " e.g. ./$0 download 3.0.7+quic1" | ||
exit 1 | ||
fi | ||
|
||
OPENSSL_VERSION=$1 | ||
echo "Making temporary workspace..." | ||
WORKSPACE=$(mktemp -d 2> /dev/null || mktemp -d -t 'tmp') | ||
|
||
|
||
cd "$WORKSPACE" | ||
|
||
echo "Fetching OpenSSL source archive..." | ||
curl -sL "https://api.github.com/repos/quictls/openssl/tarball/openssl-$OPENSSL_VERSION" | tar xzf - | ||
mv quictls-openssl-* openssl | ||
|
||
echo "Replacing existing OpenSSL..." | ||
rm -rf "$DEPS_DIR/openssl/openssl" | ||
mv "$WORKSPACE/openssl" "$DEPS_DIR/openssl/" | ||
|
||
echo "All done!" | ||
echo "" | ||
echo "Please git add openssl, and commit the new version:" | ||
echo "" | ||
echo "$ git add -A deps/openssl/openssl" | ||
echo "$ git commit -m \"deps: upgrade openssl sources to quictls/openssl-$OPENSSL_VERSION\"" | ||
echo "" | ||
} | ||
|
||
regenerate() { | ||
command -v perl >/dev/null 2>&1 || { echo >&2 "Error: 'Perl' required but not installed."; exit 1; } | ||
command -v nasm >/dev/null 2>&1 || { echo >&2 "Error: 'nasm' required but not installed."; exit 1; } | ||
command -v as >/dev/null 2>&1 || { echo >&2 "Error: 'GNU as' required but not installed."; exit 1; } | ||
perl -e "use Text::Template">/dev/null 2>&1 || { echo >&2 "Error: 'Text::Template' Perl module required but not installed."; exit 1; } | ||
|
||
echo "Regenerating platform-dependent files..." | ||
|
||
make -C "$DEPS_DIR/openssl/config" clean | ||
# Needed for compatibility with nasm on 32-bit Windows | ||
# See https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-openssl.md#2-execute-make-in-depsopensslconfig-directory | ||
sed -i 's/#ifdef/%ifdef/g' "$DEPS_DIR/openssl/openssl/crypto/perlasm/x86asm.pl" | ||
sed -i 's/#endif/%endif/g' "$DEPS_DIR/openssl/openssl/crypto/perlasm/x86asm.pl" | ||
make -C "$DEPS_DIR/openssl/config" | ||
|
||
echo "All done!" | ||
echo "" | ||
echo "Please commit the regenerated files:" | ||
echo "" | ||
echo "$ git add -A deps/openssl/config/archs deps/openssl/openssl" | ||
echo "$ git commit -m \"deps: update archs files for openssl\"" | ||
echo "" | ||
} | ||
|
||
help() { | ||
echo "Shell script to update OpenSSL in the source tree to a specific version" | ||
echo "Sub-commands:" | ||
printf "%-23s %s\n" "help" "show help menu and commands" | ||
printf "%-23s %s\n" "download" "download and replace OpenSSL source code with new version" | ||
printf "%-23s %s\n" "regenerate" "regenerate platform-specific files" | ||
echo "" | ||
exit "${1:-0}" | ||
} | ||
|
||
main() { | ||
if [ ${#} -eq 0 ]; then | ||
help 0 | ||
fi | ||
|
||
trap cleanup INT TERM EXIT | ||
|
||
BASE_DIR=$(cd "$(dirname "$0")/../.." && pwd) | ||
DEPS_DIR="$BASE_DIR/deps" | ||
|
||
case ${1} in | ||
help | download | regenerate ) | ||
$1 "${2}" | ||
;; | ||
* ) | ||
echo "unknown command: $1" | ||
help 1 | ||
exit 1 | ||
;; | ||
esac | ||
} | ||
|
||
main "$@" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this block the automatic backport. We won't be able to update v14 and v16 at same time considering the race condition of the branch creation.
So, I'll include the labels
dont-land-on-vX
and after landing you will need to manually backport and change the branch name toactions/tools-update-openssl-vX
.Does that make sense to you?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the same logic used for the other deps. See this line from the current dependency update action:
node/.github/workflows/tools.yml
Line 153 in 9b1ed04
As far as I understand, this would be already an issue for
corepack
,eslint
, etc right? Since the branches created for them are calledtools-update-corepack
andtools-update-eslint
without mentioning the Node version.Do you know how those are handled currently?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we have different versions of eslint/corepack in each release line. Could you check if the bot has ever opened PR targeting other branches instead of
main
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The bot will not because the automatic scheduled runs only happen on the default branch:
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule