From 7aabbacd2e8cc25acded2d98b00b6e0871704b7a Mon Sep 17 00:00:00 2001 From: Koichiro Iwao Date: Fri, 20 Mar 2020 18:13:04 +0900 Subject: [PATCH 1/7] mark mavlev as EoL as the last commit of mavlev is over 3 years ago Discussed at: https://github.com/rbenv/ruby-build/pull/1402#issuecomment-596162716 --- share/ruby-build/maglev-1.0.0 | 2 +- share/ruby-build/maglev-1.1.0-dev | 2 +- share/ruby-build/maglev-2.0.0-dev | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/share/ruby-build/maglev-1.0.0 b/share/ruby-build/maglev-1.0.0 index d6da116e1..5f3b6b2eb 100644 --- a/share/ruby-build/maglev-1.0.0 +++ b/share/ruby-build/maglev-1.0.0 @@ -1 +1 @@ -install_package "MagLev-1.0.0" "http://seaside.gemtalksystems.com/maglev/MagLev-1.0.0.tar.gz#73401e9e69a336c2ca8369cc72e0d7f3ed867283252c385aea12ef44648c39be" maglev +install_package "MagLev-1.0.0" "http://seaside.gemtalksystems.com/maglev/MagLev-1.0.0.tar.gz#73401e9e69a336c2ca8369cc72e0d7f3ed867283252c385aea12ef44648c39be" warn_eol maglev diff --git a/share/ruby-build/maglev-1.1.0-dev b/share/ruby-build/maglev-1.1.0-dev index dfee15843..bd796940a 100644 --- a/share/ruby-build/maglev-1.1.0-dev +++ b/share/ruby-build/maglev-1.1.0-dev @@ -1 +1 @@ -install_git "maglev-1.1.0-dev" "https://github.com/MagLev/maglev.git" "master" maglev +install_git "maglev-1.1.0-dev" "https://github.com/MagLev/maglev.git" "master" warn_eol maglev diff --git a/share/ruby-build/maglev-2.0.0-dev b/share/ruby-build/maglev-2.0.0-dev index 87a110341..614ba7a8b 100644 --- a/share/ruby-build/maglev-2.0.0-dev +++ b/share/ruby-build/maglev-2.0.0-dev @@ -1 +1 @@ -install_git "maglev-2.0.0-dev" "https://github.com/MagLev/maglev.git" "master" maglev +install_git "maglev-2.0.0-dev" "https://github.com/MagLev/maglev.git" "master" warn_eol maglev From b034b8cd370fb0638c65c27ddf0c1f03b171dc6c Mon Sep 17 00:00:00 2001 From: Mingxiang Xue Date: Sat, 18 Jul 2020 00:36:59 +0800 Subject: [PATCH 2/7] Add RUBY_BUILD_MIRROR_PACKAGE_URL to specify the complete mirror URL Signed-off-by: Mingxiang Xue --- README.md | 49 ++++++++++++++++++++++------------------ bin/ruby-build | 6 +++-- test/mirror.bats | 59 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index da8e4c234..c7ff73cdb 100644 --- a/README.md +++ b/README.md @@ -64,28 +64,29 @@ definitions. The build process may be configured through the following environment variables: -| Variable | Function | -| ------------------------ | ------------------------------------------------------------------------------------------------ | -| `TMPDIR` | Where temporary files are stored. | -| `RUBY_BUILD_BUILD_PATH` | Where sources are downloaded and built. (Default: a timestamped subdirectory of `TMPDIR`) | -| `RUBY_BUILD_CACHE_PATH` | Where to cache downloaded package files. (Default: `~/.rbenv/cache` if invoked as rbenv plugin) | -| `RUBY_BUILD_HTTP_CLIENT` | One of `aria2c`, `curl`, or `wget` to use for downloading. (Default: first one found in PATH) | -| `RUBY_BUILD_ARIA2_OPTS` | Additional options to pass to `aria2c` for downloading. | -| `RUBY_BUILD_CURL_OPTS` | Additional options to pass to `curl` for downloading. | -| `RUBY_BUILD_WGET_OPTS` | Additional options to pass to `wget` for downloading. | -| `RUBY_BUILD_MIRROR_URL` | Custom mirror URL root. | -| `RUBY_BUILD_SKIP_MIRROR` | Bypass the download mirror and fetch all package files from their original URLs. | -| `RUBY_BUILD_ROOT` | Custom build definition directory. (Default: `share/ruby-build`) | -| `RUBY_BUILD_DEFINITIONS` | Additional paths to search for build definitions. (Colon-separated list) | -| `CC` | Path to the C compiler. | -| `RUBY_CFLAGS` | Additional `CFLAGS` options (_e.g.,_ to override `-O3`). | -| `CONFIGURE_OPTS` | Additional `./configure` options. | -| `MAKE` | Custom `make` command (_e.g.,_ `gmake`). | -| `MAKE_OPTS` / `MAKEOPTS` | Additional `make` options. | -| `MAKE_INSTALL_OPTS` | Additional `make install` options. | -| `RUBY_CONFIGURE_OPTS` | Additional `./configure` options (applies only to Ruby source). | -| `RUBY_MAKE_OPTS` | Additional `make` options (applies only to Ruby source). | -| `RUBY_MAKE_INSTALL_OPTS` | Additional `make install` options (applies only to Ruby source). | +| Variable | Function | +| ------------------------------- | ------------------------------------------------------------------------------------------------ | +| `TMPDIR` | Where temporary files are stored. | +| `RUBY_BUILD_BUILD_PATH` | Where sources are downloaded and built. (Default: a timestamped subdirectory of `TMPDIR`) | +| `RUBY_BUILD_CACHE_PATH` | Where to cache downloaded package files. (Default: `~/.rbenv/cache` if invoked as rbenv plugin) | +| `RUBY_BUILD_HTTP_CLIENT` | One of `aria2c`, `curl`, or `wget` to use for downloading. (Default: first one found in PATH) | +| `RUBY_BUILD_ARIA2_OPTS` | Additional options to pass to `aria2c` for downloading. | +| `RUBY_BUILD_CURL_OPTS` | Additional options to pass to `curl` for downloading. | +| `RUBY_BUILD_WGET_OPTS` | Additional options to pass to `wget` for downloading. | +| `RUBY_BUILD_MIRROR_URL` | Custom mirror URL root. | +| `RUBY_BUILD_MIRROR_PACKAGE_URL` | Custom complete mirror URL (e.g. http://mirror.example.com/package-1.0.0.tar.gz). | +| `RUBY_BUILD_SKIP_MIRROR` | Bypass the download mirror and fetch all package files from their original URLs. | +| `RUBY_BUILD_ROOT` | Custom build definition directory. (Default: `share/ruby-build`) | +| `RUBY_BUILD_DEFINITIONS` | Additional paths to search for build definitions. (Colon-separated list) | +| `CC` | Path to the C compiler. | +| `RUBY_CFLAGS` | Additional `CFLAGS` options (_e.g.,_ to override `-O3`). | +| `CONFIGURE_OPTS` | Additional `./configure` options. | +| `MAKE` | Custom `make` command (_e.g.,_ `gmake`). | +| `MAKE_OPTS` / `MAKEOPTS` | Additional `make` options. | +| `MAKE_INSTALL_OPTS` | Additional `make install` options. | +| `RUBY_CONFIGURE_OPTS` | Additional `./configure` options (applies only to Ruby source). | +| `RUBY_MAKE_OPTS` | Additional `make` options (applies only to Ruby source). | +| `RUBY_MAKE_INSTALL_OPTS` | Additional `make install` options (applies only to Ruby source). | #### Applying Patches @@ -134,6 +135,10 @@ will fall back to downloading the package from the original location if: You may specify a custom mirror by setting `RUBY_BUILD_MIRROR_URL`. +If a mirror site doesn't conform to the above URL format, you can specify the +complete URL by setting `RUBY_BUILD_MIRROR_PACKAGE_URL`. It behaves the same as +`RUBY_BUILD_MIRROR_URL` except being a complete URL. + The default ruby-build download mirror is sponsored by [Basecamp](https://basecamp.com/). diff --git a/bin/ruby-build b/bin/ruby-build index 54be8ebb9..004054184 100755 --- a/bin/ruby-build +++ b/bin/ruby-build @@ -365,6 +365,8 @@ fetch_tarball() { if [[ -z "$RUBY_BUILD_DEFAULT_MIRROR" || $package_url != */cache.ruby-lang.org/* ]]; then mirror_url="${RUBY_BUILD_MIRROR_URL}/$checksum" fi + elif [ -n "$RUBY_BUILD_MIRROR_PACKAGE_URL" ]; then + mirror_url="$RUBY_BUILD_MIRROR_PACKAGE_URL" fi fi @@ -1377,7 +1379,7 @@ else unset RUBY_BUILD_CACHE_PATH fi -if [ -z "$RUBY_BUILD_MIRROR_URL" ]; then +if [ -z "$RUBY_BUILD_MIRROR_URL" -a -z "$RUBY_BUILD_MIRROR_PACKAGE_URL" ]; then RUBY_BUILD_MIRROR_URL="https://dqw8nmjcqpjn7.cloudfront.net" RUBY_BUILD_DEFAULT_MIRROR=1 else @@ -1386,7 +1388,7 @@ else fi if [ -n "$RUBY_BUILD_SKIP_MIRROR" ] || ! has_checksum_support compute_sha2; then - unset RUBY_BUILD_MIRROR_URL + unset RUBY_BUILD_MIRROR_URL RUBY_BUILD_MIRROR_PACKAGE_URL fi ARIA2_OPTS="${RUBY_BUILD_ARIA2_OPTS} ${IPV4+--disable-ipv6=true} ${IPV6+--disable-ipv6=false}" diff --git a/test/mirror.bats b/test/mirror.bats index 87b18fcd3..e1b8d2e10 100755 --- a/test/mirror.bats +++ b/test/mirror.bats @@ -126,3 +126,62 @@ DEF unstub curl unstub shasum } + + +@test "package is fetched from complete mirror URL" { + export RUBY_BUILD_MIRROR_URL= + export RUBY_BUILD_MIRROR_PACKAGE_URL=http://mirror.example.com/package-1.0.0.tar.gz + local checksum="ba988b1bb4250dee0b9dd3d4d722f9c64b2bacfc805d1b6eba7426bda72dd3c5" + + stub shasum true "echo $checksum" + stub curl "-*I* $RUBY_BUILD_MIRROR_PACKAGE_URL : true" \ + "-q -o * -*S* $RUBY_BUILD_MIRROR_PACKAGE_URL : cp $FIXTURE_ROOT/package-1.0.0.tar.gz \$3" + + install_fixture definitions/with-checksum + + assert_success + assert [ -x "${INSTALL_ROOT}/bin/package" ] + + unstub curl + unstub shasum +} + + +@test "package is fetched from original URL if complete mirror URL download fails" { + export RUBY_BUILD_MIRROR_URL= + export RUBY_BUILD_MIRROR_PACKAGE_URL=http://mirror.example.com/package-1.0.0.tar.gz + local checksum="ba988b1bb4250dee0b9dd3d4d722f9c64b2bacfc805d1b6eba7426bda72dd3c5" + + stub shasum true "echo $checksum" + stub curl "-*I* $RUBY_BUILD_MIRROR_PACKAGE_URL : false" \ + "-q -o * -*S* http://example.com/* : cp $FIXTURE_ROOT/\${5##*/} \$3" + + install_fixture definitions/with-checksum + + assert_success + assert [ -x "${INSTALL_ROOT}/bin/package" ] + + unstub curl + unstub shasum +} + + +@test "package is fetched from original URL if complete mirror URL download checksum is invalid" { + export RUBY_BUILD_MIRROR_URL= + export RUBY_BUILD_MIRROR_PACKAGE_URL=http://mirror.example.com/package-1.0.0.tar.gz + local checksum="ba988b1bb4250dee0b9dd3d4d722f9c64b2bacfc805d1b6eba7426bda72dd3c5" + + stub shasum true "echo invalid" "echo $checksum" + stub curl "-*I* $RUBY_BUILD_MIRROR_PACKAGE_URL : true" \ + "-q -o * -*S* $RUBY_BUILD_MIRROR_PACKAGE_URL : cp $FIXTURE_ROOT/package-1.0.0.tar.gz \$3" \ + "-q -o * -*S* http://example.com/* : cp $FIXTURE_ROOT/\${5##*/} \$3" + + install_fixture definitions/with-checksum + echo "$output" >&2 + + assert_success + assert [ -x "${INSTALL_ROOT}/bin/package" ] + + unstub curl + unstub shasum +} From a31896579eca380d387ef4547a13f765658f7836 Mon Sep 17 00:00:00 2001 From: Koichiro Iwao Date: Thu, 24 Sep 2020 11:18:18 +0900 Subject: [PATCH 3/7] promote 'rbenv install -L', an alias of '--list-all' As far as I search on Twitter, the short option is not well known. --- bin/rbenv-install | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/rbenv-install b/bin/rbenv-install index 9048f1814..a9e904fe5 100755 --- a/bin/rbenv-install +++ b/bin/rbenv-install @@ -83,7 +83,7 @@ for option in "${OPTIONS[@]}"; do { echo echo "Only latest stable releases for each Ruby implementation are shown." - echo "Use 'rbenv install --list-all' to show all local versions." + echo "Use 'rbenv install --list-all / -L' to show all local versions." } 1>&2 exit ;; From 7872d9f13d4088b5e96c7b4443add5d01e114948 Mon Sep 17 00:00:00 2001 From: Hiroshi SHIBATA Date: Tue, 8 Dec 2020 21:10:42 +0900 Subject: [PATCH 4/7] 3.0.0-preview2 added --- share/ruby-build/3.0.0-preview2 | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 share/ruby-build/3.0.0-preview2 diff --git a/share/ruby-build/3.0.0-preview2 b/share/ruby-build/3.0.0-preview2 new file mode 100644 index 000000000..3021f8ac7 --- /dev/null +++ b/share/ruby-build/3.0.0-preview2 @@ -0,0 +1,2 @@ +install_package "openssl-1.1.1h" "https://www.openssl.org/source/openssl-1.1.1h.tar.gz#5c9ca8774bd7b03e5784f26ae9e9e6d749c9da2438545077e6b3d755a06595d9" mac_openssl --if has_broken_mac_openssl +install_package "ruby-3.0.0-preview2" "https://cache.ruby-lang.org/pub/ruby/3.0/ruby-3.0.0-preview2.tar.gz#9de8661565c2b1007d91a580e9a7e02d23f1e8fc8df371feb15a2727aa05fd9a" ldflags_dirs enable_shared standard verify_openssl From 01d9e8a9b57f2e114f0fc7e8c59b36b85548cdd9 Mon Sep 17 00:00:00 2001 From: Hiroshi SHIBATA Date: Tue, 8 Dec 2020 21:48:05 +0900 Subject: [PATCH 5/7] Bump version to 20201208 --- bin/ruby-build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/ruby-build b/bin/ruby-build index e09ae99a7..1c4b54ce7 100755 --- a/bin/ruby-build +++ b/bin/ruby-build @@ -14,7 +14,7 @@ # --version Show version of ruby-build # -RUBY_BUILD_VERSION="20201118" +RUBY_BUILD_VERSION="20201208" OLDIFS="$IFS" From 6c28303d115f6eda63ec37c29754bc0c12fc8f84 Mon Sep 17 00:00:00 2001 From: Jason Karns Date: Sat, 1 Jun 2024 10:18:46 -0400 Subject: [PATCH 6/7] Test NODE_BUILD_MIRROR_PACKAGE_URL correctly It's unclear why ruby-build still triggers a HEAD request to mirrors but node-build doesn't. --- test/mirror.bats | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/test/mirror.bats b/test/mirror.bats index 836d39b0c..e8f34a975 100755 --- a/test/mirror.bats +++ b/test/mirror.bats @@ -4,6 +4,7 @@ load test_helper export NODE_BUILD_SKIP_MIRROR= export NODE_BUILD_CACHE_PATH= export NODE_BUILD_MIRROR_URL=http://mirror.example.com +# TODO move into tests so mirror_cmd doesn't need stubbed in _every_ test export NODE_BUILD_MIRROR_CMD=mirror_stub @@ -98,14 +99,15 @@ export NODE_BUILD_MIRROR_CMD=mirror_stub local checksum="ba988b1bb4250dee0b9dd3d4d722f9c64b2bacfc805d1b6eba7426bda72dd3c5" stub shasum true "echo $checksum" - stub curl "-*I* $NODE_BUILD_MIRROR_PACKAGE_URL : true" \ - "-q -o * -*S* $NODE_BUILD_MIRROR_PACKAGE_URL : cp $FIXTURE_ROOT/package-1.0.0.tar.gz \$3" + stub curl "-q -o * -*S* $NODE_BUILD_MIRROR_PACKAGE_URL : cp $FIXTURE_ROOT/package-1.0.0.tar.gz \$3" + stub mirror_stub ": echo $NODE_BUILD_MIRROR_PACKAGE_URL" install_fixture definitions/with-checksum assert_success assert [ -x "${INSTALL_ROOT}/bin/package" ] + unstub mirror_stub unstub curl unstub shasum } @@ -117,14 +119,16 @@ export NODE_BUILD_MIRROR_CMD=mirror_stub local checksum="ba988b1bb4250dee0b9dd3d4d722f9c64b2bacfc805d1b6eba7426bda72dd3c5" stub shasum true "echo $checksum" - stub curl "-*I* $NODE_BUILD_MIRROR_PACKAGE_URL : false" \ + stub curl "-q -o * -*S* $NODE_BUILD_MIRROR_PACKAGE_URL : false" \ "-q -o * -*S* http://example.com/* : cp $FIXTURE_ROOT/\${5##*/} \$3" + stub mirror_stub ": echo $NODE_BUILD_MIRROR_PACKAGE_URL" install_fixture definitions/with-checksum assert_success assert [ -x "${INSTALL_ROOT}/bin/package" ] + unstub mirror_stub unstub curl unstub shasum } @@ -136,9 +140,9 @@ export NODE_BUILD_MIRROR_CMD=mirror_stub local checksum="ba988b1bb4250dee0b9dd3d4d722f9c64b2bacfc805d1b6eba7426bda72dd3c5" stub shasum true "echo invalid" "echo $checksum" - stub curl "-*I* $NODE_BUILD_MIRROR_PACKAGE_URL : true" \ - "-q -o * -*S* $NODE_BUILD_MIRROR_PACKAGE_URL : cp $FIXTURE_ROOT/package-1.0.0.tar.gz \$3" \ + stub curl "-q -o * -*S* $NODE_BUILD_MIRROR_PACKAGE_URL : cp $FIXTURE_ROOT/package-1.0.0.tar.gz \$3" \ "-q -o * -*S* http://example.com/* : cp $FIXTURE_ROOT/\${5##*/} \$3" + stub mirror_stub ": echo $NODE_BUILD_MIRROR_PACKAGE_URL" install_fixture definitions/with-checksum echo "$output" >&2 @@ -146,6 +150,11 @@ export NODE_BUILD_MIRROR_CMD=mirror_stub assert_success assert [ -x "${INSTALL_ROOT}/bin/package" ] + unstub mirror_stub unstub curl unstub shasum } + +@test "package is fetched from mirror URL constructed from MIRROR_CMD" { + skip "TODO: test this scenario" +} From 9bf4613c41fc88b819448fe72cb78c278578bf1a Mon Sep 17 00:00:00 2001 From: Jason Karns Date: Sat, 1 Jun 2024 10:57:15 -0400 Subject: [PATCH 7/7] Document mirror url construction in readme --- README.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index b68dfdcff..e6168c50e 100644 --- a/README.md +++ b/README.md @@ -170,21 +170,17 @@ definition. All definitions bundled with node-build include checksums. #### Package Mirrors -By default, node-build downloads package files from the official URL specified in the definition file. +To speed up downloads, node-build can fetch package files from a mirror. +To benefit from this, the packages must specify their checksum: ```sh # example: install_package "node-v12.0.0" "https://nodejs.org/dist/v12.0.0/node-v12.0.0.tar.gz#" ``` -node-build will attempt to construct a mirror url by invoking `NODE_BUILD_MIRROR_CMD` with two arguments: `package_url` and `checksum`. -The provided command should print the desired mirror's complete package URL. -If `NODE_BUILD_MIRROR_CMD` is unset, package mirror URL construction defaults to replacing `https://nodejs.org/dist` with `NODE_BUILD_MIRROR_URL`. - node-build will first try to fetch this package from `$NODE_BUILD_MIRROR_URL/` -(note: this is the complete URL), where `` is the checksum for the file. - -It will fall back to downloading the package from the original location if: +(note: this is the complete URL), where `` is the checksum for the file. It +will fall back to downloading the package from the original location if: - the package was not found on the mirror; - the mirror is down; - the download is corrupt, i.e. the file's checksum doesn't match; @@ -197,6 +193,11 @@ If a mirror site doesn't conform to the above URL format, you can specify the complete URL by setting `NODE_BUILD_MIRROR_PACKAGE_URL`. It behaves the same as `NODE_BUILD_MIRROR_URL` except being a complete URL. +For more control over the construction of the mirror url, you can specify a command +by setting `NODE_BUILD_MIRROR_CMD`. node-build will invoke `NODE_BUILD_MIRROR_CMD` +with two arguments: `package_url` and `checksum`. The provided command should +print the desired mirror's complete package URL to `STDOUT`. + #### Keeping the build directory after installation Both `node-build` and `nodenv install` accept the `-k` or `--keep` flag, which