Updated build output (#155)

* WIPWIP * WIP * WIP * WIP * WIP * Move build_output to external module * Fix command formatting * Changelog and integration tests * Reduce boilerplate with Section::say_with_details * Update build output internals - Add module docs - Rename `header` to `buildpack_name` - Rename internal lookatme to look_at_me - Move `section::Section` and `section::RunCommand` to top level structs - Move internal timing related structs to `time` module * Add ability to get command prefix for BYO commands * Change `Quiet` to `Silent` Originally I thought "silent" and "stream" might be too close (both start with an "s") but mean basically opposite things. However, "quiet" implies there's some noise, versus "silent" indicates there's nothing to the output. * Fix no output on installing bundler I was not outputting anything when installing bundler by mistake (part of the reason for wanting to change "quiet" to "silent"). Before this commit: ``` # Heroku Ruby Buildpack - Ruby version `3.1.3` from `default` - Installing [------] (5.577s) - Bundler version `2.4.5` from `default` - Bundle install - Running `BUNDLE_BIN="/layers/heroku_ruby/gems/bin" BUNDLE_CLEAN="1" BUNDLE_DEPLOYMENT="1" BUNDLE_GEMFILE="/workspace/Gemfile" BUNDLE_PATH="/layers/heroku_ruby/gems" BUNDLE_WITHOUT="development:test" bundle install` Fetching gem metadata from https://rubygems.org/.. Fetching rake 13.0.6 Installing rake 13.0.6 Using bundler 2.4.5 Fetching rack 2.2.3 Fetching webrick 1.7.0 Installing webrick 1.7.0 Installing rack 2.2.3 Bundle complete! 3 Gemfile dependencies, 4 gems now installed. Gems in the groups 'development' and 'test' were not installed. Bundled gems are installed into `/layers/heroku_ruby/gems` - Done (6.637s) - Setting default processes(es) - Detecting gems - Running `bundle list` [-] (0.243s) - Detected rack app (`rack` gem and `config.ru` at root of application) - Rake assets install - Cannot run rake tasks (no Rakefile) ! Help: Add `Rakefile` to your project to enable - Done (finished in 13.572s) ``` After this commit: ``` # Heroku Ruby Buildpack - Ruby version `3.1.3` from `default` - Installing [----] (3.852s) - Bundler version `2.4.5` from `default` - Running `gem install bundler --version 2.4.5` [--] (1.081s) - Bundle install - Running `BUNDLE_BIN="/layers/heroku_ruby/gems/bin" BUNDLE_CLEAN="1" BUNDLE_DEPLOYMENT="1" BUNDLE_GEMFILE="/workspace/Gemfile" BUNDLE_PATH="/layers/heroku_ruby/gems" BUNDLE_WITHOUT="development:test" bundle install` Fetching gem metadata from https://rubygems.org/.. Fetching rake 13.0.6 Installing rake 13.0.6 Fetching rack 2.2.3 Fetching webrick 1.7.0 Using bundler 2.4.5 Installing webrick 1.7.0 Installing rack 2.2.3 Bundle complete! 3 Gemfile dependencies, 4 gems now installed. Gems in the groups 'development' and 'test' were not installed. Bundled gems are installed into `/layers/heroku_ruby/gems` - Done (3.929s) - Setting default processes(es) - Detecting gems - Running `bundle list` [-] (0.168s) - Detected rack app (`rack` gem and `config.ru` at root of application) - Rake assets install - Cannot run rake tasks (no `Rakefile`) ! Help: Add `Rakefile` to your project to enable - Done (finished in 9.093s) ``` I wanted to show that we're calling `--version` but not the other args as they're just noise. To do this I split up adding args into a "visible" and "invisible" section. The code is commented accordingly. * Force font weight for "normal" colors #155 (comment) * Use dots for installing marker: ``` # Heroku Ruby Buildpack - Ruby version `3.1.3` from `default` - Installing ........... (8.230s) - Bundler version `2.4.5` from `default` - Running `gem install bundler --version 2.4.5` .... (1.333s) - Bundle install - Running `BUNDLE_BIN="/layers/heroku_ruby/gems/bin" BUNDLE_CLEAN="1" BUNDLE_DEPLOYMENT="1" BUNDLE_GEMFILE="/workspace/Gemfile" BUNDLE_PATH="/layers/heroku_ruby/gems" BUNDLE_WITHOUT="development:test" bundle install` Fetching gem metadata from https://rubygems.org/.. Fetching rake 13.0.6 Installing rake 13.0.6 Using bundler 2.4.5 Fetching rack 2.2.3 Fetching webrick 1.7.0 Installing webrick 1.7.0 Installing rack 2.2.3 Bundle complete! 3 Gemfile dependencies, 4 gems now installed. Gems in the groups 'development' and 'test' were not installed. Bundled gems are installed into `/layers/heroku_ruby/gems` - Done (5.845s) - Setting default processes(es) - Detecting gems - Running `bundle list` ... (0.240s) - Detected rack app (`rack` gem and `config.ru` at root of application) - Rake assets install - Cannot run rake tasks (no `Rakefile`) ! Help: Add `Rakefile` to your project to enable - Done (finished in 15.749s) ``` * Get rid of NOCOLOR_TMP hack #155 (comment) It also turns out we're not even using it, but might in the future. * Adjust comment to match updated progress style * Output getting started guide to GH actions on push * Don't emit builder pull in output * Fix spelling
heroku · Jul 17, 2023 · be07aea · be07aea
1 parent 974f624
commit be07aea
Show file tree

Hide file tree

Showing 26 changed files with 1,118 additions and 482 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -58,3 +58,32 @@ jobs:
  - name: Run integration tests
  # Runs only tests annotated with the `ignore` attribute (which in this repo, are the integration tests).
  run: cargo test --locked -- --ignored
+
+ pack-getting-started-output:
+ runs-on: ubuntu-22.04
+ steps:
+ - name: Checkout
+ uses: actions/checkout@v3
+ - name: Install musl-tools
+ run: sudo apt-get install musl-tools --no-install-recommends
+ - name: Update Rust toolchain
+ run: rustup update
+ - name: Install Rust linux-musl target
+ run: rustup target add x86_64-unknown-linux-musl
+ - name: Rust Cache
+ uses: Swatinem/rust-cache@v2.5.1
+ - name: Install Pack CLI
+ uses: buildpacks/github-actions/setup-pack@v5.2.0
+ - name: Pull builder image
+ run: |
+ docker pull "heroku/builder:22"
+ docker pull "heroku/heroku:22-cnb"
+ - name: Clone ruby getting started guide
+ run: mkdir tmp; git clone https://github.com/heroku/ruby-getting-started tmp/ruby-getting-started
+ - name: Install libcnb-cargo for `cargo libcnb package` command
+ run: cargo install libcnb-cargo
+ - name: Compile ruby buildpack
+ run: cargo libcnb package
+ - name: Getting started guide output
+ run: | # Use `script -e -c` to pretend to be a TTY for pack terminal color support
+ script -e -c "pack build my-image --builder heroku/builder:22 --buildpack heroku/nodejs-engine --buildpack target/buildpack/debug/heroku_ruby --path tmp/ruby-getting-started --pull-policy never"
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/README.md b/README.md
@@ -233,7 +233,7 @@ Make sure it doesn't say `/usr/bin/ruby` or another system ruby location
 As a oneliner:
 
 ```
-$(set -pipefail; cd buildpacks/ruby && cargo libcnb package; cd -) &&
+cargo libcnb package &&
 docker rmi my-image --force &&
 pack build my-image --buildpack target/buildpack/debug/heroku_ruby --path buildpacks/ruby/tests/fixtures/default_ruby --verbose &&
 docker run -it --rm --entrypoint='/cnb/lifecycle/launcher' my-image 'which bundle'

diff --git a/buildpacks/ruby/CHANGELOG.md b/buildpacks/ruby/CHANGELOG.md
@@ -2,7 +2,9 @@
 
 ## [Unreleased]
 
-- Commons: EnvCommand removed, replaced with fun_run (https://github.com/heroku/buildpacks-ruby/pull/139)
+- Commons: Introduce `build_output` module (https://github.com/heroku/buildpacks-ruby/pull/155)
+- Commons: Remove `gem_list`, `rake_status`, `rake_task_detect` modules (https://github.com/heroku/buildpacks-ruby/pull/155)
+- Commons: `EnvCommand` removed, replaced with `fun_run` (https://github.com/heroku/buildpacks-ruby/pull/139)
 
 ## [2.0.0] 2023/31/01
 

diff --git a/buildpacks/ruby/Cargo.toml b/buildpacks/ruby/Cargo.toml
@@ -12,7 +12,6 @@ flate2 = "1"
 fs-err = "2"
 indoc = "2"
 libcnb = "0.13"
-libherokubuildpack = "0.13"
 rand = "0.8"
 regex = "1"
 serde = "1"
@@ -21,6 +20,7 @@ tempfile = "3"
 thiserror = "1"
 ureq = "2"
 url = "2"
+glob = "0.3"
 
 [dev-dependencies]
 libcnb-test = "0.13"

diff --git a/commons/src/gem_list.rs → buildpacks/ruby/src/gem_list.rs b/commons/src/gem_list.rs → buildpacks/ruby/src/gem_list.rs
@@ -1,5 +1,6 @@
-use crate::fun_run::{self, CmdMapExt};
-use crate::gem_version::GemVersion;
+use crate::build_output::{RunCommand, Section};
+use commons::fun_run::{self, CmdMapExt};
+use commons::gem_version::GemVersion;
 use core::str::FromStr;
 use regex::Regex;
 use std::collections::HashMap;
@@ -63,32 +64,23 @@ impl GemList {
  /// Errors if the command `bundle list` is unsuccessful.
  pub fn from_bundle_list<T: IntoIterator<Item = (K, V)>, K: AsRef<OsStr>, V: AsRef<OsStr>>(
  envs: T,
+ build_output: &Section,
  ) -> Result<Self, ListError> {
  let output = Command::new("bundle")
  .arg("list")
  .env_clear()
  .envs(envs)
- .cmd_map(|cmd| {
- let name = fun_run::display(cmd);
- cmd.output()
- .map_err(|error| fun_run::on_system_error(name.clone(), error))
- .and_then(|output| fun_run::nonzero_captured(name.clone(), output))
- })
+ .cmd_map(|cmd| build_output.run(RunCommand::inline_progress(cmd)))
  .map_err(ListError::BundleListShellCommandError)?;
 
  let stdout = String::from_utf8_lossy(&output.stdout);
  GemList::from_str(&stdout)
  }
 
  #[must_use]
- pub fn has(&self, str: &str) -> bool {
+ pub(crate) fn has(&self, str: &str) -> bool {
  self.gems.get(&str.trim().to_lowercase()).is_some()
  }
-
- #[must_use]
- pub fn version_for(&self, str: &str) -> Option<&GemVersion> {
- self.gems.get(&str.trim().to_lowercase())
- }
 }
 
 impl FromStr for GemList {
@@ -155,12 +147,6 @@ Use `bundle info` to print more detailed information about a gem
  assert!(gem_list.has("railties"));
  assert!(!gem_list.has("foo"));
 
- assert_eq!(
- gem_list.version_for("railties").unwrap(),
- &GemVersion::from_str("6.1.4.1").unwrap()
- );
- assert_eq!(gem_list.version_for("foo"), None);
-
  assert_eq!(gem_list.gems.len(), 14);
  }
 }
diff --git a/buildpacks/ruby/src/layers/bundle_download_layer.rs b/buildpacks/ruby/src/layers/bundle_download_layer.rs
@@ -1,3 +1,4 @@
+use crate::build_output::{RunCommand, Section};
 use crate::RubyBuildpack;
 use crate::RubyBuildpackError;
 use commons::fun_run::{self, CmdMapExt};
@@ -7,8 +8,6 @@ use libcnb::data::layer_content_metadata::LayerTypes;
 use libcnb::layer::{ExistingLayerStrategy, Layer, LayerData, LayerResult, LayerResultBuilder};
 use libcnb::layer_env::{LayerEnv, ModificationBehavior, Scope};
 use libcnb::Env;
-use libherokubuildpack::command::CommandExt;
-use libherokubuildpack::log as user;
 use serde::{Deserialize, Serialize};
 use std::path::Path;
 use std::process::Command;
@@ -27,6 +26,7 @@ pub(crate) struct BundleDownloadLayerMetadata {
 pub(crate) struct BundleDownloadLayer {
  pub env: Env,
  pub version: ResolvedBundlerVersion,
+ pub build_output: Section,
 }
 
 impl Layer for BundleDownloadLayer {
@@ -45,38 +45,36 @@ impl Layer for BundleDownloadLayer {
  _context: &BuildContext<Self::Buildpack>,
  layer_path: &Path,
  ) -> Result<LayerResult<Self::Metadata>, RubyBuildpackError> {
- user::log_info(format!("Installing bundler {}", self.version));
-
  let bin_dir = layer_path.join("bin");
  let gem_path = layer_path;
 
  Command::new("gem")
  .args([
  "install",
  "bundler",
- "--force",
- "--no-document", // Don't install ri or rdoc which takes extra time
- "--env-shebang", // Start the `bundle` executable with `#! /usr/bin/env ruby`
- "--version", // Specify exact version to install
+ "--version", // Specify exact version to install
  &self.version.to_string(),
- "--install-dir", // Directory where bundler's contents will live
- &layer_path.to_string_lossy(),
- "--bindir", // Directory where `bundle` executable lives
- &bin_dir.to_string_lossy(),
  ])
  .env_clear()
  .envs(&self.env)
  .cmd_map(|cmd| {
+ // Format `gem install --version <version>` without other content for display
  let name = fun_run::display(cmd);
-
- user::log_info(format!("Running $ {name}"));
-
- cmd.output_and_write_streams(std::io::stdout(), std::io::stderr())
+ // Arguments we don't need in the output
+ cmd.args([
+ "--install-dir", // Directory where bundler's contents will live
+ &layer_path.to_string_lossy(),
+ "--bindir", // Directory where `bundle` executable lives
+ &bin_dir.to_string_lossy(),
+ "--force",
+ "--no-document", // Don't install ri or rdoc documentation, which takes extra time
+ "--env-shebang", // Start the `bundle` executable with `#! /usr/bin/env ruby`
+ ]);
+ self.build_output
+ .run(RunCommand::inline_progress(cmd).with_name(name))
  .map_err(|error| {
- fun_run::annotate_which_problem(error, cmd, self.env.get("PATH").cloned())
+ fun_run::map_which_problem(error, cmd, self.env.get("PATH").cloned())
  })
- .map_err(|error| fun_run::on_system_error(name.clone(), error))
- .and_then(|output| fun_run::nonzero_streamed(name.clone(), output))
  })
  .map_err(RubyBuildpackError::GemInstallBundlerCommandError)?;
 
@@ -113,14 +111,14 @@ impl Layer for BundleDownloadLayer {
  version: self.version.clone(),
  };
  match cache_state(old.clone(), now) {
- State::NothingChanged(version) => {
- user::log_info(format!("Using bundler {version} from cache"));
+ State::NothingChanged(_version) => {
+ self.build_output.say("Using cached version");
 
  Ok(ExistingLayerStrategy::Keep)
  }
- State::BundlerVersionChanged(old, now) => {
- user::log_info(format!("Bundler version changed from {old} to {now}"));
- user::log_info("Clearing bundler from cache");
+ State::BundlerVersionChanged(_old, _now) => {
+ self.build_output
+  .say_with_details("Clearing cache", "bundler version changed");
 
  Ok(ExistingLayerStrategy::Recreate)
  }