Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support collapsed sections in long build logs #1206

Open
erhhung opened this issue Aug 9, 2024 · 1 comment
Open

Support collapsed sections in long build logs #1206

erhhung opened this issue Aug 9, 2024 · 1 comment
Labels
kind/enhancement New feature or request status/triage

Comments

@erhhung
Copy link

erhhung commented Aug 9, 2024

Description

GitHub Actions supports collapsed sections using marker outputs ::group::<Section Title> and ::endgroup:: that, as far as I know, must begin at the start of a line.

However, docker buildx build (or just docker build) always prepends the step number and step elasped time on each line, such that running echo "::group::Start Verbose Install" doesn't produce the desired effect in the GHA Workflow UI.

I usually like to start Dockerfile RUN commands like in this example:

RUN <<'EOT'
set -e
echo "::group::Build Python 3.12"
( set -euxo pipefail

  cd /tmp
  dnf install -y  openssl-devel bzip2-devel xz-devel libffi-devel \
    libuuid-devel gdbm-devel readline-devel tk-devel sqlite-devel
  VER="3.12"
  FTP="https://www.python.org/ftp/python"
  # determine the latest patch version
  ver=$(curl -s $FTP/ | sed -En 's/^.+href="('${VER/./\\.}'\..+)\/".+$/\1/p' | sort -Vr | head -1)
  curl -sL $FTP/$ver/Python-$ver.tgz | tar -xz
  cd Python*
  # https://docs.python.org/3/using/configure.html
  ./configure -q \
    --prefix=/usr/local \
    --enable-optimizations \
    --with-lto=full \
    --with-computed-gotos
  make -sj$(nproc)
  # installs into (empty) dirs under /usr/local: /bin, /lib, /share/man/man1
  make -s altinstall
  alternatives --install /usr/local/bin/python3 python3 /usr/local/bin/python$VER 1
  alternatives --install /usr/local/bin/python  python  /usr/local/bin/python3    1
  alternatives --list
  hash -r
  python3 -VV
  python3 -m pip install -U --no-cache-dir --root-user-action=ignore pip
  # must copy all new symlinks in /etc/alternatives into the final image
  alternatives --install /usr/local/bin/pip3 pip3 /usr/local/bin/pip$VER 1
  alternatives --install /usr/local/bin/pip  pip  /usr/local/bin/pip3    1
  alternatives --list
  hash -r
  pip3 -V
)
echo "::endgroup::"
EOT

This particular example from a multi-stage build produces prodigious amounts of log output, and so I'd like to show each major step in its own collapsed section when reviewing the workflow run.

But, of course and unfortunately, the build log is generated like so:

#10 [builder 3/7] RUN <<'EOT' (echo "::group::Build Python 3.12"...)
#10 0.188 ::group::Build Python 3.12
#10 0.188 + cd /tmp
#10 0.188 + dnf install -y openssl-devel bzip2-devel xz-devel libffi-devel libuuid-devel gdbm-devel readline-devel tk-devel sqlite-devel
#10 0.405 Last metadata expiration check: 0:00:27 ago on Thu Aug  8 19:45:33 2024.
#10 0.424 Package xz-devel-5.2.5-9.amzn2023.0.2.x86_64 is already installed.
#10 0.464 Dependencies resolved.
#10 0.479 ================================================================================
#10 0.479  Package                Arch    Version                      Repository    Size
#10 0.479 ================================================================================
#10 0.479 Installing:
#10 0.479  bzip2-devel            x86_64  1.0.8-6.amzn2023.0.2         amazonlinux  214 k
#10 0.479  gdbm-devel             x86_64  1:1.19-2.amzn2023.0.2        amazonlinux   58 k
#10 0.479  libffi-devel           x86_64  3.4.4-1.amzn2023.0.1         amazonlinux   30 k
#10 0.479  libuuid-devel          x86_64  2.37.4-1.amzn2023.0.4        amazonlinux   23 k
#10 0.479  openssl-devel          x86_64  1:3.0.8-1.amzn2023.0.14      amazonlinux  3.0 M
#10 0.479  readline-devel         x86_64  8.1-2.amzn2023.0.2           amazonlinux  195 k
#10 0.479  sqlite-devel           x86_64  3.40.0-1.amzn2023.0.4        amazonlinux  143 k
#10 0.479  tk-devel               x86_64  1:8.6.10-6.amzn2023.0.2      amazonlinux  484 k
#10 0.479 Installing dependencies:

It would be great if this action intentionally strips out the step number and elasped time from lines that otherwise start immediately with the ::group:: and ::endgroup:: markers (the same marker string is displayed in both the first and second lines in the above text, but only the second line should have the annotations removed).

In essence, I'd want the effect of having the docker buildx build command be executed and filtered with sed, like this:

/usr/bin/docker buildx build --iidfile /home/runner/work/_temp/docker-actions-toolkit-FIW1PL/build-iidfile-13c464f45f.txt --lots-of-other-args --progress plain . 2>&1 | \
  sed -E 's/^(#[0-9]+ [0-9.]+ )(::(end)?group::.*)$/\2/' | \
  tee build.log
@erhhung erhhung added kind/enhancement New feature or request status/triage labels Aug 9, 2024
@crazy-max
Copy link
Member

I recall we discussed with @tonistiigi about supporting a buildkit progress for GHA. I think we could just use the rawjson progress to handle that directly in the action instead of creating a new progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or request status/triage
Projects
None yet
Development

No branches or pull requests

2 participants