Skip to content

Commit

Permalink
FIX(develop): Update workflow caching to fix faiiling github action u…
Browse files Browse the repository at this point in the history
…sed before (#778)

This pull request updates the caching mechanism to fix a failing GitHub
action that was used before. The changes include updating the
environment variables for the Google Cloud Platform (GCP)
authentication, downloading and uploading the cargo build cache, and
setting up the sccache action. These updates aim to improve the
performance and reliability of the GitHub action.

---------

Co-authored-by: goncer <gonza.cerro@gmail.com>
  • Loading branch information
tenequm and goncer authored Jul 8, 2024
1 parent fa6e061 commit 405ce87
Show file tree
Hide file tree
Showing 2 changed files with 521 additions and 137 deletions.
64 changes: 51 additions & 13 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,23 +12,41 @@ jobs:
runs-on: [compile-gke]
container:
image: mangatasolutions/node-builder:multi-nightly-2023-05-22
env:
JOB_CACHE_PREFIX: node-docs-cache-1
CACHE_ARCHIVE_NAME: cache_archive.tar.zst
steps:
- uses: actions/checkout@v4

- uses: google-github-actions/auth@v2
id: auth
with:
workload_identity_provider: '${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}'
service_account: '${{ secrets.GCP_SERVICE_ACCOUNT }}'
- name: Cache the Cargo dependencies
uses: mansagroup/gcs-cache-action@v1.0.3
with:
bucket: mangata-node-ci-cache
path: |
${{ github.workspace }}/target
/usr/local/cargo/git
/usr/local/cargo/registry
~/.cache/sccache
key: node-docs-cache-1-${{ hashFiles('Cargo.lock') }}
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
- uses: google-github-actions/setup-gcloud@v2

- name: Download cargo build cache
id: cache
run: |
set -x
CACHE_KEY="${{ env.JOB_CACHE_PREFIX }}-${{ hashFiles('**/Cargo.lock') }}"
ARCHIVE_NAME="${{ env.CACHE_ARCHIVE_NAME }}"
CACHE_FOUND=false
if gcloud storage cp "gs://mangata-node-ci-cache/$CACHE_KEY/$ARCHIVE_NAME" - | zstd -d | tar -xf - -C / ; then
CACHE_FOUND=true
fi

echo "cache_found=$CACHE_FOUND" >> $GITHUB_OUTPUT
echo "cache_key=$CACHE_KEY" >> $GITHUB_OUTPUT

- uses: mozilla-actions/sccache-action@v0.0.5
- run: |
echo "SCCACHE_GCS_BUCKET=mangata-node-ci-cache" >> $GITHUB_ENV
echo "SCCACHE_GCS_RW_MODE=READ_WRITE" >> $GITHUB_ENV
echo "SCCACHE_GCS_KEY_PREFIX=${{ env.JOB_CACHE_PREFIX }}" >> $GITHUB_ENV
echo "RUSTC_WRAPPER=sccache" >> $GITHUB_ENV
echo "CARGO_INCREMENTAL=0" >> $GITHUB_ENV
- name: Build docs
run: cargo doc
Expand All @@ -40,4 +58,24 @@ jobs:
destination: mangata-docs-node
parent: false
concurrency: 50
process_gcloudignore: false
process_gcloudignore: false

- name: Upload cargo build cache
shell: bash
run: |
set -x
CACHE_KEY="${{ steps.cache.outputs.cache_key }}"
ARCHIVE_NAME="${{ env.CACHE_ARCHIVE_NAME }}"
CACHE_PATHS=(
"./target"
"/usr/local/cargo/bin/"
"/usr/local/cargo/registry/index/"
"/usr/local/cargo/registry/cache/"
"/usr/local/cargo/git/db/"
)
SECONDS=0; tar -cf - "${CACHE_PATHS[@]}" | zstd -T0 -5 > "$ARCHIVE_NAME"
echo "Compression completed in $SECONDS seconds" && echo "Archive size: $(du -h "$ARCHIVE_NAME" | cut -f1)"

SECONDS=0; gcloud storage cp "$ARCHIVE_NAME" "gs://mangata-node-ci-cache/$CACHE_KEY/$ARCHIVE_NAME"
echo "Upload completed in $SECONDS seconds"
Loading

0 comments on commit 405ce87

Please sign in to comment.