Skip to content

Caching

Caching #3499

Workflow file for this run

name: Caching
# Docs on the workflow:
# 1. GitHub cache scoping goes:
# [main branch -> PR main branch -> PR. (essentially in a tree fashion)](https://web.archive.org/web/20211125171853/https://docs.github.com/en/actions/advanced-guides/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache).
# Building & keeping caches on `master` allows
# to share the main project state cache be shared to the whole tree.
# 2. GitHub has a [default 10G cache pool limit](https://web.archive.org/web/20211125171853/https://docs.github.com/en/actions/advanced-guides/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy) per repo.
# HLS is a big monorepo codebase, which means easy cache pool
# invalidation & exhaustion because of the pool limit.
# To keep caches useful - the main state of the main branch should remain
# & so keep caching in the repo well below the limit.
# that means preferring main branch to the PR caches
# (especially internal branch ones), since PRs from internal branches -
# count into the repo 10G pool, while that cache gets used only inside of the PR,
# while exhausting the pool would bork cache for the rest of the community.
# That is a short story why `dist-newstyle` (especially full) currently is not
# includded into `master` or PR caches.
defaults:
run:
shell: bash
# See: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency.
concurrency:
group: ${{ github.head_ref }}-${{ github.workflow }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- master
schedule:
# Refresh snapshot every (02+8*x):25 UTC
# When cache is present it is a light check workflow with early termination.
# When primary cache is not hit - runs the cache generation.
# Why: GitHub repo has 10G pool & on overflow GitHub removes caches in FIFO manner.
# When internal branche PRs save into the same pool -
# their cache is accessible only inside of the scope of the PR.
# If main cache is forced out - there are no cache shared between PRs,
# which implies all PRs would start to create & save their cache.
# Reinstitution of the main chache puts it back into FIFO
# & so it gets shared across all PRs.
- cron: "25 2/8 * * *"
# Not using a explicit target to build the top level haskell-language-server package
# which make build the rest of subpackages *libs* (but shake-bench)
env:
cabalBuild: "v2-build --keep-going"
jobs:
pre_job:
runs-on: ubuntu-latest
outputs:
ghcs: ${{ steps.ghcs.outputs.ghcs }}
should_skip: ${{ steps.skip_check.outputs.should_skip }}
steps:
# Need the repo checked out in order to read the file
- uses: actions/checkout@v3
- id: ghcs
run: echo "ghcs=$(cat ./.github/workflows/supported-ghc-versions.json)" >> $GITHUB_OUTPUT
- id: skip_check
uses: fkirc/skip-duplicate-actions@v5.3.1
with:
cancel_others: false
paths_ignore: '["**/docs/**"
, "**.md"
, "**/LICENSE"
, "**.nix"
, "flake.lock"
, "**/README.md"
, "FUNDING.yml"
, ".circleci/**"
, "**/stack*.yaml"
, ".gitlab-ci.yaml"
, ".gitlab/**"
]'
caching:
if: needs.pre_job.outputs.should_skip != 'true'
needs: pre_job
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
ghc: ${{ fromJSON(needs.pre_job.outputs.ghcs) }}
os:
- ubuntu-latest
- macOS-latest
- windows-latest
exclude:
# We disable this this combo in test.yml due to long path issues, so we also need to disable it here
- os: windows-latest
ghc: "9.2"
steps:
- uses: actions/checkout@v3
- uses: ./.github/actions/setup-build
with:
ghc: ${{ matrix.ghc }}
os: ${{ runner.os }}
# Download sources for feeding build sources cache
# Fetching from github cache is faster than doing it from hackage
# Sources does not change per ghc and ghc version son only doing it
# for one matrix job (it is arbitrary)
- if: steps.compiled-deps.outputs.cache-hit != 'true' && runner.os == 'Linux' && matrix.ghc == '9.2'
name: Download sources
run: |
cabal $cabalBuild --only-download --enable-benchmarks --enable-tests
# This build agenda is not to have successful code but produce cache as much as possible
- if: steps.compiled-deps.outputs.cache-hit != 'true'
name: Build haskell-language-server
run: |
# repeating builds to workaround segfaults in windows and ghc-8.8.4
cabal $cabalBuild || cabal $cabalBuild || cabal $cabalBuild
# We build ghcide with benchs and test enabled to include its dependencies in the cache
# (including shake-bench)
# Only for the same ghc and os used in the bench workflow, so we save cache space
- if: steps.compiled-deps.outputs.cache-hit != 'true' && runner.os == 'Linux' && matrix.ghc == '9.2'
name: Build ghcide benchmark
run: |
cabal $cabalBuild ghcide --enable-benchmarks --enable-tests