-
Notifications
You must be signed in to change notification settings - Fork 702
GSOC 2025 ‐ Benchmark Metrics
- https://tylercipriani.com/blog/2022/11/19/git-notes-gits-coolest-most-unloved-feature/
- https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests
Inspired by how the GHC project stores benchmark data and helps contributor track performance changes, we want to lay the foundations for a similar mechanism for the cabal project. While it's easy to think that most of the work is done by GHC when building a cabal project, there are numerous opportunities for user-visible performance regressions that are hard to catch in typical scenarios, but become apparent at scale.
Extraneous file system operations and parallelism of VCS retrieval are examples of things that are specific to cabal and impact its daily usage. Moreover, some subsystems like the solver and performance critical, and we ought to keep track of their evolution.
Have the CI run a benchmark job that stores results in Git Notes, and fails the job if the benchmark metrics collected during the run deviate from a window that will be determined.
Two mentors will be needed, in order to share the knowledge of this infrastructure:
- Hécate