Skip to content

CI: Make cache keys depend on Cairo version #2056

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Apr 10, 2025
Merged

CI: Make cache keys depend on Cairo version #2056

merged 5 commits into from
Apr 10, 2025
+18 −26

Conversation

JulianGCalderon
Copy link
Contributor

@JulianGCalderon JulianGCalderon commented Apr 7, 2025 •
edited
Loading

CI: Make cache keys depend on Cairo version

The CI sometimes breaks when:

  • New Cairo programs are added
  • We change te Cairo compiler
  • We modify the Makefile

This PR changes the CI so that any changes to [Makefile, requirements.txt, *.cairo] invalidate the cache. This should fix the issue.

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 7, 2025 •
edited
Loading 

**Hyper Thereading Benchmark results**




hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     26.248 s ±  0.033 s    [User: 25.528 s, System: 0.719 s]
  Range (min … max):   26.225 s … 26.272 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     26.279 s ±  0.047 s    [User: 25.530 s, System: 0.747 s]
  Range (min … max):   26.245 s … 26.312 s    2 runs
 
Summary
  hyper_threading_main threads: 1 ran
    1.00 ± 0.00 times faster than hyper_threading_pr threads: 1




hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     14.595 s ±  0.040 s    [User: 25.583 s, System: 0.789 s]
  Range (min … max):   14.567 s … 14.623 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     14.599 s ±  0.035 s    [User: 25.678 s, System: 0.750 s]
  Range (min … max):   14.574 s … 14.623 s    2 runs
 
Summary
  hyper_threading_main threads: 2 ran
    1.00 ± 0.00 times faster than hyper_threading_pr threads: 2




hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     10.776 s ±  0.245 s    [User: 37.646 s, System: 0.923 s]
  Range (min … max):   10.604 s … 10.949 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     10.548 s ±  0.128 s    [User: 37.623 s, System: 0.931 s]
  Range (min … max):   10.458 s … 10.638 s    2 runs
 
Summary
  hyper_threading_pr threads: 4 ran
    1.02 ± 0.03 times faster than hyper_threading_main threads: 4




hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     10.380 s ±  0.065 s    [User: 38.060 s, System: 0.910 s]
  Range (min … max):   10.334 s … 10.426 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     10.216 s ±  0.004 s    [User: 37.920 s, System: 0.908 s]
  Range (min … max):   10.213 s … 10.219 s    2 runs
 
Summary
  hyper_threading_pr threads: 6 ran
    1.02 ± 0.01 times faster than hyper_threading_main threads: 6




hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     10.295 s ±  0.197 s    [User: 38.264 s, System: 0.952 s]
  Range (min … max):   10.156 s … 10.434 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     10.487 s ±  0.039 s    [User: 38.274 s, System: 0.903 s]
  Range (min … max):   10.459 s … 10.515 s    2 runs
 
Summary
  hyper_threading_main threads: 8 ran
    1.02 ± 0.02 times faster than hyper_threading_pr threads: 8




hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     10.332 s ±  0.009 s    [User: 38.503 s, System: 1.019 s]
  Range (min … max):   10.325 s … 10.338 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     10.316 s ±  0.314 s    [User: 38.509 s, System: 1.039 s]
  Range (min … max):   10.094 s … 10.537 s    2 runs
 
Summary
  hyper_threading_pr threads: 16 ran
    1.00 ± 0.03 times faster than hyper_threading_main threads: 16

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 7, 2025 •
edited
Loading

Benchmark Results for unmodified programs 🚀

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 2.155 ± 0.020 2.124 2.188 1.00
head big_factorial 2.174 ± 0.049 2.136 2.292 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 2.121 ± 0.032 2.083 2.178 1.00
head big_fibonacci 2.136 ± 0.052 2.095 2.270 1.01 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 7.991 ± 0.222 7.811 8.554 1.01 ± 0.03
head blake2s_integration_benchmark 7.916 ± 0.078 7.776 8.046 1.00
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 2.321 ± 0.041 2.275 2.397 1.00
head compare_arrays_200000 2.359 ± 0.086 2.243 2.519 1.02 ± 0.04
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.591 ± 0.031 1.547 1.632 1.02 ± 0.03
head dict_integration_benchmark 1.557 ± 0.028 1.507 1.603 1.00
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.286 ± 0.023 1.254 1.320 1.01 ± 0.02
head field_arithmetic_get_square_benchmark 1.276 ± 0.016 1.252 1.305 1.00
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 7.741 ± 0.154 7.594 8.130 1.02 ± 0.02
head integration_builtins 7.600 ± 0.028 7.554 7.633 1.00
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 8.039 ± 0.131 7.898 8.313 1.01 ± 0.02
head keccak_integration_benchmark 7.925 ± 0.084 7.855 8.143 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search 2.201 ± 0.020 2.180 2.236 1.00 ± 0.01
head linear_search 2.198 ± 0.020 2.176 2.247 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.564 ± 0.014 1.547 1.587 1.01 ± 0.01
head math_cmp_and_pow_integration_benchmark 1.542 ± 0.013 1.530 1.568 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.499 ± 0.008 1.489 1.512 1.01 ± 0.01
head math_integration_benchmark 1.481 ± 0.010 1.472 1.504 1.00
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.242 ± 0.017 1.225 1.274 1.01 ± 0.02
head memory_integration_benchmark 1.232 ± 0.009 1.220 1.244 1.00
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.600 ± 0.022 1.585 1.658 1.00 ± 0.02
head operations_with_data_structures_benchmarks 1.592 ± 0.011 1.578 1.616 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 539.1 ± 5.0 533.1 548.7 1.00
head pedersen 541.6 ± 2.6 539.3 548.4 1.00 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base poseidon_integration_benchmark 638.1 ± 7.9 629.2 656.4 1.00
head poseidon_integration_benchmark 639.4 ± 8.2 627.5 657.7 1.00 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.880 ± 0.014 1.866 1.912 1.00 ± 0.01
head secp_integration_benchmark 1.871 ± 0.014 1.854 1.903 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 667.9 ± 3.5 662.5 672.4 1.00
head set_integration_benchmark 668.9 ± 6.6 658.5 683.1 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 4.265 ± 0.049 4.190 4.344 1.00 ± 0.02
head uint256_integration_benchmark 4.250 ± 0.055 4.189 4.320 1.00

@codecov Codecov
Copy link

codecov bot commented Apr 7, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.57%. Comparing base (6b283d7) to head (60df9b9).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #2056   +/-   ##
=======================================
  Coverage   96.57%   96.57%           
=======================================
  Files         102      102           
  Lines       42854    42854           
=======================================
  Hits        41387    41387           
  Misses       1467     1467

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JulianGCalderon JulianGCalderon changed the base branch from starkware-development to main April 7, 2025 15:54
@JulianGCalderon JulianGCalderon changed the title Fix Cache for Starkware Development CI: Make cache keys depend on Cairo version Apr 7, 2025
@JulianGCalderon JulianGCalderon added the pipelines This PR/issue is exclusively about improving our CI label Apr 7, 2025
@JulianGCalderon JulianGCalderon marked this pull request as ready for review April 7, 2025 19:08
FrancoGiachetta
FrancoGiachetta requested changes Apr 7, 2025
View reviewed changes
gabrielbosio and others added 2 commits April 9, 2025 16:56
@gabrielbosio
Restrict caching Cairo programs
b666a23 
Revert how the Rust workflow uses the Cairo program paths to
calculate the cache keys.
This change would imply the branch only adds Makefile and
requirements.txt to the cache key computation.
@gabrielbosio
Merge branch 'main' into fix-cache
60df9b9
@gabrielbosio gabrielbosio added this pull request to the merge queue Apr 10, 2025
Merged via the queue into main with commit 5ef8405 Apr 10, 2025
93 checks passed
@gabrielbosio gabrielbosio deleted the fix-cache branch April 10, 2025 15:41
enitrat pushed a commit to kkrt-labs/cairo-vm that referenced this pull request Apr 14, 2025
@JulianGCalderon @gabrielbosio @enitrat
CI: Make cache keys depend on Cairo version (lambdaclass#2056)
4f03e9b 
* Add Makefile and requirements.txt to cache key

* Don't fetch cairo stwo exclusive programs

* Remove empty file

* Restrict caching Cairo programs

Revert how the Rust workflow uses the Cairo program paths to
calculate the cache keys.
This change would imply the branch only adds Makefile and
requirements.txt to the cache key computation.

---------

Co-authored-by: gabrielbosio <gabrielbosio95@gmail.com>
Co-authored-by: Gabriel Bosio <38794644+gabrielbosio@users.noreply.github.com>
enitrat pushed a commit to kkrt-labs/cairo-vm that referenced this pull request May 6, 2025
@JulianGCalderon @gabrielbosio @enitrat
CI: Make cache keys depend on Cairo version (lambdaclass#2056)
1d1ece9 
* Add Makefile and requirements.txt to cache key

* Don't fetch cairo stwo exclusive programs

* Remove empty file

* Restrict caching Cairo programs

Revert how the Rust workflow uses the Cairo program paths to
calculate the cache keys.
This change would imply the branch only adds Makefile and
requirements.txt to the cache key computation.

---------

Co-authored-by: gabrielbosio <gabrielbosio95@gmail.com>
Co-authored-by: Gabriel Bosio <38794644+gabrielbosio@users.noreply.github.com>
gabrielbosio added a commit that referenced this pull request May 21, 2025
@JulianGCalderon @gabrielbosio
CI: Make cache keys depend on Cairo version (#2056)
6c83fca 
* Add Makefile and requirements.txt to cache key

* Don't fetch cairo stwo exclusive programs

* Remove empty file

* Restrict caching Cairo programs

Revert how the Rust workflow uses the Cairo program paths to
calculate the cache keys.
This change would imply the branch only adds Makefile and
requirements.txt to the cache key computation.

---------

Co-authored-by: gabrielbosio <gabrielbosio95@gmail.com>
Co-authored-by: Gabriel Bosio <38794644+gabrielbosio@users.noreply.github.com>
gabrielbosio added a commit that referenced this pull request Jun 4, 2025
@gabrielbosio @FrancoGiachetta
@JulianGCalderon @enitrat @VolodymyrBg @GarmashAlex @crStiv @DiegoCivi
Sync with main (#2116)
7269a8f 
* Remove duplicated `get_val` (#2065)

* avoid installing gmp if it is already installed

* revert part of the last commit

* changelog

* remove unwanted file

* CI: Make cache keys depend on Cairo version (#2056)

* Add Makefile and requirements.txt to cache key

* Don't fetch cairo stwo exclusive programs

* Remove empty file

* Restrict caching Cairo programs

Revert how the Rust workflow uses the Cairo program paths to
calculate the cache keys.
This change would imply the branch only adds Makefile and
requirements.txt to the cache key computation.

---------

Co-authored-by: gabrielbosio <gabrielbosio95@gmail.com>
Co-authored-by: Gabriel Bosio <38794644+gabrielbosio@users.noreply.github.com>

* dev: make Memory::get `pub` (#2039)

* dev: make Memory::get `pub`

* edit changelog

* dev: expose get_maybe_relocatable instead

* add comment on borrowed variant

* add tests

* feat(breaking): add support for hint accessible scopes (#2042)

* feat: add support for hint accessible scopes

update changelog

* Update vm/src/hint_processor/hint_processor_definition.rs

Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>

---------

Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>

* feat: add support for alias identifiers destination in program serde (#2071)

* feat: add support for aliases destination in program serde

* add changelog

---------

Co-authored-by: Julián González Calderón <gonzalezcalderonjulian@gmail.com>

* fix: Add boundary tests for SecpSplitOutOfRange in secp_utils (#2062)

* fix: Add boundary tests for SecpSplitOutOfRange in secp_utils

* Update secp_utils.rs

* Update CHANGELOG.md

* Update secp_utils.rs

* remove optimizations

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* empty commit to fix CI

---------

Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>

* docs: add --run_from_cairo_pie documentation for Cairo 0 and Cairo 1 (#2077)

* docs: add --run_from_cairo_pie flag usage documentation for Cairo 0

* docs: add --run_from_cairo_pie flag usage documentation for Cairo 1

* Update README.md

Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>

* Update README.md

Co-authored-by: Gabriel Bosio <38794644+gabrielbosio@users.noreply.github.com>

* Update README.md

Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>

---------

Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>
Co-authored-by: Gabriel Bosio <38794644+gabrielbosio@users.noreply.github.com>

* Fix WRITE_DIVMOD_SEGMENT hint (#2078)

* Minor fixes - use CAIRO_PRIME

* Add error handling

* Update CHANGELOG.md

* Bump cairo-lang to 0.13.5 (#1959)

* Update cairo-lang versin

* Update changelog

* Add new hints

* Specify prime in new hints

* Only apply mod floor if prime is not CAIRO_PRIME

* Properly implement pack for a different prime

* Fix pack in compute_doubling_slope

* Add negative_points.cairo test

* Add try_get_point_from_x for negative points

* Add double_x test for negative points

* Allow to many arguments

* Add codecov to cairo-0-secp-hints feature

* Prepare for Release v3.0.0-rc.1 (#2092)

* Update version

* Update lock

* Update toolchain

* Update changelog

* Fix clippy

* Update toolchain in CI

* fix: correct cache keys for no-std test coverage in CI (#2093)

* Update rust.yml

* Update rust.yml

* Update rust toolchain to 1.87.0 (#2100)

* update rust toolchain to 1.87.0

* remove uneeded file

* update rust in workflows

* update changelog

* clippy

* clippy

* remove unneeded clippy allow

* fix Changelog link (#2104)

* Improve memory_segments coverage (#2110)

* Add tests for .gen_arg() and .write_arg()

* Add test for .is_valid_memory_value()

* Assert contents of memory segments

---------

Co-authored-by: Franco Giachetta <francogiachetta27@gmail.com>
Co-authored-by: Julian Gonzalez Calderon <gonzalezcalderonjulian@gmail.com>
Co-authored-by: Mathieu <60658558+enitrat@users.noreply.github.com>
Co-authored-by: VolodymyrBg <aqdrgg19@gmail.com>
Co-authored-by: GarmashAlex <garmasholeksii@gmail.com>
Co-authored-by: crStiv <cryptostiv7@gmail.com>
Co-authored-by: DiegoC <90105443+DiegoCivi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gabrielbosio gabrielbosio gabrielbosio approved these changes

@FrancoGiachetta FrancoGiachetta FrancoGiachetta approved these changes

Assignees
No one assigned
Labels
pipelines This PR/issue is exclusively about improving our CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@JulianGCalderon @gabrielbosio @FrancoGiachetta