Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Use configurable !benchmark PR comment #308

Merged
merged 1 commit into from
Feb 10, 2024

Conversation

samuelburnham
Copy link
Member

Configurable !benchmark PR comment

Uses the reusable workflow from argumentcomputer/ci-workflows#38. This replaces the existing !gpu-benchmark command with
!benchmark, as shown in the following examples:

!benchmark

This will run the default benchmark settings, specified in the workflow file of this PR.
Alternatively:

!benchmark --bench supernova-ci --bench pcs --features cuda
ARECIBO_BENCH_NUM_CONS=16384,1048576

This will run with each of the input benchmarks, features, and env vars.

Important

  • If you want to benchmark with GPU acceleration enabled, you must specify --features cuda in the comment body (see above example). Otherwise, the benchmarks will still run on the same GPU-equipped benchmark machine, but will run using the CPU only.

Details

  • The previous !gpu-benchmark syntax will no longer work. The command must start with !benchmark.
  • Each benchmark specified with --bench will spawn a new CI job, which means they will each output a separate comment.

The default env vars specified in this PR are

ARECIBO_BENCH_OUTPUT=pr-comment
ARECIBO_BENCH_NUM_CONS=16384,524288

If identically named env vars are specified in the !benchmark comment, they will overwrite the above defaults.

See also argumentcomputer/lurk-beta#1114

Copy link
Member

@huitseeker huitseeker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@samuelburnham samuelburnham added this pull request to the merge queue Feb 10, 2024
Merged via the queue into argumentcomputer:dev with commit 82b55c1 Feb 10, 2024
9 checks passed
@samuelburnham samuelburnham deleted the pr-bench-config branch February 10, 2024 14:56
huitseeker pushed a commit to huitseeker/arecibo that referenced this pull request Feb 17, 2024
…uter#308)

* move KZG engine to provider module; update paths everywhere

* add a sparse matrix entry only if the coefficient is non-zero; update digests

* move test code to place where it is used

* move asm to default and add a note

* update constants

* update link

* simplify test_pp_digest
huitseeker pushed a commit to huitseeker/arecibo that referenced this pull request Feb 17, 2024
…uter#308)

* move KZG engine to provider module; update paths everywhere

* add a sparse matrix entry only if the coefficient is non-zero; update digests

* move test code to place where it is used

* move asm to default and add a note

* update constants

* update link

* simplify test_pp_digest
huitseeker pushed a commit that referenced this pull request Feb 17, 2024
* move KZG engine to provider module; update paths everywhere

* add a sparse matrix entry only if the coefficient is non-zero; update digests

* move test code to place where it is used

* move asm to default and add a note

* update constants

* update link

* simplify test_pp_digest
huitseeker pushed a commit to huitseeker/arecibo that referenced this pull request Feb 19, 2024
…uter#308)

* move KZG engine to provider module; update paths everywhere

* add a sparse matrix entry only if the coefficient is non-zero; update digests

* move test code to place where it is used

* move asm to default and add a note

* update constants

* update link

* simplify test_pp_digest
github-merge-queue bot pushed a commit that referenced this pull request Feb 20, 2024
* update README; update version (#303)

* Miscellaneous improvements (details in commit messages) (#308)

* move KZG engine to provider module; update paths everywhere

* add a sparse matrix entry only if the coefficient is non-zero; update digests

* move test code to place where it is used

* move asm to default and add a note

* update constants

* update link

* simplify test_pp_digest

* chore: update expect tests

* Improvements to error handling and naming (#309)

* rename and introduce checks about length

* introduce a test about public IO

* chore: remove needless annotation

---------

Co-authored-by: Srinath Setty <srinath@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants