Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Use configurable !benchmark PR comment #1114

Merged
merged 1 commit into from
Feb 10, 2024

Conversation

samuelburnham
Copy link
Member

Configurable !benchmark PR comment

Uses the reusable workflow from argumentcomputer/ci-workflows#38. This replaces the existing !gpu-benchmark command with
!benchmark, as shown in the following examples:

!benchmark

This will run the default benchmark settings, specified in the workflow file of this PR.
Alternatively:

!benchmark --bench fibonacci --bench end2end --features cuda
LURK_RC=100,600,900
LURK_PERF="fully-sequential"

This will run with each of the input benchmarks, features, and env vars.

Important

  • If you want to benchmark with GPU acceleration enabled, you must specify --features cuda in the comment body (see above example). Otherwise, the benchmarks will still run on the same GPU-equipped benchmark machine, but will run using the CPU only.

Details

  • The previous !gpu-benchmark syntax will no longer work. The command must start with !benchmark.
  • Each benchmark specified with --bench will spawn a new CI job, which means they will each output a separate comment.

The default env vars specified in this PR are

LURK_BENCH_OUTPUT=pr-comment
LURK_PERF=100,600

If identically named env vars are specified in the !benchmark comment, they will overwrite the above defaults.

Copy link
Member

@huitseeker huitseeker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so much work went into this, thanks a lot! 🙏

@samuelburnham samuelburnham added this pull request to the merge queue Feb 10, 2024
Merged via the queue into argumentcomputer:main with commit 5796bb9 Feb 10, 2024
11 checks passed
@samuelburnham samuelburnham deleted the pr-bench-config branch February 10, 2024 23:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants