Skip to content

WIP: Multiturn Benchmarking Support #211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

WIP: Multiturn Benchmarking Support #211

wants to merge 3 commits into from

Conversation

sjmonson
Copy link
Collaborator

@sjmonson sjmonson commented Jun 27, 2025

PR Status

  • Rework scheduler to unify RPS and concurrent code pathways
  • Support scheduling request multi-turn conversions where each request feeds into the next
  • Support generating synthetic multi-turn conversations
  • Support parsing external multi-turn datasets
  • Add "users" variant of concurrent; where each concurrent thread handles a single multi-turn conversion at a time

Closes #199

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/guidellm/actions/runs/15934063548/artifacts/3421762392.
They will be retained for up to 30 days.

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/guidellm/actions/runs/15934227250/artifacts/3421819489.
They will be retained for up to 30 days.

@sjmonson sjmonson changed the title WIP: Multiturn benchmark support WIP: Multiturn Benchmarking Support Jun 27, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/guidellm/actions/runs/15935134867/artifacts/3422151072.
They will be retained for up to 30 days.

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/guidellm/actions/runs/15935412052/artifacts/3422245922.
They will be retained for up to 30 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Multi-turn benchmarking
2 participants