[Bugfix] Fix bug when dataset is json

[Chenyaaang]

In benchmark_serving_structured_output.py, when dataset is json, use the original loaded schema instead of creating json_schemas.

The sample command provided in benchmark_serving_structured_output.py doesn't work because it fails at sample_requests() for json dataset.
python benchmarks/benchmark_serving_structured_output.py
--backend
--model <your_model>
--dataset json
--structured-output-ratio 1.0
--structured-output-backend xgrammar
--request-rate 10
--num-prompts 1000

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[yaochengji]

To better understand the fix, could you please provide a test case that demonstrates the bug you've addressed?

[DarkLight1337]

I think it would be cleaner to just add another if-else branch.

elif args.dataset == "json":
    json_schemas = [schema]

That way we don't have to modify gen_prompt and get_schema at all

[Chenyaaang]

json_schemas

len(json_schemas) should be same as prompt size, so it means we need to copy schema num_prompts times (that's unncessary)

[Chenyaaang]

To better understand the fix, could you please provide a test case that demonstrates the bug you've addressed?

Added in the description.

[DarkLight1337]

json_schemas

len(json_schemas) should be same as prompt size, so it means we need to copy schema num_prompts times (that's unncessary)

How about json_schemas = [schema] * args.num_prompts?

Edit: actually it doesn't look like they need to be of the same length since we apply modulus operator before indexing the list of schemas.

[Chenyaaang]

json_schemas

len(json_schemas) should be same as prompt size, so it means we need to copy schema num_prompts times (that's unncessary)

How about json_schemas = [schema] * args.num_prompts?

Edit: actually it doesn't look like they need to be of the same length since we apply modulus operator before indexing the list of schemas.

Thanks, fixed. PTAL.

[DarkLight1337]

Thanks for bearing with me!