Skip to content

Commit f338f28

Browse files
committed
benchmarks: Simplify test jsonschema
PR vllm-project#13288 changed this schema from an object to an array. The real issue was that the original schema used features unsupported by xgrammar. However, by switching to an array, I saw the failure rate go up a lot in my benchmark runs. This is because as an unbounded array, we would often cut off generation in the middle of a json response, so it failed to be parsed. This change replaces it with a trivial jsonschema. This should be effective in focusing benchmarks more on the overhead vs the generation. The change also broke the `json-unique` option, since the code for inserting the unique property assumbed the top level was an object. This change makes `json-unique` work again. Signed-off-by: Russell Bryant <rbryant@redhat.com>
1 parent b0746fa commit f338f28

File tree

1 file changed

+2
-16
lines changed

1 file changed

+2
-16
lines changed
Lines changed: 2 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,11 @@
11
{
2-
"type": "array",
3-
"items": {
42
"type": "object",
53
"properties": {
64
"name": { "type": "string" },
7-
"race": { "type": "string" },
8-
"class": { "type": "string" },
9-
"level": { "type": "integer" },
10-
"background": { "type": "string" },
11-
"alignment": { "type": "string" },
12-
"backstory": { "type": "string" }
5+
"email": { "type": "string" }
136
},
147
"required": [
158
"name",
16-
"race",
17-
"class",
18-
"level",
19-
"background",
20-
"alignment",
21-
"backstory"
9+
"email"
2210
]
23-
}
2411
}
25-

0 commit comments

Comments
 (0)