[BUG] xgrammar does not follow the constraint

[merrymercy]

xgrammar does not follow the integer constraint and generate a floating number for an integer filed.

Schema

sglang/test/srt/test_json_constrained.py

Line 30 in 1f45146

"population": {"type": "integer"},

Error

======================================================================
FAIL: test_json_openai (__main__.TestJSONConstrained)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/sglang/test/srt/test_json_constrained.py", line 109, in test_json_openai
    assert isinstance(js_obj["population"], int), f"{js_obj=}"
AssertionError: js_obj={'name': 'Paris', 'population': 2.0}

Reproduce:

branch: https://github.com/sgl-project/sglang/tree/xgrammar-fail

python3 test/srt/test_json_constrained.py TestJSONConstrained.test_json_openai

[merrymercy]

cc @Ubospica @DarkSharpness

[roG0d]

Hi, I'm researching inference-serving frameworks that implement Grammar-Constrained Decoding (GCD) for Context Free Grammar (CFG) and you guys are the first to include the recent xgrammar. I'm trying to reproduce this error with the specified branch but I get the following new one:

File "/home/rog0d/miniconda3/envs/sglang/lib/python3.10/site-packages/sglang/srt/constrained/bnf_cache.py", line 44, in __init__
self.grammar_cache = GrammarMatcherInitContextCache(
TypeError: Dummy() takes no arguments

When deploying the container provided here with xgrammar as the backend,

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<hf_token>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server --model-path meta-llama/Llama-3.2-3B-Instruct --host 0.0.0.0 --port 30000 --grammar-backend xgrammar

I get the same Dummy() error. Is there something different I'm missing? I've traced it back to this PR.
I would like to test SGLang with CFG (apart from JSON and Regex) using outlines and xgrammar, if I manage to test it I'll gladly contribute it here.

One possible solution could be to reproduce vLLM OpenAI compatible server and its workflow as they have a guided_grammar parameter.

[merrymercy]

@roG0d You need to compile and install xgrammar: https://github.com/mlc-ai/xgrammar

pip install pybind11 ninja

mkdir build
cd build
cmake .. -G Ninja
ninja
cd ../python
pip install -e .

[Ubospica]

@roG0d Yes it is not uploaded to pypi yet. We will further integrate it soon and avoid this compile step.