[CI Failure]: test_struct_output_generate.py::test_structured_output - json.decoder.JSONDecodeError

[markmc]

Name of failing test

v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output[mistralai/Ministral-8B-Instruct-2410-lm-format-enforcer-auto-None]

Basic information

• Flaky test
• Can reproduce locally
• Caused by external libraries (e.g. bug in transformers)

🧪 Describe the failing test

Prompt: 'Generate a description of a frog using 50 characters. Make the response as short as possible.', Generated text: '{"description": "Frog: Green, leaps, croak. Darting eyes, webbed, 능'
...
 >           output_json = json.loads(generated_text)

 v1/entrypoints/llm/test_struct_output_generate.py:443:

📝 History of failing test

Still failing as of commit eafa8dc

CC List.

No response

[markmc]

#24404 disabled some of these tests, but not this one:

    ("mistralai/Ministral-8B-Instruct-2410", "lm-format-enforcer", "auto",
     None),

[markmc]

Oh, I think that's because #24380 fixed it?