[Bug]: Error using OpenAI interface with TGI: logit_bias: invalid type: map, expected a sequence

[jvstme]

Steps to reproduce

> cat .dstack.yml
type: service

image: ghcr.io/huggingface/text-generation-inference:2.2.0
env:
  - HUGGING_FACE_HUB_TOKEN
  - MODEL_ID=mistralai/Mistral-7B-Instruct-v0.2
commands:
  - text-generation-launcher --port 8000 --trust-remote-code
port: 8000

resources:
  gpu: 24GB

model:
  format: openai
  type: chat
  name: mistralai/Mistral-7B-Instruct-v0.2

> dstack apply -y

...

> curl https://gateway.mydomain.example/chat/completions -H 'Authorization: Bearer *****' -H 'Content-Type: application/json' -d '{"model":"mistralai/Mistral-7B-Instruct-v0.2", "messages": [{"role":"user", "content":"Hi"}]}'

Actual behaviour

> curl https://gateway.mydomain.example/chat/completions -H 'Authorization: Bearer *****' -H 'Content-Type: application/json' -d '{"model":"mistralai/Mistral-7B-Instruct-v0.2", "messages": [{"role":"user", "content":"Hi"}]}'
{"error":"GatewayError","message":"Failed to deserialize the JSON body into the target type: logit_bias: invalid type: map, expected a sequence at line 1 column 137"}

Expected behaviour

A successful response from the model.

dstack version

master

Server logs

No response

Additional information

Workaround: Use format: tgi in the service configuration when launching models with TGI.