[Feature] Add support for `logit_bias`

[creatorrr]

Support just landed in transformers lib for this. See:

• SequenceBiasLogitsProcessor
• Corresponding GenerationConfig
• discussion

[creatorrr]

@WoosukKwon Any tips for implementing this? Where can I start if I were to take a stab at this?

[benbot]

It's be great to see this added. The only blocker for lmql to add vLLM support!

[benbot]

Am I right in thinking that logit_bias like behavior already exists?

https://github.com/vllm-project/vllm/blob/acbed3ef40f015fcf64460e629813922fab90380/vllm/model_executor/layers/sampler.py#L94C33-L94C33

I think this is the whole logit bias calculation, right?

[benbot]

@creatorrr @WoosukKwon can we just expose that embedding_bias option to the api?

[simon-mo]

Closed by #3027 and usage of logits processors