Add outlines vLLM OpenAI server

[Soufiane-Ra]

To use with Langchain / OpenAI add extra_body={"schema": yourSchema} or extra_body={"regex": yourRegex} example:

llm = ChatOpenAI ( openai_api_key="EMPTY",
                   openai_api_base="http://localhost:8000/v1",
                   model_name=model_path,
                   extra_body={"schema": json_schema },
                 )

[lapp0]

Thanks for the PR! I see two issues this would close

#512

#594

[lapp0]

Good job integrating vllms openai module with outlines!

Could you document usage and availability in vllm.md (or a new markdown file openai_endpoint.md)?

I'll do thorough smoke testing later this week.

[lapp0]

Please ensure any new dependencies are added are part of pyproject.toml

[rlouf]

Wouldn't they be installed when installing vLLM?

[lapp0]

Yes, I should have pulled before comparing to my local openai_server.py...

[lapp0]

Beam search functionality depends on #539

[jamestwhedbee]

I'd expect this to extend OpenAI's existing response_format field to include the schema, rather than use an extra_body field. Thoughts?

[rlouf]

Thank you for opening a PR, this is impressive and I will need a few days to review it. In the meantime please make sure that the checks in the CI are passing.

[7flash]

Wow looks amazing, so I understand now after I deploy outlines to my server, then I can just add another line into my litellm config pointing to outlines, and then it just works compatible with my existing UI?

[lapp0]

I tried smoke testing, but got

  File "/root/outlines/outlines/serve/openai_server.py", line 86
    parser.add_argument(
IndentationError: unexpected indent

[rlouf]

@Soufiane-Ra could you solve the formatting issues?

[danielchalef]

@Soufiane-Ra Great work! We'd love to use this. It looks like there is another indentation error here. Could you fix it? Thx

[findalexli]

Following this and would love to see any update

[olaf-beh]

I'd expect this to extend OpenAI's existing response_format field to include the schema, rather than use an extra_body field. Thoughts?

I cannot agree. To be openai compatible we have to use extra_body. They specified extra_body field exactly for the use case to pass custom application fields.

E.g. if you use openai library for sending requests to vllm.engine.openai.entrypoints.openai.apiserver and you use an extra JSON field which is not specified by official openai API, then the openai python library will raise an error (I tried that a couple of weeks ago).

Just my two cents ...

Just saw this PR in vLLM: vllm-project/vllm#3211
It integrates oulines in vllm.engine.openai.entrypoints.openai.apiserver. This solves this PR, doesn't it?

[rlouf]

Thank you for contributing! In the meantime vLLM integrated Outlines, and you can now use structured generation directly from there.