feat(api_server): add options for users to control output

[aarnphm]

cc @WoosukKwon when you have bandwidth. This allows users to have more fine tuned control on how the API server yield back the text. Default behaviour are still the same, but it will address #1612

Signed-off-by: Aaron 29749331+aarnphm@users.noreply.github.com

[Tostino]

@aarnphm I implemented somewhat similar functionality for the OpenAI API chat endpoint (here: #1493) for completing a model response. I used return_full_response as the parameter name. I did not add it for the generation endpoint there though, so I figured we should have consistent implementations of this type of feature throughout.

[aarnphm]

@Tostino I believe for OpenAI there is a echo feature right?

[Tostino]

Correct, for their completions (legacy) API they support echo. Not for their chat completions API.

The current support looks like:

if request.echo:
        # We do not support echo since the vLLM engine does not
        # currently support getting the logprobs of prompt tokens.
        return create_error_response(HTTPStatus.BAD_REQUEST,
                                     "echo is not currently supported")

[simon-mo]

Currently, we are minimizing the functionality of api server. API server is not designed for production and mostly for demo purpose.

If you can add echo to OpenAI server that would be great. I believe we are support prompt log prob now.

[aarnphm]

Thats fair. I will close this PR then and will leave it to the OpenAI to implement echo support.