Responses max_output_tokens

[iamemilio]

🚀 Describe the new functionality needed

Responses and Completions have a max_output_tokens field. It is currently missing from the create and response object in Responses API.

Requirements

• Implement max_output_tokens field as an input argument for Responses API, and add it to the response object
• Pass this through to the completions request translation, and ensure that it gets returned

💡 Why is this needed? What if we don't build it?

This is a feature gap

Other thoughts

Good First Issue

[abhibongale]

Hey @iamemilio , I would like to work on it, can you please help me with good starting point, files to look at?.

thank you.

/assign

[iamemilio]

Of course @abhibongale!

Responses API types are stored here, which is where you can find the response object . This should be updated to contain max_output_tokens based on the openAI spec. I also like to check the openai python types to get a sense for the best way to implement that field in python.

Once you do that, you can add it to the responses create request.

Finally, you will have to pass it through to the providers. Responses in llama stack today does a translation from responses API to completions. In completions, there is a similar field, max_tokens which we can use to pass this through in the inline implementation of responses API here.

Once you finish your changes, make sure to run:

openapi type generation

 uv run python -m docs.openapi_generator.generate docs/_static

unit tests

  uv run --group unit pytest -sv llama-stack/tests/unit/

pre-commit

uv run pre-commit run

If those pass, then you're looking pretty good and are ready to update the integration tests: https://github.com/llamastack/llama-stack/tree/main/tests/integration

Hope that helps!