Just to link another relevant discussion. #1202 perhaps we can have a convenient way to create a structured response generator. This seems particularly useful for non OpenAI model and helps them to support function calling. Not sure where that code would live but it can be a huge benefit. Could be designed as something similar to a decorator like what we did for function calling.
Originally posted by @ekzhu in #490 (comment)