Request: specifying response schema to `Docs.query`

[jamesbraza]

PaperQA is not set up to be a general purpose LLM where we can say "answer with this schema". PaperQA is more aimed to provide answers using citations.

However, there are times where we want more precise answers, for example when answering true/false, yes/no, or multiple choice questions. Currently, this is done using prompt engineering followed by a verifier/grader.

A more precise form of this would be specifying a JSON or XML schema. Note that in the past, our attempts to force adherence to a schema have resulted in decreased system performance.

The request here is to allow specifying a response schema, mainly to ease unit testing burdens and simple applications. And note that for SOTA performance, perhaps using a response schema is a bad call.

[jacknguyen0810]

Hi, has there been any updates on this? Specifically, if there is a way to enforce a certain answer format (without prompt engineering) for multiple choice questions.