Support for JSON examples in PromptTemplate

[timrwilliams]

It's common when prompt engineering to require the LLM to return results in JSON format and to include an example JSON response within the prompt. The current Prompt variable parser can't handle this as it interprets the example JSON as an input variable and then complains that it is missing:

require "langchain"
simple_template = "Tell me a {adjective} joke. Return in JSON in the format {{joke: 'The joke'}}"
Prompt::Base.extract_variables_from_template(simple_template)
=> ["adjective", "joke: 'The joke'"]

The current Python f-string parser handles this by allowing you to escape a curly brace with a double curly brace, as shown in the example above. Python f-string spec

Looking at the regex its clearly trying to do something with double curly brackets so before I try and fix it can you shed some light on the original implementation?

[andreibondarev]

@rafaelqfigueiredo Any chance that you could chime in here?

[andreibondarev]

@timrwilliams Implementing something like this: https://github.com/1rgs/jsonformer would be amazing. It's built on top of the Hugging Face transformers library and I'm not sure that we have an equivalent in the Ruby world yet.

/cc @alchaplinsky

[timrwilliams]

Interesting. I tried the jsonformer example notebook and it appeared to download a 5.6GB model before doing the JSON generation which is probably not something we want to drag into langchainrb!

Looks like they are multiple people taking different approaches to getting structured JSON (e.g. Microsoft's guidance: https://github.com/microsoft/guidance#guidance-acceleration-notebook) so might be premature to dive into anything too complex.

In my experience if you are dealing with relatively simple JSON structures then GPT-3 models can respond pretty well.