What is the difference between Kor and the StructuredOutputs from Langchain?

[alberduris]

Can someone clarify the distinctions between Kor and StructuredOutputs in Langchain? I'm trying to understand the nuances and functionalities of both solutions. Any insights or experiences would be greatly appreciated!

[eyurtsev]

Hi @alberduris!

Kor and Langhcain StructuredParsers are similar when no examples are provided. Code is included below for reference.

Use Kor in situations where including examples helps with quality of results. The API helps with:

1. Adding examples to the prompt
2. Changing the encoders used (JSON, CSV or bring your own) for encoding the examples. Useful because csv seems to work better when extracting multiple results from a long piece of text.

Kor uses TypeScript to describe the schema by default while Langchain uses JSONSchema. Unclear which one works better, but TypeScript uses less tokens typically.

Providing examples helps the model learn how to use the schema and match the encoding correctly, reducing encoding errors.

Some guidelines are included here: https://eyurtsev.github.io/kor/guidelines.html

---

Zero-shot example

Below is a zero-shot example. I recommend generally using examples when easy to specify as they will likely improve performance.

import enum
from typing import Optional, List

import langchain
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field, validator

from kor import from_pydantic, create_extraction_chain

## Model

model_name = 'text-davinci-003'
temperature = 0.0
llm = OpenAI(model_name=model_name, temperature=temperature)

Schema

The MusicRequest schema below has examples that are commented out. Uncomment that field to see how that affects results when using Kor (careful not to use the examples field w/ the Langchain output parser since it ends up in the instruction segment as part of the JSONSchema, and could lead to worse results with the Langchain output parser.).

class Action(enum.Enum):
    play = "play"
    stop = "stop"
    previous = "previous"
    next_ = "next"


class MusicRequest(BaseModel):
    song: Optional[List[str]] = Field(
        description="The song(s) that the user would like to be played."
    )
    album: Optional[List[str]] = Field(
        description="The album(s) that the user would like to be played."
    )
    artist: Optional[List[str]] = Field(
        description="The artist(s) whose music the user would like to hear.",
        examples=[("Songs by paul simon", "paul simon")],
    )
    action: Optional[Action] = Field(
        description="The action that should be taken; one of `play`, `stop`, `next`, `previous`",
        examples=[
            # ("Please stop the music", "stop"),
            # ("play something", "play"),
            # ("play a song", "play"),
            # ("next song", "next"),
        ],
    )

Kor

query = "play a bob dylan song"

schema, validator = from_pydantic(MusicRequest)   
chain = create_extraction_chain(
    llm, schema, encoder_or_encoder_class="json", validator=validator
)
chain.predict_and_parse(text=query)['validated_data']

OutputParser

parser = PydanticOutputParser(pydantic_object=MusicRequest)

prompt = PromptTemplate(
    template="Determine what music request the user is making.\n{format_instructions}\n{query}\n",
    input_variables=["query"],
    partial_variables={"format_instructions": parser.get_format_instructions()}
)

_input = prompt.format_prompt(query=query)

output = llm(_input.to_string())

parser.parse(output)

[alberduris]

Thanks for the magnificent and detailed explanation @eyurtsev!! This should be on the docs <3