Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

result_type as List #523

Open
fils opened this issue Dec 22, 2024 · 0 comments
Open

result_type as List #523

fils opened this issue Dec 22, 2024 · 0 comments

Comments

@fils
Copy link

fils commented Dec 22, 2024

I don't know if this is related to #242 or not.

I am trying to replicate this from the Ollama examples (ref: https://ollama.com/blog/structured-outputs)

from openai import OpenAI
import openai
from pydantic import BaseModel

client = OpenAI(base_url="http://192.168.202.137:11434/v1", api_key="ollama")

class Pet(BaseModel):
    name: str
    animal: str
    age: int
    color: str | None
    favorite_toy: str | None

class PetList(BaseModel):
    pets: list[Pet]

try:
    completion = client.beta.chat.completions.parse(
        temperature=0,
        model='llama3.2:3b',
        messages=[
            {"role": "user", "content": '''
                I have two pets.
                A cat named Luna who is 5 years old and loves playing with yarn. She has grey fur.
                I also have a 2 year old black cat named Loki who loves tennis balls.
            '''}
        ],
        response_format=PetList,
    )

    pet_response = completion.choices[0].message
    if pet_response.parsed:
        print(pet_response.parsed)
    elif pet_response.refusal:
        print(pet_response.refusal)
except Exception as e:
    if type(e) == openai.LengthFinishReasonError:
        print("Too many tokens: ", e)
        pass
    else:
        print(e)
        pass

In pydantic-al I try

from pydantic import BaseModel
from pydantic_ai import Agent
from pydantic_ai.models.ollama import OllamaModel

ollama_model = OllamaModel(
    model_name='llama3.2:3b',
    base_url='http://192.168.202.137:11434/v1'
)

class Pet(BaseModel):
  name: str
  animal: str
  age: int
  color: str | None
  favorite_toy: str | None

class PetList(BaseModel):
  pets: list[Pet]


# # Create a system prompt to guide the model
SYSTEM_PROMPT = """
You are a helper that extracts pet information from text and formats it as a list.
For each pet mentioned, extract:
- name
- animal type
- age
- color (if mentioned)
- favorite toy (if mentioned)

Format as a JSON object with a 'pets' array containing each pet's details.
"""

agent3 = Agent(model=ollama_model, result_type=PetList, retries=3)
result3 = agent3.run_sync(('I have two pets. A cat named Luna who is 5 years old'
                           ' and loves playing with yarn. She has grey fur. I also '
                           'have a 2 year old black cat named Loki who loves tennis balls.'))
pet_data = result3.data
print(pet_data)

passing PetList to the result_type. This will fail.

If I just pass Pet, ie, result_type=Pet it will sometimes work, getting only 1 cat of course, but also fail sometimes.

Any guidance on how to address this would be appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant