Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add Groq client support #84

Merged
merged 10 commits into from
Nov 24, 2024
Merged

add Groq client support #84

merged 10 commits into from
Nov 24, 2024

Conversation

ricklamers
Copy link
Contributor

  • Added GroqModel class to interact with Groq API.
  • Implemented structured and text-based responses for Groq models.
  • Added example and tests for Groq model integration.
  • Updated dependencies to include groq package.

@ricklamers ricklamers changed the title feat(models): add initial support for Groq models add support for Groq models Nov 22, 2024
@ricklamers ricklamers changed the title add support for Groq models add Groq client support Nov 22, 2024
@ricklamers
Copy link
Contributor Author

One thing I'm not sure of is how to let users override temperature, I've defaulted to 0.0 here because that's what most open source models need to adhere to function calling requests

@ricklamers
Copy link
Contributor Author

Not sure how to fix CI for

0s
Run make docs-insiders
Error: PPPR_TOKEN is not set, can't install insiders packages
make: *** [Makefile:80: .docs-insiders-install] Error [1](https://github.com/pydantic/pydantic-ai/actions/runs/11975670356/job/33389567928?pr=84#step:6:1)
Error: Process completed with exit code 2.

@samuelcolvin
Copy link
Member

This is awesome, thank you.

CI is my problem, I'll fix that — you're the first person contributing form outside pydantic (from a fork), hence you're seeing the issues.

I'm flat out on calls today, but I'll review ASAP.

samuelcolvin
samuelcolvin reviewed Nov 22, 2024
View reviewed changes
pyproject.toml Outdated Show resolved Hide resolved
tests/test_live.py Outdated
@@ -90,3 +91,34 @@ async def test_gemini_structured(http_client: httpx.AsyncClient):
print('Gemini structured cost:', result.cost())
cost = result.cost()
assert cost.total_tokens is not None and cost.total_tokens > 0


async def test_groq(http_client: httpx.AsyncClient):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I'll need to set an API key for this?

I can look it up, but is the process trivial?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Free API key can be created, trivial just go to https://console.groq.com/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix this once the initial PR is merged, test-live will no longer run on PRs from forks once you rebase.

pydantic_ai/models/groq.py Show resolved Hide resolved
@samuelcolvin
Copy link
Member

As of #87, PRs should work from forks. You'll need to rebase or merge with main.

@ricklamers
feat(models): add initial support for Groq models
07caf70 
- Added GroqModel class to interact with Groq API.
- Implemented structured and text-based responses for Groq models.
- Added example and tests for Groq model integration.
- Updated dependencies to include groq package.
@ricklamers
chore: lint
01d6cf3
@ricklamers
fix(models): make agent_model method asynchronous in GroqModel
fa31e4f 
Converted agent_model to async for better performance with asynchronous I/O operations.
hyperlint-ai[bot]
hyperlint-ai bot reviewed Nov 24, 2024
View reviewed changes
Copy link
Contributor

@hyperlint-ai hyperlint-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The style guide flagged several spelling errors that seemed like false positives. We skipped posting inline suggestions for the following words:

  • Groq

@samuelcolvin samuelcolvin merged commit 710b776 into pydantic:main Nov 24, 2024
15 checks passed
@samuelcolvin
Copy link
Member

amazing, thank you so much.

@samuelcolvin
Copy link
Member

Thanks so much for this @ricklamers.

Related to the documentation: it would be really nice if Groq published an objects.inv file for their python SDK, so we can link directly to entities in the package, as we do with other packages:

pydantic-ai/mkdocs.yml

Lines 155 to 160 in d29d5e0

- url: https://docs.python.org/3/objects.inv
# TODO use /latest/ not /dev/ when Pydantic 2.10 is released
- url: https://docs.pydantic.dev/dev/objects.inv
- url: https://fastapi.tiangolo.com/objects.inv
- url: https://typing-extensions.readthedocs.io/en/latest/objects.inv
- url: https://rich.readthedocs.io/en/stable/objects.inv

I asked @rattrayalex about this in relation to the OpenAI SDK, and hi said it wasn't an immediate priority, but I can't helping thinking that it wouldn't that hard for SDKs built with stainless to generate API docs with mkdocstrings and get a objects.inv definition for free.

@rattrayalex
Copy link

Hey @samuelcolvin , sorry I don't recall that convo?

@RobertCraigie would be the one to take this on – we could generate a objects.inv definition, but not 100% straightforward and also not sure were it should go?

@RobertCraigie
Copy link

would providing an objects.inv file be helpful without also hosting docs?

@samuelcolvin samuelcolvin mentioned this pull request Dec 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hyperlint-ai hyperlint-ai[bot] hyperlint-ai[bot] left review comments

@samuelcolvin samuelcolvin samuelcolvin left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ricklamers @samuelcolvin @rattrayalex @RobertCraigie