Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2c3b9cb
add hf inference providers support
hanouticelina Jun 4, 2025
537a657
update dependencies
hanouticelina Jun 4, 2025
af602a5
nit
hanouticelina Jun 4, 2025
1f3f7a2
update docstring
hanouticelina Jun 4, 2025
bea050c
add tests
hanouticelina Jun 9, 2025
40aef2e
add docs and known models for hf
hanouticelina Jun 9, 2025
af6fa42
Merge branch 'main' of github.com:hanouticelina/pydantic-ai into hf-i…
hanouticelina Jun 9, 2025
7a4b9a4
fix imports in test
hanouticelina Jun 9, 2025
a153081
fix tests
hanouticelina Jun 9, 2025
2f0ec51
fix provider test
hanouticelina Jun 9, 2025
69aee55
adapt cli test
hanouticelina Jun 9, 2025
f68dace
re-record vcr cassettes
hanouticelina Jun 9, 2025
cc982e5
fix token name
hanouticelina Jun 9, 2025
00da46e
fix examples test
hanouticelina Jun 9, 2025
bb20a34
Merge remote-tracking branch 'origin/main' into hf-inference-providers
Kludex Jun 25, 2025
922fd13
Add API docs and refactor a bit the wording
Kludex Jun 25, 2025
c779503
Merge branch 'main' of github.com:hanouticelina/pydantic-ai into hf-i…
hanouticelina Jul 2, 2025
873f090
Merge branch 'hf-inference-providers' of github.com:hanouticelina/pyd…
hanouticelina Jul 2, 2025
adfc254
review suggestions
hanouticelina Jul 2, 2025
e4af59e
more tests
hanouticelina Jul 7, 2025
fbead3d
Merge branch 'main' of github.com:hanouticelina/pydantic-ai into hf-i…
hanouticelina Jul 7, 2025
cd76d78
fix test
hanouticelina Jul 7, 2025
13ebbf9
fix another test
hanouticelina Jul 7, 2025
b96b710
add more vcr tests
hanouticelina Jul 9, 2025
9b0edee
split thinking part
hanouticelina Jul 9, 2025
6da1cf2
fix tests
hanouticelina Jul 9, 2025
d546e04
add more context to hugging face models page
burtenshaw Jul 9, 2025
789c261
Update docs/models/huggingface.md
hanouticelina Jul 9, 2025
17b74d2
Merge pull request #1 from burtenshaw/docs/inference-providers-docs
hanouticelina Jul 9, 2025
96a9c96
add more tests
hanouticelina Jul 9, 2025
0c37e41
Merge branch 'hf-inference-providers' of github.com:hanouticelina/pyd…
hanouticelina Jul 9, 2025
60e5c74
coverage
hanouticelina Jul 9, 2025
f2a74dd
remove no-cover
hanouticelina Jul 9, 2025
eb4af3f
Merge remote-tracking branch 'origin/main' into hf-inference-providers
Kludex Jul 16, 2025
9539909
replace exception handling
Kludex Jul 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/api/models/huggingface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# `pydantic_ai.models.huggingface`

## Setup

For details on how to set up authentication with this model, see [model configuration for Hugging Face](../../models/huggingface.md).

::: pydantic_ai.models.huggingface
2 changes: 2 additions & 0 deletions docs/api/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,5 @@
::: pydantic_ai.providers.github.GitHubProvider

::: pydantic_ai.providers.openrouter.OpenRouterProvider

::: pydantic_ai.providers.huggingface.HuggingFaceProvider
95 changes: 95 additions & 0 deletions docs/models/huggingface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Hugging Face

[Hugging Face](https://huggingface.co/) is an AI platform with all major open source models, datasets, MCPs, and demos. You can use [Inference Providers](https://huggingface.co/docs/inference-providers) to run open source models like DeepSeek R1 on scalable serverless infrastructure.

## Install

To use `HuggingFaceModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `huggingface` optional group:

```bash
pip/uv-add "pydantic-ai-slim[huggingface]"
```

## Configuration

To use [Hugging Face](https://huggingface.co/) inference, you'll need to set up an account which will give you [free tier](https://huggingface.co/docs/inference-providers/pricing) allowance on [Inference Providers](https://huggingface.co/docs/inference-providers). To setup inference, follow these steps:

1. Go to [Hugging Face](https://huggingface.co/join) and sign up for an account.
2. Create a new access token in [Hugging Face](https://huggingface.co/settings/tokens).
3. Set the `HF_TOKEN` environment variable to the token you just created.

Once you have a Hugging Face access token, you can set it as an environment variable:

```bash
export HF_TOKEN='hf_token'
```

## Usage

You can then use [`HuggingFaceModel`][pydantic_ai.models.huggingface.HuggingFaceModel] by name:

```python
from pydantic_ai import Agent

agent = Agent('huggingface:Qwen/Qwen3-235B-A22B')
...
```

Or initialise the model directly with just the model name:

```python
from pydantic_ai import Agent
from pydantic_ai.models.huggingface import HuggingFaceModel

model = HuggingFaceModel('Qwen/Qwen3-235B-A22B')
agent = Agent(model)
...
```

By default, the [`HuggingFaceModel`][pydantic_ai.models.huggingface.HuggingFaceModel] uses the
[`HuggingFaceProvider`][pydantic_ai.providers.huggingface.HuggingFaceProvider] that will select automatically
the first of the inference providers (Cerebras, Together AI, Cohere..etc) available for the model, sorted by your
preferred order in https://hf.co/settings/inference-providers.

## Configure the provider

If you want to pass parameters in code to the provider, you can programmatically instantiate the
[`HuggingFaceProvider`][pydantic_ai.providers.huggingface.HuggingFaceProvider] and pass it to the model:

```python
from pydantic_ai import Agent
from pydantic_ai.models.huggingface import HuggingFaceModel
from pydantic_ai.providers.huggingface import HuggingFaceProvider

model = HuggingFaceModel('Qwen/Qwen3-235B-A22B', provider=HuggingFaceProvider(api_key='hf_token', provider_name='nebius'))
agent = Agent(model)
...
```

## Custom Hugging Face client

[`HuggingFaceProvider`][pydantic_ai.providers.huggingface.HuggingFaceProvider] also accepts a custom
[`AsyncInferenceClient`][huggingface_hub.AsyncInferenceClient] client via the `hf_client` parameter, so you can customise
the `headers`, `bill_to` (billing to an HF organization you're a member of), `base_url` etc. as defined in the
[Hugging Face Hub python library docs](https://huggingface.co/docs/huggingface_hub/package_reference/inference_client).

```python
from huggingface_hub import AsyncInferenceClient

from pydantic_ai import Agent
from pydantic_ai.models.huggingface import HuggingFaceModel
from pydantic_ai.providers.huggingface import HuggingFaceProvider

client = AsyncInferenceClient(
bill_to='openai',
api_key='hf_token',
provider='fireworks-ai',
)

model = HuggingFaceModel(
'Qwen/Qwen3-235B-A22B',
provider=HuggingFaceProvider(hf_client=client),
)
agent = Agent(model)
...
```
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ nav:
- api/models/gemini.md
- api/models/google.md
- api/models/groq.md
- api/models/huggingface.md
- api/models/instrumented.md
- api/models/mistral.md
- api/models/test.md
Expand Down
14 changes: 13 additions & 1 deletion pydantic_ai_slim/pydantic_ai/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,14 @@
'heroku:claude-3-7-sonnet',
'heroku:claude-4-sonnet',
'heroku:claude-3-haiku',
'huggingface:Qwen/QwQ-32B',
'huggingface:Qwen/Qwen2.5-72B-Instruct',
'huggingface:Qwen/Qwen3-235B-A22B',
'huggingface:Qwen/Qwen3-32B',
'huggingface:deepseek-ai/DeepSeek-R1',
'huggingface:meta-llama/Llama-3.3-70B-Instruct',
'huggingface:meta-llama/Llama-4-Maverick-17B-128E-Instruct',
'huggingface:meta-llama/Llama-4-Scout-17B-16E-Instruct',
Comment on lines +230 to +237
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we keep the list of those models up-to-date? Do you folks have an endpoint that we can call to list a lot of them, or something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes! we've just shipped https://router.huggingface.co/v1/models that returns the list of models (sorted by trending score on the Hugging Face Hub). Not sure how can I update KnownModelName dynamically? i can do that for [LatestHuggingFaceModelNames](https://github.com/hanouticelina/pydantic-ai/blob/873f090197b6c7b8c2ae1c0760db17ef54814f86/pydantic_ai_slim/pydantic_ai/models/huggingface.py#L69, but KnownModelName needs to be update as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can create a test that: if run locally, it would call that endpoint and it will check against this list of models.

In CI it should use the cassette.

'mistral:codestral-latest',
'mistral:mistral-large-latest',
'mistral:mistral-moderation-latest',
Expand Down Expand Up @@ -560,7 +568,7 @@ def override_allow_model_requests(allow_model_requests: bool) -> Iterator[None]:
ALLOW_MODEL_REQUESTS = old_value # pyright: ignore[reportConstantRedefinition]


def infer_model(model: Model | KnownModelName | str) -> Model:
def infer_model(model: Model | KnownModelName | str) -> Model: # noqa: C901
"""Infer the model from the name."""
if isinstance(model, Model):
return model
Expand Down Expand Up @@ -624,6 +632,10 @@ def infer_model(model: Model | KnownModelName | str) -> Model:
from .bedrock import BedrockConverseModel

return BedrockConverseModel(model_name, provider=provider)
elif provider == 'huggingface':
from .huggingface import HuggingFaceModel

return HuggingFaceModel(model_name, provider=provider)
else:
raise UserError(f'Unknown model: {model}') # pragma: no cover

Expand Down
Loading