Skip to content

Commit 0c990ee

Browse files
authored
Warn about implicit retries on the FallbackModel docs
Closes #3267 Added note about disabling provider SDK retries when using FallbackModel.
1 parent 660c59a commit 0c990ee

File tree

1 file changed

+18
-12
lines changed

1 file changed

+18
-12
lines changed

docs/models/overview.md

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,8 @@ Pydantic AI uses a few key terms to describe how it interacts with different LLM
4747
vendor-SDK-agnostic API, ensuring a single Pydantic AI agent is portable to different LLM vendors without
4848
any other code changes just by swapping out the Model it uses. Model classes are named
4949
roughly in the format `<VendorSdk>Model`, for example, we have `OpenAIChatModel`, `AnthropicModel`, `GoogleModel`,
50-
etc. When using a Model class, you specify the actual LLM model name (e.g., `gpt-5`,
51-
`claude-sonnet-4-5`, `gemini-2.5-flash`) as a parameter.
50+
etc. When using a Model class, you specify the actual LLM model name (e.g., `gpt-4o`,
51+
`claude-3-5-sonnet-latest`, `gemini-1.5-flash`) as a parameter.
5252
- **Provider**: This refers to provider-specific classes which handle the authentication and connections
5353
to an LLM vendor. Passing a non-default _Provider_ as a parameter to a Model is how you can ensure
5454
that your agent will make requests to a specific endpoint, or make use of a specific approach to
@@ -61,7 +61,7 @@ Pydantic AI uses a few key terms to describe how it interacts with different LLM
6161
and the same schema transformer needs to be used for Gemini models whether you're using `GoogleModel`
6262
with model name `gemini-2.5-pro-preview`, or `OpenAIChatModel` with `OpenRouterProvider` and model name `google/gemini-2.5-pro-preview`.
6363

64-
When you instantiate an [`Agent`][pydantic_ai.Agent] with just a name formatted as `<provider>:<model>`, e.g. `openai:gpt-5` or `openrouter:google/gemini-2.5-pro-preview`,
64+
When you instantiate an [`Agent`][pydantic_ai.Agent] with just a name formatted as `<provider>:<model>`, e.g. `openai:gpt-4o` or `openrouter:google/gemini-2.5-pro-preview`,
6565
Pydantic AI will automatically select the appropriate model class, provider, and profile.
6666
If you want to use a different provider or profile, you can instantiate a model class directly and pass in `provider` and/or `profile` arguments.
6767

@@ -86,6 +86,12 @@ You can use [`FallbackModel`][pydantic_ai.models.fallback.FallbackModel] to atte
8686
in sequence until one successfully returns a result. Under the hood, Pydantic AI automatically switches
8787
from one model to the next if the current model returns a 4xx or 5xx status code.
8888

89+
!!! note
90+
91+
The provider SDKs on which Models are based (like OpenAI, Anthropic, etc.) often have built-in retry logic that can delay the `FallbackModel` from activating.
92+
93+
When using `FallbackModel`, it's recommended to disable provider SDK retries to ensure immediate fallback, for example by setting `max_retries=0` on a [custom OpenAI client](openai.md#custom-openai-client).
94+
8995
In the following example, the agent first makes a request to the OpenAI model (which fails due to an invalid API key),
9096
and then falls back to the Anthropic model.
9197

@@ -97,8 +103,8 @@ from pydantic_ai.models.anthropic import AnthropicModel
97103
from pydantic_ai.models.fallback import FallbackModel
98104
from pydantic_ai.models.openai import OpenAIChatModel
99105

100-
openai_model = OpenAIChatModel('gpt-5')
101-
anthropic_model = AnthropicModel('claude-sonnet-4-5')
106+
openai_model = OpenAIChatModel('gpt-4o')
107+
anthropic_model = AnthropicModel('claude-3-5-sonnet-latest')
102108
fallback_model = FallbackModel(openai_model, anthropic_model)
103109

104110
agent = Agent(fallback_model)
@@ -121,7 +127,7 @@ print(response.all_messages())
121127
),
122128
ModelResponse(
123129
parts=[TextPart(content='Paris', part_kind='text')],
124-
model_name='claude-sonnet-4-5',
130+
model_name='claude-3-5-sonnet-latest',
125131
timestamp=datetime.datetime(...),
126132
kind='response',
127133
provider_response_id=None,
@@ -147,11 +153,11 @@ from pydantic_ai.models.openai import OpenAIChatModel
147153

148154
# Configure each model with provider-specific optimal settings
149155
openai_model = OpenAIChatModel(
150-
'gpt-5',
156+
'gpt-4o',
151157
settings=ModelSettings(temperature=0.7, max_tokens=1000) # Higher creativity for OpenAI
152158
)
153159
anthropic_model = AnthropicModel(
154-
'claude-sonnet-4-5',
160+
'claude-3-5-sonnet-latest',
155161
settings=ModelSettings(temperature=0.2, max_tokens=1000) # Lower temperature for consistency
156162
)
157163

@@ -179,8 +185,8 @@ contains all the exceptions encountered during the `run` execution.
179185
from pydantic_ai.models.fallback import FallbackModel
180186
from pydantic_ai.models.openai import OpenAIChatModel
181187

182-
openai_model = OpenAIChatModel('gpt-5')
183-
anthropic_model = AnthropicModel('claude-sonnet-4-5')
188+
openai_model = OpenAIChatModel('gpt-4o')
189+
anthropic_model = AnthropicModel('claude-3-5-sonnet-latest')
184190
fallback_model = FallbackModel(openai_model, anthropic_model)
185191

186192
agent = Agent(fallback_model)
@@ -211,8 +217,8 @@ contains all the exceptions encountered during the `run` execution.
211217
print(exc)
212218

213219

214-
openai_model = OpenAIChatModel('gpt-5')
215-
anthropic_model = AnthropicModel('claude-sonnet-4-5')
220+
openai_model = OpenAIChatModel('gpt-4o')
221+
anthropic_model = AnthropicModel('claude-3-5-sonnet-latest')
216222
fallback_model = FallbackModel(openai_model, anthropic_model)
217223

218224
agent = Agent(fallback_model)

0 commit comments

Comments
 (0)