fix forward type reference in Pydantic schemas

[eli-bl]

This small change addresses an issue that, unfortunately, I don't have a good test case to demonstrate. I've only been able to reproduce it with one specific API spec file... and only in my fork. Ordinarily I wouldn't submit an upstream change due to something that's only ever happened in my fork. However, in this case:

1. I never changed any code in openapi_schema_pydantic/ in this fork.
2. The code path that is raising the error also has no changes. It's this line in schemas.py, where it's trying to construct a Parameter instance. The error message ("Parameter is not fully defined") indicates that the problem is not the arguments to the Parameter initializer—it's the Parameter class itself, which Pydantic considers to be invalid. The message also indicates that the "not fully defined" part is a forward reference to the Header class.
3. The forward reference (in encoding.py) existed in order to avoid an import cycle between Parameter, Encoding, and Header. That should be fine, and was working fine till now with exactly the same code in these Pydantic models. However...
4. Per the Pydantic docs, the logic for lazily resolving forward references is extremely complicated, and in some hard-to-define cases it can fail. In that case they advise doing what I've done here: calling model_rebuild() on the class that's getting the error, after the previously-undefined class has been defined.

Because of all that, my theory is that there is some extremely subtle interaction between my other changes in the fork which, under some very specific conditions related to the particular spec I was testing with, maybe led to modules being imported in a different order... or something like that. This apparently had no effect on anything else, but made a difference in Pydantic's lazy-resolve mechanism. I can't characterize it any better than that, and I don't know how to write a test for it (although this fix does solve my use case). But I do believe this workaround is valid according to the docs and doesn't cause problems under any other circumstances. And I suspect that the circumstances under which it was working were brittle enough that some other completely unrelated change could've eventually caused it to manifest in an equally confusing way, so you could see this as a fix-in-advance for that.

(The problem is not that header.py hadn't actually been imported before that line in schemas.py executed. I verified 100% for sure that it had been. It's something more subtle than that.)

[texhnolyze]

This error happened to me after upgrading to version 0.21.7 and still persisted with other 0.21.* versions I tested. Even with 0.21.5, which I was previously using, I got the issue.
I also tested with diff different JSON OpenAPI specs, to ensure this is not just an issue with my personal spec.
E.g. this also happens with the default pet store example from: https://editor.swagger.io/
for me.

This led me to the assumption that it must be some change/regression in pydantic itself.
Manually downgrading to pydantic==2.10.1 in the pipx venv fixes this problem for me.

I took a look at the pydantic diff: pydantic/pydantic@v2.10.1...v2.10.2, but without having a deeper understanding of the pydantic code base I can't really pinpoint what changes might lead to this issue.
There also seem to be a few fixes still outstanding: pydantic/pydantic#10910

Manually adding your changes also fixes the issue, but I guess it should be clarified if there should be a fix in pydantic instead and potentially the pinning of pydantic==2.10.1 in the meantime.

[dedehas]

Can confirm, was also struggling with this.
PydanticUserError: Parameter is not fully defined; you should
define Header, then call Parameter.model_rebuild().
I believe it happens when the parameters include references to components.
Changing to pydantic 2.10.1 fixed the issue

[eli-bl]

It sounds likely that pydantic's behavior did change with that patch release, but I'm not sure that that's the whole story. The pydantic upgrade has been present for a couple of months; my fork includes that change, and I've been working on my fork a lot during that time, and testing with specs that do include a reference from parameters to components. So it may be that there's some other subtle factor.

But even if it's only pydantic >2.10.1 that has the problem in these cases, I have to assume, based on the docs I linked to, that this was already a known issue in some cases: they specifically tell you that if you have a circular reference and it fails to resolve automatically, the workaround is to call model_resolve(). There are online discussions about that going back several years. So I would consider this change to be an appropriate defensive measure regardless of the version.

[Viicos]

Hi, Pydantic maintainer here.

In 2.10, we did a complete refactor of the forward annotations evaluation logic to be more correct: it supports more edge cases, but also removes support for invalid use cases. This created some churn in libraries having invalid use cases implemented, but fixing it will result in safer code and better consistency.

Regarding the openapi-python-client, there's a couple places where things can be fixed. Taking the simplest example:

openapi-python-client/openapi_python_client/schema/openapi_schema_pydantic/encoding.py

Lines 7 to 22 in 861ef56

	if TYPE_CHECKING: # pragma: no cover
	from .header import Header
	else:
	Header = "Header"
	class Encoding(BaseModel):
	"""A single encoding definition applied to a single schema property.
	References:
	- https://swagger.io/docs/specification/describing-request-body/
	- https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.3.md#encodingObject
	"""
	contentType: Optional[str] = None
	headers: Optional[dict[str, Union[Header, Reference]]] = None

Here, Header is imported in an if TYPE_CHECKING: block to avoid a circular import. At runtime, TYPE_CHECKING is False, so Pydantic has no way of knowing what it represents (and only sees the "Header" string, what we call a forward reference). To evaluate forward references, Pydantic provides what we call a namespace: a mapping of strings to objects. When Optional[dict[str, Union["Header", Reference]]] is encountered, we look into the namespace for "Header" to replace it with the actual type.

Before 2.10, this used to somehow work because our namespace management was poorly designed, and symbols that shouldn't resolve (like in the example above, where Header is not available at runtime in the module) were still present in the provided namespace (in most cases, this was when building Encoding while in the process of building another model, or when subclassing). In rare cases, this messy namespace management could be dangerous, as you could have different types of the same name in different modules and Pydantic could use the wrong one ¹.

To fix the issue, you can either:

• Fix the circular import. This often requires a complete refactor of the module structure and is sometimes even not possible to do (and unless you flatten everything by defining models in a single module, this seems to be your case).
• Call model_rebuild() in the right place, once everything is imported. This is the path of least resistance but you have to think about adding these calls whenever you define a model using an unknown forward annotation.

I'm working on a PR to suggest changes following the second option. @eli-bl, do note that benchling#223 is not a complete fix. You might want to apply the same changes from my PR on the fork.

Footnotes

1. For instance, see https://github.com/langchain-ai/langchain-google/issues/610#issuecomment-2493831993. ↩

[eli-bl]

@Viicos, thanks for all of that context, and for submitting the alternate PR! I'll close this one and I'll also update the fix on our fork as per your advice.