Migrate to pydantic v2

[einarwar]

Hi, and thanks for an awesome library. It has really helped my current project with automating integration- and end-to-end tests using docker compose.

We are looking to migrate to pydantic v2, which python-on-whales does not yet support. This is of course understandable since the first stable pydantic v2 released june 30 (two days when writing this).

If you would like me to submit a PR for this migration i would gladly help :)

[gabrieldemarmiesse]

I would be open to such a PR :) python-on-whales should be compatible with both versions

[einarwar]

So if i understand correctly, in requirements.txt it would look like e.g pydantic>=1.5,<3 ?
Most changes from pydantic v1 to v2 involves renames, which will work but give DeprecationWarnings when using v2.

I found there are mainly two things in the repo that would need to be changed to be able to support both pydantic v1 and v2:

• the @all_fields_optional decorator can no longer be used, because of changes to the FieldInfo-class. To solve this, it looks like I have to change all pydantic-classes from e.g

  @all_fields_optional
  class ImageGraphDriver(DockerCamelModel):
      name: str
      data: Any

  to

  class ImageGraphDriver(DockerCamelModel):
      name: Optional[str] = None
      data: Optional[Any] = None

  This also seems to be the direction pydantic will go in the future, as stated in the migration guide

• The pydantic.error_wrappers are removed in v2. I believe importing it as pydantic.ValidationError can be changed here

[gabrieldemarmiesse]

it would look like e.g pydantic>=1.5,<3 ?

Yes

the @all_fields_optional decorator can no longer be used

That makes me really sad. Isn't there really any way to recreate it? That would be a massive change to put all field optional.

The pydantic.error_wrappers are removed in v2. I believe importing it as pydantic.ValidationError can be changed here

That's fine by me :)

[einarwar]

the @all_fields_optional decorator can no longer be used

That makes me really sad. Isn't there really any way to recreate it? That would be a massive change to put all field optional.

Yeah, me too. In my current project we have several pydantic models where we want all the different attributes to be optional, and we went ahead and did it the way i described above. Pydantic v2 is still young, and a lot of the docs has not been updated for the new version yet, so there hopefully might be some way of accomplishing something similar in v2. But if one wants to support both v1 and v2 it seems like there is some breaking changes to the FieldInfo, e.g the attributes used by the decorator, field.required and field.allow_none no longer exists

[LewisGaul]

With my limited experience with pydantic, I can see some benefit from "explicit over implicit" (this did previously confuse me on a previous contribution to this project).

[gabrieldemarmiesse]

A first step: #454

[gabrieldemarmiesse]

Second and last step (hopefully): #455

[gabrieldemarmiesse]

We are currently blocked by pydantic/pydantic#6512

[einarwar]

Sorry for ghosting you @gabrieldemarmiesse. Vacation and IRL hit hard. Nice to see that you're working on it :)

[gabrieldemarmiesse]

No worries. Nobody's in a hurry in open source :)

[gabrieldemarmiesse]

IT IS DONE

Can you try the latest version of master? I'd like some feedback before an official release

Note: pydantic v2.0.x won't work. Please install 2.1.x

[einarwar]

Good job! Did some testing on our test suite, and it seems like python-on-whales worked as it used to before supporting pydantic v2. Only nitpick i have is some deprecation warnings being raised, eg

python-on-whales/python_on_whales/components/container/cli_wrapper.py

Lines 95 to 96 in e054f1e

	def _parse_json_object(self, json_object: Dict[str, Any]):
	return ContainerInspectResult.parse_obj(json_object)

will raise a PydanticDeprecatedSince20 warning since parse_obj is deprecated. I don't think one can get around things like this if one wants to support pydantic v1 and v2 :)

I was not able to do extensive testing, since my project also needed to support pydantic v2 in order to test this and i had to do some quick and dirty fixes to get things up and running.

[gabrieldemarmiesse]

That seems feasible. I'm sure with a if-else on the version we can do that

[gabrieldemarmiesse]

@einarwar today is your lucky day: #460

[gabrieldemarmiesse]

you shouldn't have any warnings now, feel free to try the latest master

[einarwar]

Awesome! Tested and got no warnings :)

[gabrieldemarmiesse]

The new release is here, feel free to test it and report any bugs. I'll close this issue!