chore: bump-pydantic to 2.3.0 #375

ThibaultFy · 2023-07-24T16:12:46Z

Companion PR

closes FL-1068

ThibaultFy · 2023-08-31T10:25:13Z

/e2e

ThibaultFy · 2023-08-31T11:57:08Z

/e2e --refs substrafl=chore/bump-to-pydantic-2

Owlfred · 2023-08-31T11:57:13Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon: ⏭️
Dispatch Jobs: ✔️
Distributed / distributed-sdk: ❌
MNIST: ⏭️
Standalone / standalone-sdk: ❌

Sorry, try again.

ThibaultFy · 2023-08-31T14:55:14Z

/e2e --refs substrafl=chore/bump-to-pydantic-2 substra=chore/bump-to-pydantic-2 substra-tests=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmark mnist,camelyon

Owlfred · 2023-08-31T14:55:16Z

Something happened while processing your message: error: unknown option '--benchmark'
(Did you mean --benchmarks?)

ThibaultFy · 2023-08-31T14:55:28Z

/e2e --refs substrafl=chore/bump-to-pydantic-2 substra=chore/bump-to-pydantic-2 substra-tests=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon

Owlfred · 2023-08-31T14:55:30Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed / distributed-sdk,substrafl: ❌
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

“You shall not pass!” ― Gandalf, The Lord of the Rings, The Fellowship of the Ring

ThibaultFy · 2023-08-31T15:19:18Z

/e2e --refs substrafl=chore/bump-to-pydantic-2 substra=chore/bump-to-pydantic-2 substra-tests=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon

Owlfred · 2023-08-31T15:19:20Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed / distributed-sdk,substrafl: ❌
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

“Success is not final; failure is not fatal: It is the courage to continue that counts.” ―- Winston S. Churchill

ThibaultFy · 2023-08-31T15:48:35Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon

Owlfred · 2023-08-31T15:48:37Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed / distributed-sdk,substrafl: ❌
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

Too bad.

ThibaultFy · 2023-08-31T16:16:47Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2,substra-backend=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon

Owlfred · 2023-08-31T16:16:49Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed / distributed-sdk,substrafl: ❌
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

Too bad.

ThibaultFy · 2023-09-01T09:10:10Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2,substra-backend=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon

Owlfred · 2023-09-01T09:10:13Z

End to end tests: ❌ FAILURE

“You shall not pass!” ― Gandalf, The Lord of the Rings, The Fellowship of the Ring

ThibaultFy · 2023-09-01T15:26:48Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2,substra-backend=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon

Owlfred · 2023-09-01T15:26:50Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed / distributed-sdk,substrafl: ❌
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

“It’s just a flesh wound.” ― The Black Knight, Monty Python and the Holy Grail

ThibaultFy · 2023-09-06T07:54:04Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2,substra-backend=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon --mode standalone

Owlfred · 2023-09-06T07:54:07Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed: ⏭️
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

Sorry, try again.

linear · 2023-09-06T08:10:44Z

FL-1068 Support pydantic v2.0

Context

Following the recent release of pydantic v2.0 and related changelog, the code need to be adapted. In particular, some methods have been deprecated like validator and root_validator, resulting in multiple CI failures.

More details here.

Specification

Usages of pydantic in substra SDK need to be adapted so they comply with v2.0
Usages of pydantic in substrafl library need to be adapted so they comply with v2.0

Acceptance criteria

Tests passing on substra, substrafl with pydantic v2.0

Documentation examples relying on both substra and substrafl running with pydantic v2.0

CI passing on e2e distributed, standalone, benchmark tests and documentation examples with pydantic v2.0

ThibaultFy · 2023-09-06T11:39:55Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon --mode standalone

Owlfred · 2023-09-06T11:39:58Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed: ⏭️
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

Oh well.

ThibaultFy · 2023-09-06T14:23:57Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2,substra-backend=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon --mode standalone

Owlfred · 2023-09-06T14:24:03Z

End to end tests: ❌ FAILURE

Jobs status:

Camelyon / standalone-camelyon: ❌
Dispatch Jobs: ✔️
Distributed: ⏭️
MNIST / standalone-mnist: ❌
Standalone / standalone-sdk,substrafl: ❌

Not this time.

ThibaultFy · 2023-09-06T15:16:27Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2,substra-backend=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist,camelyon --mode standalone

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

SdgJlbl

Thanks for the upgrade 🙏

setup.py

SdgJlbl · 2023-09-07T08:57:13Z

substra/__version__.py

@@ -1 +1 @@
-__version__ = "0.47.0"
+__version__ = "0.47.0.dev2"


Comment to change that in the merged version (I guess it's to get e2e tests passing)

substra/sdk/backends/local/backend.py

substra/sdk/models.py

substra/sdk/schemas.py

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

ThibaultFy · 2023-09-07T12:12:21Z

/e2e --refs substrafl=chore/bump-to-pydantic-2,substra=chore/bump-to-pydantic-2,substra-tests=chore/bump-to-pydantic-2,substra-backend=chore/bump-to-pydantic-2 --tests sdk,substrafl --benchmarks mnist --mode standalone

Owlfred · 2023-09-07T12:12:23Z

End to end tests: ✔️ SUCCESS

Awesome sauce!

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

SdgJlbl · 2023-09-07T14:17:24Z

substra/sdk/backends/local/backend.py

@@ -602,7 +602,10 @@ def _output_from_spec(outputs: Dict[str, schemas.ComputeTaskOutputSpec]) -> Dict


 def _schemas_list_to_models_list(inputs: Any, model: Any) -> Any:
-    return [model.model_validate(input_schema.model_dump()) for input_schema in inputs]
+    if inputs:


can inputs be None?
If it's always an iterable, you don't need the conditional

Yes, for instance in TaskSpec, inputs is given as "optional". In practice, it does not looks lite to be none sometimes but I added the condition to match the typing

minor but I feel the flow is slightly more readable with

if not inputs: return [] return [model.model_validate(input_schema.model_dump()) for input_schema in inputs]

(reduce nested statements)

SdgJlbl

Thanks, don't forget to adjust the version number

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

# Companion PR - Substra/substra#375 (main) - Substra/substrafl#159 - Substra/substra-tests#272 - #718 --------- Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

ThibaultFy force-pushed the chore/bump-to-pydantic-2 branch from 429fc44 to ac833b6 Compare August 30, 2023 09:24

ThibaultFy changed the title ~~chore: bump-pydantic tool~~ chore: bump-pydantic to 2.3.0 Aug 31, 2023

This comment was marked as outdated.

Sign in to view

This was referenced Aug 31, 2023

chore: bump pydantic to 2.3.0 Substra/substrafl#159

Merged

chore: bump to pydantic 2.3.0 Substra/substra-tests#272

Merged

ThibaultFy added 13 commits September 7, 2023 10:28

bump-pydantic tool

b679c23

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

bump to pydantic 2.0

ab2b9a6

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: pin pydantic

78364fc

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: isort

87d00f0

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

fix: tests

ff92bca

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

doc

2ca4b11

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: changelog

9c2b539

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

temp

8b399a9

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: release dev1

dff9916

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: more bumping

23c531a

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: release dev2

83140f5

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: metatada only string

46fa5d3

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: mchangelog

1110146

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

ThibaultFy force-pushed the chore/bump-to-pydantic-2 branch from 011535a to 1110146 Compare September 7, 2023 08:29

ThibaultFy marked this pull request as ready for review September 7, 2023 08:36

ThibaultFy requested a review from a team as a code owner September 7, 2023 08:36

ThibaultFy mentioned this pull request Sep 7, 2023

chore: bump to pydantic 2.3.0 Substra/substra-backend#718

Merged

SdgJlbl reviewed Sep 7, 2023

View reviewed changes

chore: metadata only string

127921e

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

fix: schemas list to model list

4dee597

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

ThibaultFy requested a review from SdgJlbl September 7, 2023 14:10

SdgJlbl reviewed Sep 7, 2023

View reviewed changes

SdgJlbl approved these changes Sep 8, 2023

View reviewed changes

ThibaultFy added 2 commits September 8, 2023 09:59

chore: rewrite schema list to model list

0607eb7

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

chore: back to normal version

a8223b4

Signed-off-by: ThibaultFy <50656860+ThibaultFy@users.noreply.github.com>

ThibaultFy merged commit 08c99ad into main Sep 8, 2023

ThibaultFy deleted the chore/bump-to-pydantic-2 branch September 8, 2023 14:00

		@@ -1 +1 @@
		__version__ = "0.47.0"
		__version__ = "0.47.0.dev2"

chore: bump-pydantic to 2.3.0 #375

chore: bump-pydantic to 2.3.0 #375

Conversation

ThibaultFy commented Jul 24, 2023 • edited Loading

Companion PR

ThibaultFy commented Aug 31, 2023

This comment was marked as outdated.

ThibaultFy commented Aug 31, 2023

Owlfred commented Aug 31, 2023 • edited Loading

ThibaultFy commented Aug 31, 2023

Owlfred commented Aug 31, 2023

ThibaultFy commented Aug 31, 2023

Owlfred commented Aug 31, 2023 • edited Loading

ThibaultFy commented Aug 31, 2023

Owlfred commented Aug 31, 2023 • edited Loading

ThibaultFy commented Aug 31, 2023

Owlfred commented Aug 31, 2023 • edited Loading

ThibaultFy commented Aug 31, 2023

Owlfred commented Aug 31, 2023 • edited Loading

ThibaultFy commented Sep 1, 2023

Owlfred commented Sep 1, 2023 • edited Loading

ThibaultFy commented Sep 1, 2023

Owlfred commented Sep 1, 2023 • edited Loading

ThibaultFy commented Sep 6, 2023

Owlfred commented Sep 6, 2023 • edited Loading

linear bot commented Sep 6, 2023

Context

Specification

Acceptance criteria

ThibaultFy commented Sep 6, 2023

Owlfred commented Sep 6, 2023 • edited Loading

ThibaultFy commented Sep 6, 2023

Owlfred commented Sep 6, 2023 • edited Loading

ThibaultFy commented Sep 6, 2023

SdgJlbl left a comment

Choose a reason for hiding this comment

SdgJlbl Sep 7, 2023

Choose a reason for hiding this comment

ThibaultFy commented Sep 7, 2023

Owlfred commented Sep 7, 2023 • edited Loading

SdgJlbl Sep 7, 2023

Choose a reason for hiding this comment

ThibaultFy Sep 7, 2023

Choose a reason for hiding this comment

SdgJlbl Sep 8, 2023

Choose a reason for hiding this comment

SdgJlbl left a comment

Choose a reason for hiding this comment

ThibaultFy commented Jul 24, 2023 •

edited

Loading

Owlfred commented Aug 31, 2023 •

edited

Loading

Owlfred commented Aug 31, 2023 •

edited

Loading

Owlfred commented Aug 31, 2023 •

edited

Loading

Owlfred commented Aug 31, 2023 •

edited

Loading

Owlfred commented Aug 31, 2023 •

edited

Loading

Owlfred commented Sep 1, 2023 •

edited

Loading

Owlfred commented Sep 1, 2023 •

edited

Loading

Owlfred commented Sep 6, 2023 •

edited

Loading

Owlfred commented Sep 6, 2023 •

edited

Loading

Owlfred commented Sep 6, 2023 •

edited

Loading

Owlfred commented Sep 7, 2023 •

edited

Loading