feat: upgrade to pydantic v2 #160

thoas · 2023-07-01T09:06:42Z

Tests passed locally but there are still two warnings:

taskiq/serialization.py:299
  /Users/thoas/Sites/Python/taskiq/taskiq/serialization.py:299: PydanticDeprecatedSince20: The `validate_arguments` method is deprecated; use `validate_call` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.0/migration/
    @pydantic.validate_arguments(config={"arbitrary_types_allowed": True})

taskiq/serialization.py:314
  /Users/thoas/Sites/Python/taskiq/taskiq/serialization.py:314: PydanticDeprecatedSince20: The `validate_arguments` method is deprecated; use `validate_call` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.0/migration/
    @pydantic.validate_arguments(config={"arbitrary_types_allowed": True})

I can't get pydantic.validate_call to work properly, it fails with this traceback:

ImportError while loading conftest '/Users/thoas/Sites/Python/taskiq/tests/conftest.py'.
tests/conftest.py:5: in <module>
    from taskiq.abc.broker import AsyncBroker
taskiq/__init__.py:4: in <module>
    from taskiq.abc.broker import AsyncBroker, AsyncTaskiqDecoratedTask
taskiq/abc/__init__.py:2: in <module>
    from taskiq.abc.broker import AsyncBroker
taskiq/abc/broker.py:32: in <module>
    from taskiq.result_backends.dummy import DummyResultBackend
taskiq/result_backends/dummy.py:3: in <module>
    from taskiq.abc.result_backend import AsyncResultBackend
taskiq/abc/result_backend.py:4: in <module>
    from taskiq.result import TaskiqResult
taskiq/result.py:8: in <module>
    from taskiq.serialization import exception_to_python, prepare_exception, ExceptionRepr
taskiq/serialization.py:314: in <module>
    @pydantic.validate_call(config={"arbitrary_types_allowed": True})
.venv/lib/python3.11/site-packages/pydantic/validate_call.py:46: in validate
    return _validate_call.ValidateCallWrapper(function, config, validate_return)  # type: ignore
.venv/lib/python3.11/site-packages/pydantic/_internal/_validate_call.py:67: in __init__
    simplified_schema = inline_schema_defs(schema)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:515: in inline_schema_defs
    return _simplify_schema_references(schema, inline=True)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:473: in _simplify_schema_references
    schema = walk_core_schema(schema, count_refs)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:392: in walk_core_schema
    return f(schema.copy(), _dispatch)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:461: in count_refs
    return recurse(s, count_refs)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:188: in walk
    return f(schema.copy(), self._walk)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:461: in count_refs
    return recurse(s, count_refs)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:191: in _walk
    schema = self._schema_type_to_method[schema['type']](schema, f)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:369: in handle_call_schema
    schema['arguments_schema'] = self.walk(schema['arguments_schema'], f)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:188: in walk
    return f(schema.copy(), self._walk)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:461: in count_refs
    return recurse(s, count_refs)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:191: in _walk
    schema = self._schema_type_to_method[schema['type']](schema, f)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:359: in handle_arguments_schema
    replaced_param['schema'] = self.walk(param['schema'], f)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:188: in walk
    return f(schema.copy(), self._walk)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:461: in count_refs
    return recurse(s, count_refs)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:191: in _walk
    schema = self._schema_type_to_method[schema['type']](schema, f)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:199: in _handle_other_schemas
    schema['schema'] = self.walk(schema['schema'], f)  # type: ignore
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:188: in walk
    return f(schema.copy(), self._walk)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:461: in count_refs
    return recurse(s, count_refs)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:191: in _walk
    schema = self._schema_type_to_method[schema['type']](schema, f)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:279: in handle_union_schema
    schema['choices'] = [self.walk(v, f) for v in schema['choices']]
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:279: in <listcomp>
    schema['choices'] = [self.walk(v, f) for v in schema['choices']]
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:188: in walk
    return f(schema.copy(), self._walk)
.venv/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:469: in count_refs
    recurse(all_defs[ref], count_refs)
E   KeyError: 'taskiq.serialization.ExceptionRepr:5589281440'

and it annoys me 😒

codecov-commenter · 2023-07-01T21:51:25Z

Codecov Report

Merging #160 (4c6808e) into master (04dd68d) will increase coverage by 6.88%.
The diff coverage is 74.65%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##           master     #160      +/-   ##
==========================================
+ Coverage   61.94%   68.82%   +6.88%     
==========================================
  Files          37       45       +8     
  Lines         938     1360     +422     
==========================================
+ Hits          581      936     +355     
- Misses        357      424      +67

Impacted Files	Coverage Δ
taskiq/abc/middleware.py	`100.00% <ø> (ø)`
taskiq/abc/result_backend.py	`100.00% <ø> (ø)`
taskiq/cli/scheduler/run.py	`0.00% <0.00%> (ø)`
taskiq/cli/watcher.py	`0.00% <ø> (ø)`
taskiq/cli/worker/log_collector.py	`100.00% <ø> (ø)`
taskiq/cli/worker/process_manager.py	`0.00% <0.00%> (ø)`
taskiq/cli/worker/run.py	`0.00% <0.00%> (ø)`
taskiq/events.py	`100.00% <ø> (ø)`
taskiq/result/v1.py	`0.00% <0.00%> (ø)`
taskiq/result_backends/dummy.py	`100.00% <ø> (ø)`
... and 26 more

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

s3rius · 2023-07-01T22:57:02Z

@thoas, hi and thanks for this pull request. Tests pass fine, but this PR contains some minor styling issues.

You can automatically fix them using command pre-commit run -a. It will fix what it can and show you places where you need to make changes manually.

I really appreciate your contribution, but I have a question. Can we support both pydanticV1 and pydanticV2? Because not all users will migrate to newer version.

thoas · 2023-07-02T16:51:06Z

Thank you for the review, I will update the styling and you are right we should keep the support for pydantic v1, I will work on that.

thoas · 2023-07-03T09:09:27Z

@s3rius I have added a compatibility layer for both pydantic v1 and v2.

Let me know if you need anything else.

PS: I think you should authorize by default github actions

s3rius

I really like the way you did it. But I have some suggestions to make it even 🌠 better 🌠 .

Hope you're going to like it. I really appreciate your contributions. Thanks.

taskiq/compat.py

taskiq/result.py

s3rius · 2023-07-03T23:33:11Z

@thoas, about gh actions. You'll be able to run it automatically after first merged PR. Hope it will happen soon, btw. ❤️

thoas · 2023-07-04T07:22:36Z

@thoas, about gh actions. You'll be able to run it automatically after first merged PR. Hope it will happen soon, btw. ❤️

I made the changes, thank you again for your review and don't hesitate.

s3rius

Looks really good. One small issue with pydantic version. With current constraint it won't allow you to install pydantic with version bigger than 2.0.

pyproject.toml

thoas · 2023-07-05T20:05:41Z

Looks really good. One small issue with pydantic version. With current constraint it won't allow you to install pydantic with version bigger than 2.0.

fixed and I also fixed the typing compatibility for Python 3.9

thoas · 2023-07-05T20:11:02Z

... and here we go again for Python 3.8 ❤️

sorry for the noise (I only work with Python 3.11)

s3rius

Congratulation with your first merged request in taskiq. Thank you for your contribution!

s3rius · 2023-07-05T20:18:00Z

I will release it after we merge some other requests.

thoas · 2023-07-05T20:20:33Z

Thank you @s3rius for your time and review, I can now run gh actions by myself!

feat: upgrade to pydantic v2

3c692a8

thoas added 2 commits July 3, 2023 09:12

chore: linters

9e3b9a4

feat: add compat for both pydantic v1 and v2

f4d0007

s3rius requested changes Jul 3, 2023

View reviewed changes

taskiq/compat.py Outdated Show resolved Hide resolved

taskiq/compat.py Outdated Show resolved Hide resolved

taskiq/result.py Outdated Show resolved Hide resolved

thoas force-pushed the feat/pydantic-v2 branch 2 times, most recently from cd86248 to ea90e72 Compare July 4, 2023 07:20

thoas requested a review from s3rius July 4, 2023 07:29

chore: review

924d7df

thoas force-pushed the feat/pydantic-v2 branch from ea90e72 to 924d7df Compare July 4, 2023 07:35

s3rius requested changes Jul 5, 2023

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

thoas added 2 commits July 5, 2023 22:00

fix: pydantic version

4159ce3

fix: typing compatibility

6f9d5e4

thoas requested a review from s3rius July 5, 2023 20:05

fix: typing compatibility (python 3.8)

4c6808e

s3rius changed the base branch from master to develop July 5, 2023 20:14

s3rius approved these changes Jul 5, 2023

View reviewed changes

s3rius merged commit f323fcd into taskiq-python:develop Jul 5, 2023

thoas deleted the feat/pydantic-v2 branch July 5, 2023 20:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: upgrade to pydantic v2 #160

feat: upgrade to pydantic v2 #160

thoas commented Jul 1, 2023 •

edited

Loading

codecov-commenter commented Jul 1, 2023 •

edited

Loading

s3rius commented Jul 1, 2023

thoas commented Jul 2, 2023

thoas commented Jul 3, 2023 •

edited

Loading

s3rius left a comment

s3rius commented Jul 3, 2023 •

edited

Loading

thoas commented Jul 4, 2023

s3rius left a comment

thoas commented Jul 5, 2023

thoas commented Jul 5, 2023 •

edited

Loading

s3rius left a comment

s3rius commented Jul 5, 2023

thoas commented Jul 5, 2023

feat: upgrade to pydantic v2 #160

feat: upgrade to pydantic v2 #160

Conversation

thoas commented Jul 1, 2023 • edited Loading

codecov-commenter commented Jul 1, 2023 • edited Loading

Codecov Report

s3rius commented Jul 1, 2023

thoas commented Jul 2, 2023

thoas commented Jul 3, 2023 • edited Loading

s3rius left a comment

Choose a reason for hiding this comment

s3rius commented Jul 3, 2023 • edited Loading

thoas commented Jul 4, 2023

s3rius left a comment

Choose a reason for hiding this comment

thoas commented Jul 5, 2023

thoas commented Jul 5, 2023 • edited Loading

s3rius left a comment

Choose a reason for hiding this comment

s3rius commented Jul 5, 2023

thoas commented Jul 5, 2023

thoas commented Jul 1, 2023 •

edited

Loading

codecov-commenter commented Jul 1, 2023 •

edited

Loading

thoas commented Jul 3, 2023 •

edited

Loading

s3rius commented Jul 3, 2023 •

edited

Loading

thoas commented Jul 5, 2023 •

edited

Loading