Skip to content

fix(redis): make xadd fields use SupportsItems #10780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Feb 17, 2024

Conversation

sileht
Copy link
Contributor

@sileht sileht commented Sep 26, 2023

Using an InvariantMapping is wrong, the type of each values of the dict
can be different.

This change uses the SupportsItems Protocol instead to match as much as possible the
code.

@github-actions

This comment has been minimized.

@AlexWaygood
Copy link
Member

Why?

@sileht
Copy link
Contributor Author

sileht commented Sep 26, 2023

Why?

You're too fast :) I didn't finish to write the description correctly.

That said, I just saw a discussion to drop types-redis, so not sure you continue to accept change to this typing.

(btw, redis 5.x builtin typing is far to be usable due to redis/redis-py#2933)

@srittau
Copy link
Collaborator

srittau commented Sep 26, 2023

While I'm fine with replacing Mapping with an appropriate covariant protocol (we have a few of those available in _typeshed),we not should just revert this commit and replace the existing annotations with no annotations.

@sileht sileht changed the title revert: "redis: Improve typing of xadd/xdel (#10531)" fix(redis): make xadd fields CovariantMapping Sep 27, 2023
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@sileht sileht marked this pull request as draft September 27, 2023 08:59
@sileht sileht force-pushed the cleanup-xadd branch 2 times, most recently from 98eed3a to 3744db3 Compare September 27, 2023 09:07
@sileht sileht changed the title fix(redis): make xadd fields CovariantMapping fix(redis): make xadd fields use SupportItems Sep 27, 2023
@sileht sileht changed the title fix(redis): make xadd fields use SupportItems fix(redis): make xadd fields use SupportsItems Sep 27, 2023
@sileht sileht marked this pull request as ready for review September 27, 2023 09:09
@sileht
Copy link
Contributor Author

sileht commented Sep 27, 2023

I propose another solution.

@github-actions

This comment has been minimized.

@srittau
Copy link
Collaborator

srittau commented Sep 27, 2023

Hmm, redis-py seems to explicitly check for a dict:

https://github.com/redis/redis-py/blob/cc4bc1a544d1030aec1696baef2861064fa8dd1c/redis/commands/core.py#L3547

@sileht Can you give an example where type checkers have problems with the current annotations?

@AlexWaygood
Copy link
Member

AlexWaygood commented Sep 27, 2023

Hmm, redis-py seems to explicitly check for a dict:

FWIW, it does look to me like the current annotations we have are possibly the worst of both worlds. We don't get nice covariant types, since Mapping is invariant in its key types -- but we're also not particularly accurate, since redis-py does actually require a dict (for some reason) rather than just any old Mapping. So I think we should either change the stubs to be more accurate (Mapping -> dict), or change them to be more user-friendly (Mapping -> SupportsItems, or some other protocol). The status quo doesn't seem great.

I'd also be curious for @sileht's use case where the current annotations are causing a problem, though; it's hard to solve a problem if we don't know exactly what the problem is.

@sileht
Copy link
Contributor Author

sileht commented Sep 27, 2023

Currently, you can't write thing like:

class RedisStreamData(typing.TypedDict):
    foo: str
    bar: bytes

redis.xadd("stream", RedisStreamData({"foo": "bar", "bar": b"foo"}))

Copy link
Collaborator

@srittau srittau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering @sileht's valid use case and the fact that we prefer false negatives over false positives, I think this change is the best we can do.

@sileht Could you add a comment before the fields argument, something like "Only accepts dict objects, but for variance reasons we use a looser annotation." (Maybe you can come up with a better wording.)

Using an InvariantMapping is wrong, the type of each values of the dict
can be different.

This change uses a Protocol instead to match as much as possible the
code:

```
if not isinstance(fields, dict) or len(fields) == 0:
    raise DataError("XADD fields must be a non-empty dict")
for pair in fields.items():
    pieces.extend(pair)
```
@github-actions

This comment has been minimized.

This comment has been minimized.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

This comment has been minimized.

This comment has been minimized.

@JelleZijlstra
Copy link
Member

Pyright is now unhappy because xadd doesn't have a return type. If I'm reading https://redis.io/commands/xadd/ right, it should return a str most of the time, but possibly None if nomkstream is True.

Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the easier solution :)

@AlexWaygood
Copy link
Member

Pyright is now unhappy because xadd doesn't have a return type. If I'm reading redis.io/commands/xadd right, it should return a str most of the time, but possibly None if nomkstream is True.

Oh I just ignored it for now. But if you have a better fix, feel free to push it :)

Copy link
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

@AlexWaygood AlexWaygood merged commit 522f4bc into python:main Feb 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants