-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Fix SQLite Aggregation Protocols #12192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
335e5ef
feat: Improve/fix sqlite aggregration protocols
max-muoto c76c643
Remove default
max-muoto d417ee5
Merge branch 'main' into improve-sqllite-agg-types
max-muoto 694f7b4
Add default back
max-muoto 6b9e3a1
Add test
max-muoto d52702a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 5377741
Fix test
max-muoto 6169219
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 58fc21a
Finalize tests
max-muoto 4e5b605
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 7f41490
Tweak
max-muoto c1ca821
Fix tests
max-muoto 8bea1e0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
import sqlite3 | ||
import sys | ||
|
||
|
||
class WindowSumInt: | ||
def __init__(self) -> None: | ||
self.count = 0 | ||
|
||
def step(self, param: int) -> None: | ||
self.count += param | ||
|
||
def value(self) -> int: | ||
return self.count | ||
|
||
def inverse(self, param: int) -> None: | ||
self.count -= param | ||
|
||
def finalize(self) -> int: | ||
return self.count | ||
|
||
|
||
con = sqlite3.connect(":memory:") | ||
cur = con.execute("CREATE TABLE test(x, y)") | ||
values = [("a", 4), ("b", 5), ("c", 3), ("d", 8), ("e", 1)] | ||
cur.executemany("INSERT INTO test VALUES(?, ?)", values) | ||
|
||
if sys.version_info >= (3, 11): | ||
con.create_window_function("sumint", 1, WindowSumInt) | ||
|
||
con.create_aggregate("sumint", 1, WindowSumInt) | ||
cur.execute( | ||
""" | ||
SELECT x, sumint(y) OVER ( | ||
ORDER BY x ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING | ||
) AS sum_y | ||
FROM test ORDER BY x | ||
""" | ||
) | ||
con.close() | ||
|
||
|
||
def _create_window_function() -> WindowSumInt: | ||
return WindowSumInt() | ||
|
||
|
||
# A callable should work as well. | ||
if sys.version_info >= (3, 11): | ||
con.create_window_function("sumint", 1, _create_window_function) | ||
con.create_aggregate("sumint", 1, _create_window_function) | ||
|
||
# With num_args set to 1, the callable should not be called with more than one. | ||
|
||
|
||
class WindowSumIntMultiArgs: | ||
def __init__(self) -> None: | ||
self.count = 0 | ||
|
||
def step(self, *args: int) -> None: | ||
self.count += sum(args) | ||
|
||
def value(self) -> int: | ||
return self.count | ||
|
||
def inverse(self, *args: int) -> None: | ||
self.count -= sum(args) | ||
|
||
def finalize(self) -> int: | ||
return self.count | ||
|
||
|
||
if sys.version_info >= (3, 11): | ||
con.create_window_function("sumint", 1, WindowSumIntMultiArgs) | ||
con.create_window_function("sumint", 2, WindowSumIntMultiArgs) | ||
|
||
con.create_aggregate("sumint", 1, WindowSumIntMultiArgs) | ||
con.create_aggregate("sumint", 2, WindowSumIntMultiArgs) | ||
|
||
|
||
class WindowSumIntMismatchedArgs: | ||
def __init__(self) -> None: | ||
self.count = 0 | ||
|
||
def step(self, *args: str) -> None: | ||
self.count += 34 | ||
|
||
def value(self) -> int: | ||
return self.count | ||
|
||
def inverse(self, *args: int) -> None: | ||
self.count -= 34 | ||
|
||
def finalize(self) -> str: | ||
return str(self.count) | ||
|
||
|
||
# Since the types for `inverse`, `step`, `finalize`, and `value` are not compatible, the following should fail. | ||
if sys.version_info >= (3, 11): | ||
con.create_window_function("sumint", 1, WindowSumIntMismatchedArgs) # type: ignore | ||
con.create_window_function("sumint", 2, WindowSumIntMismatchedArgs) # type: ignore | ||
|
||
|
||
class AggMismatchedArgs: | ||
def __init__(self) -> None: | ||
self.count = 0 | ||
|
||
def step(self, *args: str) -> None: | ||
self.count += 34 | ||
|
||
def finalize(self) -> int: | ||
return self.count | ||
|
||
|
||
# Since the types for `step` and `finalize` are not compatible, the following should fail. | ||
con.create_aggregate("sumint", 1, AggMismatchedArgs) # type: ignore | ||
con.create_aggregate("sumint", 2, AggMismatchedArgs) # type: ignore |
File renamed without changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From testing things out, it doesn't seem this protocol really works as intended in either MyPy or Pyright. Unless you actually were annotating a lambda perhaps. Due to this, I went ahead and removed it.
Some examples of how it might not work as you would expect:
Pyright Playground
MyPy Playground
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has been part of the initial commit in #7625, while the other protocols already used a function. Maybe @JelleZijlstra remembers why we used an attribute instead of a function here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really remember what I was thinking when I wrote that code, but the annotations proposed in this PR mean that protocol implementations must take
*args
. I am not familiar with how these things are used, but I'd expect concrete implementations to only accept a fixed number of parameters. Maybe that's why I chose to useCallable[...
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried messing around with TypeVarTuples to do that, but had some issues there as well, there might not be a great way, but I'll see if I can figure something out.