For itertools.accumulate, erroneously: requires func if initial specified; allows initial to be positional

[tonyelewis]

Based on the documentation for the ≥3.8 version of itertools.accumulate, I'd expect this to be valid:

itertools.accumulate([4, 5, 6], initial=0)

…and this to be invalid (due to initial being a keyword-only argument):

itertools.accumulate([4, 5, 6], lambda x, y: x + y, 0)

…and this expectation is reflected by running it under Python 3.8.5. But mypy (0.782) rejects the former with:

a.py:6: error: No overload variant of "accumulate" matches argument types "List[int]", "int"
a.py:6: note: Possible overload variants:
a.py:6: note:     def [_T] accumulate(iterable: Iterable[_T], func: Callable[[_T, _T], _T] = ...) -> Iterator[_T]
a.py:6: note:     def [_T, _S] accumulate(iterable: Iterable[_T], func: Callable[[_S, _T], _S], initial: Optional[_S]) -> Iterator[_S]
Found 1 error in 1 file (checked 1 source file)

…and accepts the latter. The above error disappears if func is specified.

I'm a type-annotation novice, but it looks to me like this could be addressed with:

diff --git a/stdlib/3/itertools.pyi b/stdlib/3/itertools.pyi
index 8a918bdc..f7142495 100644
--- a/stdlib/3/itertools.pyi
+++ b/stdlib/3/itertools.pyi
@@ -23,7 +23,7 @@ if sys.version_info >= (3, 8):
     @overload
     def accumulate(iterable: Iterable[_T], func: Callable[[_T, _T], _T] = ...) -> Iterator[_T]: ...
     @overload
-    def accumulate(iterable: Iterable[_T], func: Callable[[_S, _T], _S], initial: Optional[_S]) -> Iterator[_S]: ...
+    def accumulate(iterable: Iterable[_T], func: Callable[[_S, _T], _S] = ..., *, initial: Optional[_S]) -> Iterator[_S]: ...
 
 else:
     def accumulate(iterable: Iterable[_T], func: Callable[[_T, _T], _T] = ...) -> Iterator[_T]: ...

That seems to fix both issues for me. Does it look right?

I cloned the repo and started looking at the test setup, but I can't see: is there a way I can add testcases that reproduce the false positive and false negative that I'm aiming to fix?

[hauntsaninja]

Looks almost right. It should be def accumulate(iterable: Iterable[_T], func: Optional[Callable[[_S, _T], _S]] = ..., *, initial: Optional[_S]) -> Iterator[_S]: ..., since list(itertools.accumulate([4, 5, 6], None)) is valid.

typeshed doesn't really do tests like that, see #1339 and #754 for some discussion. We run stubtest, which catches this issue but we've whitelisted the large backlog of errors while we whittle them down (a lot of progress has been made + this is an easy way to contribute to typeshed if you're looking!):

λ python3.9 -m mypy.stubtest itertools --concise
itertools.accumulate is inconsistent, runtime argument "func" has a default value of type None, which is incompatible with stub argument type Union[def (_T`-1, _T`-1) -> _T`-1, def (_S`-2, _T`-1) -> _S`-2]. This is often caused by overloads failing to account for explicitly passing in the default value.
itertools.accumulate is inconsistent, stub argument "initial" is not keyword-only
...

[tonyelewis]

Yep - my motivating cases now work for me. Thanks very much @hauntsaninja and @srittau for getting this all sorted. Apologies I didn't respond sooner.