ENH: Support groupby.ewm operations

[mroeschke]

• closes ENH: support .groupby().ewm() directly #16037
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

Additionally enables a Numba engine for groupby(...).ewm(...).mean(...)

In [1]: df = pd.DataFrame({"A": range(10_000), "B": range(10_000)})

In [2]: gb_ewm = df.groupby("A").ewm(com=1.0)

--cache time first
In [3]: %timeit -r 1 -n 1 gb_ewm.mean(engine='numba')
1.02 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)

In [4]: %timeit gb_ewm.mean(engine='numba')
578 ms ± 28.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [5]: %timeit df.groupby('A').apply(lambda x: x.ewm(com=1.0).mean())
3.43 s ± 208 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [6]: %timeit gb_ewm.mean(engine='cython')
4.19 s ± 204 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

[pep8speaks]

Hello @mroeschke! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-11-17 21:58:52 UTC

[jreback]

initial scan looks good, can you add some asv's for this

[jreback]

can you show a timeit result of the new groupby.ewm vs groupby.apply(.....ewm) for posterity in the top of this PR

[mroeschke]

Posted the timings. I think there is a slight slowdown using the cython engine with the convenience wrapper since we have to reconstruct the EWM object in the apply loop.

[jreback]

what is the timing for master (before this PR for the groupby apply)

i would have expected the numba one to be way after here

[mroeschke]

Updated the snippet above; the numba timing including a caching run which skewed the benchmark.

The groupby apply timing is similar on master.

[jreback]

question and request for tests

[jreback]

can you also compare to groupby('A').apply(lambda x: getattr(x.ewm(com=1.0), method)) ?

[jreback]

hah i think its expected actually :->

[jreback]

or if you think its a bug, then open an issue :->

[mroeschke]

Ah right ewm acts as a transform so the index should be the same.

[jreback]

thanks @mroeschke really nice