BUG: {expanding,rolling}_{cov,corr} don't handle arguments with different index sets properly

[seth-p]

Closes #7512.

[jreback]

needs a release note in v0.14.1 (in bug fix section)

[jreback]

why was this changed?

[seth-p]

The expanding_xxx() functions need window = len(arg1) + len(arg2) rather than = max(len(arg1), len(arg2)) because once arg1 and arg2 are merged the result could be of length up to len(arg1) + len(arg2) if there is no overlap between the index (or major_axis) of arg1 and arg2. If the index of one is a subset of the index of the other, then max() will be fine; but in general when neither is a subset of the other max() will be too small.

[seth-p]

For example, without this fix, see the following; whereas the final value should be 1.

In [2]: s1 = Series([7,8,10], index=[0,1,3])

In [3]: s2 = Series([7,9,10], index=[0,2,3])

In [4]: expanding_corr(s1, s2)
Out[4]:
0   NaN
1   NaN
2   NaN
3   NaN
dtype: float64

There isn't quite a test for this. I think I should add one.

[jreback]

why would u not align the 2 objects before you start, then all this is moot

eg try doing

df1, df2 = df1.align(df2)

and see what that results

[seth-p]

No idea. This is the way it was written, and I didn't want to modify things any more than necessary.
At the very least I will add a test, and will see if the .align() thing works. Probably won't get it done until tomorrow.

[seth-p]

Replacing:

    window = len(arg1) + len(arg2)

with:

    arg1, arg2 = arg1.align(arg2)
    window = len(arg1)

in expanding_cov/corr() works for series, but not for data frames: it not only merges the index, but also the columns, thus producing the wrong result for pairwise=True calls -- e.g. if arg1.columns=['A','B'] and arg2.columns=['X','Y'], then while the result should have major/minor axes ['A','B'] and ['X','Y'], instead it ends up with both being ['A','B','X','Y'].

At any rate, since the two objects end up being aligned anyway in _flex_binary_moment(), aligning them in expanding_cov/corr is just redundant.

I updated the code to include several additional tests, but left window = len(arg1) + len(arg2) unchanged.

[jreback]

ok, just ensuring that this is a bug fix only (and not actually changing the result; which is technically wrong currently when the indices are not aligned).

[jreback]

thanks