Elegant way to generate indexable sliding window timeseries

[spacegoing]

To implement pytorch's DataSet class __get_item__() method, it requires to support the indexing such that dataset[i] can be used to get ith sample.

Say I have a time-series ser:

2017-12-29 14:44:00  69.90
2017-12-29 14:45:00  69.91
2017-12-29 14:46:00  69.87
2017-12-29 14:47:00  69.85
2017-12-29 14:48:00  69.86
2017-12-29 14:49:00  69.92
2017-12-29 14:50:00  69.90
2017-12-29 14:51:00  70.00
2017-12-29 14:52:00  69.97
2017-12-29 14:53:00  69.99
2017-12-29 14:54:00  69.99
2017-12-29 14:55:00  69.85 

Since I need to index into the rolling window. I generate a window length 3 time-series by using:

l3_list = list()
def t(x):
  l3_list.append(x.copy())
ser.rolling(3).apply(t)

l3_list becomes:

[array([69.9 , 69.91, 69.87]),
 array([69.91, 69.87, 69.85]),
 array([69.87, 69.85, 69.86]),
 array([69.85, 69.86, 69.92]),
 array([69.86, 69.92, 69.9 ]),
 array([69.92, 69.9 , 70.  ]),
 array([69.9 , 70.  , 69.97]),
 array([70.  , 69.97, 69.99]),
 array([69.97, 69.99, 69.99]),
 array([69.99, 69.99, 69.85])]

So that I can index in l3_list. Namely l3_list[i] is the ith sliding window. Is there a more memory efficient way to do this?

[TomAugspurger]

IIUC, an API to get values for the ith window would suite your needs? This would be analgous to get_group from groupby.

I'm not sure if that information is available. You could look through the pandas/core/window.py source code to see if it's doable.

[mroeschke]

Since there is a rolling.__iter__ method now, you can get the ith window by iterating over the object and yielding at the ith element.

I believe that should suffice the requirement and not necessary worth an auxiliary method. Can reopen this issue if more discussion is needed.