Series.shift() doesn't work for categorical type

[lminer]

Not sure if this is intentional, but Series.shift() won't run with categorical dtypes:

ser = pd.Series(['a', 'b', 'c', 'd'], dtype="category")
ser.shift(1)
Traceback (most recent call last):

  File "<ipython-input-15-1a7536b0af06>", line 1, in <module>
    ser.shift(1)

  File "/.../pandas/core/generic.py", line 3394, in shift
    new_data = self._data.shift(periods=periods, axis=block_axis)

  File "/.../pandas/core/internals.py", line 2533, in shift
    return self.apply('shift', **kwargs)

  File "/.../pandas/core/internals.py", line 2497, in apply
    applied = getattr(b, f)(**kwargs)

  File "/.../pandas/core/internals.py", line 893, in shift
    new_values, fill_value = com._maybe_upcast(self.values)

  File "/.../pandas/core/common.py", line 1218, in _maybe_upcast
    new_dtype, fill_value = _maybe_promote(dtype, fill_value)

  File "/.../pandas/core/common.py", line 1124, in _maybe_promote
    if issubclass(np.dtype(dtype).type, compat.string_types):

TypeError: data type not understood

[shoyer]

This simply hasn't been implemented, but otherwise was not intentional. Help would be appreciated if you're interested in putting together a PR. The place to get started (I believe) would be to implement the shift method on CategoricalBlock in pandas.core.internals.

[jreback]

here's basically what you would do

In [2]: s = Series(list('aabbcde'),dtype='category')

In [3]: s
Out[3]: 
0    a
1    a
2    b
3    b
4    c
5    d
6    e
dtype: category
Categories (5, object): [a < b < c < d < e]

In [4]: s.values
Out[4]: 
[a, a, b, b, c, d, e]
Categories (5, object): [a < b < c < d < e]

In [5]: s.values.codes  
Out[5]: array([0, 0, 1, 1, 2, 3, 4], dtype=int8)

In [6]: np.roll(s.values.codes,len(s)-1,axis=0)
Out[6]: array([0, 1, 1, 2, 3, 4, 0], dtype=int8)

In [7]: codes = np.roll(s.values.codes,len(s)-1,axis=0)

In [8]: codes[-1] = -1

In [11]: pd.Categorical(codes,categories=s.values.categories,fastpath=True)
Out[11]: 
[a, b, b, c, d, e, NaN]
Categories (5, object): [a, b, c, d, e]

you would use the Block.shift method (and pass the codes to it for the actual shifting), then wrap it back to a catetgorical (their is a method for that too). Should be pretty straightforward.