BUG:Pandas 1.0.3 → 1.1.1 behavior change on DataFrame.apply() whith raw option and func returning string

[m-hunsicker]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

df_1 = pd.DataFrame({'A': ["aa","bbb"]})
df_2 = pd.DataFrame({'A': ["bbb","aa"]})

def get_value(array):
        return array[0]

r_1 = df_1.apply(get_value, axis=1, raw=True)
r_2 = df_2.apply(get_value, axis=1, raw=True)

print(r_1)
print(r_2)

Output

0 aa
1 bb
dtype: object
0 bbb
1 aa
dtype: object

Problem description

The results are truncated when the smallest strings is first. However, when the result (eg. array[0]) is printed before the return of the func, it's displays the correct value.
(This issue occurred when using apply with the raw option for a function using several columns)

Expected Output

0 aa
1 bbb
dtype: object
0 bbb
1 aa
dtype: object

Output of pd.show_versions()

Pandas 1.1.1
Numpy 1.19.1

[asishm]

#34913

cc @jbrockmendel

7d0ee96 is the first bad commit
commit 7d0ee96
Author: jbrockmendel jbrockmendel@gmail.com
Date: Sat Jun 20 16:16:12 2020 -0700

REF: dont use compute_reduction (#34913)

[incredibleray]

fascinating, I can reproduce this on master. (I did python setup.py install on the latest master, not sure if this is the right way)

Like @m-hunsicker said, make the first string longer, or setting raw=False will fix it

@m-hunsicker If you are willing, can you share what are you trying to do when you discovered this bug? Thanks.

[simonjayhawkins]

it appears that the underlying issue is numpy/numpy#8352.

> c:\users\simon\pandas\pandas\core\apply.py(220)apply_raw()
-> result = np.apply_along_axis(self.f, self.axis, self.values)
(Pdb) self.f
<function get_value at 0x000001F31A9E7280>
(Pdb) self.axis
1
(Pdb) self.values
array([['aa'],
       ['bbb']], dtype=object)
(Pdb) np.apply_along_axis(self.f, self.axis, self.values)
array(['aa', 'bb'], dtype='<U2')

values is indeed passed as object dtype to np.apply_along_axis.

As a workaround, the function can force an object dtype result with

>>> pd.__version__
'1.2.0.dev0+487.g27aae225e8'
>>>
>>> def get_value(array):
...     return np.array(array[0], dtype=object)
...
>>>
>>> df_1 = pd.DataFrame({"A": ["aa", "bbb"]})
>>> df_1.apply(get_value, axis=1, raw=True)
0     aa
1    bbb
dtype: object
>>>