BUG: Inconsistent return type for groupby-resample-count

[JenDobson]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df1 = pd.DataFrame(index=pd.DatetimeIndex(['2022-1-1 1:00','2022-1-2 1:00','2022-1-2 1:00']),
    data={'category':['A','A','B']})
res1 = df1.groupby(['category']).resample('1D').size()
print(res1)

df2 = pd.DataFrame(index=pd.DatetimeIndex(['2022-1-1 1:00','2022-1-2 1:00','2022-1-2 1:00',
    '2022-1-1 1:00']),data={'category':['A','A','B','B']})

res2 = df2.groupby(['category']).resample('1D').size()
print(res2)

Issue Description

The two dataframes above, df1 and df2, are identical except for one additional data point in df2.

In df1, category A has samples on the first and second day and category B has a sample only on the second day. When I groupby-resample-count I get a Series with multiindex on category and time:

category
A 2022-01-01 1
2022-01-02 1
B 2022-01-02 1

df2 has one additional data point on an existing category and day, so that category B now has a sample on day 1, corresponding to the first time bin in the resample. The return value is now a dataframe with single-index in category and columns corresponding to the time bins:

      2022-01-01  2022-01-02

category
A 1 1
B 1 1

Expected Behavior

I expect res1 and res2 to have the same type: either a multiindex series like res1 or a dataframe like res2.

Installed Versions

INSTALLED VERSIONS

commit : 4bfe3d0
python : 3.8.8.final.0
python-bits : 64
OS : Linux
OS-release : 5.13.0-1021-aws
Version : #23~20.04.2-Ubuntu SMP Thu Mar 31 11:36:15 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.4.2
numpy : 1.20.1
pytz : 2021.3
dateutil : 2.8.2
pip : 21.0.1
setuptools : 58.0.4
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : 1.3.8
lxml.etree : 4.8.0
html5lib : None
pymysql : 1.0.2
psycopg2 : None
jinja2 : 3.1.1
IPython : 7.22.0
pandas_datareader: None
bs4 : 4.9.3
bottleneck : 1.3.4
brotli :
fastparquet : None
fsspec : None
gcsfs : None
markupsafe : 2.0.1
matplotlib : 3.4.3
numba : None
numexpr : 2.8.1
odfpy : None
openpyxl : 3.0.7
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.6.2
snappy : None
sqlalchemy : 1.4.15
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None

[dicristina]

This is present on main. The result should be a series with a MultiIndex.

INSTALLED VERSIONS

commit : ca3e0c8
python : 3.10.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.15.0-25-generic
Version : #25-Ubuntu SMP Wed Mar 30 15:54:22 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.0.0.dev0+971.gca3e0c875f
numpy : 1.23.5
pytz : 2022.7
dateutil : 2.8.2
setuptools : 65.6.3
pip : 22.3.1
Cython : 0.29.32
pytest : 7.2.0
hypothesis : 6.61.0
sphinx : 4.5.0
blosc : 1.11.1
feather : None
xlsxwriter : 3.0.3
lxml.etree : 4.9.2
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : 2.9.5
jinja2 : 3.1.2
IPython : 8.7.0
pandas_datareader: None
bs4 : 4.11.1
bottleneck : 1.3.5
brotli :
fastparquet : 2022.12.0
fsspec : 2022.11.0
gcsfs : 2022.11.0
matplotlib : 3.6.2
numba : 0.56.4
numexpr : 2.8.4
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : 9.0.0
pyreadstat : 1.2.0
pyxlsb : 1.0.10
s3fs : 2022.11.0
scipy : 1.9.3
snappy :
sqlalchemy : 1.4.45
tables : 3.7.0
tabulate : 0.9.0
xarray : 2022.12.0
xlrd : 2.0.1
zstandard : 0.19.0
tzdata : 2022.7
qtpy : None
pyqt5 : None

[dicristina]

The difference between the two examples is how the results are wrapped. Eventually we get to DataFrameGroupBy._wrap_applied_output_series:

pandas/pandas/core/groupby/generic.py

Lines 1339 to 1351 in ca3e0c8

	all_indexed_same = all_indexes_same(x.index for x in values)
	if not all_indexed_same:
	# GH 8467
	return self._concat_objects(
	values,
	not_indexed_same=True,
	is_transform=is_transform,
	)
	# Combine values
	# vstack+constructor is faster than concat and handles MI-columns
	stacked_values = np.vstack([np.asarray(v) for v in values])

Here we test whether the indices are all the same. If they are not the same (as in the first example) _concat_objects is called; if they are the same (as in the second example) the result is constructed with np.vstack.

A workaround could be to use pd.Grouper(freq="...") instead of resample. Like so:

df2 = pd.DataFrame(index=pd.DatetimeIndex(['2022-1-1 1:00','2022-1-2 1:00','2022-1-2 1:00',
    '2022-1-1 1:00']),data={'category':['A','A','B','B']})
 
res2 = df2.groupby(['category', pd.Grouper(freq='1D')]).size()
print(res2)

I'm not sure if using pd.Grouper in this way should be the same as df.groupby(...).resample(...) for all cases but it has been the same for the examples that I have tried; except for this issue of course.

---

As far as I can tell this only applies to the use of df.groupby(...).resample(...).size(). Someone please edit the title so that it reads "groupby-resample-size" instead of "groupby-resample-count".