BUG: Segmentation fault when JSON serializing a PeriodIndex

[jgmarcel]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import faulthandler; faulthandler.enable()
import pandas as pd
s = pd.Series(["2022-04-06", "2022-04-07"])
p = pd.PeriodIndex(s, freq='D')
df = pd.DataFrame(index=p)
df.to_json()

Issue Description

$ python repro.py 
Fatal Python error: Segmentation fault

Current thread 0x000000011dfc5dc0 (most recent call first):
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/io/json/_json.py", line 172 in write
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/io/json/_json.py", line 110 in to_json
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 2621 in to_json
  File "/opt/src/origin/repro.py", line 6 in <module>
Segmentation fault: 11

Expected Behavior

The code above should run without segfaulting.

Installed Versions

INSTALLED VERSIONS
------------------
commit           : 4bfe3d07b4858144c219b9346329027024102ab6
python           : 3.9.7.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 19.6.0
Version          : Darwin Kernel Version 19.6.0: Tue Feb 15 21:39:11 PST 2022; root:xnu-6153.141.59~1/RELEASE_X86_64
machine          : x86_64
processor        : i386
byteorder        : little
LC_ALL           : en_US.UTF-8
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.4.2
numpy            : 1.22.3
pytz             : 2021.1
dateutil         : 2.8.2
pip              : 22.0.4
setuptools       : 58.0.4
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : 2.9.2
jinja2           : None
IPython          : 7.27.0
pandas_datareader: None
bs4              : None
bottleneck       : None
brotli           : 
fastparquet      : None
fsspec           : 2021.08.1
gcsfs            : None
markupsafe       : 2.0.1
matplotlib       : 3.4.3
numba            : None
numexpr          : None
odfpy            : None
openpyxl         : 3.0.9
pandas_gbq       : None
pyarrow          : None
pyreadstat       : None
pyxlsb           : None
s3fs             : 2021.08.1
scipy            : 1.8.0
snappy           : None
sqlalchemy       : 1.4.34
tables           : None
tabulate         : None
xarray           : None
xlrd             : 2.0.1
xlwt             : None
zstandard        : None

[simonjayhawkins]

Thanks @jgmarcel for the report

Expected Behavior

The code above should run without segfaulting.

working in pandas-1.3.5

first bad commit: [8456833] Bug tojson memleak (#45489)

cc @vernetya

[roberthdevries]

The Py_DECREF() call causes the reference count to drop to 0 for some index types (as observed in the debugger). Fix is to assign the result of PyObject_CallMethod to a temporary object, then decrement the ref count and assign the temporary object to values again.

[roberthdevries]

take

[davidgilbertson]

I note this is closed but to_json() still throws an error with Period in 2.0.3.

import pandas as pd

df = pd.DataFrame(
    dict(
        A=[1, 2, 12],
        Date=pd.period_range("2000-01-01", periods=3),
    )
)

df.to_json()

Result:

  File "/home/davidg/.virtualenvs/learning/lib/python3.10/site-packages/pandas/core/generic.py", line 2532, in to_json
    return json.to_json(
  File "/home/davidg/.virtualenvs/learning/lib/python3.10/site-packages/pandas/io/json/_json.py", line 191, in to_json
    ).write()
  File "/home/davidg/.virtualenvs/learning/lib/python3.10/site-packages/pandas/io/json/_json.py", line 244, in write
    return dumps(
OverflowError: Maximum recursion level reached

Is this a regression, should I create a new issue?

[bquistorff]

That appears to be a separate issue, where serializing a dataframe with a PeriodArray errors. Serializing a PeriodIndex has continued to work (the fix included a test for a period in the index). I've started a separate bug report for PeriodArray at #58236.