BUG: LSAN Detected Memory Leaks

[WillAyd]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

Invoke meson via pip install -ve . --no-build-isolation --config-settings=builddir="asan" --config-settings=setup-args="-Db_sanitize=address"

Also create an LSAN "suppression file" with these contents:

leak:unicode_decode_utf8
leak:PyObject_Malloc

Name that file and provide it via an absolute path to the below command:

LSAN_OPTIONS=suppressions=<absolute_path_to_suppression_file> LD_PRELOAD=$(gcc -print-file-name=libasan.so) python -m pytest pandas/tests/io/json/

Issue Description

You have to sift through a lot, but ASAN detects direct leaks in our codebase from at least the following locations (just running JSON test suite above):

Direct leak of 1440 byte(s) in 36 object(s) allocated from:
    #0 0x7f953ea95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f94d4f543df in __pyx_pw_6pandas_5_libs_9hashtable_17PyObjectHashTable_1__init__ (/home/willayd/clones/pandas/asan/pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so+0x3543df)
    #2 0x55b4f55a1dba in type_call /usr/local/src/conda/python-3.10.12/Objects/typeobject.c:1135

Direct leak of 480 byte(s) in 12 object(s) allocated from:
    #0 0x7f953ea95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f94d4f301ca in __pyx_tp_new_6pandas_5_libs_9hashtable_Int64HashTable (/home/willayd/clones/pandas/asan/pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so+0x3301ca)
    #2 0x55b4f55a1d32 in type_call /usr/local/src/conda/python-3.10.12/Objects/typeobject.c:1123
    #3 0x7f94cc8c3da2 in __pyx_f_6pandas_5_libs_5index_11Int64Engine__make_hash_table (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0xaada2)
    #4 0x7f94cc9262e3 in __pyx_f_6pandas_5_libs_5index_11IndexEngine__ensure_mapping_populated (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0x10d2e3)
    #5 0x7f94cc8cbfae in __pyx_f_6pandas_5_libs_5index_11IndexEngine__do_unique_check (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0xb2fae)
    #6 0x7f94cc8c5973 in __pyx_getprop_6pandas_5_libs_5index_11IndexEngine_is_unique (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0xac973)
    #7 0x55b4f55a6da9 in _PyObject_GenericGetAttrWithDict /usr/local/src/conda/python-3.10.12/Objects/object.c:1254

Direct leak of 240 byte(s) in 6 object(s) allocated from:
    #0 0x7f953ea95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f94d4f7b65a in __pyx_tp_new_6pandas_5_libs_9hashtable_UInt64HashTable (/home/willayd/clones/pandas/asan/pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so+0x37b65a)
    #2 0x55b4f55a1d32 in type_call /usr/local/src/conda/python-3.10.12/Objects/typeobject.c:1123
    #3 0x7f94cc8a5442 in __pyx_f_6pandas_5_libs_5index_12UInt64Engine__make_hash_table (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0x8c442)
    #4 0x7f94cc9262e3 in __pyx_f_6pandas_5_libs_5index_11IndexEngine__ensure_mapping_populated (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0x10d2e3)
    #5 0x7f94cc8cbfae in __pyx_f_6pandas_5_libs_5index_11IndexEngine__do_unique_check (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0xb2fae)
    #6 0x7f94cc8c5973 in __pyx_getprop_6pandas_5_libs_5index_11IndexEngine_is_unique (/home/willayd/clones/pandas/asan/pandas/_libs/index.cpython-310-x86_64-linux-gnu.so+0xac973)
    #7 0x55b4f55a6da9 in _PyObject_GenericGetAttrWithDict /usr/local/src/conda/python-3.10.12/Objects/object.c:1254

Expected Behavior

No leaks

Installed Versions

INSTALLED VERSIONS

commit : 29873e4
python : 3.10.12.final.0
python-bits : 64
OS : Linux
OS-release : 6.2.0-26-generic
Version : #26-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 10 23:39:54 UTC 2023
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.1.0rc0+254.g29873e442c
numpy : 1.22.4
pytz : 2023.3
dateutil : 2.8.2
setuptools : 68.1.2
pip : 23.2.1
Cython : 0.29.33
pytest : 7.4.0
hypothesis : 6.82.7
sphinx : 6.2.1
blosc : None
feather : None
xlsxwriter : 3.1.2
lxml.etree : 4.9.3
html5lib : 1.1
pymysql : 1.4.6
psycopg2 : 2.9.6
jinja2 : 3.1.2
IPython : 8.14.0
pandas_datareader : None
bs4 : 4.12.2
bottleneck : 1.3.7
dataframe-api-compat: None
fastparquet : 2023.7.0
fsspec : 2023.6.0
gcsfs : 2023.6.0
matplotlib : 3.7.2
numba : 0.57.1
numexpr : 2.8.4
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : 13.0.0
pyreadstat : 1.2.3
pyxlsb : 1.0.10
s3fs : 2023.6.0
scipy : 1.11.2
sqlalchemy : 2.0.20
tables : 3.8.0
tabulate : 0.9.0
xarray : 2023.8.0
xlrd : 2.0.1
zstandard : 0.19.0
tzdata : 2023.3
qtpy : None
pyqt5 : None
None

[WillAyd]

progress towards #52990

[lithomas1]

Does this work with clang?

Your bitmask PR reminded me to try out asan again, but no leaks seem to be reported (on macos, I don't think asan is loaded in correctly).

[WillAyd]

It should. At the very least you should be getting a lot of (false?) positives from Python. Do you get any output at all?

[WillAyd]

Here are more detailed tracebacks from the leaks above with optimization level 0

Direct leak of 1440 byte(s) in 36 object(s) allocated from:
    #0 0x7f561de95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f55b406e882 in traced_calloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:41
    #2 0x7f55b4084d7d in kh_init_pymap ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:393
    #3 0x7f55b41fccc0 in __pyx_pf_6pandas_5_libs_9hashtable_17PyObjectHashTable___init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:97063
    #4 0x7f55b41fcc0d in __pyx_pw_6pandas_5_libs_9hashtable_17PyObjectHashTable_1__init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:97034
    #5 0x55b22d97fdba in type_call /usr/local/src/conda/python-3.10.12/Objects/typeobject.c:1135

Direct leak of 480 byte(s) in 12 object(s) allocated from:
    #0 0x7f561de95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f55b406e882 in traced_calloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:41
    #2 0x7f55b4073374 in kh_init_int64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:688
    #3 0x7f55b40fd9c0 in __pyx_pf_6pandas_5_libs_9hashtable_14Int64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:40064
    #4 0x7f55b40fd90a in __pyx_pw_6pandas_5_libs_9hashtable_14Int64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:40035
    #5 0x7f55b435d993 in __pyx_tp_new_6pandas_5_libs_9hashtable_Int64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:161069
    #6 0x55b22d97fd32 in type_call /usr/local/src/conda/python-3.10.12/Objects/typeobject.c:1123
    #7 0x7f55abd5eeae in __Pyx__PyObject_CallOneArg pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:85628
    #8 0x7f55abd5ef97 in __Pyx_PyObject_CallOneArg pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:85647
    #9 0x7f55abc81ca7 in __pyx_f_6pandas_5_libs_5index_11Int64Engine__make_hash_table pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:22390
    #10 0x7f55abc47115 in __pyx_f_6pandas_5_libs_5index_11IndexEngine__ensure_mapping_populated pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:9667
    #11 0x7f55abc445eb in __pyx_f_6pandas_5_libs_5index_11IndexEngine__do_unique_check pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:8788
    #12 0x7f55abc4443b in __pyx_pf_6pandas_5_libs_5index_11IndexEngine_9is_unique___get__ pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:8718
    #13 0x7f55abc44345 in __pyx_pw_6pandas_5_libs_5index_11IndexEngine_9is_unique_1__get__ pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:8684
    #14 0x7f55abd4533d in __pyx_getprop_6pandas_5_libs_5index_11IndexEngine_is_unique pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:77428
    #15 0x55b22d984da9 in _PyObject_GenericGetAttrWithDict /usr/local/src/conda/python-3.10.12/Objects/object.c:1254

Direct leak of 240 byte(s) in 6 object(s) allocated from:
    #0 0x7f561de95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f55b406e882 in traced_calloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:41
    #2 0x7f55b4074af5 in kh_init_uint64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:689
    #3 0x7f55b40e433b in __pyx_pf_6pandas_5_libs_9hashtable_15UInt64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:34277
    #4 0x7f55b40e4285 in __pyx_pw_6pandas_5_libs_9hashtable_15UInt64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:34248
    #5 0x7f55b435d577 in __pyx_tp_new_6pandas_5_libs_9hashtable_UInt64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:160929
    #6 0x55b22d97fd32 in type_call /usr/local/src/conda/python-3.10.12/Objects/typeobject.c:1123
    #7 0x7f55abd5eeae in __Pyx__PyObject_CallOneArg pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:85628
    #8 0x7f55abd5ef97 in __Pyx_PyObject_CallOneArg pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:85647
    #9 0x7f55abc92b77 in __pyx_f_6pandas_5_libs_5index_12UInt64Engine__make_hash_table pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:27630
    #10 0x7f55abc47115 in __pyx_f_6pandas_5_libs_5index_11IndexEngine__ensure_mapping_populated pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:9667
    #11 0x7f55abc445eb in __pyx_f_6pandas_5_libs_5index_11IndexEngine__do_unique_check pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:8788
    #12 0x7f55abc4443b in __pyx_pf_6pandas_5_libs_5index_11IndexEngine_9is_unique___get__ pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:8718
    #13 0x7f55abc44345 in __pyx_pw_6pandas_5_libs_5index_11IndexEngine_9is_unique_1__get__ pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:8684
    #14 0x7f55abd4533d in __pyx_getprop_6pandas_5_libs_5index_11IndexEngine_is_unique pandas/_libs/index.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/index.pyx.c:77428
    #15 0x55b22d984da9 in _PyObject_GenericGetAttrWithDict /usr/local/src/conda/python-3.10.12/Objects/object.c:1254

[WillAyd]

Interestingly enough I couldn't get valgrind to detect these. I have a hunch that LSAN may be misreporting bytes that Cython releases in any __dealloc__ calls when they are constructed via pytest parameters. Haven't verified that is the case for these in particular, but something worth further investigation

[WillAyd]

As far as a suppression file goes I am having luck with:

leak:obmalloc.c
leak:stringio.c
leak:pyo3
leak:crypto
leak:unicodeobject.c
leak:bytesobject.c
leak:listobject.c
leak:gcmodule.c
leak:PyThread
leak:arrow
leak:numpy

I was somewhat expecting this to be so wide of a net cast of suppressions that it wouldn't yield any results, but looks like there are still plenty:

$ LSAN_OPTIONS=suppressions=lsan_suppr.txt LD_PRELOAD=$(gcc -print-file-name=libasan.so) python -m pytest pandas/tests/io/json/test_pandas.py 
==122102==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 600 byte(s) in 15 object(s) allocated from:
    #0 0x7f9723e95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f96b7e55254 in traced_calloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:37
    #2 0x7f96b7e8789c in kh_init_pymap ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:356
    #3 0x7f96b81246a9 in __pyx_pf_6pandas_5_libs_9hashtable_17PyObjectHashTable___init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130346
    #4 0x7f96b8124510 in __pyx_pw_6pandas_5_libs_9hashtable_17PyObjectHashTable_1__init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130311
    #5 0x5586c60a39ea in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1135
    #6 0x5586c60a39ea in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Direct leak of 520 byte(s) in 1 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96ba57a129 in parser_trim_buffers ../pandas/_libs/src/parser/tokenizer.c:1211
    #2 0x7f96a4a56606 in __pyx_pf_6pandas_5_libs_7parsers_10TextReader_12read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:29031
    #3 0x7f96a4a5364c in __pyx_pw_6pandas_5_libs_7parsers_10TextReader_13read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:28559
    #4 0x7f96a4b6bca9 in __Pyx_CyFunction_Vectorcall_FASTCALL_KEYWORDS pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:57975
    #5 0x5586c609aa3b in _PyObject_VectorcallTstate /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:114
    #6 0x5586c609aa3b in PyObject_Vectorcall /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:123
    #7 0x5586c609aa3b in call_function /usr/local/src/conda/python-3.10.13/Python/ceval.c:5893
    #8 0x5586c609aa3b in _PyEval_EvalFrameDefault /usr/local/src/conda/python-3.10.13/Python/ceval.c:4198

Direct leak of 520 byte(s) in 1 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96ba579faa in parser_trim_buffers ../pandas/_libs/src/parser/tokenizer.c:1207
    #2 0x7f96a4a56606 in __pyx_pf_6pandas_5_libs_7parsers_10TextReader_12read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:29031
    #3 0x7f96a4a5364c in __pyx_pw_6pandas_5_libs_7parsers_10TextReader_13read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:28559
    #4 0x7f96a4b6bca9 in __Pyx_CyFunction_Vectorcall_FASTCALL_KEYWORDS pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:57975
    #5 0x5586c609aa3b in _PyObject_VectorcallTstate /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:114
    #6 0x5586c609aa3b in PyObject_Vectorcall /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:123
    #7 0x5586c609aa3b in call_function /usr/local/src/conda/python-3.10.13/Python/ceval.c:5893
    #8 0x5586c609aa3b in _PyEval_EvalFrameDefault /usr/local/src/conda/python-3.10.13/Python/ceval.c:4198

Direct leak of 160 byte(s) in 4 object(s) allocated from:
    #0 0x7f9723e95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f96b7e55254 in traced_calloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:37
    #2 0x7f96b7e6356d in kh_init_uint64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:733
    #3 0x7f96b7f5fb14 in __pyx_pf_6pandas_5_libs_9hashtable_15UInt64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59122
    #4 0x7f96b7f5f98c in __pyx_pw_6pandas_5_libs_9hashtable_15UInt64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59087
    #5 0x7f96b8333020 in __pyx_tp_new_6pandas_5_libs_9hashtable_UInt64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187181
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Direct leak of 80 byte(s) in 2 object(s) allocated from:
    #0 0x7f9723e95749 in __interceptor_calloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f96b7e55254 in traced_calloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:37
    #2 0x7f96b7e5fe2c in kh_init_int64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:732
    #3 0x7f96b7f88e92 in __pyx_pf_6pandas_5_libs_9hashtable_14Int64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65682
    #4 0x7f96b7f88d0a in __pyx_pw_6pandas_5_libs_9hashtable_14Int64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65647
    #5 0x7f96b8333683 in __pyx_tp_new_6pandas_5_libs_9hashtable_Int64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187347
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96ba57abe0 in parser_trim_buffers ../pandas/_libs/src/parser/tokenizer.c:1252
    #2 0x7f96a4a56606 in __pyx_pf_6pandas_5_libs_7parsers_10TextReader_12read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:29031
    #3 0x7f96a4a5364c in __pyx_pw_6pandas_5_libs_7parsers_10TextReader_13read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:28559
    #4 0x7f96a4b6bca9 in __Pyx_CyFunction_Vectorcall_FASTCALL_KEYWORDS pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:57975
    #5 0x5586c609aa3b in _PyObject_VectorcallTstate /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:114
    #6 0x5586c609aa3b in PyObject_Vectorcall /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:123
    #7 0x5586c609aa3b in call_function /usr/local/src/conda/python-3.10.13/Python/ceval.c:5893
    #8 0x5586c609aa3b in _PyEval_EvalFrameDefault /usr/local/src/conda/python-3.10.13/Python/ceval.c:4198

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96ba57ad21 in parser_trim_buffers ../pandas/_libs/src/parser/tokenizer.c:1258
    #2 0x7f96a4a56606 in __pyx_pf_6pandas_5_libs_7parsers_10TextReader_12read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:29031
    #3 0x7f96a4a5364c in __pyx_pw_6pandas_5_libs_7parsers_10TextReader_13read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:28559
    #4 0x7f96a4b6bca9 in __Pyx_CyFunction_Vectorcall_FASTCALL_KEYWORDS pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:57975
    #5 0x5586c609aa3b in _PyObject_VectorcallTstate /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:114
    #6 0x5586c609aa3b in PyObject_Vectorcall /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:123
    #7 0x5586c609aa3b in call_function /usr/local/src/conda/python-3.10.13/Python/ceval.c:5893
    #8 0x5586c609aa3b in _PyEval_EvalFrameDefault /usr/local/src/conda/python-3.10.13/Python/ceval.c:4198

Indirect leak of 480 byte(s) in 15 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96b7e552a9 in traced_realloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:45
    #2 0x7f96b7e883ea in kh_resize_pymap ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:356
    #3 0x7f96b8124c05 in __pyx_pf_6pandas_5_libs_9hashtable_17PyObjectHashTable___init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130385
    #4 0x7f96b8124510 in __pyx_pw_6pandas_5_libs_9hashtable_17PyObjectHashTable_1__init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130311
    #5 0x5586c60a39ea in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1135
    #6 0x5586c60a39ea in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 480 byte(s) in 15 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96b7e552a9 in traced_realloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:45
    #2 0x7f96b7e882d2 in kh_resize_pymap ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:356
    #3 0x7f96b8124c05 in __pyx_pf_6pandas_5_libs_9hashtable_17PyObjectHashTable___init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130385
    #4 0x7f96b8124510 in __pyx_pw_6pandas_5_libs_9hashtable_17PyObjectHashTable_1__init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130311
    #5 0x5586c60a39ea in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1135
    #6 0x5586c60a39ea in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 257 byte(s) in 1 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96ba57a437 in parser_trim_buffers ../pandas/_libs/src/parser/tokenizer.c:1226
    #2 0x7f96a4a56606 in __pyx_pf_6pandas_5_libs_7parsers_10TextReader_12read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:29031
    #3 0x7f96a4a5364c in __pyx_pw_6pandas_5_libs_7parsers_10TextReader_13read_low_memory pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:28559
    #4 0x7f96a4b6bca9 in __Pyx_CyFunction_Vectorcall_FASTCALL_KEYWORDS pandas/_libs/parsers.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/parsers.pyx.c:57975
    #5 0x5586c609aa3b in _PyObject_VectorcallTstate /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:114
    #6 0x5586c609aa3b in PyObject_Vectorcall /usr/local/src/conda/python-3.10.13/Include/cpython/abstract.h:123
    #7 0x5586c609aa3b in call_function /usr/local/src/conda/python-3.10.13/Python/ceval.c:5893
    #8 0x5586c609aa3b in _PyEval_EvalFrameDefault /usr/local/src/conda/python-3.10.13/Python/ceval.c:4198

Indirect leak of 128 byte(s) in 4 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96b7e552a9 in traced_realloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:45
    #2 0x7f96b7e640ae in kh_resize_uint64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:733
    #3 0x7f96b7f60070 in __pyx_pf_6pandas_5_libs_9hashtable_15UInt64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59161
    #4 0x7f96b7f5f98c in __pyx_pw_6pandas_5_libs_9hashtable_15UInt64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59087
    #5 0x7f96b8333020 in __pyx_tp_new_6pandas_5_libs_9hashtable_UInt64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187181
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 128 byte(s) in 4 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96b7e552a9 in traced_realloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:45
    #2 0x7f96b7e63f96 in kh_resize_uint64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:733
    #3 0x7f96b7f60070 in __pyx_pf_6pandas_5_libs_9hashtable_15UInt64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59161
    #4 0x7f96b7f5f98c in __pyx_pw_6pandas_5_libs_9hashtable_15UInt64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59087
    #5 0x7f96b8333020 in __pyx_tp_new_6pandas_5_libs_9hashtable_UInt64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187181
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 64 byte(s) in 2 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96b7e552a9 in traced_realloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:45
    #2 0x7f96b7e6096d in kh_resize_int64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:732
    #3 0x7f96b7f893ee in __pyx_pf_6pandas_5_libs_9hashtable_14Int64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65721
    #4 0x7f96b7f88d0a in __pyx_pw_6pandas_5_libs_9hashtable_14Int64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65647
    #5 0x7f96b8333683 in __pyx_tp_new_6pandas_5_libs_9hashtable_Int64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187347
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 64 byte(s) in 2 object(s) allocated from:
    #0 0x7f9723e94dd2 in __interceptor_realloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x7f96b7e552a9 in traced_realloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:45
    #2 0x7f96b7e60855 in kh_resize_int64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:732
    #3 0x7f96b7f893ee in __pyx_pf_6pandas_5_libs_9hashtable_14Int64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65721
    #4 0x7f96b7f88d0a in __pyx_pw_6pandas_5_libs_9hashtable_14Int64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65647
    #5 0x7f96b8333683 in __pyx_tp_new_6pandas_5_libs_9hashtable_Int64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187347
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 60 byte(s) in 15 object(s) allocated from:
    #0 0x7f9723e955f8 in __interceptor_malloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f96b7e55205 in traced_malloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:29
    #2 0x7f96b7e88187 in kh_resize_pymap ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:356
    #3 0x7f96b8124c05 in __pyx_pf_6pandas_5_libs_9hashtable_17PyObjectHashTable___init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130385
    #4 0x7f96b8124510 in __pyx_pw_6pandas_5_libs_9hashtable_17PyObjectHashTable_1__init__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:130311
    #5 0x5586c60a39ea in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1135
    #6 0x5586c60a39ea in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 16 byte(s) in 4 object(s) allocated from:
    #0 0x7f9723e955f8 in __interceptor_malloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f96b7e55205 in traced_malloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:29
    #2 0x7f96b7e63e4b in kh_resize_uint64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:733
    #3 0x7f96b7f60070 in __pyx_pf_6pandas_5_libs_9hashtable_15UInt64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59161
    #4 0x7f96b7f5f98c in __pyx_pw_6pandas_5_libs_9hashtable_15UInt64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:59087
    #5 0x7f96b8333020 in __pyx_tp_new_6pandas_5_libs_9hashtable_UInt64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187181
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

Indirect leak of 8 byte(s) in 2 object(s) allocated from:
    #0 0x7f9723e955f8 in __interceptor_malloc ../../../../libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f96b7e55205 in traced_malloc ../pandas/_libs/include/pandas/vendored/klib/khash_python.h:29
    #2 0x7f96b7e6070a in kh_resize_int64 ../pandas/_libs/include/pandas/vendored/klib/khash.h:732
    #3 0x7f96b7f893ee in __pyx_pf_6pandas_5_libs_9hashtable_14Int64HashTable___cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65721
    #4 0x7f96b7f88d0a in __pyx_pw_6pandas_5_libs_9hashtable_14Int64HashTable_1__cinit__ pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:65647
    #5 0x7f96b8333683 in __pyx_tp_new_6pandas_5_libs_9hashtable_Int64HashTable pandas/_libs/hashtable.cpython-310-x86_64-linux-gnu.so.p/pandas/_libs/hashtable.pyx.c:187347
    #6 0x5586c60a3908 in type_call /usr/local/src/conda/python-3.10.13/Objects/typeobject.c:1123
    #7 0x5586c60a3908 in _PyObject_MakeTpCall /usr/local/src/conda/python-3.10.13/Objects/call.c:215

-----------------------------------------------------
Suppressions used:
  count      bytes template
   7213   13128616 obmalloc.c
      2         16 stringio.c
      3         96 pyo3
    182      10066 crypto
    247     676487 unicodeobject.c
    386     179166 bytesobject.c
      9      28224 listobject.c
     37      26920 gcmodule.c
     25        800 PyThread
    957     117662 arrow
    286       5103 numpy
-----------------------------------------------------

SUMMARY: AddressSanitizer: 3597 byte(s) leaked in 89 allocation(s).

[WillAyd]

FWIW some of these leaks I could only get when running under pytest. Not sure if it is a pytest/pytest-xdist thing