Possible bug: pdb causes exception

[cklb]

BPO	46535
Nosy	@cklb

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-01-26.11:20:07.032>
labels = ['type-bug', 'library', '3.10']
title = 'Possible bug: pdb causes exception'
updated_at = <Date 2022-01-27.15:39:46.150>
user = 'https://github.com/cklb'

bugs.python.org fields:

activity = <Date 2022-01-27.15:39:46.150>
actor = 'cklb'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2022-01-26.11:20:07.032>
creator = 'cklb'
dependencies = []
files = []
hgrepos = []
issue_num = 46535
keywords = []
message_count = 1.0
messages = ['411738']
nosy_count = 1.0
nosy_names = ['cklb']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46535'
versions = ['Python 3.10']

[cklb]

Consider a script called snippet.py, containing the lines

------------------------------------------

import numpy as np
import pandas as pd

np.finfo(float)
idx = pd.MultiIndex.from_tuples([(1, 2)])
np.finfo(float)
print("Done")

---

When running via 'python snippet.py' no errors occur. However when running via 'python -m pdb snippet.py' the following happens:
> snippet.py(1)<module>()
-> import numpy as np
(Pdb) r
--Return--
> snippet.py(6)<module>()->None
-> np.finfo(float)
(Pdb) c
Traceback (most recent call last):
  File "VENV/lib/python3.10/site-packages/numpy/core/getlimits.py", line 459, in __new__
    dtype = numeric.dtype(dtype)
TypeError: 'NoneType' object is not callable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/pdb.py", line 1723, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib/python3.10/pdb.py", line 1583, in _runscript
    self.run(statement)
  File "/usr/lib/python3.10/bdb.py", line 597, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "snippet.py", line 6, in <module>
    np.finfo(float)
  File "VENV/lib/python3.10/site-packages/numpy/core/getlimits.py", line 462, in __new__
    dtype = numeric.dtype(type(dtype))
TypeError: 'NoneType' object is not callable
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program

Commenting the MultiIndex line will will get rid of the problem but I am not quite sure why.

Please note that this error only occurs if the script is run via 'r' in pdb. Running via 'c' does not cause any problems.
Therefore, I suspect that the problem may be with pdb instead of the external packages.

My setup:
$ python --version
Python 3.10.1
$ pip list | grep numpy
numpy 1.22.1
$ pip list | grep pandas
pandas 1.4.0

[damarvin]

Hi,
I see this when debugging in PyCharm, and I cannot downgrade to python 3.9.

(besides commenting out the try-except in numpy/core/getlimits.py::finfo.__new__)
Is there a workaround?

[iritkatriel]

@cklb

Are you still seeing this problem? I'm unable to reproduce it with Python 3.10.4. (numpy-1.23.0, pandas-1.4.3).

iritkatriel@Irits-MBP cpython-654 % python3.10 -m pdb xx.py         
> /Users/iritkatriel/src/cpython-654/xx.py(1)<module>()
-> import numpy as np
(Pdb) r
Done
--Return--
> /Users/iritkatriel/src/cpython-654/xx.py(7)<module>()->None
-> print("Done")
(Pdb) c
The program finished and will be restarted
> /Users/iritkatriel/src/cpython-654/xx.py(1)<module>()
-> import numpy as np
(Pdb) list
  1  ->	import numpy as np
  2  	import pandas as pd
  3  	
  4  	np.finfo(float)
  5  	idx = pd.MultiIndex.from_tuples([(1, 2)])
  6  	np.finfo(float)
  7  	print("Done")
  8  	

[cklb]

@iritkatriel sorry for the delay

I just reran the snippet with the following configuration:

Python 3.10.5 (main, Jun  6 2022, 18:49:26) [GCC 12.1.0] on linux
numpy==1.23.1
pandas==1.4.0

and still get the following output:

python -m pdb snippet.py
> /PATH/snippet.py(1)<module>()
-> import numpy as np
(Pdb) r
--Return--
> /PATH/snippet.py(6)<module>()->None
-> np.finfo(float)
(Pdb) c
Traceback (most recent call last):
  File "/VENV/lib/python3.10/site-packages/numpy/core/getlimits.py", line 460, in __new__
    dtype = numeric.dtype(dtype)
TypeError: 'NoneType' object is not callable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/pdb.py", line 1726, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib/python3.10/pdb.py", line 1586, in _runscript
    self.run(statement)
  File "/usr/lib/python3.10/bdb.py", line 597, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "/PATH/snippet.py", line 6, in <module>
    np.finfo(float)
  File "/VENV/lib/python3.10/site-packages/numpy/core/getlimits.py", line 463, in __new__
    dtype = numeric.dtype(type(dtype))
TypeError: 'NoneType' object is not callable
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /VENVl/lib/python3.10/site-packages/numpy/core/getlimits.py(463)__new__()
-> dtype = numeric.dtype(type(dtype))
(Pdb) 

[iritkatriel]

Have you reported this on the numpy bug tracker?

[cklb]

Negative. I also did not open an issue in the pandas repo. However, if you think that this help I'll gladly do so.

[iritkatriel]

The exception is coming from numpy code, so I would start there. If it's a cpython bug they will know how to narrow it down to a small reproducer that doesn't involve so many dependencies.

[cklb]

I just had a look at the numpy bugtracker and it seems like this actually was a cython issue:

• BUG: finfo not working, issue related to numeric.dtype numpy/numpy#21693
• BUG: TypeError: 'NoneType' object is not callable numpy/numpy#21519

My fault was to only update numpy, but pandas was also still using the affected cython version:

• BUG: Debugging problems in Python 3.10 due to use of old Cython pandas-dev/pandas#47429

After updating to v1.4.3 of pandas the code now behaves as expected in the debugger 😄

Thank you very much for looking into this.

[Kaliteli]

.traceback (most recent call last):
File "/usr/lib/python3.10/pdb.py", line 1726, in main
pdb._runscript(mainpyfile)
File "/usr/lib/python3.10/pdb.py", line 1586, in _runscript
self.run(statement)
File "/usr/lib/python3.10/bdb.py", line 597, in run
exec(cmd, globals, locals)
File "", line 1, in
ValueError: source code string cannot contain null bytes
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program

[iritkatriel]

@Kaliteli can you give some context for your comment?

[Kaliteli]

Anlayamıyorum

[isidentical]

@Kaliteli can you please check versions of the local libraries you are using (pip freeze) and ensure that they are up to date? Also, let's try to keep the communication in this tracker in English.

Lütfen kullandığınız yerel kütüphanelerin sürümlerini ("pip freeze" ve "python --version") kontrol edip güncel olduklarından emin olur musunuz? Ayrıca GitHub'daki iletişimi ingilizce tutmaya çalışalım.