Meaning of tracebacklimit differs between sys.tracebacklimit and traceback module

[cfbolz]

BPO	38197
Nosy	@cfbolz, @iritkatriel
Files	x.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-09-17.10:34:55.146>
labels = ['3.8', '3.9', '3.10']
title = 'Meaning of tracebacklimit differs between sys.tracebacklimit and traceback module'
updated_at = <Date 2020-11-04.19:48:39.522>
user = 'https://github.com/cfbolz'

bugs.python.org fields:

activity = <Date 2020-11-04.19:48:39.522>
actor = 'Carl.Friedrich.Bolz'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2019-09-17.10:34:55.146>
creator = 'Carl.Friedrich.Bolz'
dependencies = []
files = ['48610']
hgrepos = []
issue_num = 38197
keywords = []
message_count = 3.0
messages = ['352628', '380352', '380355']
nosy_count = 2.0
nosy_names = ['Carl.Friedrich.Bolz', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue38197'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

• gh-82378: fix sys.tracebacklimit in pyrepl #122452
• gh-82378 fix sys.tracebacklimit in pyrepl #123062
• [3.13] gh-82378 fix sys.tracebacklimit in pyrepl (GH-123062) #123252
• gh-82378: document the difference between sys.tracebacklimit and the limit arguments #123286
• [3.13] gh-82378: Document the difference between sys.tracebacklimit and the limit arguments (GH-123286) #123325
• [3.12] gh-82378: Document the difference between sys.tracebacklimit and the limit arguments (GH-123286) #123326

[cfbolz]

The meaning of sys.tracebacklimit seems to be different than the meaning of the various limit parameters in the traceback module. One shows the top n stack frames, the other the bottom n.

Is this intentional, and if yes, is that difference documented somewhere? (it came up because PyPy just uses the traceback module and has no equivalent of PyTraceBack_Print).

See the attached script to understand the problem. The script formats the same exception twice, once with the traceback module, once by the interpreter. I would have expected them to look the same for all limits, but instead:

$ ./python /tmp/x.py 3
limit 3
from traceback module:
Traceback (most recent call last):
  File "/tmp/x.py", line 19, in <module>
    main()
  File "/tmp/x.py", line 16, in main
    x3()
  File "/tmp/x.py", line 14, in x3
    x2()
ZeroDivisionError: division by zero

from interpreter:
Traceback (most recent call last):
  File "/tmp/x.py", line 14, in x3
    x2()
  File "/tmp/x.py", line 12, in x2
    x1()
  File "/tmp/x.py", line 10, in x1
    1 / 0
ZeroDivisionError: division by zero

[iritkatriel]

If you change your script to do
sys.tracebacklimit = abs(limit)
and run it with arg -3, then you get the output you expect:

C:\Users\User\src\cpython>python.bat x.py -3
Running Release|Win32 interpreter...
limit -3
from traceback module:
Traceback (most recent call last):
  File "C:\Users\User\src\cpython\x.py", line 14, in x3
    x2()
  File "C:\Users\User\src\cpython\x.py", line 12, in x2
    x1()
  File "C:\Users\User\src\cpython\x.py", line 10, in x1
    1 / 0
ZeroDivisionError: division by zero

from interpreter:
Traceback (most recent call last):
  File "C:\Users\User\src\cpython\x.py", line 14, in x3
    x2()
  File "C:\Users\User\src\cpython\x.py", line 12, in x2
    x1()
  File "C:\Users\User\src\cpython\x.py", line 10, in x1
    1 / 0
ZeroDivisionError: division by zero

The documentation for traceback mentions the possibility of using negative limits and their meaning:

Print up to limit stack trace entries from traceback object tb (starting from the caller’s frame) if limit is positive. Otherwise, print the last abs(limit) entries.

https://docs.python.org/3/library/traceback.html#traceback.print_tb

[cfbolz]

It's still inconsistent between the two ways to get a traceback, and the inconsistency is not documented.

[cfbolz]

This issue also becomes more relevant now, with pyrepl using code, which uses the traceback module. Here's a behaviour difference between cpy3.12 and cpy3.13 (current 3.13 branch):

cfbolz@triacontahedron:~/projects/cpython$ cat x.py 
def x1():
    1 / 0
def x2():
    x1()
def x3():
    x2()
cfbolz@triacontahedron:~/projects/cpython$ python3.12 -i x.py
>>> x3()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cfbolz/projects/cpython/x.py", line 6, in x3
    x2()
  File "/home/cfbolz/projects/cpython/x.py", line 4, in x2
    x1()
  File "/home/cfbolz/projects/cpython/x.py", line 2, in x1
    1 / 0
    ~~^~~
ZeroDivisionError: division by zero
>>> import sys
>>> sys.tracebacklimit = 2
>>> x3()
Traceback (most recent call last):
  File "/home/cfbolz/projects/cpython/x.py", line 4, in x2
    x1()
  File "/home/cfbolz/projects/cpython/x.py", line 2, in x1
    1 / 0
    ~~^~~
ZeroDivisionError: division by zero
>>> 
cfbolz@triacontahedron:~/projects/cpython$ ./python -i x.py 
>>> x3()
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    x3()
    ~~^^
  File "/home/cfbolz/projects/cpython/x.py", line 6, in x3
    x2()
    ~~^^
  File "/home/cfbolz/projects/cpython/x.py", line 4, in x2
    x1()
    ~~^^
  File "/home/cfbolz/projects/cpython/x.py", line 2, in x1
    1 / 0
    ~~^~~
ZeroDivisionError: division by zero
>>> import sys
>>> sys.tracebacklimit = 2
>>> x3()
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    x3()
    ~~^^
  File "/home/cfbolz/projects/cpython/x.py", line 6, in x3
    x2()
    ~~^^
ZeroDivisionError: division by zero
>>> 

note how on 3.12 the traceback is missing the frame for x3 when sys.tracebacklimit is set, and 3.13 is missing the frame for x1.

I would still argue that sys.tracebacklimit has basically been broken since essentially forever in the traceback module. It might still be too late to fix this, of course, because people have come to rely on this undocumented behavior.

[iritkatriel]

I would still argue that sys.tracebacklimit has basically been broken since essentially forever in the traceback module. It might still be too late to fix this, of course, because people have come to rely on this undocumented behavior.

Yes, we'd need to add a new limit and deprecate this one.

[cfbolz]

Ok, but that means we still have a very clear behavior change in the repl from 3.12 to 3.13 if someone is setting sys.tracebacklimit

[iritkatriel]

CC @pablogsal @ambv re repl.

[cfbolz]

ah, I see that @pablogsal already had to work around this problem in traceback.py in e7331365b488. It should be relatively easy to use this facility in Lib/_pyrepl/console.py in InteractiveColoredConsole.showtraceback, I think.

[pablogsal]

Ok, I think we are done with this one. Thanks @cfbolz for the PRs and @serhiy-storchaka for the reviews!

[cfbolz]

@pablogsal I kind of would like to still add a paragraph to the traceback module docs somewhere that mentions the differences between the meaning of limit in that module and the meaning of sys.tracebacklimit. I just don't quite know where in the page to put it. Maybe simply in the text after print_tb where the meaning of limit is explained for the first time?

I'll re-open the issue to make sure I don't forget (or should I open a new one for the doc change?).

[pablogsal]

or should I open a new one for the doc change?

No, we can reuse this one 👍

Maybe simply in the text after print_tb where the meaning of limit is explained for the first time?

I think that would be a good place. Alternatively we can add it at the start after the "The module defines the following functions:
" as a note