Improve conditional check for test_gdb

[jdemeyer]

BPO	37382
Nosy	@vstinner, @benjaminp, @jdemeyer, @stratakis
PRs	bpo-37382: check for debug info before testing GDB #14331 bpo-33926: enable GDB tests on Travis CI #14395

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-06-24.08:31:13.482>
labels = ['type-feature', 'tests', '3.9']
title = 'Improve conditional check for test_gdb'
updated_at = <Date 2019-07-02.10:01:02.710>
user = 'https://github.com/jdemeyer'

bugs.python.org fields:

activity = <Date 2019-07-02.10:01:02.710>
actor = 'jdemeyer'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2019-06-24.08:31:13.482>
creator = 'jdemeyer'
dependencies = []
files = []
hgrepos = []
issue_num = 37382
keywords = ['patch']
message_count = 15.0
messages = ['346358', '346359', '346600', '346601', '346608', '346609', '346612', '346619', '346624', '346625', '346627', '346630', '346633', '346711', '347125']
nosy_count = 4.0
nosy_names = ['vstinner', 'benjamin.peterson', 'jdemeyer', 'cstratak']
pr_nums = ['14331', '14395']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue37382'
versions = ['Python 3.9']

[jdemeyer]

Many tests in test_gdb are not run if python_is_optimized() is true. However, it's not entirely clear why we have this condition. For example, on my system, with the default CPython configuration, Python *is* optimized but still all gdb tests pass if I artificially enable them.

Maybe we need debug symbols, but the existence of debug symbols and optimization are orthogonal.

[jdemeyer]

The tests FAIL when CPython is compiled with "-O0 -g0" (without optimizations, without debug info) but tests PASS when compiled with "-O3 -g", showing that the check for "-g" is the right one.

[vstinner]

Depending on the compiler optimization level, gdb may or may not br able to read variables and function arguments. IHMO it is fine to skip test_gdb if Python "is optimized". I don't think it is worth it to check the exact optimizatio level.

The worst debugging experience is provided by PGO build.

[jdemeyer]

Depending on the compiler optimization level, gdb may or may not br able to read variables and function arguments.

I haven't seen that problem. As I said, all GDB tests pass when compiled with "-O3 -g".

IHMO it is fine to skip test_gdb if Python "is optimized".

The fact that these GDB tests are not run in any CI build is bad. For example, we had a buildbot failure after merging PEP-590 because of this. We should try to catch test breakage as early as possible, the buildbot should be the last resort.

[vstinner]

I haven't seen that problem. As I said, all GDB tests pass when compiled with "-O3 -g".

Again, it depends on the optimization level, but IMHO it's not worth it to implement an heuristic to check if gdb has enough info to run tests.

The fact that these GDB tests are not run in any CI build is bad.

gdb is not installed on Travis CI, but test_gdb is run on almost all buildbots.

We should try to catch test breakage as early as possible, the buildbot should be the last resort.

Maybe open an issue to attempt to install gdb on Travis CI?

test_gdb breakage are quiet rare. If you modify code which can impact test_gdb, you can run test_gdb locally. Because breakages are rare, I never invested time on this issue: there are more blocking issues in my agenda ;-)

[jdemeyer]

If you modify code which can impact test_gdb, you can run test_gdb locally

I do run test_gdb locally but it's not very useful since most tests are skipped by default because python_is_optimized() is True. That's the whole point of this issue: tests are skipped without a good reason.

[vstinner]

I do run test_gdb locally but it's not very useful since most tests are skipped by default because python_is_optimized() is True.

Ah, I forgot to mention that I always compile Python using:

./configure -C --with-pydebug CFLAGS="-O0" && make

-O0 is not really needed, but sometimes -Og is too aggressive and fails to read some symbols. Or maybe it's just me who is not used to -Og yet :-)

I prefer -O0 because the compilation is way faster! But don't use this config for benchmarks ;-)

[jdemeyer]

The worst debugging experience is provided by PGO build.

That's true but unrelated to this issue. We already skip test_gdb completely when PGO is enabled.

[vstinner]

That's true but unrelated to this issue. We already skip test_gdb completely when PGO is enabled.

What do you mean? support.PGO is something different: it's only used while profiling Python before Python is recompiled a second time.

[jdemeyer]

I meant this part in test_gdb.py:

if ((sysconfig.get_config_var('PGO_PROF_USE_FLAG') or 'xxx') in
(sysconfig.get_config_var('PY_CORE_CFLAGS') or '')):
raise unittest.SkipTest("test_gdb is not reliable on PGO builds")

[vstinner]

If you want to understand why tests are currently skipped when Python is optimized, you should dig into the Git history. Not simply remove the decorator because it looks wrong to you. There is likely a story behind it.

https://en.wikipedia.org/wiki/Wikipedia:Chesterton's_fence

[jdemeyer]

These python_is_optimized() tests were added in bpo-8605 and bpo-13628. Those issues talk about tests failing when CPython is compiled with -O2 or -O3. But, as I said here, I see no problems today with gcc -O3. So I'm guessing that either gcc or gdb improved such that these tests pass with more recent versions.