bpo-46329: Reduce default recursion to 800 if Py_DEBUG is enabled.

[markshannon]

Prevents C stack overflow on Clang debug builds.

This is a temporary fix to get the buildbots working again.
This should be reverted in time for the 3.11 beta.

@pablogsal
As it looks like #31011 has failed, would this be an acceptable, temporary fix?

https://bugs.python.org/issue46329

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit b2d5700 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[pablogsal]

I am ok to go this way if we don't manage to properly fix the issue today or tomorrow.

[pablogsal]

We should also consider asking clang for more stack space as someone also suggested.

[markshannon]

I think you and I have different ideas about what "properly" is in this context.
To me, that means handling C stack overflows, and that isn't happening today or tomorrow. Soon hopefully, but not that soon.

Even if #31011 had passed the tests, we would be very close to stack overflow on the clang debug build, such that any number of otherwise harmless changes could cause a stack overflow. Any "proper" fix, would need to provide a decent safety margin.

Asking clang for more stack space would work, if it has such an option.

[pablogsal]

I think you and I have different ideas about what "properly" is in this context.
To me, that means handling C stack overflows, and that isn't happening today or tomorrow. Soon hopefully, but not that soon.

Sorry, I think I didn't explain myself correctly. You are talking about some fix to avoid the C stack overflow but I'm taking about just getting the buildbots green without introducing any user visible changes (such as reducing the recursion depth in debug builds).

The issue of resolving the stack overflow problem in general still stands and indeed is worth a more propper solution as you describe 👍

[lazka]

Asking clang for more stack space would work, if it has such an option.

We use this in the mingw port for both gcc/clang: msys2-contrib@6fcebb6 to mirror a6a116c

[markshannon]

This is no longer needed, as #31035 did the job.

[vstinner]

For the specific case of clang with ./configure --with-pydebug, I wrote PR #31052 which reduces the f() stack usage from 9,104 bytes to 736 bytes.

See also https://bugs.python.org/issue46600