test_frame fails when running with -R 3:3 argument

[Eclips4]

Bug report

Bug description:

./python.exe -m test -R 3:3 test_frame
Using random seed: 617081239
0:00:00 load avg: 38.69 Run 1 test sequentially
0:00:00 load avg: 38.69 [1/1] test_frame
beginning 6 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123:456
Xtest test_frame failed -- Traceback (most recent call last):
  File "/Users/admin/Projects/cpython/Lib/test/test_frame.py", line 353, in test_sneaky_frame_object
    self.assertIs(g.gi_frame, sneaky_frame_object)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: <frame at 0x1063d5020, file '/Users/admin/Projects/cpython/Lib/test/test_frame.py', line 318, code f> is not <frame at 0x10637b9b0, file '/Users/admin/Projects/cpython/Lib/test/test_frame.py', line 353, code test_sneaky_frame_object>

test_frame failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_frame

Total duration: 146 ms
Total tests: run=21 failures=1
Total test files: run=1/1 failed=1
Result: FAILURE

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-116098: Revert "gh-107674: Improve performance of sys.settrace (GH-114986)" #116178
• gh-116098: Clean up frame allocation code and remove the invalid sneaky frame test #116687

[Eclips4]

Bisected to 0a61e23
cc @gaogaotiantian

[gaogaotiantian]

Okay it took me a while, but I don't believe this is a "bug" the commit brings - more like a very deligate(fragile) test is broken because it depends on some undocumented(undefined) behavior.

I believe @brandtbucher is the author for the test. If I understand it correctly, the test requires a gc triggers in trace function - which is not guaranteed. Normally RESUME and INSTRUMENTED_RESUME triggers the gc, but RESUME_CHECK does not. I think it's intentional @markshannon ? The commit mentioned (correctly?) converts the RESUME instruction for trace to RESUME_CHECK which is not able to trigger gc anymore, thus fails the test.

So we need to confirm a couple of things before fixing this:

1. Should we fix the test, or the code?
2. Is it reasonable for the test to require a gc in trace?
3. Is it intentional that RESUME_CHECK does not trigger gc? Is that okay?
4. Is it valid to have RESUME_CHECK for trace instead of RESUME or INSTRUMENTED_RESUME?

[gaogaotiantian]

If we need further discussion for this and can't come up with a fix today, I'll revert this commit by the end of today and wait for the final decision.

[encukou]

Thank you for the revert.
Sorry that it's your commit that got caught here. Reverting is a way to get the buildbots green so they can catch more issues, not a judgement of your work.

[gaogaotiantian]

I understand - it's important to keep the test clean otherwise the test lost its meaning. I'll work on the fix and hopefully we can come up with something and check it back in.

[Eclips4]

@gaogaotiantian can it be closed now?

[gaogaotiantian]

Yes. The root cause was that the test is not valid anymore and we removed the test.