gh-106368: Add tests for permutation helpers in Argument Clinic

[erlend-aasland]

Added new test class PermutationTests()

• Issue: Increase Argument Clinic test coverage #106368

[sobolevn]

Looks good to me! 👍

[miss-islington]

Thanks @erlend-aasland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[bedevere-bot]

GH-106409 is a backport of this pull request to the 3.12 branch.

[bedevere-bot]

GH-106410 is a backport of this pull request to the 3.11 branch.

[erlend-aasland]

Hm, I should have put these in the existing ClinicGroupPermuterTest test case. Let's keep that in mind and merge those two later, when we get the chance.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Arch Linux TraceRefs 3.x has failed when building commit 8f6df5e.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/484/builds/3602) and take a look at the build logs.
4. Check if the failure is related to this commit (8f6df5e) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/484/builds/3602

Failed tests:

• test__xxsubinterpreters

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 34, done.        
remote: Counting objects:   2% (1/34)        
remote: Counting objects:   5% (2/34)        
remote: Counting objects:   8% (3/34)        
remote: Counting objects:  11% (4/34)        
remote: Counting objects:  14% (5/34)        
remote: Counting objects:  17% (6/34)        
remote: Counting objects:  20% (7/34)        
remote: Counting objects:  23% (8/34)        
remote: Counting objects:  26% (9/34)        
remote: Counting objects:  29% (10/34)        
remote: Counting objects:  32% (11/34)        
remote: Counting objects:  35% (12/34)        
remote: Counting objects:  38% (13/34)        
remote: Counting objects:  41% (14/34)        
remote: Counting objects:  44% (15/34)        
remote: Counting objects:  47% (16/34)        
remote: Counting objects:  50% (17/34)        
remote: Counting objects:  52% (18/34)        
remote: Counting objects:  55% (19/34)        
remote: Counting objects:  58% (20/34)        
remote: Counting objects:  61% (21/34)        
remote: Counting objects:  64% (22/34)        
remote: Counting objects:  67% (23/34)        
remote: Counting objects:  70% (24/34)        
remote: Counting objects:  73% (25/34)        
remote: Counting objects:  76% (26/34)        
remote: Counting objects:  79% (27/34)        
remote: Counting objects:  82% (28/34)        
remote: Counting objects:  85% (29/34)        
remote: Counting objects:  88% (30/34)        
remote: Counting objects:  91% (31/34)        
remote: Counting objects:  94% (32/34)        
remote: Counting objects:  97% (33/34)        
remote: Counting objects: 100% (34/34)        
remote: Counting objects: 100% (34/34), done.        
remote: Compressing objects:   5% (1/17)        
remote: Compressing objects:  11% (2/17)        
remote: Compressing objects:  17% (3/17)        
remote: Compressing objects:  23% (4/17)        
remote: Compressing objects:  29% (5/17)        
remote: Compressing objects:  35% (6/17)        
remote: Compressing objects:  41% (7/17)        
remote: Compressing objects:  47% (8/17)        
remote: Compressing objects:  52% (9/17)        
remote: Compressing objects:  58% (10/17)        
remote: Compressing objects:  64% (11/17)        
remote: Compressing objects:  70% (12/17)        
remote: Compressing objects:  76% (13/17)        
remote: Compressing objects:  82% (14/17)        
remote: Compressing objects:  88% (15/17)        
remote: Compressing objects:  94% (16/17)        
remote: Compressing objects: 100% (17/17)        
remote: Compressing objects: 100% (17/17), done.        
remote: Total 20 (delta 16), reused 6 (delta 3), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '8f6df5e9cbc3a1689601714192aa6ecbb23e1927'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 8f6df5e9cb gh-106368: Add tests for permutation helpers in Argument Clinic (#106407)
Switched to and reset branch 'main'

Objects/object.c:2197: _Py_ForgetReference: Assertion failed: invalid object chain
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x7fc2dd670ee0
object refcount : 0
object type     : 0x55b810834bc0
object type name: str
object repr     : <refcnt 0 at 0x7fc2dd670ee0>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007fc2ef2de740 (most recent call first):
  <no Python frame>
Debug memory block at address p=0x7fc2ef205e00: API '�'
    18302063728033390045 bytes originally requested
    The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd):
        at p-7: 0xdd *** OUCH
        at p-6: 0xdd *** OUCH
        at p-5: 0xdd *** OUCH
        at p-4: 0xdd *** OUCH
        at p-3: 0xdd *** OUCH
        at p-2: 0xdd *** OUCH
        at p-1: 0xdd *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 8 pad bytes at tail=0xfdfe7dc0ed1e3bdd are Fatal Python error: Segmentation fault

Current thread 0x00007fc2ef2de740 (most recent call first):
  <no Python frame>

Extension modules: _testcapi, _xxsubinterpreters (total: 2)
make: *** [Makefile:2010: buildbottest] Segmentation fault (core dumped)

Cannot open file '/buildbot/buildarea/3.x.pablogsal-arch-x86_64/build/test-results.xml' for upload

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 LTO 3.11 has failed when building commit 9f8aa7a.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/924/builds/913) and take a look at the build logs.
4. Check if the failure is related to this commit (9f8aa7a) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/924/builds/913

Failed tests:

• test___all__
• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script
• test_all - test.test___all__.AllTest.test_all

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

413 tests OK.

10 slowest tests:

• test_tools: 3 min 39 sec
• test_concurrent_futures: 2 min 27 sec
• test_multiprocessing_spawn: 2 min 14 sec
• test_multiprocessing_forkserver: 1 min 24 sec
• test_multiprocessing_fork: 1 min 14 sec
• test_tokenize: 1 min 9 sec
• test_asyncio: 1 min 4 sec
• test_peg_generator: 49.5 sec
• test_signal: 48.1 sec
• test_unparse: 41.5 sec

1 test failed:
test_tools

20 tests skipped:
test_devpoll test_gdb test_idle test_ioctl test_kqueue
test_launcher test_msilib test_smtpnet test_ssl test_startfile
test_tcl test_tix test_tk test_ttk_guionly test_ttk_textonly
test_turtle test_winconsoleio test_winreg test_winsound
test_zipfile64

2 re-run tests:
test___all__ test_tools

Total duration: 9 min 58 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_tools/test_freeze.py", line 28, in test_freeze_simple_script
    executable = helper.freeze(python, scriptfile, outdir)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 198, in freeze
    _run_quiet([MAKE, '-C', os.path.dirname(scriptfile)])
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpuzyw3yoh']' returned non-zero exit status 2.


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_79b4645b'


Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/tmpuzyw3yoh/cpython/Lib/sysconfig.py", line 715, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/tmpuzyw3yoh/cpython/Lib/sysconfig.py", line 670, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/tmpuzyw3yoh/cpython/Lib/sysconfig.py", line 531, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata__linux_s390x-linux-gnu'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_tools/test_freeze.py", line 28, in test_freeze_simple_script
    executable = helper.freeze(python, scriptfile, outdir)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 198, in freeze
    _run_quiet([MAKE, '-C', os.path.dirname(scriptfile)])
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmp4po6o4wf']' returned non-zero exit status 2.


Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/tmp4po6o4wf/cpython/Lib/sysconfig.py", line 715, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/tmp4po6o4wf/cpython/Lib/sysconfig.py", line 670, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/tmp4po6o4wf/cpython/Lib/sysconfig.py", line 531, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata__linux_s390x-linux-gnu'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test___all__.py", line 129, in test_all
    self.check_all(modname)
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test___all__.py", line 43, in check_all
    with warnings_helper.check_warnings(
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/support/warnings_helper.py", line 185, in _filterwarnings
    raise AssertionError("unhandled warning %s" % reraise[0])
AssertionError: unhandled warning {message : DeprecationWarning("invalid escape sequence '\\?'"), category : 'DeprecationWarning', filename : '/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_httpservers.py', lineno : 445, line : None}