gh-100238: Use setuptools in peg-generator and reenable tests

[lysnikolaou]

• Issue: Update test_peg_generator to not need distutils & reenable it #100238

[pablogsal]

@lysnikolaou you need to probably rebase the changes in the PR that removes the endings from lines (also two windows tests are failing).

[pablogsal]

We still have the failing windows tests. This may be something regarding the name of the binary perhaps?

[lysnikolaou]

It appears so and I'm not sure why. The only other test that uses this (test_cppext) is skipped on Windows, so we don't really have something to compare.

[pablogsal]

It appears so and I'm not sure why

Do you know if python_d.exe is correct on Windows? I think that otherwise this line is kind of wrong:

Run: D:\a\cpython\cpython\PCbuild\win32\python_d.exe -X dev -m venv venv

[pablogsal]

Also that looks odd because there should be a temporary directory there no (as in /tmp/.../venv)?

[lysnikolaou]

Also that looks odd because there should be a temporary directory there no (as in /tmp/.../venv)?

That's okay, since that's the command that creates the venv.

[pablogsal]

Great job @lysnikolaou ❤️

[lysnikolaou]

Probabaly no backport for this, right?

[pablogsal]

Probabaly no backport for this, right?

I don't think so, but I also don't think it would hurt

[hroncok]

@pablogsal I built #105056 on top. I think it would be reasonable to backport this to 3.12, so I can backport mine as is. WDYT?

[miss-islington]

Thanks @lysnikolaou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

[bedevere-bot]

GH-105135 is a backport of this pull request to the 3.12 branch.

[hroncok]

I've asked Petr to label this with the 3.12 label so we know the CI results before a decision is made.

[encukou]

(I pushed the button to open the 3.12 PR for Miro; I'm not implying it should be merged.)

[lysnikolaou]

I'm okay with backporting, since this is just re-enabling a test, so it really wouldn't hurt. I'll leave it up to @pablogsal though.

[pablogsal]

I'm okay with backporting, since this is just re-enabling a test, so it really wouldn't hurt. I'll leave it up to @pablogsal though.

The problem with the back port is that we have ref leaks with this PR on main no? If we back port without fixing them we are going to have them also on 3.12

[lysnikolaou]

I'm okay with backporting, since this is just re-enabling a test, so it really wouldn't hurt. I'll leave it up to @pablogsal though.

The problem with the back port is that we have ref leaks with this PR on main no? If we back port without fixing them we are going to have them also on 3.12

That's right, I meant after fixing the refleaks.

[pablogsal]

I'm okay with backporting, since this is just re-enabling a test, so it really wouldn't hurt. I'll leave it up to @pablogsal though.

The problem with the back port is that we have ref leaks with this PR on main no? If we back port without fixing them we are going to have them also on 3.12

That's right, I meant after fixing the refleaks.

In that case I am ok back porting 👍

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows11 Bigmem 3.12 has failed when building commit 6371029.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1117/builds/131) and take a look at the build logs.
4. Check if the failure is related to this commit (6371029) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1117/builds/131

Summary of the results of the build (if available):

== Tests result: ENV CHANGED ==

431 tests OK.

10 slowest tests:

• test_bigmem: 43 min 23 sec
• test_lzma: 34 min 18 sec
• test_bz2: 20 min 50 sec
• test_array: 7 min 23 sec
• test_zlib: 5 min 5 sec
• test_hashlib: 4 min 3 sec
• test_multiprocessing_spawn: 2 min 1 sec
• test_math: 1 min 32 sec
• test_pickle: 1 min 18 sec
• test_concurrent_futures: 1 min 14 sec

1 test altered the execution environment:
test_peg_generator

35 tests skipped:
test.test_asyncio.test_unix_events test_curses test_dbm_gnu
test_dbm_ndbm test_devpoll test_epoll test_fcntl test_fork1
test_gdb test_grp test_ioctl test_kqueue test_multiprocessing_fork
test_multiprocessing_forkserver test_nis test_openpty
test_ossaudiodev test_perf_profiler test_perfmaps test_pipes
test_poll test_posix test_pty test_pwd test_readline test_resource
test_spwd test_syslog test_threadsignals test_wait3 test_wait4
test_xxlimited test_xxtestfuzz test_zipfile64 test_zoneinfo

Total duration: 43 min 51 sec

Click to see traceback logs

remote: Enumerating objects: 13, done.        
remote: Counting objects:   8% (1/12)        
remote: Counting objects:  16% (2/12)        
remote: Counting objects:  25% (3/12)        
remote: Counting objects:  33% (4/12)        
remote: Counting objects:  41% (5/12)        
remote: Counting objects:  50% (6/12)        
remote: Counting objects:  58% (7/12)        
remote: Counting objects:  66% (8/12)        
remote: Counting objects:  75% (9/12)        
remote: Counting objects:  83% (10/12)        
remote: Counting objects:  91% (11/12)        
remote: Counting objects: 100% (12/12)        
remote: Counting objects: 100% (12/12), done.        
remote: Compressing objects:  11% (1/9)        
remote: Compressing objects:  22% (2/9)        
remote: Compressing objects:  33% (3/9)        
remote: Compressing objects:  44% (4/9)        
remote: Compressing objects:  55% (5/9)        
remote: Compressing objects:  66% (6/9)        
remote: Compressing objects:  77% (7/9)        
remote: Compressing objects:  88% (8/9)        
remote: Compressing objects: 100% (9/9)        
remote: Compressing objects: 100% (9/9), done.        
remote: Total 13 (delta 3), reused 8 (delta 3), pack-reused 1        
From https://github.com/python/cpython
 * branch                  3.12       -> FETCH_HEAD
Note: switching to '637102980d12e5ff99b69ddffd5f05ecfe0adeb6'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 637102980d [3.12] gh-100238: Use setuptools in peg-generator and reenable tests (GH-104798) (#105135)
Switched to and reset branch '3.12'

Could Not Find R:\buildarea\3.12.ambv-bb-win11.bigmem\build\Lib\*.pyc
The system cannot find the file specified.
Could Not Find R:\buildarea\3.12.ambv-bb-win11.bigmem\build\PCbuild\python*.zip

Could Not Find R:\buildarea\3.12.ambv-bb-win11.bigmem\build\PCbuild\python*.zip