gh-121468: Show asyncio information in pdb #124367

gaogaotiantian · 2024-09-23T19:01:31Z

This is a draft because we still need to decide on the format.

I'm aiming for something short, the original repr of asyncio task is just too long. For now it's like

gaotian@gaotian-mba cpython % ./python.exe example.py             
Task-1: <main pending>
> /Users/gaotian/programs/cpython/example.py(11)main()
-> breakpoint()

If pdb stops in an asyncio task, it will print an extra line showing the name of the task (Task-1), the name of the coroutine (main) and the status (pending).

As of now I think it will always be pending because we hit a breakpoint inside. However, we will add the feature to list all asyncio tasks in the future and allow switching between those, so we will have more status and it will be some important information.

I will work on the docs and the tests as long as the format is okay.

Issue: Make pdb asyncio aware #121468

Wulian233 · 2024-09-23T19:34:02Z

Lib/pdb.py

@@ -79,6 +79,7 @@
 import codeop
 import pprint
 import signal
+import asyncio


Line 614 has deferred import of asyncio.base_futures, so we could consider adding a lazy import of asyncio at line 606 to improve import time?

#109653
#118761

asyncio will be used as long as pdb is used. pdb is not an module that is often imported explicitly or implicitly, I don't think import time of pdb is specifically critical.

You're right. I didn't realize that at the time

kumaraditya303 · 2024-11-01T16:42:44Z

Lib/pdb.py

@@ -596,6 +602,38 @@ def _hold_exceptions(self, exceptions):
            self._chained_exceptions = tuple()
            self._chained_exception_index = 0

+    def _get_asyncio_loop_and_task(self):
+        try:
+            loop = asyncio.get_event_loop()


You can replace this with direct call to current_task and if it exists get loop from it, using get_event_loop can have unwanted consequences

Yeah I will change this.

gaogaotiantian · 2025-03-11T22:59:02Z

Okay I want to move this forward so I decided to do baby step. For this PR, I only added the $_asynctask variable. The extra print could break some tests so I removed it for now. We'll see if we ever want to implement that. We can put the information for asynctasks command in the future. This PR should be really benign as the first step to support asyncio.

Misc/NEWS.d/next/Library/2025-03-11-21-52-33.gh-issue-121468.WsEP02.rst

bedevere-bot · 2025-03-14T19:05:37Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.x (tier-3) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1594/builds/1586) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1594/builds/1586

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.14/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/data/user/0/org.python.testbed/files/python/lib/python3.14/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T19:10:34Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x (tier-3) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1380/builds/2883) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1380/builds/2883

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/E71C4EE5-FC1E-4FD1-9774-EAA918A785F6/data/Containers/Bundle/Application/940EE364-2EBF-4FC4-9C96-18A2FD8CEA2A/iOSTestbed.app/python/lib/python3.14/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/Users/buildbot/Library/Developer/XCTestDevices/E71C4EE5-FC1E-4FD1-9774-EAA918A785F6/data/Containers/Bundle/Application/940EE364-2EBF-4FC4-9C96-18A2FD8CEA2A/iOSTestbed.app/python/lib/python3.14/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T19:23:47Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Android 3.x (tier-3) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1591/builds/1479) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1591/builds/1479

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.14/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/data/user/0/org.python.testbed/files/python/lib/python3.14/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T20:00:43Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 Refleaks 3.x (tier-3) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/75/builds/2315) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/75/builds/2315

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T20:01:45Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 CentOS9 NoGIL Refleaks 3.x (tier-1) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1610/builds/1028) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1610/builds/1028

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.refleak.nogil/build/Lib/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.refleak.nogil/build/Lib/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T20:02:36Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL9 Refleaks 3.x (tier-3) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1589/builds/1115) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1589/builds/1115

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T20:03:28Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 MacOS M1 Refleaks NoGIL 3.x (tier-2) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1368/builds/2817) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1368/builds/2817

Failed tests:

test_pdb
test.test_multiprocessing_spawn.test_processes

Failed subtests:

test_release_task_refs - test.test_multiprocessing_spawn.test_processes.WithProcessesTestPool.test_release_task_refs

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_pdb.py", line 2063, in test_pdb_asynctask


Traceback (most recent call last):
  File "/Users/ec2-user/buildbot/buildarea/3.x.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/_test_multiprocessing.py", line 3052, in test_release_task_refs
    self.assertEqual(CountedObject.n_instances, 0)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 5 != 0

bedevere-bot · 2025-03-14T20:05:43Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 Refleaks 3.x (tier-1) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/259/builds/2320) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/259/builds/2320

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T20:22:33Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Refleaks 3.x (tier-3) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1613/builds/957) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1613/builds/957

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/test/test_pdb.py", line 2063, in test_pdb_asynctask

bedevere-bot · 2025-03-14T21:06:58Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Refleaks 3.x (tier-1) has failed when building commit 27fc62c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/320/builds/2113) and take a look at the build logs.
Check if the failure is related to this commit (27fc62c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/320/builds/2113

Failed tests:

test_pdb

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/doctest.py", line 2327, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_asynctask
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/test/test_pdb.py", line 2063, in test_pdb_asynctask

freakboy3742 · 2025-03-17T05:07:10Z

The iOS and Android test appear to be passing on a re-run; trying to see if I can identify the cause.

freakboy3742 · 2025-03-17T06:07:38Z

@gaogaotiantian I can confirm this isn't an iOS/Android issue - it's an issue with test ordering.

I can reproduce the test failure on macOS by running python -m test test_asyncio.test_unix_events test_pdb. I'm guessing any test that touches asyncio in the same thread as test_pdb would trigger the bug. I haven't tested this on non-Apple platforms, but I suspect the behavior would exist.

The failure isn't a failure of the test itself - it's a failure caused by the test modifying the test environment:

0:09:33 load avg: 904.14 [302/485] test_pdb
Warning -- asyncio.events._event_loop_policy was modified by test_pdb
Warning --   Before: None
Warning --   After:  <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x1044be9f0> 
0:09:42 load avg: 901.17 [302/485/1] test_pdb failed (env changed)

On a re-run, the test passes, because on entry, the test already has an event loop policy. What isn't clear to me is why this isn't a failure when you run the test by itself.

I'm not completely sure what the right fix is here. Calling asyncio.events._set_event_loop_policy(None) in the __exit__ of PdbTestInput seems to work, but I'm not sure if that would have other unintended consequences (plus, it's a deprecated method).

gaogaotiantian · 2025-03-17T16:23:10Z

@freakboy3742 Thank you for finding this. I checked the other code using asyncio.run, and they all have a policy cleanup at the end of the test. So my question is - what are we going to do with those tests? Is there a replacement for the deprecated method? What are the rules for future asyncio tests? Should we just avoid using asyncio.run in our tests (do it in a separate process)?

…n pdb (python#124367)

freakboy3742 · 2025-03-18T00:14:37Z

g> @freakboy3742 Thank you for finding this. I checked the other code using asyncio.run, and they all have a policy cleanup at the end of the test. So my question is - what are we going to do with those tests? Is there a replacement for the deprecated method? What are the rules for future asyncio tests?

I won't profess to being an asyncio expert - but my understanding is that the method is being deprecated because the underlying feature is no longer required, so there's no replacement API. My guess is that when the deprecation process is finalised, this "policy cleanup" bug will also disappear - because there's no state that will be inconsistent between the start and end of the test.

I'll push up a PR that implements this fix, and tag someone with more async experience to make sure I'm not missing anything.

Should we just avoid using asyncio.run in our tests (do it in a separate process)?

FWIW - that approach won't work (or, at least, if you use that approach, the test won't run be able to run at all on iOS, Android or Emscripten) - mobile and web platforms don't support subprocesses.

…n pdb (python#124367)

Show asyncio information in pdb

5c52a71

gaogaotiantian requested a review from iritkatriel September 23, 2024 19:01

bedevere-app bot mentioned this pull request Sep 23, 2024

Make pdb asyncio aware #121468

Open

Wulian233 reviewed Sep 23, 2024

View reviewed changes

iritkatriel requested a review from kumaraditya303 September 23, 2024 20:30

kumaraditya303 reviewed Nov 1, 2024

View reviewed changes

gaogaotiantian and others added 3 commits March 11, 2025 17:29

Merge branch 'main' into pdb-asyncio-basic

a51ba04

Remove the print

ae16c8b

📜🤖 Added by blurb_it.

490a6d9

gaogaotiantian marked this pull request as ready for review March 11, 2025 22:02

bedevere-app bot added the awaiting core review label Mar 11, 2025

gaogaotiantian added 2 commits March 11, 2025 18:04

Add test

297a684

Skip asyncio test on wasi

dcb96df

kumaraditya303 reviewed Mar 14, 2025

View reviewed changes

Misc/NEWS.d/next/Library/2025-03-11-21-52-33.gh-issue-121468.WsEP02.rst Outdated Show resolved Hide resolved

Update 2025-03-11-21-52-33.gh-issue-121468.WsEP02.rst

9c66d9b

kumaraditya303 approved these changes Mar 14, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Mar 14, 2025

gaogaotiantian merged commit 27fc62c into python:main Mar 14, 2025
43 checks passed

bedevere-app bot removed the awaiting merge label Mar 14, 2025

gaogaotiantian deleted the pdb-asyncio-basic branch March 14, 2025 18:46

gaogaotiantian mentioned this pull request Mar 14, 2025

gh-121468: Fix the doctest failure for asyncio test on pdb #131258

Merged

plashchynski pushed a commit to plashchynski/cpython that referenced this pull request Mar 17, 2025

pythongh-121468: Add current asyncio task as a convenience variable i…

7d1d4d0

…n pdb (python#124367)

freakboy3742 mentioned this pull request Mar 18, 2025

gh-121468: Ensure PDB cleans up event loop policies after using asyncio. #131388

Merged

seehwan pushed a commit to seehwan/cpython that referenced this pull request Apr 16, 2025

pythongh-121468: Add current asyncio task as a convenience variable i…

a810304

…n pdb (python#124367)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-121468: Show asyncio information in pdb #124367

gh-121468: Show asyncio information in pdb #124367

gaogaotiantian commented Sep 23, 2024 •

edited by bedevere-app bot

Loading

Wulian233 Sep 23, 2024

gaogaotiantian Sep 23, 2024

Wulian233 Sep 23, 2024

kumaraditya303 Nov 1, 2024

gaogaotiantian Nov 1, 2024

gaogaotiantian commented Mar 11, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

bedevere-bot commented Mar 14, 2025

freakboy3742 commented Mar 17, 2025

freakboy3742 commented Mar 17, 2025

gaogaotiantian commented Mar 17, 2025

freakboy3742 commented Mar 18, 2025

gh-121468: Show asyncio information in pdb #124367

gh-121468: Show asyncio information in pdb #124367

Conversation

gaogaotiantian commented Sep 23, 2024 • edited by bedevere-app bot Loading

Wulian233 Sep 23, 2024

Choose a reason for hiding this comment

gaogaotiantian Sep 23, 2024

Choose a reason for hiding this comment

Wulian233 Sep 23, 2024

Choose a reason for hiding this comment

kumaraditya303 Nov 1, 2024

Choose a reason for hiding this comment

gaogaotiantian Nov 1, 2024

Choose a reason for hiding this comment

gaogaotiantian commented Mar 11, 2025

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Mar 14, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

freakboy3742 commented Mar 17, 2025

freakboy3742 commented Mar 17, 2025

gaogaotiantian commented Mar 17, 2025

freakboy3742 commented Mar 18, 2025

gaogaotiantian commented Sep 23, 2024 •

edited by bedevere-app bot

Loading