[Bug]: async_playwright context manager hangs on exit if you ran something in a ProcessPool

[rhettlunn]

Playwright version

1.17.2

Operating system

Linux

What browsers are you seeing the problem on?

Chromium

Other information

Tested in python 3.7.3 and python 3.8.10

What happened? / Describe the bug

Observed behavour:
I'm developing something that is working with both playwright and other cpu intensive processing at the same time, so with playwright open I call out to run something in a process pool. That works, but then when I'm leaving the playwright context block, playwright hangs.

Expected behaviour:
Running something in a process pool while playwright is open will have no effect on playwright itself, and it will just leave the context block as normal.

For the code snippet:
You can comment/uncomment whichever asyncio.run you want in the if __name__ == "__main__": - the function names just describe if they hang or not.
The two lines at the top of the file are sort of a workaround, in that if you uncomment them it doesn't hang, but 'spawn' has other problems, and the default ('fork') is preferred.

If I do run with set_start_method('spawn', True), I get the following output, which may be useful in debugging:

playwright is open
<_UnixSelectorEventLoop running=True closed=False debug=False>
sync_function: args
Result of sync_function
We _do_ get here
We don't get here
/usr/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 5 leaked semaphores to clean up at shutdown
  len(cache))
/usr/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 5 leaked semaphores to clean up at shutdown
  len(cache))
/usr/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 5 leaked semaphores to clean up at shutdown
  len(cache))
/usr/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 5 leaked semaphores to clean up at shutdown
  len(cache))

Actually the last function ( yet_another_thing_that_does_not_hang ) may be the best clue as to what's happening: if I run something in the process pool before entering async_playwright() then running a thing inside there doesn't fork a new process, and the context exits cleanly

Code snippet to reproduce your bug

# import multiprocessing
# multiprocessing.set_start_method('spawn', True)

import asyncio
import concurrent.futures

from playwright.async_api import async_playwright

process_pool = concurrent.futures.ProcessPoolExecutor(max_workers=4)
thread_pool = concurrent.futures.ThreadPoolExecutor(max_workers=4)


async def async_function_processpool(args):
    loop = asyncio.get_event_loop()
    print(loop)
    return await loop.run_in_executor(
        process_pool,
        sync_function,
        args,
    )


async def async_function_threadpool(args):
    loop = asyncio.get_event_loop()
    print(loop)
    return await loop.run_in_executor(
        thread_pool,
        sync_function,
        args,
    )


def sync_function(args):
    print("sync_function:", args)
    return "Result of sync_function"


async def this_hangs():
    print("Begin hanging procedure")
    async with async_playwright():
        print("playwright is open")
        print(await async_function_processpool("args"))
        print("We _do_ get here")
    print("We don't get here")


async def this_does_not_hang():
    await async_function_processpool("args")


async def this_also_does_not_hang():
    async with async_playwright():
        pass


async def this_also_also_does_not_hang():
    async with async_playwright():
        print(await async_function_threadpool("args"))


async def yet_another_thing_that_does_not_hang():
    await async_function_processpool("args")
    async with async_playwright():
        print(await async_function_processpool("args"))


if __name__ == "__main__":
    asyncio.run(this_hangs())
    # asyncio.run(this_does_not_hang())
    # asyncio.run(this_also_does_not_hang())
    # asyncio.run(this_also_also_does_not_hang())
    # asyncio.run(yet_another_thing_that_does_not_hang())

Relevant log output

No response

[rwoll]

Closing as part of triage due to inactivity. Feel free to re-file if this issue blocks the usage of Playwright