Some tests fail randomly when using the trio runner

[gsalgado]

test_open_proc_SIGINT_can_be_ignored, test_open_proc_SIGINT_while_running and test_open_proc_with_trio_KeyboardInterrupt_while_running all fail randomly when using the open_in_process_with_trio() runner. That happens because sometimes the runner raises a RuntimeError like shown below

<function test_open_proc_SIGINT_while_running.<locals>.do_sleep_forever at 0x7f214a0c10d0> raised an unexpected exception: Cancel scope stack corrupted: attempted to exit <trio.CancelScope at 0x7f214a0f6280, active, cancelled> in <Task '__main__._do_async_fn' at 0x7f214a0f28e0> that's still within its child <trio.CancelScope at 0x7f214a0f6460, active, cancelled>

This is probably a bug in your code, that has caused Trio's internal state to
become corrupted. We'll do our best to recover, but from now on there are
no guarantees.

Typically this is caused by one of the following:
  - yielding within a generator or async generator that's opened a cancel
    scope or nursery (unless the generator is a @contextmanager or
    @asynccontextmanager); see https://github.com/python-trio/trio/issues/638
  - manually calling __enter__ or __exit__ on a trio.CancelScope, or
    __aenter__ or __aexit__ on the object returned by trio.open_nursery();
    doing so correctly is difficult and you should use @[async]contextmanager
    instead, or maybe [Async]ExitStack
  - using [Async]ExitStack to interleave the entries/exits of cancel scopes
    and/or nurseries in a way that couldn't be achieved by some nesting of
    'with' and 'async with' blocks
  - using the low-level coroutine object protocol to execute some parts of
    an async function in a different cancel scope/nursery context than
    other parts
If you don't believe you're doing any of these things, please file a bug:
https://github.com/python-trio/trio/issues/new

[pipermerriam]

Ick. I've run into this error from trio elsewhere and I never figured out how to properly address it... I think I ended dropping some async generators I was using in favor of other mechanisms.

[gsalgado]

Yeah, I've had to deal the same issue in https://github.com/ethereum/trinity/blob/d80a3b1fe5661d5108c9f10ddd077312ad4661a9/p2p/discovery.py#L1349-L1368

Just need to find what's causing it here

[gsalgado]

I can't seem to find out what is causing this. At first I thought it was the signal asynciterator in

asyncio-run-in-process/asyncio_run_in_process/_child_trio.py

Line 45 in d0c0d16

with trio.open_signal_receiver(*SHUTDOWN_SIGNALS) as signal_aiter:

but even after removing that it still happens

[gsalgado]

One thing that makes the failure disappear is changing the sleep(0) below to, say, a sleep(0.01):

asyncio-run-in-process/tests/core/test_open_in_process.py

Lines 93 to 97 in d0c0d16

	@pytest.mark.asyncio
	async def test_open_proc_SIGINT_while_running(open_in_proc):
	async def do_sleep_forever():
	while True:
	await sleep(0)

I've no idea why, though, but it looks like another race condition that's only triggered by our tests that run not-very-realistic functions in the subprocess and kill the subprocess immediately after they're started