Assertion failure in uv__io_stop

[PromyLOPh]

• uvloop version: uvloop==0.14.0
• Python version: Python 3.7.5 (3.7.5-2~19.10)
• Platform: Ubuntu 19.10
• Can you reproduce the bug with PYTHONASYNCIODEBUG in env?: Yes.
• Does uvloop behave differently from vanilla asyncio? How?: Vanilla asyncio works fine.

The attached testcase test.zip crashes uvloop with the following message:

python: src/unix/core.c:932: uv__io_stop: Assertion `loop->watchers[w->fd] == w' failed.

This looks similar to #88. Here is a stacktrace using gdb: stacktrace.txt

The real-world application tries to read kadmin’s stdout and write to its stdin. The included mock.py replaces kadmin, but – for this issue – behaves the same. Vanilla asyncio works exactly as expected.

edit: Upon further investigation replacing the two lines

proc.stdin.write (password.encode ('utf-8'))
proc.stdin.write (b'\n')

with a single write

proc.stdin.write (password.encode ('utf-8') + b'\n')

solves the issue, which indicates a race condition somewhere inside uvloop.

[1st1]

Thanks for reporting this. We'll take a look soon.

[fantix]

This is highly-possibly caused by the last comment I put in libuv/libuv#2058, and #317 for the same reason.

[alviezhang]

@fantix
Hi, I'm using uvloop 0.15.2 version with starlette and uvicorn, and I still got coredump with message

python: src/unix/core.c:930: uv__io_stop: Assertion `loop->watchers[w->fd] == w' failed.

I tried to use gdb to print stack as below

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ff4784bb859 in __GI_abort () at abort.c:79
#2  0x00007ff4784bb729 in __assert_fail_base (fmt=0x7ff478651588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=0x7ff471c60fe5 "loop->watchers[w->fd] == w", file=0x7ff471c60f75 "src/unix/core.c", line=930,
    function=<optimized out>) at assert.c:92
#3  0x00007ff4784ccf36 in __GI___assert_fail (assertion=assertion@entry=0x7ff471c60fe5 "loop->watchers[w->fd] == w",
    file=file@entry=0x7ff471c60f75 "src/unix/core.c", line=line@entry=930,
    function=function@entry=0x7ff471c611e0 <__PRETTY_FUNCTION__.9565> "uv__io_stop") at assert.c:101
#4  0x00007ff471c3d6c1 in uv__io_stop (loop=<optimized out>, w=w@entry=0x58869a8, events=events@entry=1) at src/unix/core.c:938
#5  0x00007ff471c440d7 in uv_read_stop (stream=0x5886920) at src/unix/stream.c:1595
#6  0x00007ff471b8801a in __pyx_f_6uvloop_4loop_8UVStream__stop_reading (__pyx_v_self=0x6c5f7a0) at uvloop/loop.c:90370
#7  __pyx_f_6uvloop_4loop_8UVStream__stop_reading (__pyx_v_self=0x6c5f7a0) at uvloop/loop.c:24774
#8  0x00007ff471c2b173 in __pyx_pf_6uvloop_4loop_15UVBaseTransport_6_force_close (__pyx_v_exc=None, __pyx_v_self=0x6c5f7a0)
    at uvloop/loop.c:86079
#9  __pyx_pw_6uvloop_4loop_15UVBaseTransport_7_force_close (__pyx_v_self=<uvloop.loop.TCPTransport at remote 0x6c5f7a0>,
    __pyx_v_exc=None) at uvloop/loop.c:20461
#10 0x00007ff471b78b5d in __Pyx_PyObject_CallMethO (arg=None,
    func=<built-in method _force_close of uvloop.loop.TCPTransport object at remote 0x6c5f7a0>) at uvloop/loop.c:166801
#11 __Pyx_PyObject_CallOneArg (arg=<optimized out>,
    func=<built-in method _force_close of uvloop.loop.TCPTransport object at remote 0x6c5f7a0>) at uvloop/loop.c:35782
#12 __pyx_f_6uvloop_4loop_11SSLProtocol__abort (__pyx_v_self=<optimized out>, __pyx_v_exc=None) at uvloop/loop.c:3017
#13 0x00007ff471b5a469 in __pyx_f_6uvloop_4loop_11SSLProtocol__start_shutdown (__pyx_v_self=0x7ff450f18300,
    __pyx_optional_args=<optimized out>) at uvloop/loop.c:132598
#14 0x00007ff471bef798 in __pyx_pf_6uvloop_4loop_21_SSLProtocolTransport_10close (__pyx_v_self=0x7ff4426c33c0)
    at uvloop/loop.c:125246
#15 __pyx_pw_6uvloop_4loop_21_SSLProtocolTransport_11close (
    __pyx_v_self=<uvloop.loop._SSLProtocolTransport at remote 0x7ff4426c33c0>, unused=<optimized out>) at uvloop/loop.c:59656
#16 0x00007ff471b7ae51 in __Pyx_PyObject_CallMethO (arg=0x0,
    func=<built-in method close of uvloop.loop._SSLProtocolTransport object at remote 0x7ff4426c33c0>) at uvloop/loop.c:166801
#17 __Pyx_PyObject_CallNoArg (func=<built-in method close of uvloop.loop._SSLProtocolTransport object at remote 0x7ff4426c33c0>)
    at uvloop/loop.c:35755
#18 0x00007ff471c191e1 in __pyx_gb_6uvloop_4loop_4Loop_60generator4 (__pyx_generator=0x7ff4480224c0, __pyx_tstate=0x2830b80,
    __pyx_sent_value=<optimized out>) at /opt/python/cp38-cp38/include/python3.8/object.h:478
#19 0x00007ff471b48645 in __Pyx_Coroutine_SendEx (self=self@entry=0x7ff4480224c0, value=0x0, closing=closing@entry=0)
    at uvloop/loop.c:169733
#20 0x00007ff471b4efa3 in __Pyx_Coroutine_FinishDelegation (gen=0x7ff4480224c0) at uvloop/loop.c:169776
#21 __Pyx__Coroutine_Throw (self=self@entry=<iterable_coroutine at remote 0x7ff4480224c0>,
    typ=<CancelledError at remote 0x7ff448c10640>, val=0x0, tb=0x0, args=args@entry=(<CancelledError at remote 0x7ff448c10640>,),
    close_on_genexit=1) at uvloop/loop.c:38943
#22 0x00007ff471b4f19d in __Pyx_Coroutine_Throw (self=<iterable_coroutine at remote 0x7ff4480224c0>, args=(<CancelledError at remote 0x7ff448c10640>,)) at uvloop/loop.c:170029
#23 0x00000000005f305a in cfunction_call_varargs (kwargs=<optimized out>, args=<optimized out>, func=<built-in method throw of coroutine_wrapper object at remote 0x7ff450b0f760>) at ../Include/internal/pycore_pyerrors.h:13
#24 PyCFunction_Call (func=<built-in method throw of coroutine_wrapper object at remote 0x7ff450b0f760>, args=<optimized out>, kwargs=<optimized out>) at ../Objects/call.c:772
#25 0x00000000005f3446 in _PyObject_MakeTpCall (callable=<built-in method throw of coroutine_wrapper object at remote 0x7ff450b0f760>, args=<optimized out>, nargs=<optimized out>, keywords=<optimized out>) at ../Include/internal/pycore_pyerrors.h:13
#26 0x00000000005f3b6d in _PyObject_Vectorcall (kwnames=0x0, nargsf=1, args=0x7ffd31ad5300, callable=<built-in method throw of coroutine_wrapper object at remote 0x7ff450b0f760>) at ../Include/cpython/abstract.h:125
#27 _PyObject_Vectorcall (kwnames=0x0, nargsf=1, args=0x7ffd31ad5300, callable=<built-in method throw of coroutine_wrapper object at remote 0x7ff450b0f760>) at ../Include/cpython/abstract.h:115
#28 _PyObject_FastCall (nargs=1, args=0x7ffd31ad5300, func=<built-in method throw of coroutine_wrapper object at remote 0x7ff450b0f760>) at ../Include/cpython/abstract.h:147
#29 object_vacall (base=<optimized out>, callable=<built-in method throw of coroutine_wrapper object at remote 0x7ff450b0f760>, vargs=<optimized out>) at ../Objects/call.c:1186
#30 0x00000000005f3cf5 in PyObject_CallFunctionObjArgs (callable=<optimized out>) at ../Objects/call.c:1259
#31 0x0000000000613119 in _gen_throw (gen=0x7ff448b98040, close_on_genexit=1, typ=<optimized out>, val=<optimized out>, tb=<optimized out>) at ../Objects/genobject.c:442
#32 0x000000000061318e in _gen_throw (gen=0x7ff470181540, close_on_genexit=1, typ=<optimized out>, val=<optimized out>, tb=<optimized out>) at ../Objects/genobject.c:427
#33 0x000000000061318e in _gen_throw (gen=0x7ff4529676c0, close_on_genexit=1, typ=<optimized out>, val=<optimized out>, tb=<optimized out>) at ../Objects/genobject.c:427
...

I cannot get useful information from the stack, can you help me to confirm is it relative about this ticket? Or how can I reproduce this error?