Using uvloop

[jakirkham]

I'm curious to know if we have looked into using uvloop with Distributed recently (particularly on the Scheduler). There are some older threads that mention this, but some of them are 2-3 years old. Here's one on the worker ( #2156 ), communication ( #2162 ) ( #3145 ), and some others can be found as well (usually thinking about communication). What I'm more interested in knowing than the communication bit, is if having enough of the async bits Cythonized ends up helping overall Scheduler runtime?

[jakirkham]

Occasionally Sanic comes up in this context as an alternative to Tornado. Though IIUC outside of using uvloop, which Tornado could also use, it uses httptools to parse URLs in C, which is faster, but probably not very important to us (unless I'm missing something). So not sure a Sanic vs. Tornado comparison would be interesting, but could be wrong about that.

[mrocklin]

I've used uvloop before. It works fine and is easy to integrate. In my experience the best thing about it was reducing effective network latency. If you want to add it to the benchmarking/profiling process I think that that would be sensible. If you wanted to make it a config variable and turn it on automatically if that variable is set then that might be an easy thing to do. distributed: event-loop: "tornado" # "uvloop"

…

On Wed, Dec 16, 2020 at 12:16 PM jakirkham ***@***.***> wrote: Occasionally Sanic comes up in this context as an alternative to Tornado. Though IIUC outside of using uvloop, which Tornado could also use, it uses httptools to parse URLs in C, which is faster, but probably not very important to us (unless I'm missing something). So not sure a Sanic vs. Tornado comparison would be interesting, but could be wrong about that. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4369 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTBIRQ3ECRVTO5QFDODSVEISNANCNFSM4U6NIBTA> .

[jakirkham]

This is being addressed in PR ( #4448 )