fix: gevent_uwsgi

[ForJobsGit]

Fix delay in message processing.

python-engineio/src/engineio/async_drivers/gevent_uwsgi.py

Line 49 in a49ed4a

sel.register(fd, selectors.EVENT_READ)

EVENT_READ => EVENT_WRITE

To Reproduce

requirements.txt

uwsgi==2.0.20
gevent==21.8.0
greenlet==1.1.2
Flask==2.0.2
Werkzeug==2.0.2
Jinja2==3.0.2
Flask-SocketIO==5.1.1
python-socketio[client]==5.4.1

server.py

import gevent.monkey
gevent.monkey.patch_all()

from datetime import datetime

from flask import Flask
from flask_socketio import SocketIO

flask_server = Flask(__name__)
flask_server.config['SECRET_KEY'] = 'test'

socket_io = SocketIO(flask_server, async_mode='gevent_uwsgi')


@socket_io.on('upload_data')
def upload(*args):
    print(f'{datetime.utcnow()} upload_data {args}')

client.py

from datetime import datetime
import time
import socketio

sio = socketio.Client()
sio.connect(url='http://localhost:8080')

for i in range(0, 30):
    now = datetime.utcnow()
    sio.emit(event='upload_data', data={'emit_at': str(now)})  
    print(f'Emit i={i} at {now}')

run

uwsgi  --http 127.0.0.1:8080 --gevent 1000 --http-websockets --wsgi-file server.py --callable flask_server

server logs before fix

2021-10-19 07:42:25.841836 upload_data ({'emit_at': '2021-10-19 07:42:25.840544'},)
2021-10-19 07:42:25.842579 upload_data ({'emit_at': '2021-10-19 07:42:25.840761'},)
2021-10-19 07:42:25.843259 upload_data ({'emit_at': '2021-10-19 07:42:25.840909'},)
2021-10-19 07:42:25.843980 upload_data ({'emit_at': '2021-10-19 07:42:25.840993'},)
2021-10-19 07:42:25.844401 upload_data ({'emit_at': '2021-10-19 07:42:25.841103'},)
2021-10-19 07:42:25.844773 upload_data ({'emit_at': '2021-10-19 07:42:25.841217'},)
2021-10-19 07:42:25.845167 upload_data ({'emit_at': '2021-10-19 07:42:25.841285'},)
2021-10-19 07:42:25.845536 upload_data ({'emit_at': '2021-10-19 07:42:25.841323'},)
2021-10-19 07:42:28.847918 upload_data ({'emit_at': '2021-10-19 07:42:25.841426'},)
2021-10-19 07:42:30.852169 upload_data ({'emit_at': '2021-10-19 07:42:25.841483'},)
2021-10-19 07:42:30.853161 upload_data ({'emit_at': '2021-10-19 07:42:25.841520'},)
2021-10-19 07:42:30.853798 upload_data ({'emit_at': '2021-10-19 07:42:25.841595'},)
2021-10-19 07:42:30.854293 upload_data ({'emit_at': '2021-10-19 07:42:25.841651'},)
2021-10-19 07:42:30.854778 upload_data ({'emit_at': '2021-10-19 07:42:25.841688'},)
2021-10-19 07:42:30.855260 upload_data ({'emit_at': '2021-10-19 07:42:25.841777'},)
2021-10-19 07:42:30.855748 upload_data ({'emit_at': '2021-10-19 07:42:25.841856'},)
2021-10-19 07:42:30.856231 upload_data ({'emit_at': '2021-10-19 07:42:25.841919'},)
2021-10-19 07:42:30.856728 upload_data ({'emit_at': '2021-10-19 07:42:25.841975'},)
2021-10-19 07:42:30.857196 upload_data ({'emit_at': '2021-10-19 07:42:25.842069'},)
2021-10-19 07:42:30.857664 upload_data ({'emit_at': '2021-10-19 07:42:25.842161'},)
2021-10-19 07:42:30.858147 upload_data ({'emit_at': '2021-10-19 07:42:25.842337'},)
2021-10-19 07:42:30.858617 upload_data ({'emit_at': '2021-10-19 07:42:25.842407'},)
2021-10-19 07:42:30.859088 upload_data ({'emit_at': '2021-10-19 07:42:25.842498'},)
2021-10-19 07:42:30.859557 upload_data ({'emit_at': '2021-10-19 07:42:25.842566'},)
2021-10-19 07:42:30.860018 upload_data ({'emit_at': '2021-10-19 07:42:25.842616'},)
2021-10-19 07:42:30.860499 upload_data ({'emit_at': '2021-10-19 07:42:25.842703'},)
2021-10-19 07:42:30.860975 upload_data ({'emit_at': '2021-10-19 07:42:25.842790'},)
2021-10-19 07:42:30.861438 upload_data ({'emit_at': '2021-10-19 07:42:25.842887'},)
2021-10-19 07:42:30.861931 upload_data ({'emit_at': '2021-10-19 07:42:25.842954'},)
2021-10-19 07:42:30.862416 upload_data ({'emit_at': '2021-10-19 07:42:25.843041'},)

server logs after fix

2021-10-19 07:43:43.963762 upload_data ({'emit_at': '2021-10-19 07:43:43.960985'},)
2021-10-19 07:43:43.964624 upload_data ({'emit_at': '2021-10-19 07:43:43.962026'},)
2021-10-19 07:43:43.965339 upload_data ({'emit_at': '2021-10-19 07:43:43.962137'},)
2021-10-19 07:43:43.966081 upload_data ({'emit_at': '2021-10-19 07:43:43.962209'},)
2021-10-19 07:43:43.966541 upload_data ({'emit_at': '2021-10-19 07:43:43.962272'},)
2021-10-19 07:43:43.966939 upload_data ({'emit_at': '2021-10-19 07:43:43.962335'},)
2021-10-19 07:43:43.967345 upload_data ({'emit_at': '2021-10-19 07:43:43.962487'},)
2021-10-19 07:43:43.967726 upload_data ({'emit_at': '2021-10-19 07:43:43.962655'},)
2021-10-19 07:43:43.968191 upload_data ({'emit_at': '2021-10-19 07:43:43.962832'},)
2021-10-19 07:43:43.968551 upload_data ({'emit_at': '2021-10-19 07:43:43.962988'},)
2021-10-19 07:43:43.968906 upload_data ({'emit_at': '2021-10-19 07:43:43.963052'},)
2021-10-19 07:43:43.969321 upload_data ({'emit_at': '2021-10-19 07:43:43.963107'},)
2021-10-19 07:43:43.969961 upload_data ({'emit_at': '2021-10-19 07:43:43.963206'},)
2021-10-19 07:43:43.970574 upload_data ({'emit_at': '2021-10-19 07:43:43.963283'},)
2021-10-19 07:43:43.971074 upload_data ({'emit_at': '2021-10-19 07:43:43.963371'},)
2021-10-19 07:43:43.971600 upload_data ({'emit_at': '2021-10-19 07:43:43.963442'},)
2021-10-19 07:43:43.972203 upload_data ({'emit_at': '2021-10-19 07:43:43.963531'},)
2021-10-19 07:43:43.972738 upload_data ({'emit_at': '2021-10-19 07:43:43.963596'},)
2021-10-19 07:43:43.973208 upload_data ({'emit_at': '2021-10-19 07:43:43.963684'},)
2021-10-19 07:43:43.973685 upload_data ({'emit_at': '2021-10-19 07:43:43.963757'},)
2021-10-19 07:43:43.974224 upload_data ({'emit_at': '2021-10-19 07:43:43.963802'},)
2021-10-19 07:43:43.974922 upload_data ({'emit_at': '2021-10-19 07:43:43.963950'},)
2021-10-19 07:43:43.975585 upload_data ({'emit_at': '2021-10-19 07:43:43.964699'},)
2021-10-19 07:43:43.976247 upload_data ({'emit_at': '2021-10-19 07:43:43.964848'},)
2021-10-19 07:43:43.976932 upload_data ({'emit_at': '2021-10-19 07:43:43.965028'},)
2021-10-19 07:43:43.977655 upload_data ({'emit_at': '2021-10-19 07:43:43.965140'},)
2021-10-19 07:43:43.978312 upload_data ({'emit_at': '2021-10-19 07:43:43.965270'},)
2021-10-19 07:43:43.978902 upload_data ({'emit_at': '2021-10-19 07:43:43.965367'},)
2021-10-19 07:43:43.979506 upload_data ({'emit_at': '2021-10-19 07:43:43.965482'},)
2021-10-19 07:43:43.979927 upload_data ({'emit_at': '2021-10-19 07:43:43.965562'},)

[miguelgrinberg]

I believe your fix is incorrect. The select call that you changed is supposed to wait until there is incoming data. Waiting for write on that socket will pretty much return immediately all the time, causing the thread to spin its wheels and consume CPU with no purpose.

I do understand that there is a bug with those delays though, but just by looking at the code I cannot really find it. Your example code is going to be very useful in debugging this.

[ForJobsGit]

Thank you @miguelgrinberg for your prompt response! Close this pull request?