-
-
Notifications
You must be signed in to change notification settings - Fork 588
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RuntimeError: Second simultaneous read on fileno #13
Comments
Include the full stack trace of the error please, and make sure you have monkey patched the standard library. |
I did the monkey_patch dance.
|
Do you know how many active clients did you have when this happened? |
I opened 1 browser window. The problem might come from the changes to _listen. If I make the following changes:
The problem goes aways. |
Okay, so your solution is basically to read and write to the queue using different connections. Can't find any references in the documentation regarding the thread safety of a kombu connection, and none of the examples seem to have the producer and the consumer working concurrently on the same process. Thanks for your analysis, I'll probably go about this with your solution. |
@NotSureAbout Unfortunately your fix does not work for me. The error goes away, but messages that the server sends do not make it to the client because the I'll continue investigating. I haven't really found any similar working examples, pretty much all Kombu examples I have found have the producer and the consumer in different processes, not sure how well tested is the multithreaded approach that I'm using. |
@NotSureAbout Can I ask you to retest with the current master branch to confirm the problem is addressed for you? |
Works for me. |
I am using the latest release of the great flask-socketio library, along with eventlet. I came across an issue in my code that I can't quite figure out but it seems that it may be related to this issue. I have the following code
In a separate process (a celery task; celery is run with eventlet enabled) I call:
This task gets called quite frequently and so it is not surprising that multiple threads ended up calling the above line simultaneously. When that happens I end up with the following stack trace:
This error message seems fairly similar to the one listed by @NotSureAbout. I am very interested to know if you have any insight as to what I might be doing wrong. Thanks in advance. |
As an update, I have tried two things.
|
@pmarti28 the error is the same, but your problem is different, I think. In your case, there actually is a legit simultaneous access of a handle by two or more threads, because you are emitting with the same Your solution removes the stored file handle, and creates a new one each time a write is made. Clearly this addresses the problem, but I'm concerned about the performance cost of constantly having to open new handles. As a short term solution I think your change is fine, but I'm not sure how I feel about penalizing the majority of users who do not have a set up like yours. An alternative solution that I have used, is to not run celery under eventlet. In my opinion there is no reason to switch Celery to eventlet, in general Celery works better with a pool of process workers of a fixed size. Running Celery w/o eventlet will address this problem, because each worker process will have a separate instance of A long term solution for this problem is, I think, something that works similarly to context variables in Flask would make sense. That would allow a different file handle to be kept for each thread, all transparently done for you. |
@miguelgrinberg thanks for the thorough explanation. I will look into switching Celery away from eventlet. For the long term solution, I will come back here if I think of anything that might be useful. |
Does not seem to work for me.
I launch the app with:
I would more than welcome any pointers to where I have messed up.
The text was updated successfully, but these errors were encountered: