Avoid terminating connections prematurely #1511
Conversation
While sending very large messages (far beyond what fits in a the tcp buffer, so it takes multiple sendto system calls for it to finish), zmq_close will close the connection regardless of ZMQ_LINGER. In case no engine is attached, a pipe->check_read() is needed to look for the delimiter in the pipe and ultimately trigger the pipe termination. However, if there *is* an engine attached, the check_read() looks ahead and finds the delimiter and terminates the connection even though the engine might actually still be in the middle of sending a message. This happens because while the io_thread is still busy sending the data, the pipe can get terminated and the io thread ends up being terminated.
Avoid terminating connections prematurely
|
Thanks! I would have left a comment on https://zeromq.jira.com/browse/LIBZMQ-554 and https://zeromq.jira.com/browse/LIBZMQ-551 asking them to test, but I can't seem to create an account on Jira. It says my domain is not allowed. |
|
AFAIK Jira is not used anymore and all issue tracking happens on Github. On Thu, Aug 6, 2015 at 8:54 PM, Soren Hansen notifications@github.com
Kapp Arnaud - Xaqq |
|
Oh, ok. Thanks! |
|
Does anyone know the status of this fix, as it relates to a zmq release? Is it in a release, or not yet in a release? I see this fix was committed Aug 6, 2015. The company product I work on makes extensive use of ZMQ 3 Push Pull, sometimes in a connect-send-disconnect model where because of this bug messages are lost. We are VERY interested in upgrading to a version of ZMQ that contains this fix. Our workarounds for this bug are proving to be unreliable. |
|
@mikedevx if it helps, we can push forwards with a 4.2 stable release. |
|
Sure. It's also reasonable to make a release candidate with some pieces still needing work. |
|
Thank you! Thank you again! From: Pieter Hintjens [mailto:notifications@github.com] @mikedevxhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_mikedevx&d=CwMCaQ&c=8S5idjlO_n28Ko3lg6lskTMwneSC-WqZ5EBTEEvDlkg&r=KbV3rp4q2lvwTUd_oq0LWShyI451vfiOLjtzYiTyUso&m=r8Z2bVkQePtNBGjllCd9zlajuV6spFKA8Torp5MvM04&s=BivhrGCph1cr8EaQVp8vAjDjDtau9CHc7YF2-ka7HG4&e= if it helps, we can push forwards with a 4.2 stable release. — The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt. |
|
Good stuff.
|
|
@hintjens - What about backporting this fix to 4.0 and 4.1? Seems easy enough, I can do that if you want. |
|
@bluca yes, nice. We have an issue, so if you could do that and update the NEWS in each case, that'd be awesome... |
|
Sure! Is LIBZMQ-554 the issue? There was no mention of one in this PR |
|
Ah, I was confusing #1511 with an issue. We need an explicit issue then. |
|
Will open one. |
While sending very large messages (far beyond what fits in a the tcp
buffer, so it takes multiple sendto system calls for it to finish),
zmq_close will close the connection regardless of ZMQ_LINGER.
In case no engine is attached, a pipe->check_read() is needed to look
for the delimiter in the pipe and ultimately trigger the pipe
termination.
However, if there is an engine attached, the check_read() looks ahead
and finds the delimiter and terminates the connection even though the
engine might actually still be in the middle of sending a message.
This happens because while the io_thread is still busy sending the data,
the pipe can get terminated and the io thread ends up being terminated.