Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close socket when connection was not accepted #977

Closed
wants to merge 1 commit into from

Conversation

mikepurvis
Copy link
Member

Retry #960, from @mgrrx.

We haven't been able to get a MWE for the problems we saw with this merged, but it came up in our crash reporter originating from unit test runs, workstations, and also robots. An example stack trace:

  File "../nptl/sysdeps/unix/sysv/linux/raise.c", line 56, in __GI_raise
  File "abort.c", line 89, in __GI_abort
  File "../sysdeps/posix/libc_fatal.c", line 175, in __libc_message
  File "fortify_fail.c", line 38, in __GI___fortify_fail
  File "chk_fail.c", line 28, in __GI___chk_fail
  File "fdelt_chk.c", line 25, in __fdelt_chk
  File "../../../../../ros-comm-f6da1ec87d6ed4653e65ac2dbae1325f7bc1238d/utilities/xmlrpcpp/src/XmlRpcDispatch.cpp", line 100, in XmlRpc::XmlRpcDispatch::work
  File "../../../../../ros-comm-f6da1ec87d6ed4653e65ac2dbae1325f7bc1238d/utilities/xmlrpcpp/src/XmlRpcServer.cpp", line 133, in XmlRpc::XmlRpcServer::work
  File "../../../../../ros-comm-f6da1ec87d6ed4653e65ac2dbae1325f7bc1238d/clients/roscpp/src/libros/xmlrpc_manager.cpp", line 272, in ros::XMLRPCManager::serverThreadFunc
  File "/usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0", in ?? ()
  File "pthread_create.c", line 312, in start_thread
  File "../sysdeps/unix/sysv/linux/x86_64/clone.S", line 111, in clone

It's possible that the crashes were coming up during node shutdown, but that still fails affected tests and is a regression from previously.

trainman419 added a commit to trainman419/ros_comm that referenced this pull request Nov 22, 2017
Add integration tests for the XmlRpcServer interacting with an
XmlRpcClient in the same process, particularly around the behavior when
the server process runs out of available file handles.

Update the XmlRpcServer with two different mitigations for file handle
exhaustion (Fixes ros#914, replaces ros#960 and ros#977):
 * Measure the number of free file handles and reject incoming
connections if the pool of free file descriptors is too small. This
actively rejects incoming clients instead of waiting for complete file
handle exhaustion, which would leave clients in a pending state until a
file descriptor becomes free.
 * If accept fails due to complete file handle exhaustion, temporarily
stop calling accept on this socket. This prevents a busy-loop where
poll() believes the listening socket is readable, but accept() fails to
allocate a file descriptor and leaves the socket in a readable state.
trainman419 added a commit to trainman419/ros_comm that referenced this pull request Dec 14, 2017
Add integration tests for the XmlRpcServer interacting with an
XmlRpcClient in the same process, particularly around the behavior when
the server process runs out of available file handles.

Update the XmlRpcServer with two different mitigations for file handle
exhaustion (Fixes ros#914, replaces ros#960 and ros#977):
 * Measure the number of free file handles and reject incoming
connections if the pool of free file descriptors is too small. This
actively rejects incoming clients instead of waiting for complete file
handle exhaustion, which would leave clients in a pending state until a
file descriptor becomes free.
 * If accept fails due to complete file handle exhaustion, temporarily
stop calling accept on this socket. This prevents a busy-loop where
poll() believes the listening socket is readable, but accept() fails to
allocate a file descriptor and leaves the socket in a readable state.
dirk-thomas pushed a commit that referenced this pull request Dec 19, 2017
* Tests and bug fixes for XmlRpcServer

Add integration tests for the XmlRpcServer interacting with an
XmlRpcClient in the same process, particularly around the behavior when
the server process runs out of available file handles.

Update the XmlRpcServer with two different mitigations for file handle
exhaustion (Fixes #914, replaces #960 and #977):
 * Measure the number of free file handles and reject incoming
connections if the pool of free file descriptors is too small. This
actively rejects incoming clients instead of waiting for complete file
handle exhaustion, which would leave clients in a pending state until a
file descriptor becomes free.
 * If accept fails due to complete file handle exhaustion, temporarily
stop calling accept on this socket. This prevents a busy-loop where
poll() believes the listening socket is readable, but accept() fails to
allocate a file descriptor and leaves the socket in a readable state.

* Add test depend on boost

* Run astyle
@dirk-thomas
Copy link
Member

Closing due to long time of inactivity. Please consider opening a new PR targeting the current default branch if you are still interested in getting this patch merged.

@dirk-thomas dirk-thomas closed this Feb 4, 2020
@dirk-thomas dirk-thomas deleted the pr960 branch February 4, 2020 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants