A test request from an older client can permanently freeze the server

[annualp]

I have found that an older client sending a test request will make the iperf3 server (version 3.0.9) completely unresponsive. The test request from the client does not work and the server will no longer answer any more test requests. When I do an strace and attach to the server when this problem is happening I see that it is stuck in a read attempt.

strace -p 792
Process 792 attached - interrupt to quit
read(4,

I have tested this with the server running on two different machines.

first machine:
Linux dev-ftp 3.2.0-40-generic-pae #64-Ubuntu SMP Mon Mar 25 21:44:41 UTC 2013 i686 i686 i386 GNU/Linux

second machine:
Linux G06079 2.6.32.13-0 #1 SMP Fri May 14 10:01:26 CDT 2010 i686 i686 i386 GNU/Linux

The older client I am using is a Windows port. I am aware that Windows is not a supported platform, but if you Google "iperf3 windows" the result on sourceforge.net (http://sourceforge.net/projects/iperf3cygwingui/) is one of the top results. It is quite likely someone would try and use this. I downloaded iperf3cygwingui-trunk318-03-src.zip and this is the version information it reports when I run iperf3 from the Windows command line:

iperf version 3.0b4-trunk318-03-20120720 (2 Aug 2010)

Note: this problem seems similar to the (closed) problem #202 (Connecting to server process during an existing test freezes the server). However, with that problem the server's functionality was restored after the connection attempt failed. With this problem the server remains unresponsive.

[bmah888]

The server should be more robust than that. Can you give command lines for both the client and server that you're using?

I'm having some difficulty reproducing this problem because I so far have not been able to get a working iperf 3.0b4 binary on CentOS 6.

[annualp]

I would be happy to provide more detail. You actually don't need to get a working iperf 3.0b4 binary on CentOS 6. I'm running the iperf 3.0b4 binary as a client on the Windows 7 machine. The 3.0.9 server is running on the CentOS PC and an Ubuntu PC.
I'm actually seeing different behavior now and I'm not sure why. The issue I'm seeing is still similar to problem #202. I'll explain below.

I made some version information appear and then I started the server on the Ubuntu machine:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.5 LTS
Release: 12.04
Codename: precise

$ iperf3 -v
iperf 3.0.9
Linux dev-ftp 3.2.0-40-generic-pae #64-Ubuntu SMP Mon Mar 25 21:44:41 UTC 2013 i686 i686 i386 GNU/Linux

$ iperf3 -s -D

Then on the Windows 7 Professional PC I run the client. I replaced the IP address with "IP" in this email:

iperf3 -v
iperf version 3.0b4-trunk318-03-20120720 (2 Aug 2010)

iperf3 -c IP
Connecting to host IP, port 5201

The client then hangs. No other tests can be performed against the server while the client is in this state. I left the client like this for 15 minutes and kept trying to perform tests from 3.0.9 Linux client but the tests did not work. Once I killed the Windows client the Linux 3.0.9 client was able to run tests again. So the good news is that the 3.0.9 server works as expected when the Windows client disconnects, but it can remain unresponsive for a long time while the Windows client stays connected. I'm not necessarily concerned with getting a working Windows client. I just want to make sure that the server that I'm running can't be disabled.

[bmah888]

It's easier to get a CentOS binary than run somebody's pre-compiled Windows binary because I have no Windows hosts, and no desire to set any up. :-)

The test results you are now seeing are consistent with what I was (finally) able to reproduce. Perhaps some sort of timeout on the server side might be in order. I need to look at this code a little more carefully.

[annualp]

I do think that a timeout on the server would be a good idea. If you need me to help with testing anything I would be happy to.

[rpodgorny]

any progress on this?

[bmah888]

Not really. On a marginally related note, we do know that there was "something" that happened between the iperf3 betas and the GA release that broke compatibility.

[bmah888]

This is still an issue...trivially easy to do by firing up an iperf3 server and then connecting to the server's port (i.e. with telnet or nc) and then not doing anything. One thing that's a little complicated is that the client can stop talking to the server at any point and cause the server to stall out.

[loxK]

I have this issue too. I have scheduled bandwidth monitoring between two nodes. When the connection fails during a bandwidth measurement the iperf3 server node just hangs forever.

[davidBar-On]

Submitted PR #1101 with proposed fix to the issue that should prevent the server from getting stack because of client or network problems.

[annualp]

Thank you for addressing this!