Error 24: Too many open files

[ArthurDents]

Hi
I have cross compiled the uftpd for use on my embedded arm platform, and I have started doing some stress tests. The test script repeatedly:

• logs in to the ftp server
• runs “get testfile.txt”
• close the connection.

When the script reaches approximately round 1015 the uftpd server generates an error message:
Failed opening data server socket. Error 24: Too many open files
and eventually hangs with a never ending:
Failed accepting FTP client connection. Error 24: Too many open files

My system has a limit on 1024 open files.
I have not been able to figure out why this happens, but I did add some extra debug output around fopen and fclose:

fp = fopen(path, "rb"); 
INFO("fopen fp no:%d.", fileno(fp));

INFO("Closing fp no:%d.", fileno(ctrl->fp));
fclose(ctrl->fp);

The fopen and fclose seems to work, but the fileno keeps increasing util it reaches the 1k limit.

uftpd version 2.12. Options used:
/sbin/uftpd_terminal_compile_org/uftpd -n -l debug -o ftp=9013,tftp=0 /mnt/ramdisk

Any ideas on how to fix this?

BR
AD

From the debug output (original code) when it starts to fail:

12604> Created new client session as PID 13623
13623> Client connection from 192.168.1.213
13623> Sent: 220 uftpd (2.12) ready.

13623> Recv: SYST
13623> Sent: 215 UNIX Type: L8

13623> Recv: USER Testuser
13623> Sent: 331 Login OK, please enter password.

13623> Recv: PASS Testuser
13623> User Testuser login from 192.168.1.213
13623> Sent: 230 Guest login OK, access restrictions apply.

13623> Recv: TYPE I
13623> Sent: 200 Type set to I.

13623> Recv: PASV
13623> Failed opening data server socket. Error 24: Too many open files
13623> Sent: 426 Internal server error.

13623> Recv: QUIT
13623> Sent: 221 Goodbye.

13623> Client exiting, bye
13623> FTP Client session ended.
12604> Created new client session as PID 13624
13624> Client connection from 192.168.1.213
13624> Sent: 220 uftpd (2.12) ready.

13624> Recv: SYST
13624> Sent: 215 UNIX Type: L8

13624> Recv: USER Testuser
13624> Sent: 331 Login OK, please enter password.

13624> Recv: PASS Testuser
13624> User Testuser login from 192.168.1.213
13624> Sent: 230 Guest login OK, access restrictions apply.

13624> Recv: TYPE I
13624> Sent: 200 Type set to I.

13624> Recv: PASV
13624> Failed opening data server socket. Error 24: Too many open files
13624> Sent: 426 Internal server error.

13624> Recv: QUIT
13624> Sent: 221 Goodbye.

13624> Client exiting, bye
13624> FTP Client session ended.
12604> Created new client session as PID 13625
13625> Client connection from 192.168.1.213
13625> Sent: 220 uftpd (2.12) ready.

13625> Recv: SYST
13625> Sent: 215 UNIX Type: L8

13625> Recv: USER Testuser
13625> Sent: 331 Login OK, please enter password.

13625> Recv: PASS Testuser
13625> User Testuser login from 192.168.1.213
13625> Sent: 230 Guest login OK, access restrictions apply.

13625> Recv: TYPE I
13625> Sent: 200 Type set to I.

13625> Recv: PASV
13625> Failed opening data server socket. Error 24: Too many open files
13625> Sent: 426 Internal server error.

13625> Recv: QUIT
13625> Sent: 221 Goodbye.

13625> Client exiting, bye
13625> FTP Client session ended.
12604> Created new client session as PID 13626
13626> Client connection from 192.168.1.213
13626> Sent: 220 uftpd (2.12) ready.

13626> Client exiting, bye
13626> FTP Client session ended.
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
12604> Failed accepting FTP client connection. Error 24: Too many open files
…

[troglobit]

Interesting, never tested this before! I'll give you a bit of background and some facts, maybe we can track this one down together:

1. For every new FTP connection the server accept()s it on a new client socket
2. Sockets are also files in UNIX (same shared file descriptor space)
3. For each new client connection a new child is forked (new PID)
4. Each client has a linger feature, if a client never disconnects we kill the connection/child after inactivity timeout

My guess is that in your case you also have a lot of uftpd child processes. Possibly the children are not "reaped", or their client socket is not closed properly. It could of course be something else, but let's start there.

[ArthurDents]

Thanks for your fast reply.
I have run:
pgrep -P <pid of uftpd>
which only shows one or zero child processes.

I have added my test scripts:

• ftp_get_testfile.sh is the loop
• ftp_get_testfile includes the ftp commands

ftp_get_testfile.sh:

#!/bin/bash
x=1
while [ 1  ]
do
  echo "Get testfile.txt round: $x"
  ./ftp_get_testfile
  x=$(( $x + 1 ))
  #sleep 1
  echo "-----------------------------"
done

ftp_get_testfile:

#!/bin/bash
ftp -n 192.168.1.131 9013 <<END
verbose on
user Testuser Testuser
bin
get testfile.txt
bye
END

Test script output (when ok):

-----------------------------
Get testfile.txt round: 43
Verbose mode on.
331 Login OK, please enter password.
230 Guest login OK, access restrictions apply.
local: testfile.txt remote: testfile.txt
227 Entering Passive Mode (192,168,1,131,232,77)
125 Data connection already open; transfer starting.
226 Transfer complete.
5 bytes received in 0.0019 secs (2.64 Kbytes/sec)
221 Goodbye.
-----------------------------
Get testfile.txt round: 44

BR
AD

[troglobit]

Thanks for the scripts! I've reproduced the problem now. Looking into it.

[troglobit]

There, fixed! Thank you for the report and the really great scripts to help reproduce this! :)

I'll see about getting a v2.13 out soonish

[troglobit]

v2.13 released now

[ArthurDents]

Thank you for fixing this so quickly. It’s greatly appreciated!
BR
AD