The app runs out of file descriptors when using prepared statements

[ernestoalejo]

I've adapted the main.go file of the example to show my error:
https://gist.github.com/ernestokarim/5033404

In the iteration 1021 the app returns an unable to open database file error. Running this in another terminal tab (that's the reason of the initial sleep, to find the correct PID):

 while [ 1 ] ; do ls /proc/6559/fd | wc -l ; sleep 1; done

reveals the number of open file descriptors is increasing all the time, even if I'm closing correctly the prepared statements.

Using a solution taken from other unrelated issue (#39 (comment)) I was able to keep the number of open files around 4 / 5 (in a comment, in the gist)

Is this a known behaviour of the library? Or are we leaking descriptors? There is a better solution to it?

[mattn]

I got following result.

$ ps aux | grep fuf
mattn    20516  0.0  1.1 807012  1468 pts/4    Sl+  09:17   0:00 ./fuf
$ while [ 1 ] ; do ls /proc/20516/fd | wc -l ; sleep 1; done
3
3
3
3
3
3
3
ls: cannot access /proc/20516/fd: No such file or directory
0
ls: cannot access /proc/20516/fd: No such file or directory
0
ls: cannot access /proc/20516/fd: No such file or directory
0

Are you using gccgo?

[ernestoalejo]

Using 8g, Fedora 17, 32 bits. Using the sqlite3.7.11-3.fc17 package.

My results look something like this (after introducing a time.Sleep of 20 Milliseconds to obtain more measurements):

3
3
3
3
5
5
11
16
21
27
32
37
43
49
.........more results here....
981
987
993
999
1004
1010
1016
1021     // here the program fails
0
0
0

Let me know if I can provide some more information to debug the issue.

[mattn]

Hmm, do you using latest version of go & sqlite3-devel?

I tried above on CentOS6 32bit.

[ernestoalejo]

Go installed from source:
devel +ca83aeaa55e0 Fri Feb 22 12:23:19 2013 +1100

SQLite3 is the latest version Fedora has (3.7.11), but not the current one (3.7.15.2).

I was about to try the same example using C, to see if it happens there too.

[ernestoalejo]

I've tested a similar example ported to C with the functions you use and no leak of descriptors.

Then I downgraded my Go version to 1, (hg update go1.0.3). The gist code produced no leaks there. After updating to the latest one (hg pull && hg update default) the loop stopped at iteration i=1021 with no file descriptors remaining. Same code in both cases.

Have you some distribution of Go around to test if it is a regression?

[mattn]

I've tested on tip.

CentOS6

2.6.32-042stab061.2 #1 SMP Fri Aug 24 09:07:21 MSK 2012 i686 i686 i386 GNU/Linux

[ernestoalejo]

I've used hg bisect to find the change that starts reproducing the error. This one fails for me:

https://code.google.com/p/go/source/detail?r=ddb9e6365e57

I'll investigate further

[mattn]

Could you please file this issue to https://code.google.com/p/go/issues/list ?

[ernestoalejo]

https://code.google.com/p/go/issues/detail?id=4935

[mattn]

Thank you.

[ernestoalejo]

It's now fixed:
https://code.google.com/p/go/issues/detail?id=4902
https://code.google.com/p/go/source/detail?r=3486a9feedda

[mattn]

Thank you.