runtime: probably excessive garbage collection on objects using finalizer

[mikioh]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?
   tip through go1.4
2. What operating system and processor architecture are you using (go env)?
   {dragonfly,freebsd,linux,openbsd,netbsd}/amd64
3. What did you do?
   ran the following snippet:

package main

import (
        "fmt"
        "net"
        "runtime"
        "syscall"
)

func main() {
        c, err := net.ListenPacket("udp4", "0.0.0.0:0")
        if err != nil {
                fmt.Println(err)
                return
        }
        f, err := c.(*net.UDPConn).File()
        c.Close()
        if err != nil {
                fmt.Println(err)
                return
        }
        //defer f.Close()
        s := int(f.Fd())
        for i := 0; ; i++ {
                soerr, err := syscall.GetsockoptInt(s, syscall.SOL_SOCKET, syscall.SO_ERROR)
                if err != nil {
                        fmt.Println(err)
                        return
                }
                fmt.Println(i, soerr)
                runtime.GC()
        }
}

1. What did you expect to see?
   no error output
2. What did you see instead?

0 0
1 0
[...]
bad file descriptor

[davecheney]

This is not a bug, IMO. f is not live by the time the loop starts, and it
is collected, triggering the finalisers.

On Tue, 22 Mar 2016, 19:47 Mikio Hara, notifications@github.com wrote:

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?
tip through go1.4
2.

What operating system and processor architecture are you using (go env
)?
{dragonfly,freebsd,linux,openbsd,netbsd}/amd64
3.

What did you do?
ran the following snippet:

package main

import (
"fmt"
"net"
"runtime"
"syscall"
)

func main() {
c, err := net.ListenPacket("udp4", "0.0.0.0:0")
if err != nil {
fmt.Println(err)
return
}
f, err := c.(*net.UDPConn).File()
c.Close()
if err != nil {
fmt.Println(err)
return
}
//defer f.Close()
s := int(f.Fd())
for i := 0; ; i++ {
soerr, err := syscall.GetsockoptInt(s, syscall.SOL_SOCKET, syscall.SO_ERROR)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(i, soerr)
runtime.GC()
}
}

What did you expect to see?
no error output
2.

What did you see instead?

0 0
1 0
[...]
bad file descriptor

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#14910

[mikioh]

It works fine when we add a reference for keeping os.File live like defer f.Cose(). Inlining? escape analysis? go tool compile -m -live blah.go just says:

gc.go:23: inlining call to Fd
gc.go:13: err escapes to heap
gc.go:16: c.(*net.UDPConn).conn escapes to heap
gc.go:19: err escapes to heap
gc.go:27: err escapes to heap
gc.go:30: i escapes to heap
gc.go:30: soerr escapes to heap
gc.go:13: main ... argument does not escape
gc.go:19: main ... argument does not escape
gc.go:27: main ... argument does not escape
gc.go:30: main ... argument does not escape
gc.go:25: main: autotmp_22 (type [2]interface {}) is ambiguously live
gc.go:11: live at call to ListenPacket: autotmp_22
gc.go:16: live at call to (*conn).File: autotmp_22 autotmp_25 autotmp_26
gc.go:17: live at indirect call: autotmp_22 autotmp_27 autotmp_28 os.f·2
gc.go:25: live at call to GetsockoptInt: autotmp_22
gc.go:30: live at call to convT2E: autotmp_22
gc.go:30: live at call to writebarrierptr: autotmp_22
gc.go:30: live at call to convT2E: autotmp_22
gc.go:30: live at call to writebarrierptr: autotmp_22
gc.go:30: live at call to Println: autotmp_22
gc.go:31: live at call to GC: autotmp_22
gc.go:27: live at call to convI2E: autotmp_22 autotmp_17
gc.go:27: live at call to writebarrierptr: autotmp_22 autotmp_17
gc.go:27: live at call to Println: autotmp_22 autotmp_17
gc.go:19: live at call to convI2E: autotmp_22 autotmp_11
gc.go:19: live at call to writebarrierptr: autotmp_22 autotmp_11
gc.go:19: live at call to Println: autotmp_22 autotmp_11
gc.go:16: live at call to panicdottype: autotmp_22
gc.go:13: live at call to convI2E: autotmp_22 autotmp_6
gc.go:13: live at call to writebarrierptr: autotmp_22 autotmp_6
gc.go:13: live at call to Println: autotmp_22 autotmp_6

[davecheney]

I know it won't happen, but I think we should remove the finalisers from
os.File

On Tue, 22 Mar 2016, 19:54 Mikio Hara, notifications@github.com wrote:

It works fine when we add a reference for keeping os.File live like defer
f.Cose(). Inlining? escape analysis? go tool compile -m -live blah.go
just says:

gc.go:23: inlining call to Fd
gc.go:13: err escapes to heap
gc.go:16: c.(_net.UDPConn).conn escapes to heap
gc.go:19: err escapes to heap
gc.go:27: err escapes to heap
gc.go:30: i escapes to heap
gc.go:30: soerr escapes to heap
gc.go:13: main ... argument does not escape
gc.go:19: main ... argument does not escape
gc.go:27: main ... argument does not escape
gc.go:30: main ... argument does not escape
gc.go:25: main: autotmp_22 (type [2]interface {}) is ambiguously live
gc.go:11: live at call to ListenPacket: autotmp_22
gc.go:16: live at call to (_conn).File: autotmp_22 autotmp_25 autotmp_26
gc.go:17: live at indirect call: autotmp_22 autotmp_27 autotmp_28 os.f·2
gc.go:25: live at call to GetsockoptInt: autotmp_22
gc.go:30: live at call to convT2E: autotmp_22
gc.go:30: live at call to writebarrierptr: autotmp_22
gc.go:30: live at call to convT2E: autotmp_22
gc.go:30: live at call to writebarrierptr: autotmp_22
gc.go:30: live at call to Println: autotmp_22
gc.go:31: live at call to GC: autotmp_22
gc.go:27: live at call to convI2E: autotmp_22 autotmp_17
gc.go:27: live at call to writebarrierptr: autotmp_22 autotmp_17
gc.go:27: live at call to Println: autotmp_22 autotmp_17
gc.go:19: live at call to convI2E: autotmp_22 autotmp_11
gc.go:19: live at call to writebarrierptr: autotmp_22 autotmp_11
gc.go:19: live at call to Println: autotmp_22 autotmp_11
gc.go:16: live at call to panicdottype: autotmp_22
gc.go:13: live at call to convI2E: autotmp_22 autotmp_6
gc.go:13: live at call to writebarrierptr: autotmp_22 autotmp_6
gc.go:13: live at call to Println: autotmp_22 autotmp_6

—
You are receiving this because you commented.

Reply to this email directly or view it on GitHub
#14910 (comment)

[mikioh]

Perhaps it might be better to mention that os.File and net.{Conn,PacketConn,Listener} use runtime.SetFinalizer.

[RLH]

The bug is that the program does not close f, instead it relies on the
system to perhaps figure out when f is no longer reachable and perhaps do a
best effort cleanup. I am hesitant to commit Go to a cleanup guarantees
much less that they will use finalizers as part of its implementation.

This is one of the gotchas from relying on finalizers.

On Tue, Mar 22, 2016 at 5:17 AM, Mikio Hara notifications@github.com
wrote:

Perhaps it might be better to mention that os.File and
net.{Conn,PacketConn,Listener} use runtime.SetFinalizer.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#14910 (comment)

[ianlancetaylor]

I don't see anything to do here. The doc comment for os.File.Fd says "The file descriptor is valid only until f.Close is called or f is garbage collected." That is, if you call Fd, you are responsible for explicitly keeping the os.File alive.