runtime: terminate locked OS thread if its goroutine exits

[rgooch]

I propose a new runtime.TaintOSThread function. This would mark the current thread that the calling goroutine is running on as being "tainted": not safe for use by other goroutines. When the goroutine exits or the goroutine is unpinned from the (previously pinned) OS thread, the thread is killed.

This is needed when goroutines are pinned to OS threads and the state of the thread is modified in way that is unsafe for other goroutines, such as unsharing and then modifying one of the Linux namespaces. Without this change, it is impossible to clean up OS threads: the pinning goroutines and OS threads need to stay until the programme exits, leading to unfixable goroutine and OS thread leaks.

The expected calling pattern is:

runtime.LockOSThread()
runtime.TaintOSThread()
syscall.Unshare(syscall.CLONE_NEWNS) // Example mutation of OS thread state.

[bradfitz]

Could you instead just LockOSThread + Unshare + then kill that thread yourself?

[rgooch]

How can I kill the thread safely? I don't see a function in the runtime package to do this. If I can safely kill the thread on which a goroutine is running, then that would work too, I just didn't think that would be as palatable as marking a thread as tainted.

There's actually no need to unshare (and that wouldn't help anyway, since the new namespace inherits from its ancestor).

[bradfitz]

syscall.Syscall(syscall.SYS_EXIT, 0, 0, 0)? Haven't tried it, though.

[bradfitz]

Works for me:

bradfitz@gdev:~$ cat exit.go

package main

import (
        "runtime"
        "syscall"
        "time"
)       

func main() {
        runtime.LockOSThread()
        println("Hi from ", syscall.Gettid())
        go func() {
                runtime.LockOSThread()
                println("Hi from ", syscall.Gettid())
                syscall.Syscall(syscall.SYS_EXIT, 0, 0, 0)
        }()     
        time.Sleep(500 * time.Millisecond)
        println("Bye from ", syscall.Gettid())
}    

bradfitz@gdev:~$ go run exit.go
Hi from  6876
Hi from  6878
Bye from  6876

[bradfitz]

Or, better: (showing it actually went away)

package main

import (
        "fmt"
        "os"
        "runtime"
        "strconv"
        "syscall"
        "time"
)

func main() {
        runtime.LockOSThread()
        fmt.Println("Hi from ", syscall.Gettid())
        other := make(chan int, 1)
        go func() {
                runtime.LockOSThread()
                tid := syscall.Gettid()
                other <- tid
                fmt.Println("Hi from ", tid)
                fi, err := os.Stat("/proc/" + strconv.Itoa(tid))
                fmt.Printf("proc %d: %v, %v\n", tid, fi, err)
                syscall.Syscall(syscall.SYS_EXIT, 0, 0, 0)
        }()
        time.Sleep(500 * time.Millisecond)
        tid := <-other
        fi, err := os.Stat("/proc/" + strconv.Itoa(tid))
        fmt.Printf("proc %d: %v, %v\n", tid, fi, err)
        fmt.Println("Bye from ", syscall.Gettid())
}

Hi from  7277
Hi from  7279
proc 7279: &{7279 0 2147484013 {63630648429 114580211 0x550860} 0xc20804e000}, <nil>
proc 7279: <nil>, stat /proc/7279: no such file or directory
Bye from  7277

[randall77]

I worry that the runtime would be upset at an OS thread permanently vanishing. The runtime may want notification to ensure that anything the m is holding is freed first.
Maybe the existing enterSysCall already does everything that might matter?
At the very least, the allm list will grow without bound.

[rgooch]

Yes, that was one of my worries too. Can we get some kind of guidance on whether it's considered safe to kill a thread? What happens to the goroutine that's running on it?

[randall77]

@aclements

[ianlancetaylor]

I do think we need to let the runtime know that the thread can not be used. The simple approach is

runtime.LockOSThread()
select{}

That will ensure that nothing else uses the thread, but it won't actually kill the thread. It would be an acceptable approach if you only need to do this a few times, but of course would be unsatisfactory if you need to do it continually.

Another approach would be to go through C. Have C code create a new thread, which could then call into Go. That gives you a thread that the Go runtime won't use itself, so you can simply call pthread_exit when done. Of course this is not ideal if you otherwise have a pure Go program.

[rgooch]

All my Go code so far is pure Go. I was hoping to keep it that way.

It seems to me that we need some way to cleanly kill a thread. The hack shown above is, well, a hack :-)

[ianlancetaylor]

Is runtime.ThreadExit sufficient for your purposes? I'm thinking of a function that would simply cause the thread to exit and not return. The goroutine could continue running on a different thread.

I'll note that I appreciate that you need this, but it seems awfully special purpose.

[rgooch]

Yes, runtime.ThreadExit would work fine. I'm not special, I'm just ahead of the curve :-)

[bradfitz]

@ianlancetaylor, or instead of introducing new API: just consider all threads "tainted" if they ever used LockOSThread and when the goroutine exits (implicitly or via the existing runtime.Goexit()), then kill the thread.