x/sys/windows/svc: the service terminated unexpectedly (id 7034)

[AlexSiLive]

What version of Go are you using (go version)?

$ go version
go version go1.14.4 windows/amd64

Does this issue reproduce with the latest release?

Yes.
Go version: go1.14.4
x/sys version: v0.0.0-20200625212154-ddb9806d33ae

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\alex\AppData\Local\go-build
set GOENV=C:\Users\alex\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GONOPROXY=
set GONOSUMDB=*
set GOOS=windows
set GOPATH=C:\Users\alex\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=E:\WorkGo\src\golang.org\x\sys\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\A6473~1.POG\AppData\Local\Temp\go-build406331230=/tmp/go-build -gno-record-gcc-switches
GOROOT/bin/go version: go version go1.14.4 windows/amd64
GOROOT/bin/go tool compile -V: compile version go1.14.4
gdb --version: GNU gdb (GDB) 7.9.1

What did you do?

Problem:
Building a windows service using x/sys/windows/svc on go1.14+ will cause a service to terminate unexpectedly on system shutdown (SERVICE_CONTROL_SHUTDOWN command).

How to reproduce:
Build a default example for x/sys/windows/svc on go1.13 and go1.14. The latter will be terminated unexpectedly and produce a corresponding event in the event log (7034), while the former one will stop just fine.

Remarks:

• A service can be stopped and started via service manager without an issue, the problem occurs on system shutdown or reboot.
• Reverting x/sys/windows/svc to initial commits does nothing, i.e. the problem, probably, somewhere in-between go1.13 and go1.14.
  go_svc_7034.zip

[networkimprov]

cc @alexbrainman @bcmills

@gopherbot add OS-Windows

[alexbrainman]

@AlexSiLive

Thank you for reporting the issue and the steps to reproduce the problem. Yes, I can see shutdown event sent for service compiled with go1.13.12, but not with current tip.

We need to try to use git bisect command to find commit that broken the code (it is, probably, https://go-review.googlesource.com/c/go/+/187739/ ). Unfortunately I don't have time to run bisect now, but I will do it when I have time. Unless someone beats me to it.

Alex

[AlexSiLive]

@alexbrainman

Just a quick update.
Indeed, commenting out this clause resolves the issue:
9e05d63#diff-4fcd95ba1348be2a6286fdfdd9e8393dR897

[AlexSiLive]

I was able to make a dirty fix for this without modifying runtime by adding signal.Notify handling to a wrapping package.
Something similar can be done to x/sys/windows/svc itself:
`

...
func Run(name string, handler Handler) error {
	runtime.LockOSThread()

	tid := windows.GetCurrentThreadId()

	s, err := newService(name, handler)
	if err != nil {
		return err
	}

	signals := make(chan os.Signal, 1)
	signal.Notify(signals)

	go func() {
		for sig := range signals {
			if sig == windows.SIGTERM {
				signal.Stop(signals)
				s.c <- ctlEvent{cmd: Shutdown} // todo: some checks here, i.e. for SERVICE_ACCEPT_SHUTDOWN

				return
			}
		}
	}()
...

`

Please note, that following didn't work for me (unless I'm doing something wrong):
`

signals := make(chan os.Signal, 1)
signal.Notify(signals, windows.SIGTERM) // <- specifying SIGTERM doesn't seem to work.

go func() {
	<- signals // <- this never triggers if SIGTERM is specified (mock program with SIGINT works though).
	signal.Stop(signals)
	s.c <- ctlEvent{cmd: Shutdown} // todo: some checks here, i.e. for SERVICE_ACCEPT_SHUTDOWN
}()

`

[JohanKnutzen]

@AlekSi

I believe that our issues are related. I've got another workaround for this that overrides handling in go of that particular event in Windows:
40167

[gopherbot]

Change https://golang.org/cl/242378 mentions this issue: windows/svc: restore default console shutdown notifier when starting service

[alexbrainman]

Indeed, commenting out this clause resolves the issue:

Cool. Thanks for checking that.

We just need to come up with a solution here.

Alex

[AlekSi]

@AlekSi

I'm not @AlexSiLive :)

[gopherbot]

Change https://golang.org/cl/242280 mentions this issue: runtime: detect services in signal handler

[alexbrainman]

@aclements

This issue is with golang.org/x/sys/windows/svc package.

The problem was introduced in

https://go-review.googlesource.com/c/go/+/187739/

After the CL 187739, every Windows service would not receive "shutdown" event. When computer is shutdown, Go service just exits, instead of handling "shutdown" event.

@zx2c4 sent a fix for this issue

https://golang.org/cl/242280

And the fix is in runtime package.

Both Jason and I think the fix should be back-ported to Go 1.14 (Go 1.13 is good). Should we submit CL 242280 now? Should we wait for tree to be opened first? Do you agree CL 242280 should be back-ported?

There is some risk involved. Here, in particular, we don't even have a test for the fix - you need to reboot computer to test our fix.

What do you think?

Thank you.

Alex

[networkimprov]

@aclements, most of the discussion about this is in #40167

[gopherbot]

Change https://golang.org/cl/243597 mentions this issue: runtime: do not explicitly exit on ctrl handler

[gopherbot]

Change https://golang.org/cl/244957 mentions this issue: [release-branch.go1.14] runtime: detect services in signal handler

[gopherbot]

Change https://golang.org/cl/244958 mentions this issue: [release-branch.go1.14] runtime: detect services in signal handler

[gopherbot]

Change https://golang.org/cl/244959 mentions this issue: [release-branch.go1.15] runtime: detect services in signal handler

[zx2c4]

Sorry for the churn. The two backports are https://golang.org/cl/244958 and https://golang.org/cl/244959.