cmd/go: failed to remove work dir

[zhangyoufu]

What version of Go are you using (go version)?

$ go version
go version go1.13.5 windows/amd64

What did you do?

1. clean install MSYS2
2. pacman -Syu
3. pacman -S gcc
4. go env

What did you expect to see?

go env exit normally

What did you see instead?

go env left a file with name NUL under some temporary directory, and failed to remove it.

# go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Administrator\AppData\Local\go-build
set GOENV=C:\Users\Administrator\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Administrator\go
set GOPRIVATE=
set GOPROXY=
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\msys64\tmp\go-build058580818=/tmp/go-build -gno-record-gcc-switches
go: failed to remove work dir: GetFileInformationByHandle C:\msys64\tmp\go-build058580818\NUL: Incorrect function.

---

The NUL file comes from here:

go/src/cmd/go/internal/work/exec.go

Lines 2362 to 2368 in 2ac1ca9

	// We used to write an empty C file, but that gets complicated with
	// go build -n. We tried using a file that does not exist, but that
	// fails on systems with GCC version 4.2.1; that is the last GPLv2
	// version of GCC, so some systems have frozen on it.
	// Now we pass an empty file on stdin, which should work at least for
	// GCC and clang.
	cmdArgs := str.StringList(compiler, flag, "-c", "-x", "c", "-", "-o", os.DevNull)

---

I can reproduce this issue with https://packages.msys2.org/base/gcc.
I cannot reproduce this issue with https://packages.msys2.org/package/mingw-w64-x86_64-gcc.
Different toolchains shows different behaviors when they deal with special file names.

[ianlancetaylor]

Is there a name similar to NUL for the base/gcc package? A name that causes whatever is written to that destination to be discarded? Or do we need to write to a temporary file and then remove it?

[zhangyoufu]

\\.\NUL works for both toolchains I tested. I would suggest change os.DevNull for Windows, but this requires more testing. (for anyone who don't have win32 background, please read this)

[alexbrainman]

@zhangyoufu I use gcc as suggested at

#35006 (comment)

and everything works for me:

C:\>gcc --version
gcc (GCC) 9.2.1 20190831
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


C:\>gcc -fno-caret-diagnostics -c -x c -o NIL
gcc: warning: '-x c' after last input file has no effect
gcc: error: unrecognized command line option '-fno-caret-diagnostics'
gcc: fatal error: no input files
compilation terminated.

C:\>gcc -fno-caret-diagnostics -c -x c -o nil
gcc: warning: '-x c' after last input file has no effect
gcc: error: unrecognized command line option '-fno-caret-diagnostics'
gcc: fatal error: no input files
compilation terminated.

C:\>

I suggest you change your gcc. Would that work for you?

Alex

[zhangyoufu]

@alexbrainman Thank you for your suggestion. I didn't know that musl is also available on Windows.

I do have a working toolchain. mingw-w64-x86_64-gcc works for me.

I can reproduce this issue with https://packages.msys2.org/base/gcc.
I cannot reproduce this issue with https://packages.msys2.org/package/mingw-w64-x86_64-gcc.

But I still expect golang could change os.DevNull = "NUL" on Windows, for better compatibility. I guess that gcc package (or somewhere in MSYS runtime) is using some low-level Windows API which doesn't implement special handling for DOS device names. I'm expecting a local device path like \\.\NUL should work for more situation than NUL.

Ref:

• https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html
• https://blogs.msdn.microsoft.com/jeremykuhne/2016/04/21/path-format-overview/

[zv-io]

@zhangyoufu --

@alexbrainman Thank you for your suggestion. I didn't know that musl is also available on Windows.

musl is not available for Windows; the infrastructure I use to build the musl toolchains makes it trivial to also build and test Windows toolchains of the same GCC version, so I upload those to musl.cc as well. Side note: they were updated yesterday to a newer GCC snapshot and MinGW commit.

[alexbrainman]

I'm expecting a local device path like \\.\NUL should work for more situation than NUL.

Sorry, but I am not convinced that using NUL as gcc output file is a problem. I am also not convinced that \\.\NUL is always better than NUL. I prefer to leave everything as is.

Alex

[zhangyoufu]

gcc invoked as, and create this NUL file via NtCreateFile. Caller is https://github.com/msys2/Cygwin/blob/bede85ba6d90b9383f3c83a6e99152284ca90f6a/winsup/cygwin/path.cc#L3011.

Feeding NUL into ntdll stubs cause the problem, because the win32 layer handling NUL is bypassed.

[alexbrainman]

gcc invoked as, and create this NUL file via NtCreateFile. Caller is https://github.com/msys2/Cygwin/blob/bede85ba6d90b9383f3c83a6e99152284ca90f6a/winsup/cygwin/path.cc#L3011.

@zhangyoufu Go doesn't support Cygwin.

Cygwin tools use paths that Windows don't support. Windows Go port only works with normal Windows paths.

Alex

[zhangyoufu]

Go doesn't support Cygwin.

That's fair.

---

Cygwin tools use paths that Windows don't support.

\??\C:\whatever is an absolute path that Windows kernel support. See https://stackoverflow.com/a/25099634/1016460

---

Windows Go port only works with normal Windows paths.

I would call NUL a MS-DOS device name. It is only supported in userland DLLs for compatiblity reason.

Quote from MSDN document here

To access the DosDevices namespace from user mode, specify \\.\ when you open a file name. You can open a corresponding device in user mode by calling CreateFile().

-o NUL is interpreted as a file named NUL under working directory. It's a perfectly valid filename, and I don't think gcc / cygwin / Windows kernel / NTFS filesystem is doing anything wrong.

It's insane that any program ported to Windows should check a bunch of DOS device names before they can happily use that path.

[alexbrainman]

I would call NUL a MS-DOS device name. It is only supported in userland DLLs for compatiblity reason.

If you meant Win32 API, then, yes, that is Go on Windows support.

Alex

[gopherbot]

Change https://golang.org/cl/233977 mentions this issue: src/cmd/go/internal/work: Use temporary file for output of gcc command on Windows