os: possible regression from Go 1.23 to Go 1.24 when opening DevNull with O_TRUNC

[mvdan]

What version of Go are you using (go version)?

$ go version
go version go1.24.0 linux/amd64

Does this issue reproduce with the latest release?

Yes; and it also reproduces on tip.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/mvdan/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/mvdan/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3169099861=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/home/mvdan/go/pkg/mod'
GONOPROXY='github.com/cue-unity'
GONOSUMDB='github.com/cue-unity'
GOOS='linux'
GOPATH='/home/mvdan/go'
GOPRIVATE='github.com/cue-unity'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/home/mvdan/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.24.0'
GOWORK=''
PKG_CONFIG='pkg-config'
uname -sr: Linux 6.13.2-arch1-1
LSB Version:	n/a
Distributor ID:	Arch
Description:	Arch Linux
Release:	rolling
Codename:	n/a
/usr/lib/libc.so.6: GNU C Library (GNU libc) stable release version 2.41.
gdb --version: GNU gdb (GDB) 16.2

What did you do?

Take the following program:

$ cat f.go
package main

import (
	"fmt"
	"os"
)

func main() {
	f, err := os.OpenFile(os.DevNull, os.O_TRUNC, 0)
	fmt.Println(f != nil, err)
}

Then I run it on the latest Go 1.23 and 1.24, on both Linux (the host) and Windows (via wine):

$ for goos in linux windows; do for go in go1.23.6 go1.24.0; do echo $goos $go; GOOS=$goos GOTOOLCHAIN=$go go run f.go; done; done
linux go1.23.6
true <nil>
linux go1.24.0
true <nil>
windows go1.23.6
true <nil>
windows go1.24.0
false open NUL: Invalid handle.

What did you expect to see?

Consistent behavior, or otherwise, some sort of changelog note in https://go.dev/doc/go1.24 to explain what happened and why.

What did you see instead?

It appears that opening os.DevNull, aka NUL on Windows, with O_TRUNC is now deemed an invalid operation, when before it wasn't.

It's not clear to me whether the new behavior on Windows in terms of this edge case is correct or not, but it certainly caught me by surprise. Not only because it changed from Go 1.23, but also because on Linux the operation still works OK.

I am using Wine here as a quick way to test this, but this error happens on real Windows too; see https://github.com/mvdan/sh/actions/runs/13274870177, where go test succeeded on Go 1.23 on Windows, but failed on Go 1.24 with the same "Incorrect function" errors.

I suspect one of @qmuntal's changes, perhaps https://go-review.googlesource.com/c/go/+/618835 or https://go-review.googlesource.com/c/go/+/620576, could have caused this?

My apologies for not having spotted this issue earlier; I test Go at master continuosuly, but only on Linux, as Wine only gets me so far.

[mvdan]

I should clarify that not using O_TRUNC does resolve this regression. So I could work around this regression via a check in my code like...

if path == os.DevNull {
	flag &^= os.O_TRUNC
}

but that possibly just papers over an unintentional regression that should be fixed regardless.

[gabyhelp]

Related Issues

• cmd/go: go test -o NUL fails on Windows #28035 (closed)
• os: SameFile doesn't handle invalid file IDs on Windows #62042
• os: Stat returns error with 'nul' on Windows #24482 (closed)
• os: regression with reading empty root directories on Windows with go1.21rc2 #61159 (closed)
• affected/package: os #50845 (closed)
• os: OpenFile always truncates (ignores flags) #53591 (closed)

Related Code Changes

• cmd/go: improve handling of os.DevNull on Windows
• os: return an error when the argument of Mkdir on Windows is os.DevNull
• os: remove special casing of NUL in Windows file operations
• os: return value for Stat() for files with a reserved name on windows

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

This is most likely due to https://go.dev/cl/618836 for #38225.

It's weird to pass O_TRUNC without also passing O_RDWR or O_WRONLY. On Unix in that case the O_TRUNC flag is simply ignored. Perhaps we should do the same on Windows.

[mvdan]

To be clear, the flags I was passing in the real program were os.O_WRONLY | os.O_CREATE | os.O_TRUNC. The program above was my attempt at simplifying the reproducer to the point that I could tell which flag was causing breakage. But, for the sake of not showing a nonsensical example, here is the more realistic one:

$ cat f.go
package main

import (
	"fmt"
	"os"
)

func main() {
	f, err := os.OpenFile(os.DevNull, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0)
	fmt.Println(f != nil, err)
}
$ for goos in linux windows; do for go in go1.23.6 go1.24.0; do echo $goos $go; GOOS=$goos GOTOOLCHAIN=$go go run f.go; done; done
linux go1.23.6
true <nil>
linux go1.24.0
true <nil>
windows go1.23.6
true <nil>
windows go1.24.0
false open NUL: Invalid handle.

Should this be milestoned for a backport fix to 1.24?

[ianlancetaylor]

Thanks for the O_WRONLY note. That's too bad. I was hoping this would be trivial.

We should probably backport but let's give @qmuntal a chance to take a look. It may be that we have to special case NUL, in which case we need to think about where the special case should go: os.Open or syscall.Open?

[mvdan]

Sounds good. There's no rush from my side, as thankfully I can work around the bug for now.

[qmuntal]

Weird, I can't reproduce this regression locally on a real Windows machine. Go 1.23 didn't support truncating os.DevNull, nor does Go 1.24.

>type main.go 
package main

import (
        "fmt"
        "os"
)

func main() {
        f, err := os.OpenFile(os.DevNull, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0)
        fmt.Println(f != nil, err)
}

>go1.23.6 run main.go 
false open NUL: Incorrect function.

>go1.24.0 run main.go
false open NUL: Incorrect function.

Could it be an issue with Wine? If that's the case, then IMO this issue belongs to Wine, not to Go. Another option would be to special case os.DevNull to make it consistent with the Linux behavior (what about other OSes?). I'm also good with that, although corner cases will be tricky to handle (e.g. NULL.txt is also a path to a null device on some Windows versions).

[mvdan]

I don't think this issue relates to Wine; I originally uncovered it on GitHub Actions via their windows-2022 runner, which as far as I can tell runs real Windows.

See https://github.com/mvdan/sh/actions/runs/13274870177/job/37062363963, succeeding on Go 1.23.5, and https://github.com/mvdan/sh/actions/runs/13274870177/job/37062364367, failing on Go 1.24.0.

Perhaps the way I reduced the failing test made it only show the regression on Wine, but not on real Windows? If so, try running go test ./interp on that repository at commit mvdan/sh@df624fe.

My only other guess is that, somehow, this regression affects Wine and the version of Windows used by GitHub Actions, but not the one you have installed.

[qmuntal]

Perhaps the way I reduced the failing test made it only show the regression on Wine, but not on real Windows? If so, try running go test ./interp on that repository at commit mvdan/sh@df624fe.

Thanks for the pointer, I can reproduce it now. Setting the S_IWRITE permission bit in the os.OpenFile call made the trick:

package main

import (
	"fmt"
	"os"
)

func main() {
	f, err := os.OpenFile(os.DevNull, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
	fmt.Println(f != nil, err)
}

Worth noting that the function fails if os.O_CREATE is not set, as in such case Windows does the same as we are doing in Go 1.24: truncate the file using SetFileInformationByHandle, which returns ERROR_INVALID_FUNCTION when the handle is the NUL device.

Will investigate and report back.

[mvdan]

Thanks! FWIW I don't oppose a conclusion like "never use O_CREATE or O_TRUNC on DevNull as it may fail on some platforms". This is just not something that is documented or warned about right now.

[qmuntal]

I've submitted CL 650276, which makes syscall.OpenFile behave as in Go 1.23 without losing the improvements added in CL 618836. The changes are small to facilitate backporting. @mvdan can you check it also makes Wine happy?