net: sendfile on macOS doesn't send large files

[eikemeier]

Go version

go version go1.23.2 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='~/Library/Caches/go-build'
GOENV='~/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='~/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='~/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.23.2/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.23.2/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.23.2'
GODEBUG=''
GOTELEMETRY='on'
GOTELEMETRYDIR='~/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOARM64='v8.0'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='~/sendfiletest/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/n9/_x_dtls50d54kkjvqf934s580000gn/T/go-build860831848=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

Run the sample from github.com/eikemeier/sendfiletest:

git clone https://github.com/eikemeier/sendfiletest.git
cd sendfiletest
go test .

What did you see happen?

--- FAIL: TestSendfile (0.22s)
    sendfile_test.go:111: received 354712352 bytes; expected 1073741824
    sendfile_test.go:115: retrieved data hash did not match
    sendfile_test.go:119: sent 354712352 bytes; expected 1073741824
FAIL
FAIL	github.com/eikemeier/sendfiletest	0.419s
FAIL

What did you expect to see?

ok  	github.com/eikemeier/sendfiletest	0.823s

If you prevent the usage of sendfile by commenting in the line f := io.NopCloser(f) (hiding the underlying os.File), the test runs though. Also, when you send smaller files.

The test is a simplified copy of go1.23.2/src/net/sendfile_test.go, but creates and sends a larger test file than the one used in the repository.

[gabyhelp]

Related Issues and Documentation

• net/http: short writes with FileServer on macos #70000 (closed)
• net: sendFile can't handle os.fileWithoutWriteTo #66988 (closed)
• net: fix sendfile regression with io.Copy on macOS
• net: add benchmark test for SendFile
• net.sendFile does not use sendfile() on linux in all the cases in which it is available #7005 (closed)
• net/http: TestLinuxSendfile is flaky #17805 (closed)
• net: TestSendfilePipe failures with "Read did not time out" #41327
• internal/poll: make SendFile work with large files on Windows
• net: handle >=2GiB files with sendfile on Windows
• net: interleaving sendfile and read calls are very slow #45256

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[diwasrimal]

I guess this is a duplicate of #70000 which has been fixed and is coming on 1.23.3

[seankhliao]

Duplicate of #70000