os: copy_file_range system call fails on some file systems

[ianlancetaylor]

On GNU/Linux systems we currently use the copy_file_range system call when copying from one open file to another via io.Copy or similar mechanisms. This seems to be exactly what copy_file_range is for. Unfortunately, on current Linux kernel versions this fails to work on certain file systems. The failure mode is that copy_file_range returns 0 even if the file actually contains data. A sample file system for which this occurs is the procfs file system that implements /proc.

For example, this program fails on Linux kernel version 5.7.17.

package main

import (
	"bytes"
	"fmt"
	"io"
	"io/ioutil"
	"os"
)

func main() {
	tmpfile, err := ioutil.TempFile("", "copy_file_range")
	if err != nil {
		fmt.Fprint(os.Stderr, err)
		os.Exit(1)
	}
	status := copy(tmpfile)
	os.Remove(tmpfile.Name())
	os.Exit(status)
}

func copy(tmpfile *os.File) int {
	cmdline, err := os.Open("/proc/self/cmdline")
	if err != nil {
		fmt.Fprintln(os.Stderr, err)
		return 1
	}
	defer cmdline.Close()
	if _, err := io.Copy(tmpfile, cmdline); err != nil {
		fmt.Fprintf(os.Stderr, "copy failed: %v\n", err)
		return 1
	}
	if err := tmpfile.Close(); err != nil {
		fmt.Fprintln(os.Stderr, err)
		return 1
	}
	old, err := ioutil.ReadFile("/proc/self/cmdline")
	if err != nil {
		fmt.Fprintln(os.Stderr, err)
		return 1
	}
	new, err := ioutil.ReadFile(tmpfile.Name())
	if err != nil {
		fmt.Fprintln(os.Stderr, err)
		return 1
	}
	if !bytes.Equal(old, new) {
		fmt.Fprintf(os.Stderr, "got %q want %q\n", new, old)
		return 1
	}
	return 0
}

It seems likely, though not certain, that we can detect this case by treating a zero return from copy_file_range as indicating that copy_file_range is not supported for that file. In that case we can fall back to using read and write as usual. For the case of an empty file this will introduce additional system calls, but at least it will work correctly.

There is a mailing list discussion about this problem at https://lore.kernel.org/linux-fsdevel/20210126233840.GG4626@dread.disaster.area/T/#m05753578c7f7882f6e9ffe01f981bc223edef2b0.

[ianlancetaylor]

@gopherbot Please open backport to Go 1.15.

This bug can cause file corruption when using io.Copy with files on certain file systems. There is no reasonable workaround other than avoiding io.Copy.

[gopherbot]

Backport issue(s) opened: #44273 (for 1.15).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[gopherbot]

Change https://golang.org/cl/291989 mentions this issue: internal/poll: if copy_file_range returns 0, assume it failed

[gopherbot]

Change https://golang.org/cl/292289 mentions this issue: [release-branch.go1.15] internal/poll: if copy_file_range returns 0, assume it failed