Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: max/min builtin broken when used with string(byte) conversions #64565

Closed
mighmi opened this issue Dec 5, 2023 · 10 comments
Closed
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@mighmi
Copy link

mighmi commented Dec 5, 2023

Go version

go1.21.0 linux/amd64

What operating system and processor architecture are you using (go env)?

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/eggplant/.cache/go-build'
GOENV='/home/eggplant/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'

What did you do?

Here is a proof https://go.dev/play/p/Dr26MV2mvGW from https://www.reddit.com/r/golang/comments/18be9xl/max_and_assign_issue/

Credit to /u/ar1819 and /u/YamadaAnna

What did you expect to see?

max() and slices.Max() should have the same results. 7 is the bigger number.

What did you see instead?

max() output numbers lower than 7 as maxes.

@mateusz834
Copy link
Member

mateusz834 commented Dec 5, 2023

Simpler reproducer:

func main() {
	num := "765"
	maxSymbol := ""
	for i := 0; i < len(num); i++ {
		newMax := max(string(num[i]), maxSymbol)
		fmt.Printf("max(%q, %q)=%q\n", string(num[i]), maxSymbol, newMax)
		maxSymbol = newMax
	}
}
$ go run main.go
max("7", "")="7"
max("6", "6")="6"
max("5", "5")="5"

Making a tmp variable fixes this issue.

func main() {
	num := "765"
	maxSymbol := ""
	for i := 0; i < len(num); i++ {
		tmp := string(num[i])
		newMax := max(tmp, maxSymbol)
		fmt.Printf("max(%q, %q)=%q\n", tmp, maxSymbol, newMax)
		maxSymbol = newMax
	}
}
$ go run main.go
max("7", "")="7"
max("6", "7")="7"
max("5", "7")="7"

EDIT: The same thing happens with min.

@mateusz834
Copy link
Member

CC @golang/compiler

@mateusz834 mateusz834 changed the title builtin: max() Broken cmd/compile: max/min builtin broken when used with string(rune) conversions Dec 5, 2023
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Dec 5, 2023
@mateusz834 mateusz834 added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 5, 2023
@gazerro
Copy link
Contributor

gazerro commented Dec 5, 2023

Note that the commit title is not accurate. string(num[i]) is not a string(rune) conversion but rather a string(byte) conversion.

@mateusz834 mateusz834 changed the title cmd/compile: max/min builtin broken when used with string(rune) conversions cmd/compile: max/min builtin broken when used with string(byte) conversions Dec 5, 2023
@mdempsky
Copy link
Contributor

mdempsky commented Dec 5, 2023

This appears to be an escape analysis issue.

@gopherbot Please backport to Go 1.21. This is a silent miscompilation / memory corruption issue.

@gopherbot
Copy link
Contributor

Backport issue(s) opened: #64567 (for 1.21).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@mauri870
Copy link
Member

mauri870 commented Dec 5, 2023

I find it weird that in this reproducer:

func main() {
	num := "765"
	maxSymbol := ""
	for i := 0; i < len(num); i++ {
		newMax := max(string(num[i]), maxSymbol)
		fmt.Printf("max(%q, %q)=%q\n", string(num[i]), maxSymbol, newMax)
		maxSymbol = newMax
	}
}

If you stop a debugger in the newMax assignment line where i == 1, maxSymbol has a value of 7, if you step to the next line maxSymbol suddenly has a value of 6.

@mdempsky mdempsky self-assigned this Dec 5, 2023
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/547715 mentions this issue: cmd/compile: fix escape analysis of string min/max

@mdempsky
Copy link
Contributor

mdempsky commented Dec 5, 2023

Thanks all for the report. FWIW, since there's some interest in the issue, here's an explanation of what's going on.

Starting from this reproducer:

package main

import "fmt"

func main() {
	m := "0"
	for _, c := range "321" {
		m = max(string(c), m)
		fmt.Println(m)
	}
}

The compiler transforms this into basically:

func main() {
        m := "0"
	for _, c := range "321" {
                var tmp = intstring(new([4]byte), int64(c))
		m = strmax(tmp, m)
		fmt.Println(m)
	}
}

where intstring and strmax are provided by the runtime.

However, the compiler was not correctly taking into consideration that strmax returns one of its arguments. That is, it didn't realize m could point to the memory allocated by new([4]byte). As a result, it mistakenly thought it was safe to reuse that memory allocation across loop iterations, so it further transformed the code into:

func main() {
        m := "0"
        var buf [4]byte
	for _, c := range "321" {
                var tmp = intstring(&buf, int64(c)) // BAD: reuses buf, even though m may refer to it
		m = strmax(tmp, m)
		fmt.Println(m)
	}
}

@mauri870
Copy link
Member

mauri870 commented Dec 5, 2023

Thank you for taking the time to provide such a detailed explanation, much appreciated!

@dmitshur dmitshur added this to the Go1.22 milestone Dec 5, 2023
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Dec 5, 2023
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/547757 mentions this issue: [release-branch.go1.21] cmd/compile: fix escape analysis of string min/max

gopherbot pushed a commit that referenced this issue Dec 6, 2023
…n/max

When I was plumbing min/max support through the compiler, I was
thinking mostly about numeric argument types. As a result, I forgot
that escape analysis would need to be aware that min/max can operate
on string values, which contain pointers.

Updates #64565.
Fixes #64567.

Change-Id: I36127ce5a2da942401910fa0f9de922726c9f94d
Reviewed-on: https://go-review.googlesource.com/c/go/+/547715
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 34416d7)
Reviewed-on: https://go-review.googlesource.com/c/go/+/547757
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
ezz-no pushed a commit to ezz-no/go-ezzno that referenced this issue Feb 18, 2024
When I was plumbing min/max support through the compiler, I was
thinking mostly about numeric argument types. As a result, I forgot
that escape analysis would need to be aware that min/max can operate
on string values, which contain pointers.

Fixes golang#64565.

Change-Id: I36127ce5a2da942401910fa0f9de922726c9f94d
Reviewed-on: https://go-review.googlesource.com/c/go/+/547715
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@golang golang locked and limited conversation to collaborators Dec 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

7 participants