Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

maps: segmentation violation in maps.Clone #69110

Closed
RaduBerinde opened this issue Aug 28, 2024 · 15 comments
Closed

maps: segmentation violation in maps.Clone #69110

RaduBerinde opened this issue Aug 28, 2024 · 15 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FixPending Issues that have a fix which has not yet been reviewed or submitted. NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@RaduBerinde
Copy link
Contributor

RaduBerinde commented Aug 28, 2024

Go version

go version go1.22.5 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE='on'
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/radu/Library/Caches/go-build'
GOENV='/Users/radu/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/radu/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/radu/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.5'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/radu/go/src/github.com/cockroachdb/pebble/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/x0/5yrxw_4s271g705vjscld23c0000gq/T/go-build2422055407=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

This is a test that adds, removes, and clones a map:

func TestMapClone(t *testing.T) {
	seed := time.Now().UnixNano()
	t.Logf("seed: %d", seed)
	rng := rand.New(rand.NewSource(seed))
	for i := 0; i < 100; i++ {
		keyRange := 1 + rng.Intn(100)
		clone := make(map[string]int)
		m := make(map[string]int)
		for j := 0; j < 10000; j++ {
			switch rng.Intn(10) {
			case 0, 1, 2, 3, 4:
				m[fmt.Sprint(rng.Intn(keyRange))] = j
			case 5, 6, 7, 8:
				delete(m, fmt.Sprint(rng.Intn(keyRange)))
			case 9:
				clone = maps.Clone(m)
			}
			if rng.Intn(100) == 0 {
				// Revert to the last clone.
				m = clone
			}
		}
	}
}

What did you see happen?

Ran it under stress (-exec 'stress -p 10') and got this (after a few thousand runs):

=== RUN   TestMapClone
    mem_fs_test.go:210: seed: 1724860257984210000
unexpected fault address 0x3731373330313337
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0x3731373330313337 pc=0x1005a5108]

goroutine 19 gp=0x14000104fc0 m=0 mp=0x100b594c0 [running]:
runtime.throw({0x10081e604?, 0x1009018c0?})
        /Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64/src/runtime/panic.go:1023 +0x40 fp=0x14000066cf0 sp=0x14000066cc0 pc=0x1005cd780
runtime.sigpanic()
        /Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64/src/runtime/signal_unix.go:895 +0x22c fp=0x14000066d50 sp=0x14000066cf0 pc=0x1005e786c
runtime.moveToBmap(0x1008d7900, 0x14000516ff0, 0x14000066e28?, 0x1005a7d40?, 0x3731373330313337)
        /Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64/src/runtime/map.go:1456 +0x38 fp=0x14000066de0 sp=0x14000066d60 pc=0x1005a5108
runtime.mapclone2(0x1008d7900, 0x14000516d80)
        /Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64/src/runtime/map.go:1538 +0x428 fp=0x14000066e80 sp=0x14000066de0 pc=0x1005a57e8
maps.clone({0x1008d7900, 0x14000516d80})
        /Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64/src/runtime/map.go:1448 +0x2c fp=0x14000066ea0 sp=0x14000066e80 pc=0x10060272c
maps.Clone[...](...)
        /Users/radu/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.5.darwin-arm64/src/maps/maps.go:46
github.com/cockroachdb/pebble/vfs.TestMapClone(0x14000135040)

What did you expect to see?

No crash.

@RaduBerinde
Copy link
Contributor Author

Note: this is similar to #62203 but it's a different code path, and this version includes that fix.

RaduBerinde added a commit to RaduBerinde/pebble that referenced this issue Aug 28, 2024
Stop using `maps.Clone` because of golang/go#69110

Fixes cockroachdb#3894
RaduBerinde added a commit to RaduBerinde/pebble that referenced this issue Aug 28, 2024
@ianlancetaylor
Copy link
Member

CC @randall77 @cuiweixie

@cuiweixie
Copy link
Contributor

cuiweixie commented Aug 28, 2024

it seems that it is hard to reproduce this bug.

➜  issue69110 git:(go1.22.5) ✗ go test . -exec 'stress -p 10'
5s: 6139 runs so far, 0 failures, 10 active
10s: 12220 runs so far, 0 failures, 10 active
15s: 17879 runs so far, 0 failures, 10 active
20s: 23313 runs so far, 0 failures, 10 active
25s: 28194 runs so far, 0 failures, 10 active
30s: 32985 runs so far, 0 failures, 10 active
35s: 37649 runs so far, 0 failures, 10 active
40s: 41962 runs so far, 0 failures, 10 active
45s: 46762 runs so far, 0 failures, 10 active
50s: 51536 runs so far, 0 failures, 10 active
55s: 55238 runs so far, 0 failures, 10 active
1m0s: 59627 runs so far, 0 failures, 10 active
1m5s: 64098 runs so far, 0 failures, 10 active
1m10s: 68611 runs so far, 0 failures, 10 active
1m15s: 72942 runs so far, 0 failures, 10 active
1m20s: 77221 runs so far, 0 failures, 10 active
1m25s: 81015 runs so far, 0 failures, 10 active
1m30s: 84453 runs so far, 0 failures, 10 active
1m35s: 87739 runs so far, 0 failures, 10 active
1m40s: 91737 runs so far, 0 failures, 10 active
1m45s: 95797 runs so far, 0 failures, 10 active
1m50s: 99831 runs so far, 0 failures, 10 active
1m55s: 103550 runs so far, 0 failures, 10 active
2m0s: 107480 runs so far, 0 failures, 10 active
2m5s: 110964 runs so far, 0 failures, 10 active
2m10s: 114569 runs so far, 0 failures, 10 active
2m15s: 118407 runs so far, 0 failures, 10 active
2m20s: 122205 runs so far, 0 failures, 10 active
2m25s: 125963 runs so far, 0 failures, 10 active
2m30s: 129655 runs so far, 0 failures, 10 active
2m35s: 133272 runs so far, 0 failures, 10 active
2m40s: 136882 runs so far, 0 failures, 10 active
2m45s: 140471 runs so far, 0 failures, 10 active
2m50s: 143684 runs so far, 0 failures, 10 active
2m55s: 147028 runs so far, 0 failures, 10 active
3m0s: 150530 runs so far, 0 failures, 10 active
3m5s: 153960 runs so far, 0 failures, 9 active
3m10s: 157431 runs so far, 0 failures, 10 active
3m15s: 160817 runs so far, 0 failures, 10 active
3m20s: 164217 runs so far, 0 failures, 10 active
3m25s: 167710 runs so far, 0 failures, 10 active
3m30s: 171181 runs so far, 0 failures, 10 active
3m35s: 174031 runs so far, 0 failures, 10 active
3m40s: 177329 runs so far, 0 failures, 10 active
3m45s: 180505 runs so far, 0 failures, 10 active
3m50s: 183907 runs so far, 0 failures, 10 active
3m55s: 187184 runs so far, 0 failures, 10 active
4m0s: 190556 runs so far, 0 failures, 10 active
4m5s: 193806 runs so far, 0 failures, 10 active
4m10s: 197141 runs so far, 0 failures, 10 active
4m15s: 200501 runs so far, 0 failures, 10 active
4m20s: 203863 runs so far, 0 failures, 9 active
4m25s: 207117 runs so far, 0 failures, 10 active
4m30s: 210477 runs so far, 0 failures, 10 active
4m35s: 213701 runs so far, 0 failures, 10 active
4m40s: 216769 runs so far, 0 failures, 10 active
4m45s: 220049 runs so far, 0 failures, 10 active
4m50s: 223113 runs so far, 0 failures, 10 active
4m55s: 225959 runs so far, 0 failures, 10 active
5m0s: 229179 runs so far, 0 failures, 10 active
5m5s: 232161 runs so far, 0 failures, 10 active
5m10s: 235190 runs so far, 0 failures, 10 active
5m15s: 238283 runs so far, 0 failures, 10 active
5m20s: 241345 runs so far, 0 failures, 10 active
5m25s: 244078 runs so far, 0 failures, 10 active
5m30s: 246975 runs so far, 0 failures, 10 active
5m35s: 249858 runs so far, 0 failures, 10 active
5m40s: 252901 runs so far, 0 failures, 10 active
5m45s: 255867 runs so far, 0 failures, 10 active
5m50s: 258893 runs so far, 0 failures, 10 active
5m55s: 261765 runs so far, 0 failures, 10 active
6m0s: 264876 runs so far, 0 failures, 10 active
6m5s: 267895 runs so far, 0 failures, 10 active
6m10s: 270915 runs so far, 0 failures, 10 active
6m15s: 273848 runs so far, 0 failures, 10 active
6m20s: 277000 runs so far, 0 failures, 10 active
6m25s: 279933 runs so far, 0 failures, 10 active
6m30s: 282925 runs so far, 0 failures, 10 active
6m35s: 285627 runs so far, 0 failures, 10 active
6m40s: 288550 runs so far, 0 failures, 10 active
6m45s: 291669 runs so far, 0 failures, 10 active
6m50s: 294740 runs so far, 0 failures, 10 active
6m55s: 297614 runs so far, 0 failures, 10 active
7m0s: 300528 runs so far, 0 failures, 10 active
7m5s: 303461 runs so far, 0 failures, 10 active
7m10s: 306369 runs so far, 0 failures, 10 active
7m15s: 309361 runs so far, 0 failures, 10 active
7m20s: 312464 runs so far, 0 failures, 10 active
7m25s: 315372 runs so far, 0 failures, 10 active
7m30s: 318256 runs so far, 0 failures, 10 active
7m35s: 321166 runs so far, 0 failures, 10 active
7m40s: 323931 runs so far, 0 failures, 10 active
7m45s: 326720 runs so far, 0 failures, 10 active
7m50s: 329500 runs so far, 0 failures, 10 active
7m55s: 331507 runs so far, 0 failures, 10 active
8m0s: 334102 runs so far, 0 failures, 10 active
8m5s: 337109 runs so far, 0 failures, 10 active
8m10s: 339943 runs so far, 0 failures, 10 active
8m15s: 342885 runs so far, 0 failures, 10 active
8m20s: 346065 runs so far, 0 failures, 10 active
8m25s: 348754 runs so far, 0 failures, 6 active
8m30s: 351610 runs so far, 0 failures, 10 active
8m35s: 354357 runs so far, 0 failures, 10 active
8m40s: 357630 runs so far, 0 failures, 10 active
8m45s: 360389 runs so far, 0 failures, 10 active
8m50s: 363106 runs so far, 0 failures, 10 active
8m55s: 365694 runs so far, 0 failures, 10 active
9m0s: 368866 runs so far, 0 failures, 10 active
9m5s: 371351 runs so far, 0 failures, 10 active
9m10s: 374514 runs so far, 0 failures, 10 active
9m15s: 377226 runs so far, 0 failures, 10 active
9m20s: 380173 runs so far, 0 failures, 10 active
9m25s: 382838 runs so far, 0 failures, 10 active
9m30s: 385835 runs so far, 0 failures, 10 active
9m35s: 388291 runs so far, 0 failures, 10 active
9m40s: 391274 runs so far, 0 failures, 10 active
9m45s: 393967 runs so far, 0 failures, 10 active
9m50s: 396820 runs so far, 0 failures, 10 active
9m55s: 399527 runs so far, 0 failures, 10 active
10m0s: 402584 runs so far, 0 failures, 10 active
10m5s: 405278 runs so far, 0 failures, 10 active
10m10s: 407812 runs so far, 0 failures, 10 active
10m15s: 410144 runs so far, 0 failures, 10 active
10m20s: 412101 runs so far, 0 failures, 10 active
10m25s: 414690 runs so far, 0 failures, 10 active
10m30s: 417384 runs so far, 0 failures, 10 active
10m35s: 420005 runs so far, 0 failures, 10 active
10m40s: 422668 runs so far, 0 failures, 10 active
10m45s: 425176 runs so far, 0 failures, 10 active
10m50s: 427715 runs so far, 0 failures, 10 active
10m55s: 430322 runs so far, 0 failures, 10 active
11m0s: 433534 runs so far, 0 failures, 10 active

@RaduBerinde
Copy link
Contributor Author

RaduBerinde commented Aug 28, 2024

@cuiweixie: are you sure it is actually running the test? On my machine (Apple M1), it can only do a couple hundred runs per 5s:

# go test ./vfs -run TestMapClone -exec 'stress -p 10' -v
194 runs so far, 0 failures, over 5s
405 runs so far, 0 failures, over 10s
^C559 runs completed, 0 failures, over 14s

P.S. A friendly request - please don't post long irrelevant backtraces, it makes the issue hard to navigate. E.g. it would have been sufficient to post just 11m0s: 433534 runs so far, 0 failures, 10 active.

@cuiweixie
Copy link
Contributor

cuiweixie commented Aug 28, 2024

Sorry, I changed the number of iterations!
Run Again!

RaduBerinde added a commit to cockroachdb/pebble that referenced this issue Aug 28, 2024
Stop using `maps.Clone` because of golang/go#69110

Fixes #3894
@RaduBerinde
Copy link
Contributor Author

It took ~7 minutes for it to crash. But I tried again now and it didn't crash in 11 minutes. It might take a long time.

Note that in cockroachdb/pebble#3894 we saw the fault on address 0 (signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x417278]). Looking at the code, I don't see how that would happen unless the src pointer is corrupted and is near the end range of uint64, leading to overflow when we access.

@cuiweixie
Copy link
Contributor

I run with you code now!
Do not reproduce! Sad!

➜  issue69110 git:(go1.22.5) ✗ go test -v . -run="TestMapClone" -exec "stress -p 10"
5s: 218 runs so far, 0 failures, 10 active
10s: 464 runs so far, 0 failures, 10 active
15s: 688 runs so far, 0 failures, 10 active
20s: 893 runs so far, 0 failures, 10 active
25s: 1113 runs so far, 0 failures, 10 active
30s: 1344 runs so far, 0 failures, 10 active
35s: 1585 runs so far, 0 failures, 10 active
40s: 1829 runs so far, 0 failures, 10 active
45s: 2067 runs so far, 0 failures, 10 active
50s: 2309 runs so far, 0 failures, 10 active
55s: 2551 runs so far, 0 failures, 10 active
1m0s: 2769 runs so far, 0 failures, 10 active
1m5s: 2968 runs so far, 0 failures, 10 active
1m10s: 3176 runs so far, 0 failures, 10 active
1m15s: 3415 runs so far, 0 failures, 10 active
1m20s: 3653 runs so far, 0 failures, 10 active
1m25s: 3892 runs so far, 0 failures, 10 active
1m30s: 4122 runs so far, 0 failures, 10 active
1m35s: 4338 runs so far, 0 failures, 10 active
1m40s: 4555 runs so far, 0 failures, 10 active
1m45s: 4779 runs so far, 0 failures, 10 active
1m50s: 5006 runs so far, 0 failures, 10 active
1m55s: 5216 runs so far, 0 failures, 10 active
2m0s: 5435 runs so far, 0 failures, 10 active
2m5s: 5668 runs so far, 0 failures, 10 active
2m10s: 5899 runs so far, 0 failures, 10 active
2m15s: 6092 runs so far, 0 failures, 10 active
2m20s: 6317 runs so far, 0 failures, 10 active
2m25s: 6553 runs so far, 0 failures, 10 active
2m30s: 6793 runs so far, 0 failures, 10 active
2m35s: 7036 runs so far, 0 failures, 10 active
2m40s: 7253 runs so far, 0 failures, 10 active
2m45s: 7481 runs so far, 0 failures, 10 active
2m50s: 7727 runs so far, 0 failures, 10 active
2m55s: 7972 runs so far, 0 failures, 10 active
3m0s: 8217 runs so far, 0 failures, 10 active
3m5s: 8467 runs so far, 0 failures, 10 active
3m10s: 8711 runs so far, 0 failures, 10 active
3m15s: 8955 runs so far, 0 failures, 10 active
3m20s: 9203 runs so far, 0 failures, 10 active
3m25s: 9448 runs so far, 0 failures, 10 active
3m30s: 9695 runs so far, 0 failures, 10 active
3m35s: 9938 runs so far, 0 failures, 10 active
3m40s: 10184 runs so far, 0 failures, 10 active
3m45s: 10430 runs so far, 0 failures, 10 active
3m50s: 10674 runs so far, 0 failures, 10 active
3m55s: 10923 runs so far, 0 failures, 10 active
4m0s: 11170 runs so far, 0 failures, 10 active
4m5s: 11413 runs so far, 0 failures, 10 active
4m10s: 11616 runs so far, 0 failures, 10 active
4m15s: 11833 runs so far, 0 failures, 10 active
4m20s: 12047 runs so far, 0 failures, 10 active
4m25s: 12243 runs so far, 0 failures, 10 active
4m30s: 12418 runs so far, 0 failures, 10 active
4m35s: 12547 runs so far, 0 failures, 10 active
4m40s: 12712 runs so far, 0 failures, 10 active
4m45s: 12917 runs so far, 0 failures, 10 active
4m50s: 13143 runs so far, 0 failures, 10 active
4m55s: 13388 runs so far, 0 failures, 10 active
5m0s: 13635 runs so far, 0 failures, 10 active
5m5s: 13877 runs so far, 0 failures, 10 active
5m10s: 14091 runs so far, 0 failures, 10 active
5m15s: 14335 runs so far, 0 failures, 10 active
5m20s: 14582 runs so far, 0 failures, 10 active
5m25s: 14829 runs so far, 0 failures, 10 active
5m30s: 15078 runs so far, 0 failures, 10 active
5m35s: 15321 runs so far, 0 failures, 10 active
5m40s: 15566 runs so far, 0 failures, 10 active
5m45s: 15808 runs so far, 0 failures, 10 active
5m50s: 16051 runs so far, 0 failures, 10 active
5m55s: 16296 runs so far, 0 failures, 10 active
6m0s: 16544 runs so far, 0 failures, 10 active
6m5s: 16784 runs so far, 0 failures, 10 active
6m10s: 17029 runs so far, 0 failures, 10 active
6m15s: 17269 runs so far, 0 failures, 10 active
6m20s: 17499 runs so far, 0 failures, 10 active
6m25s: 17744 runs so far, 0 failures, 10 active
6m30s: 17992 runs so far, 0 failures, 10 active
6m35s: 18235 runs so far, 0 failures, 10 active
6m40s: 18482 runs so far, 0 failures, 10 active
6m45s: 18722 runs so far, 0 failures, 10 active
6m50s: 18970 runs so far, 0 failures, 10 active
6m55s: 19217 runs so far, 0 failures, 10 active
7m0s: 19462 runs so far, 0 failures, 10 active
7m5s: 19705 runs so far, 0 failures, 10 active
7m10s: 19930 runs so far, 0 failures, 10 active
7m15s: 20170 runs so far, 0 failures, 10 active
7m20s: 20419 runs so far, 0 failures, 10 active
7m25s: 20664 runs so far, 0 failures, 10 active
7m30s: 20899 runs so far, 0 failures, 10 active
7m35s: 21138 runs so far, 0 failures, 10 active
7m40s: 21382 runs so far, 0 failures, 10 active
7m45s: 21627 runs so far, 0 failures, 10 active
7m50s: 21871 runs so far, 0 failures, 10 active
7m55s: 22116 runs so far, 0 failures, 10 active
8m0s: 22363 runs so far, 0 failures, 10 active
8m5s: 22609 runs so far, 0 failures, 10 active
8m10s: 22857 runs so far, 0 failures, 10 active
8m15s: 23101 runs so far, 0 failures, 10 active
8m20s: 23312 runs so far, 0 failures, 10 active
8m25s: 23560 runs so far, 0 failures, 10 active
8m30s: 23802 runs so far, 0 failures, 10 active
8m35s: 24048 runs so far, 0 failures, 10 active
8m40s: 24290 runs so far, 0 failures, 10 active
8m45s: 24507 runs so far, 0 failures, 10 active
8m50s: 24741 runs so far, 0 failures, 10 active
8m55s: 24988 runs so far, 0 failures, 10 active
9m0s: 25233 runs so far, 0 failures, 10 active
9m5s: 25477 runs so far, 0 failures, 10 active
9m10s: 25722 runs so far, 0 failures, 10 active
9m15s: 25960 runs so far, 0 failures, 10 active
9m20s: 26202 runs so far, 0 failures, 10 active
9m25s: 26430 runs so far, 0 failures, 10 active
9m30s: 26647 runs so far, 0 failures, 10 active
9m35s: 26838 runs so far, 0 failures, 10 active
9m40s: 27073 runs so far, 0 failures, 10 active
9m45s: 27287 runs so far, 0 failures, 10 active
9m50s: 27536 runs so far, 0 failures, 10 active
9m55s: 27777 runs so far, 0 failures, 10 active
10m0s: 28025 runs so far, 0 failures, 10 active
10m5s: 28267 runs so far, 0 failures, 10 active
10m10s: 28479 runs so far, 0 failures, 10 active
10m15s: 28709 runs so far, 0 failures, 10 active
10m20s: 28950 runs so far, 0 failures, 10 active
10m25s: 29193 runs so far, 0 failures, 10 active
10m30s: 29438 runs so far, 0 failures, 10 active
10m35s: 29678 runs so far, 0 failures, 10 active
10m40s: 29921 runs so far, 0 failures, 10 active
10m45s: 30158 runs so far, 0 failures, 10 active
10m50s: 30401 runs so far, 0 failures, 10 active
10m55s: 30645 runs so far, 0 failures, 10 active

@RaduBerinde
Copy link
Contributor Author

It might take a while, I'd increase the timeout to e.g. 1 hour. I ran it on many different versions on the test until I saw a crash, so it could have been a particularly lucky run.

@prattmic
Copy link
Member

cc @golang/runtime

@prattmic prattmic added compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 28, 2024
@prattmic prattmic added this to the Backlog milestone Aug 28, 2024
@randall77
Copy link
Contributor

I can reproduce, albeit rarely (~1 in 100000 runs).

1.22.5:
7h46m40s: 1850954 runs so far, 13 failures (0.00%)

tip:
1h33m35s: 377150 runs so far, 5 failures (0.00%)

I suspect something about same-sized grow. I'll keep looking.

@randall77
Copy link
Contributor

randall77 commented Aug 29, 2024

Yes, looks like it is an odd interaction between same-sized growth, regular growth, and cloning.
This causes clone to be in a state where the number of source buckets is less than the number of destination buckets. That causes the clone to walk off the end of the source bucket array.

Here's a reproducer:

package main

import (
	"maps"
	"unsafe"
)

func main() {
	for i := 0; i < 100; i++ {
		f()
	}
}

const NB = 4

func f() {
	// Make a map with NB buckets, at max capacity.
	// 6.5 entries/bucket.
	ne := NB * 13 / 2
	m := map[int]int{}
	for i := 0; i < ne; i++ {
		m[i] = i
	}

	// delete/insert a lot, to hopefully get lots of overflow buckets
	// and trigger a same-size grow.
	ssg := false
	for i := ne; i < ne+1000; i++ {
		delete(m, i-ne)
		m[i] = i
		p := *(**hmap)(unsafe.Pointer(&m))
		if p.flags&8 != 0 { // same-sized grow flag
			ssg = true
			break
		}
	}
	if !ssg {
		return
	}

	// Insert 1 more entry, which would ordinarily trigger a growth.
	// We can't grow while growing, so we instead go over our
	// target capacity.
	m[-1] = -1

	// Cloning in this state will make a map with a destination bucket
	// array twice the size of the source.
	_ = maps.Clone(m)
}

type hmap struct {
	count int
	flags uint8
}

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/609757 mentions this issue: runtime: size maps.Clone destination bucket array safely

@randall77
Copy link
Contributor

@gopherbot Please open backport issues.
This is a rare memory corruption bug in maps.Clone.

@gopherbot
Copy link
Contributor

Backport issue(s) opened: #69155 (for 1.22), #69156 (for 1.23).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. FixPending Issues that have a fix which has not yet been reviewed or submitted. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 30, 2024
@dmitshur dmitshur moved this from Todo to In Progress in Go Compiler / Runtime Aug 30, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Go Compiler / Runtime Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FixPending Issues that have a fix which has not yet been reviewed or submitted. NeedsFix The path to resolution is known, but the work has not been done.
Projects
Development

No branches or pull requests

8 participants