unique: `fatal error: getWeakHandle on invalid pointer` with `sbrk=1`

[brasic]

Go version

go1.24-0206eb9679, 1.23.1

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/cbrasic/Library/Caches/go-build'
GOENV='/Users/cbrasic/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/cbrasic/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/cbrasic/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/cbrasic/repos/goroot'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/cbrasic/repos/goroot/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='devel go1.24-0206eb9679 Mon Sep 30 19:15:39 2024 +0000'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/cbrasic/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOARM64='v8.0'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/_n/xv5mgbd173xf6p4vwlq3hmpw0000gp/T/go-build1074858196=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

I noticed that my go1.23.1 app panics when run with GODEBUG="sbrk=1" and tracked it down to net/netip, specifically this line. I saw #69458 and thought this issue might already be fixed so I rebuilt with go master but saw the same issue. Any app that imports net/netip will crash when run with GODEBUG="sbrk=1"

What did you see happen?

$ cat main.go
package main

import "unique"

type addrDetail struct {
        isV6 bool
}

var (
        z6noz = unique.Make(addrDetail{isV6: true})
)

func main() {
}
$ go build main.go; GODEBUG="sbrk=1" ./main

fatal error: getWeakHandle on invalid pointer

goroutine 1 gp=0x10b290030 m=0 mp=0x104720320 [running, locked to thread]:
runtime.throw({0x10468f442?, 0x10462a4dc?})
        /Users/cbrasic/repos/goroot/src/runtime/panic.go:1069 +0x38 fp=0x1400008cc20 sp=0x1400008cbf0 pc=0x10467f038
runtime.getWeakHandle(0x10b2d5e58)
        /Users/cbrasic/repos/goroot/src/runtime/mheap.go:2157 +0x138 fp=0x1400008cc70 sp=0x1400008cc20 pc=0x104642d58
runtime.getOrAddWeakHandle(0x10b2d5e58)
        /Users/cbrasic/repos/goroot/src/runtime/mheap.go:2098 +0x20 fp=0x1400008ccc0 sp=0x1400008cc70 pc=0x104642a50
internal/weak.runtime_registerWeakPointer(0x1400008cd01?)
        /Users/cbrasic/repos/goroot/src/runtime/mheap.go:2045 +0x1c fp=0x1400008cce0 sp=0x1400008ccc0 pc=0x10467e95c
internal/weak.Make[...](0x10b2d5e58?)
        /Users/cbrasic/repos/goroot/src/internal/weak/pointer.go:62 +0x74 fp=0x1400008cd00 sp=0x1400008cce0 pc=0x10468acb4
unique.Make[...].func1()
        /Users/cbrasic/repos/goroot/src/unique/handle.go:64 +0xb4 fp=0x1400008cd50 sp=0x1400008cd00 pc=0x10468abe4
unique.Make[...]({0x0})
        /Users/cbrasic/repos/goroot/src/unique/handle.go:74 +0x144 fp=0x1400008cdf0 sp=0x1400008cd50 pc=0x10468aab4
main.init()
        /Users/cbrasic/repos/test/main.go:10 +0x28 fp=0x1400008ce10 sp=0x1400008cdf0 pc=0x104689828
runtime.doInit1(0x10471c350)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:7303 +0xd4 fp=0x1400008cf40 sp=0x1400008ce10 pc=0x10465fd44
runtime.doInit(...)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:7270
runtime.main()
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:254 +0x340 fp=0x1400008cfd0 sp=0x1400008cf40 pc=0x104650790
runtime.goexit({})
        /Users/cbrasic/repos/goroot/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400008cfd0 sp=0x1400008cfd0 pc=0x1046847e4

goroutine 17 gp=0x10b2d52f8 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:435 +0xc8 fp=0x1400002e790 sp=0x1400002e770 pc=0x10467f118
runtime.goparkunlock(...)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:441
runtime.forcegchelper()
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:348 +0xb8 fp=0x1400002e7d0 sp=0x1400002e790 pc=0x104650a28
runtime.goexit({})
        /Users/cbrasic/repos/goroot/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400002e7d0 sp=0x1400002e7d0 pc=0x1046847e4
created by runtime.init.7 in goroutine 1
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:336 +0x24

goroutine 18 gp=0x10b2d5548 m=nil [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:435 +0xc8 fp=0x1400002ef60 sp=0x1400002ef40 pc=0x10467f118
runtime.goparkunlock(...)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:441
runtime.bgsweep(0x10b2d54c0)
        /Users/cbrasic/repos/goroot/src/runtime/mgcsweep.go:277 +0xa0 fp=0x1400002efb0 sp=0x1400002ef60 pc=0x10463cf10
runtime.gcenable.gowrap1()
        /Users/cbrasic/repos/goroot/src/runtime/mgc.go:203 +0x28 fp=0x1400002efd0 sp=0x1400002efb0 pc=0x1046310f8
runtime.goexit({})
        /Users/cbrasic/repos/goroot/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400002efd0 sp=0x1400002efd0 pc=0x1046847e4
created by runtime.gcenable in goroutine 1
        /Users/cbrasic/repos/goroot/src/runtime/mgc.go:203 +0x6c

goroutine 19 gp=0x10b2d5728 m=nil [GC scavenge wait]:
runtime.gopark(0x10b2d54c0?, 0x1046a8e88?, 0x1?, 0x0?, 0x10b2d5728?)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:435 +0xc8 fp=0x1400002f760 sp=0x1400002f740 pc=0x10467f118
runtime.goparkunlock(...)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:441
runtime.(*scavengerState).park(0x10471fba0)
        /Users/cbrasic/repos/goroot/src/runtime/mgcscavenge.go:425 +0x5c fp=0x1400002f790 sp=0x1400002f760 pc=0x10463a9ac
runtime.bgscavenge(0x10b2d54c0)
        /Users/cbrasic/repos/goroot/src/runtime/mgcscavenge.go:653 +0x44 fp=0x1400002f7b0 sp=0x1400002f790 pc=0x10463aee4
runtime.gcenable.gowrap2()
        /Users/cbrasic/repos/goroot/src/runtime/mgc.go:204 +0x28 fp=0x1400002f7d0 sp=0x1400002f7b0 pc=0x104631098
runtime.goexit({})
        /Users/cbrasic/repos/goroot/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400002f7d0 sp=0x1400002f7d0 pc=0x1046847e4
created by runtime.gcenable in goroutine 1
        /Users/cbrasic/repos/goroot/src/runtime/mgc.go:204 +0xac

goroutine 20 gp=0x10b2d5b40 m=nil [chan receive]:
runtime.gopark(0x14000032738?, 0x1046505c4?, 0x38?, 0x27?, 0x0?)
        /Users/cbrasic/repos/goroot/src/runtime/proc.go:435 +0xc8 fp=0x140000326f0 sp=0x140000326d0 pc=0x10467f118
runtime.chanrecv(0x10b2d5ac0, 0x0, 0x1)
        /Users/cbrasic/repos/goroot/src/runtime/chan.go:640 +0x408 fp=0x14000032770 sp=0x140000326f0 pc=0x104624c88
runtime.chanrecv1(0x104720320?, 0x104650850?)
        /Users/cbrasic/repos/goroot/src/runtime/chan.go:490 +0x14 fp=0x140000327a0 sp=0x14000032770 pc=0x104624874
runtime.unique_runtime_registerUniqueMapCleanup.func1(...)
        /Users/cbrasic/repos/goroot/src/runtime/mgc.go:1731
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
        /Users/cbrasic/repos/goroot/src/runtime/mgc.go:1734 +0x3c fp=0x140000327d0 sp=0x140000327a0 pc=0x104633f8c
runtime.goexit({})
        /Users/cbrasic/repos/goroot/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000327d0 sp=0x140000327d0 pc=0x1046847e4
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
        /Users/cbrasic/repos/goroot/src/runtime/mgc.go:1729 +0xa0

What did you expect to see?

no crash

[gabyhelp]

Related Issues and Documentation

• unique: panic for Make(struct{}{}) #69458 (closed)
• unique: Fatal errors (found bad pointer in Go heap, found pointer to free object) and memory corruption #69643
• unique: fatal error: found pointer to free object #69210 (closed)
• internal/concurrent: hashtriemap uses abi.NoEscape cause incorrect inlining decision #68511 (closed)
• runtime: segmentation fault on linux/arm when GODEBUG=sbrk=1 #33159 (closed)
• runtime: SIGBUS in initItab (write to readonly ITab.Fun) on unchecked type assertion with generics #65962 (closed)
• fatal error: unexpected signal during runtime execution #49543 (closed)
• cmd/compile: unreachable panic with GODEBUG=gotypesalias=1 #65778 (closed)
• runtime: GODEBUG=efence=1 causes panic during build on M1 #44993
• runtime: fatal error: found bad pointer in Go heap #29362 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

CC @mknyszek @golang/runtime

[mknyszek]

FWIW, I think GODEBUG=sbrk=1 has similar issues with finalizers, because that mode doesn't set up any runtime metadata for memory allocations. So, using os.File will cause similar issues.

This debug mode is a debugging tool to rule out the memory allocator when debugging super subtle issues with the runtime. It's not really meant to be used by developers using Go, only developers of Go. But even then, it's not used very frequently at all (I can't remember the last time someone used it to debug an isue). So just to set expectations, I don't think this is going to be fixed any time soon; it's not particularly high priority.

[gabyhelp]

Related Issues and Documentation

• unique: panic for Make(struct{}{}) #69458 (closed)
• unique: Fatal errors (found bad pointer in Go heap, found pointer to free object) and memory corruption #69643
• unique: fatal error: found pointer to free object #69210 (closed)
• internal/concurrent: hashtriemap uses abi.NoEscape cause incorrect inlining decision #68511 (closed)
• runtime: segmentation fault on linux/arm when GODEBUG=sbrk=1 #33159 (closed)
• runtime: SIGBUS in initItab (write to readonly ITab.Fun) on unchecked type assertion with generics #65962 (closed)
• fatal error: unexpected signal during runtime execution #49543 (closed)
• cmd/compile: unreachable panic with GODEBUG=gotypesalias=1 #65778 (closed)
• runtime: GODEBUG=efence=1 causes panic during build on M1 #44993
• runtime: fatal error: found bad pointer in Go heap #29362 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[mknyszek]

I was wrong in my last comment. We had to be able to handle immortal objects anyway, and now that's fixed. (#71726)

For sbrk mode, we could just always use the immortal object map, and that would resolve this issue.

[josharian]

FWIW, I find sbrk=1 helpful as an end user when I'm doing awful println debugging with lots of pointers/channels--it guarantees that there's no re-use, which makes correlating log lines more fruitful.

[gopherbot]

Change https://go.dev/cl/674656 mentions this issue: runtime: use the immortal weak handle map for sbrk mode