cmd/compile: expand checkptr to find conversions of smaller types into bigger

[ainar-g]

Per discussion in golang-dev.

The checkptr dynamic analyser doesn't find bad type conversions between types where the destination type is greater than the source type. E. g.:

var i32sink int32

type t1 struct { a, b int32 }

type t2 struct { a, b, c, d int32 }

func main() {
	for i := 0; i < 1000; i++ {
		var badt2 = *(*t2)(unsafe.Pointer(&t1{}))
		i32sink = badt2.d
	}
}

Playground link: https://play.golang.org/p/32lWMkFPvm5.

According to @mdempsky:

Detecting that is not currently supported, but I think there may at least be some cases where we can detect it. (…)

[randall77]

We do this sometimes in runtime and reflect. For example, see reflect/type.go:(*rtype).Elem. It does a switch on t.Kind() and then casts an *rtype to other types (e.g. *arrayType) for which the base type is larger.

So we'd need to check that the underlying object allocation is large enough. Just having the base type be larger is not in itself an error.

[mdempsky]

@randall77 Yeah. I was thinking that when we convert p to *T, we just check that p and p+sizeof(T)-1 both point into the same heap object, kinda like how we do for pointer arithmetic checking

[mdempsky]

It might also be worth recognizing (*[VeryBig]T)(ptr)[:n:m], since that's a common idiom for turning pointers into slices.

[gopherbot]

Change https://golang.org/cl/201778 mentions this issue: cmd/compile: detect unsafe conversions from smaller to larger types

[mdempsky]

CL 201778 can now detect the invalid conversion here:

package main

import "unsafe"

var i32sink int32

type t1 struct{ a, b int32 }

type t2 struct{ a, b, c, d int32 }

//go:noinline
func alloc() *t1 {
	return &t1{}
}

func main() {
	for i := 0; i < 1000; i++ {
		var badt2 = *(*t2)(unsafe.Pointer(alloc()))
		i32sink = badt2.d
	}
}

It still doesn't detect the original example, but because in that one &t1{} is stack allocated, and the instrumentation currently can only validate pointers to heap objects.

One solution would be to make -d=checkptr change escape analysis to treat conversions to unsafe.Pointer as an escaping operation, which would force the objects onto the heap and allow better runtime validation.

[gopherbot]

Change https://golang.org/cl/201781 mentions this issue: cmd/compile: escape unsafe.Pointer conversions when -d=checkptr

[ainar-g]

Holy Dennis Ritchie that was quick! Amazing, thanks!

[gopherbot]

Change https://golang.org/cl/201840 mentions this issue: cmd/compile: only escape unsafe.Pointer conversions when -d=checkptr=2