all: get standard library building with -d=checkptr #34972

mdempsky · 2019-10-17T22:16:21Z

As a prerequisite for #34964, the standard library's tests need to all pass when -d=checkptr is enabled:

$ go test -a -short -gcflags=all=-d=checkptr std cmd

Currently, that's not the case.

There are also some panics because of (*[Big]T)(p)[:n] expressions, like in package reflect. This code pattern should be recognized by cmd/compile.
There are a few tests that fail because CL 201781 leads them to heap allocate when before things would only stack allocate; e.g., context.TestAllocs or database/sql.TestRawBytesAllocs. Not sure how to handle this; maybe for now the escape analysis change should only be enabled for -d=checkptr=2, and -race/-msan only enable -d=checkptr=1.
In sync/atomic.hammerStoreLoadPointer, there's (in effect):
```
 new := uintptr(LoadPointer(addr))
 new += offset
 StorePointer(addr, unsafe.Pointer(new))
```
This violates package unsafe's pointer rules: pointers have to be converted to uintptr and back to unsafe.Pointer in a single expression without being stored in a uintptr-typed variable, as is being done with new. (This just needs someone to grok exactly what the test is doing, and fix it to follow pointer rules correctly.)

There seem to be other failures still that need to be diagnosed. If folks want to try running the command above, claiming a failure, and then investigating what's going on, that would be great.

The text was updated successfully, but these errors were encountered:

mdempsky · 2019-10-17T22:48:05Z

/cc @odeke-em @cuonglm

mdempsky · 2019-10-17T23:41:43Z

This code in package reflect is currently failing in checkptrAlignment, and seems to violate the safety rules because of the conversion to *[16]byte (sometimes gcdata points to <16 bytes):

go/src/reflect/type.go

Lines 2184 to 2192 in 8c6876e

    
           kmask := (*[16]byte)(unsafe.Pointer(ktyp.gcdata)) 
        
           for i := uintptr(0); i < ktyp.ptrdata/ptrSize; i++ { 
        
           	if (kmask[i/8]>>(i%8))&1 != 0 { 
        
           		for j := uintptr(0); j < bucketSize; j++ { 
        
           			word := base + j*ktyp.size/ptrSize + i 
        
           			mask[word/8] |= 1 << (word % 8) 
        
           		} 
        
           	} 
        
           }

(There's similar code for etype just below.)

odeke-em · 2019-10-17T23:56:29Z

I'll cover the sync/atomic.hammer* tests fixup :)

mdempsky · 2019-10-18T00:02:35Z

This failure looks genuinely suspicious to me:

$ go test -a -short -gcflags=all=-d=checkptr testing/quick
runtime: pointer 0xc00022a3e0 to unallocated span span.base()=0xc000222000 span.limit=0xc00023a000 span.state=3
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)

gopherbot · 2019-10-18T00:14:48Z

Change https://golang.org/cl/201841 mentions this issue: all: disable tests that fail under -d=checkptr

gopherbot · 2019-10-18T00:14:51Z

Change https://golang.org/cl/201839 mentions this issue: cmd/compile: recognize (*[Big]T)(ptr)[:n:m] pattern for -d=checkptr

mdempsky · 2019-10-18T00:18:32Z

Uploaded a couple CLs to improve -d=checkptr, and address some issues.

CL 201841 enumerates all of the remaining std cmd test failures that I'm aware of, but haven't investigated yet.

odeke-em · 2019-10-18T02:46:44Z

Hey @mdempsky in regards to the sync/atomic.HammerLoad* tests that are flagged, what they do is that for 8 goroutines, run pairwise increments of the high and low parts of an int* or *uint and panic if high != low. To save the value at the end of the function, we invoke

StorePointer(addr, newValue)

where newValue is a raw uint64 value such as 0xc0000000c0 and that value is being written to addr.

checkptr tries to find the object associated with newValue but unfortunately that's a raw value that's not attached to any object and perhaps beyond Go's address space. Writing such code is a valid usecase and here is a minimal repro to test that edge case

package main

import (
	"fmt"
	"sync/atomic"
	"unsafe"
)

func main() {
	val := uint64(820338753727)
	addr := (*unsafe.Pointer)(unsafe.Pointer(&val))
	v := uintptr(atomic.LoadPointer(addr))
	atomic.StorePointer(addr, unsafe.Pointer(v+1+1<<32))

	xs := uintptr(atomic.LoadPointer(addr))
	fmt.Printf("xs: %#x\n", xs)
}

$ go run -gcflags=all=-d=checkptr main.go 
panic: (runtime.ptrArith) (0x10c09c0,0xc00000c080)

goroutine 1 [running]:
main.main()
	/Users/emmanuelodeke/Desktop/openSrc/bugs/golang/34972/main.go:13 +0x91
exit status 2

Trying to recognize cases that do plain arithmetic is going to be a tricky one :)

gopherbot · 2019-10-18T03:23:56Z

Change https://golang.org/cl/201877 mentions this issue: syscall: fix wrong unsafe.Pointer align in anyToSockaddr

cuonglm · 2019-10-18T04:06:00Z

Another instance in reflect:

go/src/reflect/type.go

Line 1011 in 8c6876e

    
           return (*[1 << 20]*rtype)(add(unsafe.Pointer(t), uadd, "t.inCount > 0"))[:t.inCount]

and some lines below:

go/src/reflect/type.go

Line 1023 in 8c6876e

    
           return (*[1 << 20]*rtype)(add(unsafe.Pointer(t), uadd, "outCount > 0"))[t.inCount : t.inCount+outCount]

mdempsky · 2019-10-18T06:13:44Z

@odeke-em Thanks for looking into it.

Writing such code is a valid usecase

Regardless of the merit of the use case, if the example violates package unsafe's safety rules, then it's not valid Go.

Your repro case can be simplified to just:

package main

import (
	"fmt"
	"unsafe"
)

func main() {
	fmt.Println(unsafe.Pointer(uintptr(0xc000000000)))
}

This is failing because 0xc000000000 is an offset into the Go heap (at least where Go tries to allocate it by default on 64-bit OSes), and package unsafe's safety rules don't allow you to fabricate a pointer into the heap out of nothing.

To fix the test, it should either:

Exclusively use pointers outside of the Go heap. (This is challenging, because you don't know where the Go heap is, and allocating non-Go-heap memory in a portable way is tricky.)
Alternatively, allocate a large chunk of memory, find an adequately aligned pointer within that chunk of memory, and then restrict to modifying the lower bits of the address. For example:
```
 buf := make([]byte, 1 << 21) // allocate 2MB
 p := unsafe.Pointer(&buf[0])
 p = unsafe.Pointer((uintptr(p) + 1<<20 - 1) &^ (1<<20 - 1))  // align p to 1MB boundary
```
p is now 20-bit aligned, and you can change any of the lower 20 bits and be guaranteed it still points into buf.

bcmills · 2019-10-18T12:45:18Z

There are also some panics because of (*[Big]T)(p)[:n] expressions, like in package reflect. This code pattern should be recognized by cmd/compile.

I saw that one coming in #13656 (comment)!

One could easily envision a compiler that, say, sanity-checks that all pointers to arrays point to addresses that are mapped in the program's address space.

bcmills · 2019-10-18T12:50:35Z

There are a few tests that fail because CL 201781 leads them to heap allocate when before things would only stack allocate

If so, those tests are overspecified: the language spec does not make any guarantees about allocations or escapes. They should be updated to skip (or use different thresholds) when they are not using a specific compiler in a specific configuration.

Caught with: go test -a -short -gcflags=all=-d=checkptr log/syslog and: grep -rE '\*\[([^2]|.{2,})\].*\)\(unsafe.Pointer' syscall Updates #34972 Change-Id: Iafd199b3a34beb7cc3e88484bf2fbae45183f951 Reviewed-on: https://go-review.googlesource.com/c/go/+/201877 Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>

mdempsky · 2019-10-18T20:00:18Z

They should be updated to skip (or use different thresholds) when they are not using a specific compiler in a specific configuration.

Agreed, though I don't think currently there's a way for tests to check what compiler flags were used. (Keep in mind too that different compiler flags can be used for different packages.)

cuonglm · 2019-10-19T07:23:55Z

@mdempsky on OSX, I got a lot of:

fatal error: stack growth after fork

runtime stack:
runtime.throw(0x12e77be, 0x17)
	/Users/cuonglm/sources/go/src/runtime/panic.go:774 +0x72
runtime.newstack()
	/Users/cuonglm/sources/go/src/runtime/stack.go:928 +0xd04
runtime.morestack()
	/Users/cuonglm/sources/go/src/runtime/asm_amd64.s:449 +0x8f

goroutine 19 [running]:
runtime.checkptrAlignment(0xc00008a9e8, 0x126e840)
	/Users/cuonglm/sources/go/src/runtime/checkptr.go:14 +0x137 fp=0xc00008a8e8 sp=0xc00008a8e0 pc=0x1006177
syscall.funcPC(...)
	/Users/cuonglm/sources/go/src/syscall/syscall_darwin.go:469
syscall.forkAndExecInChild(0xc00008e120, 0xc000082ba0, 0x5, 0x5, 0xc00006a900, 0x2e, 0x2e, 0x0, 0x0, 0xc00008ac00, ...)
	/Users/cuonglm/sources/go/src/syscall/exec_darwin.go:65 +0xe6 fp=0xc00008aa60 sp=0xc00008a8e8 pc=0x10703d6
syscall.forkExec(0xc0000c6160, 0x20, 0xc000094a40, 0x4, 0x4, 0xc00008ac00, 0x1140b6b, 0x0, 0x0)
	/Users/cuonglm/sources/go/src/syscall/exec_unix.go:201 +0x35b fp=0xc00008ab70 sp=0xc00008aa60 pc=0x10718eb
syscall.StartProcess(...)
	/Users/cuonglm/sources/go/src/syscall/exec_unix.go:248
os.startProcess(0xc0000c6160, 0x20, 0xc000094a40, 0x4, 0x4, 0xc00008ad98, 0x0, 0x0, 0x0)
	/Users/cuonglm/sources/go/src/os/exec_posix.go:52 +0x2c0 fp=0xc00008ac58 sp=0xc00008ab70 pc=0x1093300
os.StartProcess(0xc0000c6160, 0x20, 0xc000094a40, 0x4, 0x4, 0xc00008ad98, 0x2d, 0x0, 0x0)
	/Users/cuonglm/sources/go/src/os/exec.go:102 +0x7c fp=0xc00008acb0 sp=0xc00008ac58 pc=0x1092d2c
os/exec.(*Cmd).Start(0xc0000bc2c0, 0xc00008af01, 0xc000082b70)
	/Users/cuonglm/sources/go/src/os/exec/exec.go:416 +0x50c fp=0xc00008adf0 sp=0xc00008acb0 pc=0x116cbac
os/exec.(*Cmd).Run(0xc0000bc2c0, 0xc000082b70, 0xc0000bc2c0)
	/Users/cuonglm/sources/go/src/os/exec/exec.go:338 +0x2b fp=0xc00008ae18 sp=0xc00008adf0 pc=0x116c63b
os/exec.(*Cmd).CombinedOutput(0xc0000bc2c0, 0x20, 0xc00008af30, 0x3, 0x3, 0xc0000bc2c0)
	/Users/cuonglm/sources/go/src/os/exec/exec.go:561 +0x91 fp=0xc00008ae48 sp=0xc00008ae18 pc=0x116d7a1
go/importer.TestForCompiler(0xc0000f0300)
	/Users/cuonglm/sources/go/src/go/importer/importer_test.go:23 +0x11d fp=0xc00008af70 sp=0xc00008ae48 pc=0x124dfed
testing.tRunner(0xc0000f0300, 0x12f26a0)
	/Users/cuonglm/sources/go/src/testing/testing.go:909 +0xc9 fp=0xc00008afd0 sp=0xc00008af70 pc=0x10f8249
runtime.goexit()
	/Users/cuonglm/sources/go/src/runtime/asm_amd64.s:1375 +0x1 fp=0xc00008afd8 sp=0xc00008afd0 pc=0x105cd11
created by testing.(*T).Run
	/Users/cuonglm/sources/go/src/testing/testing.go:960 +0x351

when testing std, even after 46aa835

Note

This error does not happen on Linux.

mdempsky · 2019-10-19T08:14:29Z

@cuonglm Thanks for the report.

It looks like maybe checkptrAlignment and checkptrArithmetic need to be labeled //go:nosplit.

gopherbot · 2019-10-19T08:29:25Z

Change https://golang.org/cl/202157 mentions this issue: runtime: fix unsafe.Pointer alignment on Linux

cuonglm · 2019-10-19T08:41:47Z

@cuonglm Thanks for the report.

It looks like maybe checkptrAlignment and checkptrArithmetic need to be labeled //go:nosplit.

I tried, but still fails with go:nosplit for checkptrAlignment and checkptrArithmetic. But it does pass if findObject is marked go:nosplit, too.

mdempsky · 2019-10-19T08:56:31Z

Ugh, right.

Okay, in that case, we should probably just make //go:nosplit imply //go:nocheckptr, like how //go:cgo_unsafe_args does. (See CL 201823.)

gopherbot · 2019-10-19T09:20:30Z

Change https://golang.org/cl/202158 mentions this issue: cmd/compile: disable checkptr for //go:nosplit functions

gopherbot · 2019-10-19T10:40:40Z

Change https://golang.org/cl/202177 mentions this issue: windows, unix: fix wrong unsafe.Pointer alignment in syscall

Same as CL 201877 did for package syscall. Updates golang/go#34972 Change-Id: I3929841ab32378516edafb1f02a84b1bdcc77bbd Reviewed-on: https://go-review.googlesource.com/c/sys/+/202177 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

gopherbot · 2019-10-21T22:53:23Z

Change https://golang.org/cl/202580 mentions this issue: reflect, internal/reflectlite: set capacity when slicing unsafe pointers

A common idiom for turning an unsafe.Pointer into a slice is to write: s := (*[Big]T)(ptr)[:n:m] This technically violates Go's unsafe pointer rules (rule #1 says T2 can't be bigger than T1), but it's fairly common and not too difficult to recognize, so might as well allow it for now so we can make progress on #34972. This should be revisited if #19367 is accepted. Updates #22218. Updates #34972. Change-Id: Id824e2461904e770910b6e728b4234041d2cc8bc Reviewed-on: https://go-review.googlesource.com/c/go/+/201839 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Follow the idiom for allowing -d=checkptr to recognize and verify correctness. Updates #22218. Updates #34972. Change-Id: Ib6001c6f0e6dc535a36bcfaa1ae48e29e0c737f8 Reviewed-on: https://go-review.googlesource.com/c/go/+/202580 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>

aclements · 2020-01-10T17:34:48Z

Actually, WSASendTo (with the big 'T') is definitely an unsafe API, since it takes the byte length of the sockaddr as an argument and hence could be used to make the kernel read arbitrary memory from the Go process. We try not to expose unsafe APIs from syscall, and go to some significant lengths to make APIs safe in the UNIX syscall packages (though this is certainly not the only unsafe API in the Windows syscall package).

Can we just remove WSASendTo? syscall is not covered by Go 1 compatibility, and this would fall under the security exception anyway.

alexbrainman · 2020-02-16T05:22:38Z

WSASendto calls WSASendTo (which seems a little nuts to me, but okay...),

WSASendto was added in

https://golang.org/cl/3136042

These were the days where syscall package was free for all to do as we pleased.

What if we were to directly invoke the syscall from WSASendto with the unsafe.Pointer that came from to.sockaddr(), instead of converting it through an intermediate *RawSockaddrAny?

I will try and report it back here.

Actually, WSASendTo (with the big 'T') is definitely an unsafe API, ...

You have to complain to Microsoft

https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsasendto

Even better. You should complain to people who designed Berkeley sockets in the first place.

Can we just remove WSASendTo?

We can do anything you want. But we are using the function ourselves in net package. And other people might be using it too. So we would have to replace it with something that call real WSASendTo anyway.

Alex

aclements · 2020-02-16T16:22:02Z

I will try and report it back here.

Thanks!

Actually, WSASendTo (with the big 'T') is definitely an unsafe API, ...

You have to complain to Microsoft
https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsasendto
Even better. You should complain to people who designed Berkeley sockets in the first place.

Yeah, it's not great. But WSASendto exposes a perfectly safe API by taking the Sockaddr interface. That's unsafe internally, but only the syscall package can implement its methods, so that unsafeness doesn't leak through the user API. Sendto on UNIX works the same way. I think the mistake was just exporting the unsafe raw WSASendTo API underlying it (and some related functions. The UNIX packages also have an unsafe sendto API, but it's not exported.

Can we just remove WSASendTo?

We can do anything you want. But we are using the function ourselves in net package. And other people might be using it too.

I may be totally missing something, but I don't see any calls to WSASendTo in net (or x/net). I do see WSASendto. In fact, I couldn't find any calls to WSASendTo in the GitHub corpus I have access to (though I'm really confused about how much coverage that has).

gopherbot · 2020-02-25T08:13:06Z

Change https://golang.org/cl/220544 mentions this issue: syscall: fix windows WSASendto -d=checkptr violation

alexbrainman · 2020-02-25T08:19:06Z

But WSASendto exposes a perfectly safe API by taking the Sockaddr interface. That's unsafe internally, but only the syscall package can implement its methods, so that unsafeness doesn't leak through the user API.

SGTM

I may be totally missing something, but I don't see any calls to WSASendTo in net ...

I did not explain myself properly. net package calls syscall.WSASendto and syscall.WSASendto calls syscall.WSASendTo. So net package calls syscall.WSASendTo indirectly. WSASendto Windows API is needed to implement net package. Whichever way we interface to it, but we must use WSASendto Windows API.

Anyway I fixed net.TestPacketConn in

https://go-review.googlesource.com/c/go/+/220544/

Unfortunately I discovered new broken test in net (I have no idea how I missed it before). This is the output after CL 220544 is applied

c:\Users\Alex\dev\go\src>go test -a -short -gcflags=all=-d=checkptr net
fatal error: checkptr: unsafe pointer conversion

goroutine 125 [running]:
runtime.throw(0x6a5112, 0x23)
        c:/users/alex/dev/go/src/runtime/panic.go:1112 +0x79 fp=0xc00017dc40 sp=0xc00017dc10 pc=0x4379c9
runtime.checkptrAlignment(0xc00020ced0, 0x668d00, 0x1)
        c:/users/alex/dev/go/src/runtime/checkptr.go:18 +0xbe fp=0xc00017dc70 sp=0xc00017dc40 pc=0x40689e
internal/poll.(*FD).WriteMsg(0xc000237400, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0x6eeec0, 0xc00020f020, 0x0, ...)
        c:/users/alex/dev/go/src/internal/poll/fd_windows.go:1130 +0x1f3 fp=0xc00017dce0 sp=0xc00017dc70 pc=0x4d3fa3
net.(*netFD).writeMsg(0xc000237400, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0x6eeec0, 0xc00020f020, 0xc00020f000, ...)
        c:/users/alex/dev/go/src/net/fd_windows.go:234 +0xb1 fp=0xc00017dd78 sp=0xc00017dce0 pc=0x5644d1
net.(*UDPConn).writeMsg(0xc0002100d8, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0xc000235200, 0x80, 0x80, ...)
        c:/users/alex/dev/go/src/net/udpsock_posix.go:94 +0x121 fp=0xc00017ddf0 sp=0xc00017dd78 pc=0x586621
net.(*UDPConn).WriteMsgUDP(0xc0002100d8, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0xc000235200, 0xf, 0x85eea9, ...)
        c:/users/alex/dev/go/src/net/udpsock.go:186 +0xac fp=0xc00017de90 sp=0xc00017ddf0 pc=0x58477c
net.TestUDPConnSpecificMethods(0xc000228a20)
        c:/users/alex/dev/go/src/net/protoconn_test.go:143 +0x4d5 fp=0xc00017df80 sp=0xc00017de90 pc=0x5d98b5
testing.tRunner(0xc000228a20, 0x6abdd0)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3 fp=0xc00017dfd0 sp=0xc00017df80 pc=0x4ff473
runtime.goexit()
        c:/users/alex/dev/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc00017dfd8 sp=0xc00017dfd0 pc=0x467b11
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 1 [chan receive]:
testing.(*T).Run(0xc000228a20, 0x6a155a, 0x1a, 0x6abdd0, 0x4c8701)
        c:/users/alex/dev/go/src/testing/testing.go:1044 +0x385
testing.runTests.func1(0xc00012c000)
        c:/users/alex/dev/go/src/testing/testing.go:1300 +0x7f
testing.tRunner(0xc00012c000, 0xc000071dd0)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
testing.runTests(0xc000004680, 0x887980, 0xbc, 0xbc, 0xbf8d542e05eb6248, 0x8bb3711725, 0x88b2a0, 0x10)
        c:/users/alex/dev/go/src/testing/testing.go:1298 +0x2df
testing.(*M).Run(0xc000112000, 0x0)
        c:/users/alex/dev/go/src/testing/testing.go:1210 +0x1ae
net.TestMain(0xc000112000)
        c:/users/alex/dev/go/src/net/main_test.go:52 +0x40
main.main()
        _testmain.go:478 +0x13c

goroutine 199 [chan receive]:
testing.(*T).Parallel(0xc0001a8fc0)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGoogleSRV(0xc0001a8fc0)
        c:/users/alex/dev/go/src/net/lookup_test.go:74 +0x4a
testing.tRunner(0xc0001a8fc0, 0x6aba50)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 200 [chan receive]:
testing.(*T).Parallel(0xc0001a90e0)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGmailMX(0xc0001a90e0)
        c:/users/alex/dev/go/src/net/lookup_test.go:123 +0x47
testing.tRunner(0xc0001a90e0, 0x6aba20)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 274 [chan receive]:
testing.(*T).Parallel(0xc00040c240)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestCloseUnblocksRead(0xc00040c240)
        c:/users/alex/dev/go/src/net/net_test.go:505 +0x32
testing.tRunner(0xc00040c240, 0x6ab628)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 202 [chan receive]:
testing.(*T).Parallel(0xc0001a9320)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGmailTXT(0xc0001a9320)
        c:/users/alex/dev/go/src/net/lookup_test.go:218 +0x4a
testing.tRunner(0xc0001a9320, 0x6aba30)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 201 [chan receive]:
testing.(*T).Parallel(0xc0001a9200)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGmailNS(0xc0001a9200)
        c:/users/alex/dev/go/src/net/lookup_test.go:169 +0x47
testing.tRunner(0xc0001a9200, 0x6aba28)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 275 [chan receive]:
testing.(*T).Parallel(0xc00040c360)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestNotTemporaryRead(0xc00040c360)
        c:/users/alex/dev/go/src/net/net_test.go:530 +0x32
testing.tRunner(0xc00040c360, 0x6abb48)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e
FAIL    net     4.524s
FAIL

c:\Users\Alex\dev\go\src>

I will try and fix that too.

Alex

gopherbot · 2020-03-07T01:46:15Z

Change https://golang.org/cl/222457 mentions this issue: internal/syscall/windows: change WSAMsg.Name type

gopherbot · 2020-03-11T13:33:07Z

Change https://golang.org/cl/222855 mentions this issue: sha3: mark xorInUnaligned with go:nocheckptr

It is unclear whether unaligned reads should be allowed, or if they are even actually a good idea here. However, while we figure that out, we should un-break 'go test -race' for users of this package. Updates golang/go#37644 Updates golang/go#37298 Updates golang/go#37715 Updates golang/go#34972 Updates golang/go#35128 Change-Id: I088f5703023e4f05ee274a6753e925973f12ac1b Reviewed-on: https://go-review.googlesource.com/c/crypto/+/222855 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>

WSASendto converts unsafe.Pointer to *syscall.RawSockaddrAny. But that violates every rule of https://golang.org/pkg/unsafe/#Pointer Implement WSASendto by calling Windows WSASendTo API by calling syscall.Syscall9 directly. This allows us to comply with (4) Conversion of a Pointer to a uintptr when calling syscall.Syscall rule. After this change, this commands succeeds: go test -a -short -gcflags=all=-d=checkptr -run=TestPacketConn net Updates #34972 Change-Id: Ib9a810bedf9e05251b7d3c7f69e15bfbd177ac62 Reviewed-on: https://go-review.googlesource.com/c/go/+/220544 Run-TryBot: Alex Brainman <alex.brainman@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>

The problem was discovered while running go test -a -short -gcflags=all=-d=checkptr -run=TestUDPConnSpecificMethods net WSAMsg is type defined by Windows. And WSAMsg.Name could point to two different structures for IPv4 and IPV6 sockets. Currently WSAMsg.Name is declared as *syscall.RawSockaddrAny. But that violates (1) Conversion of a *T1 to Pointer to *T2. rule of https://golang.org/pkg/unsafe/#Pointer When we convert *syscall.RawSockaddrInet4 into *syscall.RawSockaddrAny, syscall.RawSockaddrInet4 and syscall.RawSockaddrAny do not share an equivalent memory layout. Same for *syscall.SockaddrInet6 into *syscall.RawSockaddrAny. This CL changes WSAMsg.Name type to *syscall.Pointer. syscall.Pointer length is 0, and that at least makes type checker happy. After this change I was able to run go test -a -short -gcflags=all=-d=checkptr std cmd without type checker complaining. Updates #34972 Change-Id: Ic5c2321c20abd805c687ee16ef6f643a2f8cd93f Reviewed-on: https://go-review.googlesource.com/c/go/+/222457 Run-TryBot: Alex Brainman <alex.brainman@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>

alexbrainman · 2020-04-02T09:35:18Z

I think I finished all -d=checkptr related changes. Running (on top of 801cd7c )

c:\Users\Alex\dev\go\src>go test -a -short -gcflags=all=-d=checkptr std cmd
ok      archive/tar     0.648s
ok      archive/zip     0.170s
ok      bufio   0.089s
ok      bytes   0.325s
ok      compress/bzip2  0.287s
ok      compress/flate  1.147s
ok      compress/gzip   0.250s
ok      compress/lzw    0.371s
ok      compress/zlib   0.106s
ok      container/heap  0.118s
ok      container/list  0.244s
ok      container/ring  0.082s
ok      context 0.151s
ok      crypto  0.526s
ok      crypto/aes      0.220s
ok      crypto/cipher   0.554s
ok      crypto/des      0.259s
ok      crypto/dsa      0.108s
ok      crypto/ecdsa    0.235s
ok      crypto/ed25519  0.359s
?       crypto/ed25519/internal/edwards25519    [no test files]
ok      crypto/elliptic 0.208s
ok      crypto/hmac     0.833s
?       crypto/internal/randutil        [no test files]
ok      crypto/internal/subtle  0.724s
ok      crypto/md5      0.830s
ok      crypto/rand     0.146s
ok      crypto/rc4      0.127s
ok      crypto/rsa      0.257s
ok      crypto/sha1     0.116s
ok      crypto/sha256   0.132s
ok      crypto/sha512   0.079s
ok      crypto/subtle   0.074s
ok      crypto/tls      4.314s
ok      crypto/x509     1.605s
?       crypto/x509/pkix        [no test files]
ok      database/sql    0.762s
ok      database/sql/driver     0.053s
ok      debug/dwarf     1.120s
ok      debug/elf       1.395s
ok      debug/gosym     0.157s
ok      debug/macho     0.434s
ok      debug/pe        12.299s
ok      debug/plan9obj  0.182s
?       encoding        [no test files]
ok      encoding/ascii85        0.093s
ok      encoding/asn1   0.063s
ok      encoding/base32 0.120s
ok      encoding/base64 0.098s
ok      encoding/binary 0.056s
ok      encoding/csv    0.084s
ok      encoding/gob    0.505s
ok      encoding/hex    0.495s
ok      encoding/json   0.447s
ok      encoding/pem    0.140s
ok      encoding/xml    0.096s
ok      errors  0.110s
ok      expvar  0.206s
ok      flag    0.159s
ok      fmt     0.157s
ok      go/ast  0.059s
ok      go/build        19.119s
ok      go/constant     0.113s
ok      go/doc  1.058s
ok      go/format       0.075s
ok      go/importer     0.401s
ok      go/internal/gccgoimporter       2.147s
ok      go/internal/gcimporter  2.170s
ok      go/internal/srcimporter 3.413s
ok      go/parser       0.403s
ok      go/printer      0.569s
ok      go/scanner      0.094s
ok      go/token        0.158s
ok      go/types        4.196s
ok      hash    0.051s
ok      hash/adler32    0.102s
ok      hash/crc32      0.217s
ok      hash/crc64      0.159s
ok      hash/fnv        0.160s
ok      hash/maphash    0.178s
ok      html    0.048s
ok      html/template   0.139s
ok      image   0.312s
ok      image/color     0.139s
?       image/color/palette     [no test files]
ok      image/draw      0.250s
ok      image/gif       0.747s
?       image/internal/imageutil        [no test files]
ok      image/jpeg      0.573s
ok      image/png       0.268s
ok      index/suffixarray       0.274s
?       internal/bytealg        [no test files]
?       internal/cfg    [no test files]
ok      internal/cpu    0.111s
ok      internal/fmtsort        0.096s
?       internal/goroot [no test files]
?       internal/goversion      [no test files]
?       internal/lazyregexp     [no test files]
?       internal/lazytemplate   [no test files]
?       internal/nettrace       [no test files]
?       internal/obscuretestdata        [no test files]
?       internal/oserror        [no test files]
ok      internal/poll   0.165s
?       internal/race   [no test files]
ok      internal/reflectlite    0.130s
ok      internal/singleflight   0.107s
?       internal/syscall/execenv        [no test files]
ok      internal/syscall/windows        0.085s
ok      internal/syscall/windows/registry       0.207s
?       internal/syscall/windows/sysdll [no test files]
?       internal/testenv        [no test files]
?       internal/testlog        [no test files]
ok      internal/trace  0.737s
ok      internal/xcoff  1.169s
ok      io      0.239s
ok      io/ioutil       0.302s
ok      log     0.078s
?       log/syslog      [no test files]
ok      math    0.138s
ok      math/big        1.337s
ok      math/bits       0.116s
ok      math/cmplx      0.072s
ok      math/rand       0.416s
ok      mime    0.526s
ok      mime/multipart  0.931s
ok      mime/quotedprintable    0.067s
ok      net     32.163s
ok      net/http        14.201s
ok      net/http/cgi    0.608s
ok      net/http/cookiejar      0.125s
ok      net/http/fcgi   0.891s
ok      net/http/httptest       2.162s
ok      net/http/httptrace      0.289s
ok      net/http/httputil       0.451s
ok      net/http/internal       0.299s
ok      net/http/pprof  2.254s
ok      net/internal/socktest   0.061s
ok      net/mail        0.079s
ok      net/rpc 0.357s
ok      net/rpc/jsonrpc 0.649s
ok      net/smtp        0.323s
ok      net/textproto   0.116s
ok      net/url 0.113s
ok      os      7.549s
ok      os/exec 7.068s
ok      os/signal       2.068s
ok      os/user 0.121s
ok      path    0.071s
ok      path/filepath   1.260s
ok      plugin  0.114s
ok      reflect 1.269s
ok      regexp  0.754s
ok      regexp/syntax   0.519s
ok      runtime 99.905s
?       runtime/cgo     [no test files]
ok      runtime/debug   0.096s
ok      runtime/internal/atomic 0.116s
ok      runtime/internal/math   0.060s
ok      runtime/internal/sys    0.190s
ok      runtime/pprof   13.023s
ok      runtime/pprof/internal/profile  0.085s
?       runtime/race    [no test files]
ok      runtime/trace   0.765s
ok      sort    0.101s
ok      strconv 0.690s
ok      strings 0.268s
ok      sync    0.583s
ok      sync/atomic     0.117s
ok      syscall 0.072s
ok      testing 0.737s
?       testing/internal/testdeps       [no test files]
ok      testing/iotest  0.135s
ok      testing/quick   0.270s
ok      text/scanner    0.106s
ok      text/tabwriter  0.071s
ok      text/template   0.125s
ok      text/template/parse     0.055s
ok      time    2.523s
ok      unicode 0.173s
ok      unicode/utf16   0.193s
ok      unicode/utf8    0.062s
?       unsafe  [no test files]
?       vendor/golang.org/x/crypto/chacha20     [no test files]
?       vendor/golang.org/x/crypto/chacha20poly1305     [no test files]
?       vendor/golang.org/x/crypto/cryptobyte   [no test files]
?       vendor/golang.org/x/crypto/cryptobyte/asn1      [no test files]
?       vendor/golang.org/x/crypto/curve25519   [no test files]
?       vendor/golang.org/x/crypto/hkdf [no test files]
?       vendor/golang.org/x/crypto/internal/subtle      [no test files]
?       vendor/golang.org/x/crypto/poly1305     [no test files]
?       vendor/golang.org/x/net/dns/dnsmessage  [no test files]
?       vendor/golang.org/x/net/http/httpguts   [no test files]
?       vendor/golang.org/x/net/http/httpproxy  [no test files]
?       vendor/golang.org/x/net/http2/hpack     [no test files]
?       vendor/golang.org/x/net/idna    [no test files]
?       vendor/golang.org/x/net/nettest [no test files]
?       vendor/golang.org/x/sys/cpu     [no test files]
?       vendor/golang.org/x/text/secure/bidirule        [no test files]
?       vendor/golang.org/x/text/transform      [no test files]
?       vendor/golang.org/x/text/unicode/bidi   [no test files]
?       vendor/golang.org/x/text/unicode/norm   [no test files]
ok      cmd/addr2line   20.369s
ok      cmd/api 24.461s
?       cmd/asm [no test files]
?       cmd/asm/internal/arch   [no test files]
ok      cmd/asm/internal/asm    4.127s
?       cmd/asm/internal/flags  [no test files]
ok      cmd/asm/internal/lex    0.800s
?       cmd/buildid     [no test files]
?       cmd/cgo [no test files]
ok      cmd/compile     0.624s
?       cmd/compile/internal/amd64      [no test files]
?       cmd/compile/internal/arm        [no test files]
?       cmd/compile/internal/arm64      [no test files]
ok      cmd/compile/internal/gc 18.001s
ok      cmd/compile/internal/logopt     3.718s
?       cmd/compile/internal/mips       [no test files]
?       cmd/compile/internal/mips64     [no test files]
?       cmd/compile/internal/ppc64      [no test files]
?       cmd/compile/internal/riscv64    [no test files]
?       cmd/compile/internal/s390x      [no test files]
ok      cmd/compile/internal/ssa        0.668s
ok      cmd/compile/internal/syntax     0.188s
ok      cmd/compile/internal/test       0.140s [no tests to run]
ok      cmd/compile/internal/types      0.047s
?       cmd/compile/internal/wasm       [no test files]
?       cmd/compile/internal/x86        [no test files]
ok      cmd/cover       11.431s
?       cmd/dist        [no test files]
ok      cmd/doc 0.266s
ok      cmd/fix 6.998s
go test proxy running at GOPROXY=http://127.0.0.1:62204/mod
go proxy: no archive rsc.io v1.5.2: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.1.0: file does not exist
go proxy: no archive rsc.io v1.5.1: file does not exist
go proxy: no archive example.com/newcycle v1.0.0: file does not exist
go proxy: no archive rsc.io v1.5.2: file does not exist
--- FAIL: TestScript (0.01s)
    --- FAIL: TestScript/test_race_install_cgo (1.32s)
        script_test.go:205:
            # Tests Issue #10500 (1.244s)
            > [!race] skip
            > [!darwin] ! stale cmd/cgo  # The darwin builders are spuriously stale; see #33598.
            FAIL: testdata\script\test_race_install_cgo.txt:5: cmd/cgo is unexpectedly stale


FAIL
FAIL    cmd/go  201.196s
ok      cmd/go/internal/auth    0.349s
?       cmd/go/internal/base    [no test files]
?       cmd/go/internal/bug     [no test files]
ok      cmd/go/internal/cache   5.082s
?       cmd/go/internal/cfg     [no test files]
?       cmd/go/internal/clean   [no test files]
?       cmd/go/internal/cmdflag [no test files]
?       cmd/go/internal/doc     [no test files]
?       cmd/go/internal/envcmd  [no test files]
?       cmd/go/internal/fix     [no test files]
?       cmd/go/internal/fmtcmd  [no test files]
ok      cmd/go/internal/generate        0.479s
ok      cmd/go/internal/get     0.385s
?       cmd/go/internal/help    [no test files]
ok      cmd/go/internal/imports 0.065s
?       cmd/go/internal/list    [no test files]
ok      cmd/go/internal/load    0.224s
ok      cmd/go/internal/lockedfile      0.381s
ok      cmd/go/internal/lockedfile/internal/filelock    0.321s
?       cmd/go/internal/modcmd  [no test files]
ok      cmd/go/internal/modconv 0.412s
ok      cmd/go/internal/modfetch        1.165s
ok      cmd/go/internal/modfetch/codehost       0.241s
ok      cmd/go/internal/modfetch/zip_sum_test   0.772s
?       cmd/go/internal/modget  [no test files]
?       cmd/go/internal/modinfo [no test files]
ok      cmd/go/internal/modload 0.237s
ok      cmd/go/internal/mvs     0.109s
ok      cmd/go/internal/par     0.078s
ok      cmd/go/internal/renameio        15.631s
?       cmd/go/internal/robustio        [no test files]
?       cmd/go/internal/run     [no test files]
ok      cmd/go/internal/search  0.304s
?       cmd/go/internal/str     [no test files]
ok      cmd/go/internal/test    1.983s
?       cmd/go/internal/tool    [no test files]
ok      cmd/go/internal/txtar   0.180s
?       cmd/go/internal/version [no test files]
?       cmd/go/internal/vet     [no test files]
ok      cmd/go/internal/web     0.654s
ok      cmd/go/internal/work    0.682s
ok      cmd/gofmt       1.146s
?       cmd/internal/bio        [no test files]
?       cmd/internal/browser    [no test files]
ok      cmd/internal/buildid    1.306s
?       cmd/internal/diff       [no test files]
ok      cmd/internal/dwarf      0.085s
ok      cmd/internal/edit       0.134s
?       cmd/internal/gcprog     [no test files]
ok      cmd/internal/goobj      2.878s
?       cmd/internal/goobj2     [no test files]
ok      cmd/internal/moddeps    8.717s
ok      cmd/internal/obj        0.080s
?       cmd/internal/obj/arm    [no test files]
ok      cmd/internal/obj/arm64  0.592s
?       cmd/internal/obj/mips   [no test files]
ok      cmd/internal/obj/ppc64  0.404s
ok      cmd/internal/obj/riscv  0.258s
?       cmd/internal/obj/s390x  [no test files]
?       cmd/internal/obj/wasm   [no test files]
ok      cmd/internal/obj/x86    6.724s
ok      cmd/internal/objabi     0.079s
?       cmd/internal/objfile    [no test files]
ok      cmd/internal/src        0.081s
?       cmd/internal/sys        [no test files]
ok      cmd/internal/test2json  0.789s
--- FAIL: TestDWARF (0.40s)
    dwarf_test.go:39: cmd/link is stale - run go install cmd/link
FAIL
FAIL    cmd/link        19.864s
?       cmd/link/internal/amd64 [no test files]
?       cmd/link/internal/arm   [no test files]
?       cmd/link/internal/arm64 [no test files]
ok      cmd/link/internal/ld    20.019s
?       cmd/link/internal/loadelf       [no test files]
?       cmd/link/internal/loader        [no test files]
?       cmd/link/internal/loadmacho     [no test files]
?       cmd/link/internal/loadpe        [no test files]
?       cmd/link/internal/loadxcoff     [no test files]
?       cmd/link/internal/mips  [no test files]
?       cmd/link/internal/mips64        [no test files]
?       cmd/link/internal/objfile       [no test files]
?       cmd/link/internal/ppc64 [no test files]
?       cmd/link/internal/riscv64       [no test files]
?       cmd/link/internal/s390x [no test files]
ok      cmd/link/internal/sym   0.407s
?       cmd/link/internal/wasm  [no test files]
?       cmd/link/internal/x86   [no test files]
ok      cmd/nm  15.778s
ok      cmd/objdump     5.883s
ok      cmd/pack        7.192s
?       cmd/pprof       [no test files]
?       cmd/test2json   [no test files]
ok      cmd/trace       1.206s
?       cmd/vendor/github.com/google/pprof/driver       [no test files]
?       cmd/vendor/github.com/google/pprof/internal/binutils    [no test files]
?       cmd/vendor/github.com/google/pprof/internal/driver      [no test files]
?       cmd/vendor/github.com/google/pprof/internal/elfexec     [no test files]
?       cmd/vendor/github.com/google/pprof/internal/graph       [no test files]
?       cmd/vendor/github.com/google/pprof/internal/measurement [no test files]
?       cmd/vendor/github.com/google/pprof/internal/plugin      [no test files]
?       cmd/vendor/github.com/google/pprof/internal/report      [no test files]
?       cmd/vendor/github.com/google/pprof/internal/symbolizer  [no test files]
?       cmd/vendor/github.com/google/pprof/internal/symbolz     [no test files]
?       cmd/vendor/github.com/google/pprof/internal/transport   [no test files]
?       cmd/vendor/github.com/google/pprof/profile      [no test files]
?       cmd/vendor/github.com/google/pprof/third_party/d3       [no test files]
?       cmd/vendor/github.com/google/pprof/third_party/d3flamegraph     [no test files]
?       cmd/vendor/github.com/google/pprof/third_party/svgpan   [no test files]
?       cmd/vendor/github.com/ianlancetaylor/demangle   [no test files]
?       cmd/vendor/golang.org/x/arch/arm/armasm [no test files]
?       cmd/vendor/golang.org/x/arch/arm64/arm64asm     [no test files]
?       cmd/vendor/golang.org/x/arch/ppc64/ppc64asm     [no test files]
?       cmd/vendor/golang.org/x/arch/x86/x86asm [no test files]
?       cmd/vendor/golang.org/x/crypto/ed25519  [no test files]
?       cmd/vendor/golang.org/x/crypto/ed25519/internal/edwards25519    [no test files]
?       cmd/vendor/golang.org/x/crypto/ssh/terminal     [no test files]
?       cmd/vendor/golang.org/x/mod/internal/lazyregexp [no test files]
?       cmd/vendor/golang.org/x/mod/modfile     [no test files]
?       cmd/vendor/golang.org/x/mod/module      [no test files]
?       cmd/vendor/golang.org/x/mod/semver      [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb       [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb/dirhash       [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb/note  [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb/tlog  [no test files]
?       cmd/vendor/golang.org/x/mod/zip [no test files]
?       cmd/vendor/golang.org/x/sys/unix        [no test files]
?       cmd/vendor/golang.org/x/sys/windows     [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/internal/analysisflags   [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/internal/facts        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/asmdecl        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/assign [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/atomic [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/bools  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/buildtag       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/cgocall        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/composite      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/copylock       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/ctrlflow       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/errorsas       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/httpresponse   [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/ifaceassert    [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/inspect        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/loopclosure    [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel     [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/nilfunc        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/printf [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/shift  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/stdmethods     [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/stringintconv  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/structtag      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/tests  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unmarshal      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unreachable    [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unsafeptr      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unusedresult   [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/unitchecker   [no test files]
?       cmd/vendor/golang.org/x/tools/go/ast/astutil    [no test files]
?       cmd/vendor/golang.org/x/tools/go/ast/inspector  [no test files]
?       cmd/vendor/golang.org/x/tools/go/cfg    [no test files]
?       cmd/vendor/golang.org/x/tools/go/types/objectpath       [no test files]
?       cmd/vendor/golang.org/x/tools/go/types/typeutil [no test files]
?       cmd/vendor/golang.org/x/xerrors [no test files]
?       cmd/vendor/golang.org/x/xerrors/internal        [no test files]
ok      cmd/vet 29.924s
FAIL

c:\Users\Alex\dev\go\src>

There are still 2 failures. But I don't think they are related to -d=checkptr.

I am not sure what to do next. Should I send windows version of

https://go-review.googlesource.com/c/go/+/201783/

?

Thank you.

Alex

aclements · 2020-04-02T13:32:52Z

I think I finished all -d=checkptr related changes.

Yay!

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do they also happen without checkptr? I agree they certainly don't look related to checkptr, but the dashboard is clean for Windows right now.

I am not sure what to do next. Should I send windows version of https://go-review.googlesource.com/c/go/+/201783/ ?

That would be awesome.

bcmills · 2020-04-02T13:34:46Z

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do those resolve if you run go install cmd without -d=checkptr before running go test?

gopherbot · 2020-04-04T08:16:05Z

Change https://golang.org/cl/227003 mentions this issue: cmd/compile: enable -d=checkptr even on windows

alexbrainman · 2020-04-04T08:19:47Z

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do they also happen without checkptr?

Do they also happen without checkptr?

Yes. Running

go test -a -short std cmd

fails in exactly the same way as

go test -a -short -gcflags=all=-d=checkptr std cmd

command.

... Should I send windows version of https://go-review.googlesource.com/c/go/+/201783/ ?

That would be awesome.

I sent https://golang.org/cl/227003 But it fails here when I run all.bat with this error:

--- FAIL: TestLogOpt (0.63s)
    --- FAIL: TestLogOpt/Copy (0.27s)
        --- FAIL: TestLogOpt/Copy/arm (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build210684795: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/arm64 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build080355363: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/386 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build929204411: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/amd64 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build049316543: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/mips (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build251041127: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/mips64 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build593677423: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/ppc64le (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build772328963: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/s390x (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build555385451: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/wasm (0.08s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build914568571: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
FAIL
FAIL    cmd/compile/internal/logopt     0.741s

Mind you, I created CL on current tip. And current tip has new linker code. Maybe failure is related. I did not investigate it properly.

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do those resolve if you run go install cmd without -d=checkptr before running go test?

Running

go install cmd

before

go test -a -short -gcflags=all=-d=checkptr std cmd

makes things worse (I am showing just errors)

--- FAIL: TestScript (0.01s)
    --- FAIL: TestScript/test_relative_import_dash_i (0.48s)
        script_test.go:205:
            # Relative imports in go test -i
            # Run tests outside GOPATH. (0.000s)
            # Check that it's safe to pass -i (which installs dependencies in $GOPATH/pkg) to go test. (0.463s)
            > ! stale runtime # don't let test -i overwrite runtime
            FAIL: testdata\script\test_relative_import_dash_i.txt:7: runtime is unexpectedly stale


    --- FAIL: TestScript/test_race_install_cgo (2.53s)
        script_test.go:205:
            # Tests Issue #10500 (2.500s)
            > [!race] skip
            > [!darwin] ! stale cmd/cgo  # The darwin builders are spuriously stale; see #33598.
            FAIL: testdata\script\test_race_install_cgo.txt:5: cmd/cgo is unexpectedly stale


    --- FAIL: TestScript/build_package_not_stale_trailing_slash (0.16s)
        script_test.go:205:
            # Tests Issue #12690 (0.160s)
            > [gccgo] skip 'gccgo does not have GOROOT'
            > ! stale runtime
            FAIL: testdata\script\build_package_not_stale_trailing_slash.txt:5: runtime is unexpectedly stale


FAIL
FAIL    cmd/go  195.196s

and

--- FAIL: TestDWARF (1.32s)
    dwarf_test.go:39: cmd/link is stale - run go install cmd/link
FAIL
FAIL    cmd/link        16.210s

All tested on top of 801cd7c.

Alex

CL 201783 enable -d=checkptr when -race or -msan is specified everywhere but windows. But, now that all unsafe pointer conversions in the standard library are fixed, enable -d=checkptr even on windows. Updates #34964 Updates #34972 Change-Id: Id912fa83b0d5b46c6f1c134c742fd94d2d185835 Reviewed-on: https://go-review.googlesource.com/c/go/+/227003 Run-TryBot: Alex Brainman <alex.brainman@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

alexbrainman · 2020-04-05T06:37:11Z

I sent https://golang.org/cl/227003 But it fails here when I run all.bat with this error:
--- FAIL: TestLogOpt (0.63s)
    --- FAIL: TestLogOpt/Copy (0.27s)
        --- FAIL: TestLogOpt/Copy/arm (0.02s)

TestLogOpt failure is unrelated to CL 227003. I filled #38251 for that.

Alex

alexbrainman · 2020-04-05T06:38:58Z

And I submitted CL 227003. So I don't see what else to do for this current issue. Leaving it for others to close.

Alex

mdempsky · 2020-04-14T18:19:27Z

Thanks @alexbrainman!

gopherbot · 2020-04-19T07:43:19Z

Change https://golang.org/cl/228858 mentions this issue: syscall, internal/syscall/windows: remove utf16PtrToString parameter

gopherbot · 2020-04-22T21:36:21Z

Change https://golang.org/cl/225418 mentions this issue: windows: fix -d=checkptr slice failures

CL 208617 introduced syscall.utf16PtrToString and internal/syscall/windows.UTF16PtrToString functions. Original version of CL 208617 did not include syscall.utf16PtrToString and internal/syscall/windows.UTF16PtrToString max parameter. The parameter was added by Brad at the request of Ian. Ian said: "In some cases it seems at least possible that the null terminator is not present. I think it would be safer if we passed a maximum length here." The syscall.utf16PtrToString and internal/syscall/windows.UTF16PtrToString function are designed to work with only null terminated strings. So max parameter is superfluous. This change removes max parameter. Updates #34972 Change-Id: Ifea65dbd86bca8a08353579c6b9636c6f963d165 Reviewed-on: https://go-review.googlesource.com/c/go/+/228858 Run-TryBot: Alex Brainman <alex.brainman@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>

This CL fixes unsafe casts to slices that are missing length or capacity. Running tests with -d=checkptr enabled may panic on casting unsafe.Pointer to a static array of large predefined length, that is most likely much bigger than the size of the actual array in memory. Checkptr check is not satisfied if slicing operator misses length and capacity arguments `(*[(1 << 30) - 1]uint16)(unsafe.Pointer(p))[:]`, or when there is no slicing at all `(*[(1 << 30) - 1]uint16)(unsafe.Pointer(p))`. To find all potential cases I used `grep -nr ")(unsafe.Pointer(" ./windows`, then filtered out safe casts when object size is always static and known at compile time. To reproduce the issue run tests with checkptr enabled `go test -a -gcflags=all=-d=checkptr ./windows/...`. Updates golang/go#34972 Fixes golang/go#38355 Change-Id: I9dd2084b4f9fb7618cdb140fb2f38b56b6d6cc04 GitHub-Last-Rev: 73288ad GitHub-Pull-Request: #65 Reviewed-on: https://go-review.googlesource.com/c/sys/+/225418 Run-TryBot: Alex Brainman <alex.brainman@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>

CL 208617 introduced syscall.utf16PtrToString and internal/syscall/windows.UTF16PtrToString functions. Original version of CL 208617 did not include syscall.utf16PtrToString and internal/syscall/windows.UTF16PtrToString max parameter. The parameter was added by Brad at the request of Ian. Ian said: "In some cases it seems at least possible that the null terminator is not present. I think it would be safer if we passed a maximum length here." The syscall.utf16PtrToString and internal/syscall/windows.UTF16PtrToString function are designed to work with only null terminated strings. So max parameter is superfluous. This change removes max parameter. Updates golang#34972 Change-Id: Ifea65dbd86bca8a08353579c6b9636c6f963d165 Reviewed-on: https://go-review.googlesource.com/c/go/+/228858 Run-TryBot: Alex Brainman <alex.brainman@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>

mdempsky added help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 17, 2019

mdempsky added this to the Go1.14 milestone Oct 17, 2019

mdempsky mentioned this issue Oct 19, 2019

reflect: ptrAlign panic when accessing Method #35002

Closed

bcmills mentioned this issue Mar 26, 2020

proposal: Go 2: allow 3-index slices to elide the second (length) term #38081

Closed

bcmills mentioned this issue Apr 10, 2020

x/sys/windows/svc/mgr: "checkptr: converted pointer straddles multiple allocations" #38355

Closed

mdempsky closed this as completed Apr 14, 2020

rikysya mentioned this issue Apr 22, 2020

windows: fix -d=checkptr slice failures golang/sys#65

Closed

golang locked and limited conversation to collaborators Apr 22, 2021

gopherbot added the FrozenDueToAge label Apr 22, 2021

all: get standard library building with -d=checkptr #34972

all: get standard library building with -d=checkptr #34972

Comments

mdempsky commented Oct 17, 2019

mdempsky commented Oct 17, 2019

mdempsky commented Oct 17, 2019

odeke-em commented Oct 17, 2019

mdempsky commented Oct 18, 2019

gopherbot commented Oct 18, 2019

gopherbot commented Oct 18, 2019

mdempsky commented Oct 18, 2019

odeke-em commented Oct 18, 2019

gopherbot commented Oct 18, 2019

cuonglm commented Oct 18, 2019

mdempsky commented Oct 18, 2019

bcmills commented Oct 18, 2019

bcmills commented Oct 18, 2019

mdempsky commented Oct 18, 2019

cuonglm commented Oct 19, 2019 • edited Loading

mdempsky commented Oct 19, 2019 • edited Loading

gopherbot commented Oct 19, 2019

cuonglm commented Oct 19, 2019

mdempsky commented Oct 19, 2019 • edited Loading

gopherbot commented Oct 19, 2019

gopherbot commented Oct 19, 2019

gopherbot commented Oct 21, 2019

aclements commented Jan 10, 2020

alexbrainman commented Feb 16, 2020

aclements commented Feb 16, 2020

gopherbot commented Feb 25, 2020

alexbrainman commented Feb 25, 2020

gopherbot commented Mar 7, 2020

gopherbot commented Mar 11, 2020

alexbrainman commented Apr 2, 2020

aclements commented Apr 2, 2020

bcmills commented Apr 2, 2020

gopherbot commented Apr 4, 2020

alexbrainman commented Apr 4, 2020

alexbrainman commented Apr 5, 2020

alexbrainman commented Apr 5, 2020

mdempsky commented Apr 14, 2020

gopherbot commented Apr 19, 2020

gopherbot commented Apr 22, 2020

cuonglm commented Oct 19, 2019 •

edited

Loading

mdempsky commented Oct 19, 2019 •

edited

Loading

mdempsky commented Oct 19, 2019 •

edited

Loading