wgengine/magicsock: add a stress test

[josharian]

And add a peerMap validate method that checks its internal invariants.

Updates tailscale/corp#3016

Change-Id: I23708e68ed44d81986d9e2be82029d4555547592
Co-authored-by: Brad Fitzpatrick bradfitz@tailscale.com

[josharian]

Windows race failure is interesting! (But unrelated to this PR.)

==================
420
WARNING: DATA RACE
421
Read at 0x00c00022e070 by goroutine 114:
422
  runtime.racereadrange()
423
      <autogenerated>:1 +0x1b
424
  tailscale.com/wgengine/magicsock.(*Conn).enqueueCallMeMaybe·dwrap·36()
425
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock.go:2158 +0xb6
426

427
Previous write at 0x00c00022e070 by goroutine 74:
428
  tailscale.com/wgengine/magicsock.(*endpoint).updateFromNode()
429
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock.go:3713 +0x358
430
  tailscale.com/wgengine/magicsock.(*Conn).SetNetworkMap()
431
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock.go:2358 +0x810
432
  tailscale.com/wgengine/magicsock.meshStacks.func2()
433
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock_test.go:287 +0x25d
434
  tailscale.com/wgengine/magicsock.meshStacks.func3()
435
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock_test.go:321 +0x14a
436
  tailscale.com/wgengine/magicsock.meshStacks·dwrap·77()
437
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock_test.go:324 +0x47
438

439
Goroutine 114 (running) created at:
440
  tailscale.com/wgengine/magicsock.(*Conn).enqueueCallMeMaybe()
441
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock.go:2158 +0x724
442
  tailscale.com/wgengine/magicsock.(*endpoint).sendPingsLocked·dwrap·67()
443
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock.go:3696 +0x8d
444

445
Goroutine 74 (running) created at:
446
  tailscale.com/wgengine/magicsock.meshStacks()
447
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock_test.go:312 +0x453
448
  tailscale.com/wgengine/magicsock.TestDiscokeyChange()
449
      D:/a/tailscale/tailscale/wgengine/magicsock/magicsock_test.go:688 +0x63d
450
  testing.tRunner()
451
      C:/hostedtoolcache/windows/go/1.17.3/x64/src/testing/testing.go:1259 +0x22f
452
  testing.(*T).Run·dwrap·21()
453
      C:/hostedtoolcache/windows/go/1.17.3/x64/src/testing/testing.go:1306 +0x47
454
==================
455
--- FAIL: TestDiscokeyChange (10.32s)

Also interesting: Attempting to reproduce exposes a Go toolchain bug, golang/go#49655.