runtime/pprof: document block and mutex profiles

Amazingly, we seem to have nearly no in-tree documentation on the
semantics of block and mutex profiles. Add brief summaries, including
the new behavior from CL 506415 and CL 544195.

For #14689.
For #44920.
For #57071.
For #61015.

Change-Id: I1a6edce7c434fcb43f17c83eb362b1f9d1a32df1
Reviewed-on: https://go-review.googlesource.com/c/go/+/547057
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

src/runtime/pprof/pprof.go

@@ -126,6 +126,31 @@ import (
 // pprof display to -alloc_space, the total number of bytes allocated since
 // the program began (including garbage-collected bytes).
 //
+// The block profile tracks time spent blocked on synchronization primitives,
+// such as [sync.Mutex], [sync.RWMutex], [sync.WaitGroup], [sync.Cond], and
+// channel send/receive/select. Stack traces correspond to the location that
+// blocked (for example, [sync.Mutex.Lock]). Sample values correspond to
+// cumulative time spent blocked at that stack trace, subject to time-based
+// sampling specified by [runtime.SetBlockProfileRate].
+//
+// The mutex profile tracks contention on mutexes, such as [sync.Mutex],
+// [sync.RWMutex], and runtime-internal locks. Stack traces correspond to the
+// end of the critical section causing contention. For example, a lock held for
+// a long time while other goroutines are waiting to acquire the lock will
+// report contention when the lock is finally unlocked (that is, at
+// [sync.Mutex.Unlock]). Sample values correspond to the approximate cumulative
+// time other goroutines spent blocked waiting for the lock, subject to
+// event-based sampling specified by [runtime.SetMutexProfileFraction]. For
+// example, if a caller holds a lock for 1s while 5 other goroutines are
+// waiting for the entire second to acquire the lock, its unlock call stack
+// will report 5s of contention.
+//
+// In the mutex profile, runtime-internal locks are always reported at the
+// location "runtime._LostContendedRuntimeLock". More detailed stack traces for
+// runtime-internal locks can be obtained by setting
+// `GODEBUG=runtimecontentionstacks=1` (see package [runtime] docs for
+// caveats).
+//
 // The CPU profile is not available as a Profile. It has a special API,
 // the [StartCPUProfile] and [StopCPUProfile] functions, because it streams
 // output to a writer during profiling.