runtime: cpu profile is not useful due to systemstack

[dvyukov]

When profiling my program with pprof, I see that 80% of time is spent in runtime.memclr called from runtime._System. This is not useful. I have to attach gdb and sample stacks to figure out that the guilty call stack is:
#0 runtime.memclr () at src/runtime/memclr_amd64.s:13
#1 0x0000000000425b2a in runtime.heapBits.initSpan (h=..., size=131072, n=1, total=131072) at src/runtime/mbitmap.go:284
#2 0x0000000000423d27 in runtime.largeAlloc (size=124063, flag=1, ~r2=0x7f6e0004c7d8) at src/runtime/malloc.go:715
#3 0x000000000045faba in runtime.mallocgc.func3 () at src/runtime/malloc.go:631
#4 0x000000000046aa59 in runtime.systemstack () at src/runtime/asm_amd64.s:262
#5 0x0000000000441a10 in runtime.starttheworld () at src/runtime/proc1.go:643

But it is still unclear who is calling mallocgc.
Similar situation is possible for e.g. defers, for which one won't even get a hint from heap profile.

Cpu profiler should unwind through systemstack.

[dvyukov]

/cc @rsc @randall77 @RLH @aclements

[randall77]

I'm thinking that it would be generally good to combine the stack traces on either side of a systemstack call (not just for profiles). Maybe with a ----- mark as we used to have between stack segments.

[minux]

yeah, that will be very helpful.

[gopherbot]

CL https://golang.org/cl/9506 mentions this issue.