runtime: reported CPU time of STW phases is low

[aclements]

The gctrace in https://blog.cloudflare.com/go-dont-collect-my-garbage/ shows CPU times of STW phases that are only a small multiple of the wall-clock time, even though GOMAXPROCS is 48. It should be min(ncpu, GOMAXPROCS) * wallTime.

This also causes the overall GC CPU fraction reported in the gctrace to be much too low, since it's based on these CPU times.

/cc @RLH

[gopherbot]

Change https://golang.org/cl/77710 mentions this issue: runtime: fix gctrace STW CPU time and CPU fraction

[bronze1man]

Is it possible for the compiler to find that the memory just need free at some time. and use this infomation to skip some gc steps?

[aclements]

@bronze1man, I'm not sure I understand your question or why it's relevant to this issue. In general, the compiler's escape analysis already lets objects be freed without a GC in specific but very common cases. It can be made better, and we'd like to make it better, but that's a huge topic and an area of active research in the field that I can't really summarize here. :) That reduces load on the GC, but isn't related to eliminating any phases of the GC algorithm (there are only four phases, anyway).

[aclements]

Re-opening for consideration for 1.9.3.

[bronze1man]

@aclements From the html page https://blog.cloudflare.com/go-dont-collect-my-garbage/ , that guy alloc a lot object ,throw the object and return his function , he do not touch that alloc object again. I think the compiler can mark those objects can be freed for sure just like the user call free in c, and the GC algorithm do not need to find which object is not used, It can just free them or just reuse them to another object, this way should reduces load on the GC.

The compiler's escape analysis can not handle some unknow(unknow at compiler time) length objects. My suggestion may reduces load on the GC more.(may be no need to gc any more?) But i do not understand the GC algorithm in detail.

My suggestion looks like the compiler can use a *sync.Pool if it can save alloc.

[aclements]

@bronze1man, we don't know specifically what Vlad was doing in his benchmark, so I can't say how easy or hard it would be for the compiler to optimize the allocation pattern.

We'd certainly like to improve the escape analysis to be able to handle variable-length allocations (and many other things it can't currently handle). It's on our list, but, alas, it's a hard problem and there are only so many hours in a day. :)

[andybons]

CL 77710 OK for Go 1.9.3.