tidb-server cluster_L File: tidb-server File: tidb-server Type: cpu Time: Aug 22, 2018 at 8:01pm (CST) Duration: 50s, Total samples = 369.97s (739.93%) Showing nodes accounting for 333.36s, 90.10% of 369.97s total Dropped 572 nodes (cum <= 1.85s) Dropped 66 edges (freq <= 0.37s) Showing top 80 nodes out of 110 N1 github com/pingcap/tidb/executor (*HashAggPartialWorker) run 0.18s (0.049%) of 157.74s (42.64%) N5 github com/pingcap/tidb/executor (*HashAggPartialWorker) updatePartialResult 9.89s (2.67%) of 114.42s (30.93%) N1->N5 114.42s N14 github com/pingcap/tidb/executor (*HashAggPartialWorker) shuffleIntermData 2.54s (0.69%) of 30.63s (8.28%) N1->N14 30.63s N31 github com/pingcap/tidb/executor (*HashAggPartialWorker) getChildInput 1.25s (0.34%) of 12.51s (3.38%) N1->N31 12.51s N2 runtime systemstack 0.29s (0.078%) of 107.86s (29.15%) N58 runtime gcDrain 0.59s (0.16%) of 64.23s (17.36%) N2->N58 64.23s N23 runtime wbBufFlush1 10.11s (2.73%) of 23.23s (6.28%) N2->N23 23.23s N51 runtime (*mcentral) cacheSpan 0.24s (0.065%) of 5.43s (1.47%) N2->N51 5.43s N3 github com/pingcap/tidb/executor baseHashAggWorker getPartialResult 2.30s (0.62%) of 103.80s (28.06%) N11 runtime mapaccess2_faststr 27.94s (7.55%) of 46.08s (12.46%) N3->N11 46.07s N13 runtime mapassign_faststr 5.01s (1.35%) of 41.93s (11.33%) N3->N13 41.92s N34 runtime makeslice 0.75s (0.2%) of 8.95s (2.42%) N3->N34 4.36s N24 runtime newobject 0.95s (0.26%) of 14.55s (3.93%) N3->N24 3.94s N67 runtime slicebytetostring 0.53s (0.14%) of 3.14s (0.85%) N3->N67 3.13s N10 runtime gcWriteBarrier 3.59s (0.97%) of 24.04s (6.50%) N3->N10 1.44s N4 github com/pingcap/tidb/executor (*HashAggFinalWorker) consumeIntermData 5.37s (1.45%) of 91.60s (24.76%) N15 github com/pingcap/tidb/util/mvmap (*MVMap) Get 8.25s (2.23%) of 29.06s (7.85%) N4->N15 29.06s N4->N3 26.28s N26 github com/pingcap/tidb/executor (*HashAggIntermData) getPartialResultBatch 1.39s (0.38%) of 24.92s (6.74%) N4->N26 24.92s N54 github com/pingcap/tidb/util/mvmap (*MVMap) Put 0.30s (0.081%) of 3.75s (1.01%) N4->N54 3.75s N71 github com/pingcap/tidb/executor/aggfuncs (*avgPartial4Float64) MergePartialResult 2.05s (0.55%) N4->N71 2.05s N5->N3 75.86s N25 github com/pingcap/tidb/executor (*HashAggPartialWorker) getGroupKey 2.59s (0.7%) of 15.69s (4.24%) N5->N25 15.69s N5->N24 6.49s N52 github com/pingcap/tidb/executor/aggfuncs (*avgPartial4Float64) UpdatePartialResult 1.96s (0.53%) of 5.47s (1.48%) N5->N52 5.47s N6 runtime scanobject 19.62s (5.30%) of 73.41s (19.84%) N17 runtime greyobject 28.35s (7.66%) of 28.41s (7.68%) N6->N17 28.40s N9 runtime heapBitsForObject 38.30s (10.35%) N6->N9 25.39s N7 runtime mallocgc 11.97s (3.24%) of 36.50s (9.87%) N7->N2 18.28s N50 runtime heapBitsSetType 3.72s (1.01%) N7->N50 3.72s N8 github com/pingcap/tidb/executor (*HashAggFinalWorker) run 0 of 94.69s (25.59%) N8->N4 91.60s N60 github com/pingcap/tidb/executor (*HashAggFinalWorker) getFinalResult 0.26s (0.07%) of 3.09s (0.84%) N8->N60 3.09s N40 runtime wbBufFlush 0.09s (0.024%) of 23.35s (6.31%) N10->N40 20.45s N12 runtime memeqbody 26.74s (7.23%) N11->N12 15.79s N19 runtime aeshashbody 15.89s (4.29%) N11->N19 2.01s N18 runtime evacuate_faststr 6.73s (1.82%) of 24.34s (6.58%) N13->N18 24.34s N79 runtime hashGrow 0 of 9.35s (2.53%) N13->N79 8.75s N64 runtime (*hmap) newoverflow 0.85s (0.23%) of 2.39s (0.65%) N13->N64 2.12s N13->N19 1.01s N13->N10 0.57s N41 runtime stringtoslicebyte 0.31s (0.084%) of 13.94s (3.77%) N14->N41 13.94s N53 runtime mapiternext 1.73s (0.47%) of 6.55s (1.77%) N14->N53 6.55s N14->N34 3.03s N14->N10 2.19s N63 runtime growslice 0.08s (0.022%) of 2.28s (0.62%) N14->N63 1.61s N78 github com/pingcap/tidb/util/mvmap (*dataStore) get 0.51s (0.14%) of 7.69s (2.08%) N15->N78 7.69s N38 runtime mapaccess1_fast64 6.79s (1.84%) of 7.07s (1.91%) N15->N38 6.97s N15->N10 5.72s N16 runtime gcBgMarkWorker 0 of 64.23s (17.36%) N16->N2 64.23s N18->N19 11.99s N37 runtime typedmemmove 0.68s (0.18%) of 4.31s (1.16%) N18->N37 1.85s N18->N10 0.59s N20 runtime memmove 14s (3.78%) N21 runtime findrunnable 2.62s (0.71%) of 21.37s (5.78%) N48 runtime runqgrab 1.71s (0.46%) of 9.31s (2.52%) N21->N48 9.31s N57 runtime epollwait 2.94s (0.79%) N21->N57 2.89s N32 runtime lock 4.19s (1.13%) of 7.30s (1.97%) N21->N32 2.68s N35 runtime unlock 2s (0.54%) of 4.04s (1.09%) N21->N35 1.35s N22 github com/pingcap/tidb/executor (*HashAggExec) fetchChildData 0.20s (0.054%) of 20.02s (5.41%) N66 github com/pingcap/tidb/distsql (*selectResult) Next 0.09s (0.024%) of 16.67s (4.51%) N22->N66 16.65s N61 runtime chansend1 0.13s (0.035%) of 3.47s (0.94%) N22->N61 1.55s N29 runtime selectgo 2.35s (0.64%) of 9.06s (2.45%) N22->N29 1.44s N23->N9 12.91s N24->N7 13.60s N73 github com/pingcap/tidb/util/codec EncodeValue 0.60s (0.16%) of 6.17s (1.67%) N25->N73 6.17s N62 github com/pingcap/tidb/expression (*Column) Eval 1.21s (0.33%) of 5.81s (1.57%) N25->N62 5.81s N25->N37 0.44s N30 runtime mapaccess1_faststr 5.06s (1.37%) of 17.02s (4.60%) N26->N30 17.02s N26->N10 6.51s N27 github com/pingcap/tidb/util/codec (*Decoder) DecodeOne 2.73s (0.74%) of 13.98s (3.78%) N72 github com/pingcap/tidb/util/codec appendUintToChunk 0.60s (0.16%) of 2.18s (0.59%) N27->N72 2.18s N69 github com/pingcap/tidb/util/chunk (*Chunk) AppendFloat64 1.34s (0.36%) of 1.86s (0.5%) N27->N69 1.68s N28 runtime mcall 0.19s (0.051%) of 24.55s (6.64%) N76 runtime park_m 0.09s (0.024%) of 24.17s (6.53%) N28->N76 24.17s N80 runtime sellock 0.58s (0.16%) of 3.98s (1.08%) N29->N80 3.98s N29->N35 1.16s N30->N12 10.94s N30->N19 0.87s N31->N29 7.17s N31->N24 2.21s N31->N61 1.66s N42 runtime futex 4.08s (1.10%) N32->N42 0.96s N33 runtime memclrNoHeapPointers 6.01s (1.62%) N34->N7 8.20s N74 runtime futexwakeup 0.08s (0.022%) of 2.74s (0.74%) N35->N74 2.04s N36 runtime usleep 7.63s (2.06%) N44 runtime bulkBarrierPreWrite 1.43s (0.39%) of 4.33s (1.17%) N37->N44 2.58s N37->N20 1.05s N39 runtime cmpbody 7.14s (1.93%) N40->N2 23.26s N41->N20 10.37s N70 runtime rawbyteslice 0.64s (0.17%) of 3.26s (0.88%) N41->N70 3.26s N43 runtime schedule 0.76s (0.21%) of 23.73s (6.41%) N43->N21 21.37s N44->N40 2.90s N45 github com/pingcap/tidb/distsql (*selectResult) readRowsData 1.46s (0.39%) of 15.47s (4.18%) N45->N27 13.98s N46 github com/pingcap/tidb/util/chunk Row GetDatum 4.23s (1.14%) of 4.32s (1.17%) N47 runtime chansend 0.20s (0.054%) of 3.40s (0.92%) N47->N32 0.65s N47->N37 0.48s N48->N36 7.60s N49 runtime (*mheap) alloc 0.06s (0.016%) of 5.83s (1.58%) N49->N33 4.84s N51->N49 3.54s N51->N32 0.44s N51->N35 0.43s N65 github com/pingcap/tidb/expression (*Column) EvalReal 2.51s (0.68%) N52->N65 2.51s N53->N10 4.82s N75 runtime mapassign_fast64 0.33s (0.089%) of 1.95s (0.53%) N54->N75 1.94s N54->N10 1.08s N55 github com/pingcap/tidb/util/codec encode 1.57s (0.42%) of 5.58s (1.51%) N68 github com/pingcap/tidb/util/codec encodeSignedInt 1.07s (0.29%) of 3.61s (0.98%) N55->N68 3.61s N56 github com/pingcap/tidb/vendor/google golang org/grpc/transport (*http2Client) reader 0.02s (0.0054%) of 2.44s (0.66%) N56->N34 0.95s N58->N6 63.27s N59 github com/pingcap/tidb/server (*clientConn) writeChunks 0.04s (0.011%) of 2.24s (0.61%) N60->N3 1.66s N61->N47 3.34s N62->N46 4.32s N63->N37 1.32s N63->N7 0.85s N64->N24 1.36s N66->N45 15.47s N66->N63 0.56s N67->N7 2.09s N67->N20 0.52s N77 github com/pingcap/tidb/util/codec EncodeVarint 0.78s (0.21%) of 2.53s (0.68%) N68->N77 2.53s N70->N7 2.38s N73->N55 5.57s N74->N42 2.66s N75->N79 0.60s N76->N43 23.62s N77->N20 0.59s N78->N39 7.13s N79->N7 9.35s N80->N32 3.40s