may use Counter instead of Histogram for Packet IO metrics

[siddontang]

Enhancement

I found that we use Histogram to track the packet IO, see https://github.com/pingcap/tidb/blob/master/server/packetio.go#L52

But is it necessary to use Histogram here? In the Grafana, We have a Client Traffic Panel, see below, but in the panel, we can see that here we use histogram sum, and we can use a counter instead.

Counter is much faster than Histogram, I wrote a very simple benchmark here:

func BenchmarkPromHistogram(b *testing.B) {
	r := PacketIOHistogram.WithLabelValues("read")
	for i := 0; i < b.N; i++ {
		r.Observe(10)
	}
}

func BenchmarkPromCounter(b *testing.B) {
	r := QueryTotalCounter.WithLabelValues("read", "write")
	for i := 0; i < b.N; i++ {
		r.Add(10)
	}
}

cpu: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
BenchmarkPromHistogram-8   	19636056	        60.26 ns/op
BenchmarkPromCounter-8     	95965551	        10.68 ns/op

[siddontang]

If we decide to advance this, I think we should list all the histogram metrics and check whether we need to use Counter instead.

[bb7133]

Checked the current metrics by:

• The calls of prometheus.NewHistogram() / prometheus.NewHistogramVec()
• The use of {histogram_metrics}_sum / {histogram_metrics}_count

One another metric that can be improved is DistSQLCoprCacheHistogram:

tidb/metrics/distsql.go

Line 56 in d5d8e13

DistSQLCoprCacheHistogram = prometheus.NewHistogramVec(

tidb/metrics/grafana/tidb.json

Line 6660 in 27348d6

"expr": "sum(rate(tidb_distsql_copr_cache_sum{tidb_cluster=\"$tidb_cluster\"}[1m])) by (type)",

It was introduced as a histogram metric in #19979 but after #26338 and #30712(IMO the change is reasonable), it using a counter is enough.

[AricSu]

/assign