Expose request latency metric

[slaunay]

The Java client exposes two metrics about request latency (request-latency-avg as the average request latency in ms and request-latency-max as the maximum request latency in ms).
This metric is extremely valuable when optimizing the communication between a remote producer and a Kafka cluster through TCP tuning, typically when the socket buffers are too small on either side to reach the bandwidth-delay product of the connection.

Code change:

• new request-latency-in-ms histogram metric (global and per registered broker)
• addng 10ms latency to brokers in producer functional tests for validation
• updating documentation

[eapache]

Looks good to me. @wvanbergen ?

[wvanbergen]

Do we care that time.Now() is not monotonic? (golang/go#12914)

Other than that - LGTM.

[slaunay]

Non monotonic clock can lead to negative or inaccurate latency metric which might not be that big of deal I believe and it does not seem that there is an easy and portable way around it in Go.

time.Now() is also used when calling Set[Read|Write]Deadline on a TCP connection.

[eapache]

I tried to rebase this myself but got confused trying to merge the documentation... I think the name column currently in master is more correct?

[slaunay]

I did remove the type prefix typo (histogram) on the other PR and had to fix indentation on this one, so none of them are more correct than the other 😞.

AFAIK the only way around is to resolve the conflicts manually as the changes are on the same block of lines so I will rebase the changes against master...

[eapache]

Thanks!