Add usage reporting capability for Loki to (optionally) send usage stats to Grafana Labs

[slim-bean]

I'll start this by saying that I realize there are always mixed opinions about software phoning home, I often find myself opting out of usage reporting because it makes me uncomfortable at times.

However, there is a tremendous amount we could learn to benefit the project. Being able to understand what typical deployment models of Loki's users are can greatly help us understand how to improve the project.

Things like:

• Deployment mode (Single binary, SSD, Microservices)
• Throughput (what volumes people run Loki at)
• Storage in use (what index type, what object store type)
• Cache configurations
• Active streams, index size vs storage size etc.
• Query throughput (how fast are queries)

These kinds of anonymous stats let us better understand where we should focus improvements, documentation, enhancements.

For example, we are able to see query throughput now over 100GB/s on really highly parallelized and highly provisioned clusters. When we talk about Loki being fast we really feel like it is! However if you are under provisioned or misconfigured you may find your Loki executing queries on a single thread and seeing 400-1000MB/s of query throughput, but we honestly have no idea what average query performance looks like for our users, and we want everyone to have the best experience possible!

The intention is for this to be opt-out not opt-in, because if it's opt-in, we all know nobody will do it.

We want this to be transparent, just like Grafana, what's collected and sent will be clearly visible in the source

This issue is intended as a high level issue for tracking the progress of this work as well as to be as transparent as possible about what we are doing and why we are doing it!

[honganan]

@slim-bean Can others see these reports? I want to compare with others' query throughput. Thanks!

[slim-bean]

@honganan good question! I'm not sure what our plans are for sharing this info, could be kind of neat!

/cc @DanCech

[DanCech]

A great question for sure, it's early days but we're definitely interested in ways that the community can make use of the data.

[hevisko]

I'm not going to debate the phone home, but I think there is a need to have it consider an exponential back-off when it's not able to connect? Every 10seconds to throw and error is a tad much - and makes firewall admins frown