Query frontend

[owen-d]

This includes two changes.

Codec performance improvements for merging responses.

We now take advantage of the fact that split responses are already ordered and don't overlap. This includes

• Introduces an escape hatch when limit >= entries that avoid some unnecessary comparisons
• Ordering per-stream entry groups in O(n*log(n)) instead of O(m*log(m)) where n = num_intervals, m = num_entries
• Extracting limit results from a collection of streams in O(n*log(k)) where n = limit, k = num_streams by using a new priority queue impl.

The splitByInterval middleware has been changed significantly.

Previously it split time ranges twice, the first for serial execution and the second for parallel execution within the first.

Now we split one time and use a channel to determine parallelism. This avoids queries in a later interval being convoyed behind a slow (retrying, etc) earlier query. It simplifies the execution model while making it less prone to blocking.

If requested, we can also add a backpressure parameter to the splitByInterval middleware which would prevent scheduling later queries if we're still waiting on an earlier query from backpressure previous intervals.

$ go test -bench . | prettybench
goos: darwin
goarch: amd64
pkg: github.com/grafana/loki/pkg/querier/queryrange
PASS
benchmark                                                              iter     time/iter
---------                                                              ----     ---------
BenchmarkResponseMerge/mergeStreams_unlimited-8                          25   45.81 ms/op
BenchmarkResponseMerge/mergeOrderedNonOverlappingStreams_unlimited-8   1029    1.16 ms/op
BenchmarkResponseMerge/mergeStreams_limited-8                            24   47.88 ms/op
BenchmarkResponseMerge/mergeOrderedNonOverlappingStreams_limited-8       56   22.16 ms/op
ok      github.com/grafana/loki/pkg/querier/queryrange  5.983s