Loki Query Frontend #1442
Merged
Loki Query Frontend #1442
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cyriltovena
force-pushed
the
frontend-cleaned
branch
3 times, most recently
from
53170ee
to
66dda4f
Compare
owen-d
reviewed
Dec 19, 2019
owen-d
reviewed
Dec 19, 2019
sandeepsukhani
previously requested changes
Dec 31, 2019
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
* frontend codec merging optimizations
* codec benchmarks
* removes unused bounds code in queryrange ordering
* [wip] splitby uses channels instead of sub batching intervals
* splitBy channel limit test
* single allocation for merging entries from a single stream
* skip merging loki responses when limit is already hit
* removes checks for unlimited queries in queryrange
* removes splitByInterval{,.interval} spans
* removes interval_batch_size from jsonnet lib
* moves benchmark utils to own file
* renames markers -> entries
* priority queue comments
cyriltovena
force-pushed
the
frontend-cleaned
branch
from
6b23951
to
8f7c437
Compare
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
owen-d
approved these changes
Jan 6, 2020
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
|
dam windows |
cyriltovena
pushed a commit
to cyriltovena/loki
that referenced
this pull request
Jun 11, 2021
Fix the version of protobuf and pin tools in the build image.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
After many refactoring PRs in Cortex, this PR introduces a
query-frontendcomponent in loki.The
query-frontendsits in front of queriers and communicates with them to dispatch work, allowing parallelization and queries speed improvement.I have been running the frontend for a month in dev and it really shows promising result. However it definitely requires bigger queriers pool compared to Cortex.
The current implementation targets specifically slow queries.
Metrics queries
Metrics queries like cortex will be aligned, splitted by 4 hours by default and result cached. In fact we are using the same set of middlewares.For some streams 4 hours block can still be too much (high throughput), but I expect the sharding parallelization to split again those 4 hours block into 16 (Future works), in the meantime you can tweak the interval to a lower range.
Log Regex queries
Log regex queries are splitted also by interval of 4 hours, however compared to cortex they are run in sequence and each block is splitted by 16 (configurable) and run in parallel. This allows to check for every 4 hours if we already have enough result before moving to the next block.
Anything else (non regex log queries, labels, etc...) run the same way but still goes through the frontend. Except for labels names queries I actually don't plan any work since those queries are already very fast. (500~700ms)
The loki jsonnet library has been updated.
PS: I had to update to latest cortex and there were some changes related to the ring and some vendoring issue.
Next steps :
/cc @slim-bean @tomwilkie @joe-elliott @rfratto
Here is a screenshot of a trace for a regex queries on an high throughput stream:
This is not yet part of the single binary, I still need to think about how we want to design this, it could be very nice to swap grpc for direct querier library calls.