Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg/ingester: check that ingester is in LEAVING state when transferring chunks and claiming tokens. Required when using memberlist client. #1300

Merged
merged 4 commits into from
Dec 6, 2019

Conversation

pstibrany
Copy link
Member

@pstibrany pstibrany commented Nov 21, 2019

In order to work with gossiped ring correctly, ingester that is going to claim tokens from a different ingester needs to make sure that leaving ingester is indeed in the LEAVING state. Otherwise claiming tokens will fail. This change implements the check.

(Plus it also adds ring to prometheus metrics in distributor)

This is important when using gossiping ring. If source ingester is not
leaving yet, than claiming its tokens will fail (merge function
will not move ownership to the target ingester), and target ingester
will end up with no tokens in the ring.
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany pstibrany changed the title pkg/ingester: changes to work for gossiping pkg/ingester: changes to work with gossiping Nov 25, 2019
@rfratto rfratto self-requested a review November 25, 2019 21:37
Copy link
Member

@rfratto rfratto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than my one question

pkg/ingester/transfer.go Outdated Show resolved Hide resolved
There is no direct usage of logging, which is consistent
with the rest of this method.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
Copy link
Member

@sh0rez sh0rez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

What about renaming this to something like "pkg/ingester: check ingester is in leaving state" or so?

@sh0rez sh0rez added component/loki type/enhancement Something existing could be improved labels Nov 30, 2019
@pstibrany pstibrany changed the title pkg/ingester: changes to work with gossiping pkg/ingester: check that ingester is in LEAVING state when transferring chunks and claiming tokens. Required when using memberlist client. Dec 2, 2019
@cyriltovena cyriltovena merged commit dd96fa1 into grafana:master Dec 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/loki type/enhancement Something existing could be improved
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants