Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memberlist svc does not publish endpoint when not ready #322

Open
nschad opened this issue Feb 22, 2022 · 0 comments
Open

memberlist svc does not publish endpoint when not ready #322

nschad opened this issue Feb 22, 2022 · 0 comments
Labels
good first issue Good for newcomers help wanted Extra attention is needed

Comments

@nschad
Copy link
Collaborator

nschad commented Feb 22, 2022

recently I deployed just the compactor with the "shuffle-sharding" tec from cortexproject/cortex#4433.

The compactor would not come up due to the memberlist not publishing non-ready endpoints

ts=2022-02-22T10:48:56.126638511Z caller=memberlist_logger.go:74 level=warn msg="Failed to resolve cortex-fast-compact-memberlist: lookup cortex-fast-compact-memberlist on 10.64.0.10:53: no such host"

Only after I set

publishNotReadyAddresses: true

in the memberlist headless service the compactor would come up and join the ring

level=info ts=2022-02-22T10:50:23.815845998Z caller=memberlist_client.go:542 msg="joined memberlist cluster" reached_nodes=1

I'm currently not sure on the best way to solve that, maybe we should just allow the service to publish NonReadyAddresses.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant