Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup long terminating stateful pods #153

Merged
merged 3 commits into from
Jun 6, 2018

Conversation

ewoutp
Copy link
Contributor

@ewoutp ewoutp commented Jun 4, 2018

This PR extends #151

This PR adds cleaning up of members (and their Pods & PVCs) that are no longer part of a cluster.
When /_admin/cluster/health no longer shows the member ID, it is subject to removal.
To do this, a new condition is added for members that records if the member was at least once reported as member of the cluster. Without this, the member could be deleted before it even has a chance to join the cluster.

@ewoutp ewoutp added the 9 WIP label Jun 4, 2018
// Member is (still) found, skip it
if m.Conditions.Update(api.ConditionTypeMemberOfCluster, true, "", "") {
list.Update(m)
if err := r.context.UpdateStatus(status); err != nil {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this fails on one server, would that lead to subsequent server in the ForeachServerGroup not getting their last access time updated ... maybe leading to false delete?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, other member can be affected.
It would mean that other members may late with recording their "I'm a member of the cluster" state. If anything, they will be removed later, not earlier.

@ewoutp ewoutp merged commit 265397d into master Jun 6, 2018
@ewoutp ewoutp deleted the bugfix/cleanup-long-terminating-stateful-pods branch June 6, 2018 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants