Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custodian controller improvements #180

Merged
merged 5 commits into from
Jun 2, 2021

Conversation

timuthy
Copy link
Member

@timuthy timuthy commented May 26, 2021

How to categorize this PR?

/area robustness
/kind enhancement

What this PR does / why we need it:
This PR improves several aspects of the custodian and etcd controllers as a preparation for #151:

  • Adds a separate worker count flag for the custodian controller
  • Controllers now make use of the Controller-Runtime provided logger instead of instantiating another instance via logrus
  • Improves event triggers for the custodian controller, i.e., reconciliation happens on create/update/delete events for etcd and create/delete events for statefulset, as well as status update for statefulsets. Additionally, a mapper has been introduced which maps the statefulset for which an event was triggered to the corresponding etcd resource. Earlier, this has only worked because in most of the cases etcd and statefulset have the same namespace/name.
  • Custodian controller does not try to claim a statefulset anymore, since this is the job of the etcd controller.
  • The statefulset condition sync has been removed as I didn't find any statefulsets which conditions in our system and further more, we should not try to sync and information from statefulset.status to etcd by default. If a statusfulset.status.condition is really of interest, then it can be read directly from the statefulset.

I created dedicated commits to simply the review.

Which issue(s) this PR fixes:
Fixes parts of #151

Special notes for your reviewer:
Previously, the custodian controller was setup by the following instruction:
builder.For(&appsv1.StatefulSet{}).Owns(&druidv1alpha1.Etcd{})

This caused event triggers for all statefulsets (ref) (-> we don't need to enqueue for events of all statefulsets in the cluster) and etcds with an ownerReference (ref) (-> etcds don't have an owner reference).

Release note:

It is now possible to configure the count of custodian controller workers by `--custodian-workers`.
The reconciliation loop count of the custodian controller has been improved. Now, reconciliation only happens at relevant `create/update/delete` events.

@timuthy timuthy requested a review from a team as a code owner May 26, 2021 16:04
@gardener-robot gardener-robot added area/robustness Robustness, reliability, resilience related kind/enhancement Enhancement, improvement, extension needs/review Needs review size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else labels May 26, 2021
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label May 26, 2021
@gardener-robot-ci-1 gardener-robot-ci-1 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels May 26, 2021
Copy link
Collaborator

@amshuman-kr amshuman-kr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the changes and some much needed corrections! Not the least, the long-pending use of gomock ❤️

/lgtm

Comment on lines -210 to +202
filteredStatefulSets, err := m.ClaimStatefulsets(statefulSets)
err = m.cl.List(ctx, statefulSets, client.InNamespace(etcd.Namespace), client.MatchingLabelsSelector{Selector: selector})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this correction ❤️

I have created the #186 to separately address simplification of the claim logic.

Comment on lines +80 to 86
// TODO: (timuthy) remove this as it could block important health checks
if etcd.Status.LastError != nil && *etcd.Status.LastError != "" {
logger.Info(fmt.Sprintf("Requeue item because of last error: %v", *etcd.Status.LastError))
return ctrl.Result{
RequeueAfter: 30 * time.Second,
}, nil
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree 👍 This should be removed.

@gardener-robot gardener-robot added reviewed/lgtm Has approval for merging and removed needs/review Needs review needs/second-opinion Needs second review by someone else labels Jun 1, 2021
@amshuman-kr
Copy link
Collaborator

@abdasgupta Can you please give your comments? Based on that I will merge.

@abdasgupta
Copy link
Contributor

I have went through the PR. I am fine with the changes. I also tested it in local cluster. It works as it should. Thanks @timuthy for your PR.

Copy link
Contributor

@abdasgupta abdasgupta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@timuthy timuthy merged commit a9dbf5c into gardener:master Jun 2, 2021
@timuthy timuthy deleted the enhancement.etcd-status-updates branch June 2, 2021 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/robustness Robustness, reliability, resilience related kind/enhancement Enhancement, improvement, extension needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) reviewed/lgtm Has approval for merging size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants