Filter by label

[killianmuldoon]

I'm looking to filter pods by label before applying a descheduling strategy to them. This allows us to slice our descheduling operations helping with testing and with only descheduling workloads that are actively identified as deschedulable based on developer/operator knowledge. This should be opt in - i.e. with no label selector in the policy descheduler will work as now.

I'm happy to write a PR for this - but I wanted to ask for advice on where this logic should be implemented.

One option is in the ListEvictablePodsOnNode method, or one of its sub methods, - but it would need a change to the function signature which would break current uses of this package.

Another option is to implement the label selector with a call in every strategy. There's duplication of work here that may be tough to maintain in future.

A third option is to wrap the Kubernetes Client in an LabelledClient Interface that embeds the kubernetes.Interface but also holds a labelSelector. This could then be called in the ListEvictablePodsOnNode, but this requires a type assertion in that method.

I'd love to get some feedback on this - or any implementation ideas I might be missing.

[killianmuldoon]

PR to resolve this at #202

[seanmalloy]

/kind feature

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[seanmalloy]

/remove-lifecycle stale

[seanmalloy]

The proposal has been reviewed by SIG scheduling and this feature can now be implemented. Descheduling by pod label will be implemented using a strategy parameter, not a CLI option. The labelSelector field in the YAML should be a standard k8s labelSelector.

Here is an example of what the YAML might look like:

---
apiVersion: "descheduler/v1alpha1"
kind: "DeschedulerPolicy"
strategies:
  "PodLifeTime":
     enabled: true
     params:
        maxPodLifeTimeSeconds: 86400
	labelSelector:
          matchLabels:
            component: redis
          matchExpressions:
            - {key: tier, operator: In, values: [cache]}
            - {key: environment, operator: NotIn, values: [dev]}

The proposal document can be found here: https://bit.ly/k8s-descheduler-labels-namespaces

@hanumanthan you had mentioned that you were interested in implementing this feature. Feel free to start working on it. Let us know if you have any questions. Let us know if you are able to work on this or not. Thanks!

[bytetwin]

/assign @hanumanthan

[bytetwin]

@seanmalloy I couldnt find the standard implementation for labels parsing. Can you please provide any doc or code for parsing labels.
Found this https://godoc.org/k8s.io/apimachinery/pkg/labels#Selector but want to know the standard way to parse this from the input config

matchLabels:
	component: redis
matchExpressions:
	- {key: tier, operator: In, values: [cache]}
	- {key: environment, operator: NotIn, values: [dev]}

[damemi]

@hanumanthan these links might be helpful, the example config is from the k8s docs on label selectors: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#resources-that-support-set-based-requirements

So really the type that needs to be added to the config is just a metav1.LabelSelector, which can be parsed to a labels.Selector with metav1.LabelSelectorAsSelector(). Then you have all of the matching helpers you linked that come with a labels.Selector.

[ingvagabund]

@hanumanthan do you still want to work on this issue? There's a Options struct which can be easily extended with WithLabels method. ListPodsOnANode already accepts the struct so it's easy to wire the labels down to it.

https://github.com/kubernetes-sigs/descheduler/blob/master/pkg/descheduler/pod/pods.go#L30-L34

[bytetwin]

Yeah I will work on this weekend

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[seanmalloy]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[ingvagabund]

/remove-lifecycle stale

[lindhe]

Would be nice to have label filtering for namespaces too!

[lixiang233]

Hi @hanumanthan, do you still plan to work on this? If not, I'm glad to pick this up.

[seanmalloy]

@lixiang233 I say go for it if you have the time.

[lixiang233]

/unassign @hanumanthan
/assign

[seanmalloy]

This feature will be available in descheduler v0.21.0.