-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Organize Resource Deployment with Tiers #543
Conversation
Here is an example. All managed clusters have labels env:fv Initially deploy this ClusterProfile apiVersion: config.projectsveltos.io/v1alpha1
kind: ClusterProfile
metadata:
name: validation-and-monitoring
spec:
clusterSelector: env=fv
continueOnConflict: true
helmCharts:
- repositoryURL: https://kyverno.github.io/kyverno/
repositoryName: kyverno
chartName: kyverno/kyverno
chartVersion: v3.1.1
releaseName: kyverno-latest
releaseNamespace: kyverno
helmChartAction: Install
- repositoryURL: https://prometheus-community.github.io/helm-charts
repositoryName: prometheus-community
chartName: prometheus-community/prometheus
chartVersion: 23.4.0
releaseName: prometheus
releaseNamespace: prometheus
helmChartAction: Install
- repositoryURL: https://grafana.github.io/helm-charts
repositoryName: grafana
chartName: grafana/grafana
chartVersion: 6.58.9
releaseName: grafana
releaseNamespace: grafana
helmChartAction: Install Prometheus, Grafana and Kyverno are deployed in all clusters. If now we post this ClusterProfile selecting a subset of managed clusters (the one with region:west): apiVersion: config.projectsveltos.io/v1alpha1
kind: ClusterProfile
metadata:
name: kyverno
spec:
clusterSelector: region=west
helmCharts:
- repositoryURL: https://kyverno.github.io/kyverno/
repositoryName: kyverno
chartName: kyverno/kyverno
chartVersion: v3.1.4
releaseName: kyverno-latest
releaseNamespace: kyverno
helmChartAction: Install and error will be reported on this status:
dependencies: no dependencies
featureSummaries:
- failureMessage: cannot manage chart kyverno/kyverno-latest. ClusterSummary validation-and-monitoring-capi-clusterapi-workload
managing it
featureID: Helm With this PR though now we can simply set Tier behaviour. Setting Tier to 50 (so lower than Tier default value of 100) will allow kyverno ClusterProfile to win the conflict and so Kyverno gets upgraded to 3.1.4 only in the region:west clusters apiVersion: config.projectsveltos.io/v1alpha1
kind: ClusterProfile
metadata:
name: kyverno
spec:
tier: 50
clusterSelector: region=west
helmCharts:
- repositoryURL: https://kyverno.github.io/kyverno/
repositoryName: kyverno
chartName: kyverno/kyverno
chartVersion: v3.1.4
releaseName: kyverno-latest
releaseNamespace: kyverno
helmChartAction: Install
|
This PR also enhances the message displayed in case of conflict. For helm chart: featureSummaries:
- failureMessage: |
cannot manage chart kyverno/kyverno-latest. ClusterSummary validation-and-monitoring-capi-clusterapi-workload managing it.
cannot manage chart prometheus/prometheus. ClusterSummary validation-and-monitoring-capi-clusterapi-workload managing it.
cannot manage chart grafana/grafana. ClusterSummary validation-and-monitoring-capi-clusterapi-workload managing it. For resources - failureMessage: |
Object Deployment:nginx/nginx-deployment currently deployed because of ConfigMap default/nginx.
Sveltos instance currently deploying this resource: ClusterProfile deploy-resources;
Object Namespace:/nginx currently deployed because of ConfigMap default/nginx.
Sveltos instance currently deploying this resource: ClusterProfile deploy-resources; |
ClusterProfile/Profile instances let deploy add-ons and applications (Helm charts or Kubernetes resources) across a set of managed clusters. Sometimes there might be a need to tweak deployments for specific clusters (a subset of the original group) within that group. Previously, creating a new ClusterProfile/Profile targeting a subset of clusters with resources already managed by another profile resulted in conflicts. Sveltos wouldn't allow deployment for those resources. The concept of ```tier``` is introduced to manage deployment priority for resources targeted by multiple configurations. How it works: 1. Each ClusterProfile/Profile has a new property called __tier__. 2. This tier value controls the deployment order for resources targeting the same cluster element (e.g., a Kubernetes object or Helm chart). 3. By default, the first configuration to reach the cluster "wins" and deploys the resource. 4. Tiers override this behavior. When conflicts occur, the configuration with the lowest tier value takes precedence and deploys the resource. Higher tier values represent lower priority. 5. The default tier value is 100. Benefits: 1. Finer control over resource deployment: Tiers allow you to fine-tune deployments within your cluster, especially when multiple configurations manage the same resources. 2. Conflict resolution: Tiers ensure predictable outcomes when multiple configurations target the same resource. The configuration with the most critical deployment (lowest tier) takes priority. Fixes projectsveltos#305
Very helpful feature for operations. Thanks @gianlucam76! |
ClusterProfile/Profile instances let deploy add-ons and applications (Helm charts or Kubernetes resources) across a set of managed clusters.
Sometimes there might be a need to tweak deployments for specific clusters (a subset of the original group) within that group.
Previously, creating a new ClusterProfile/Profile targeting a subset of clusters with resources already managed by another profile resulted in conflicts. Sveltos wouldn't allow deployment for those resources.
The concept of
tier
is introduced to manage deployment priority for resources targeted by multiple configurations.How it works:
Benefits:
Fixes #305