Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IfNotPresent and Ignore SSA policies #943

Merged
merged 3 commits into from
Aug 11, 2023
Merged

Add IfNotPresent and Ignore SSA policies #943

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6b85c71
Update github.com/fluxcd/pkg/ssa to v0.30.0
stefanprodan Aug 9, 2023
6d32b08
Add IfNotPresent and Ignore SSA policies
stefanprodan Aug 9, 2023
743cb79
Document the apply behavior of resources
stefanprodan Aug 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion api/v1/kustomization_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ const (
MaxConditionMessageLength = 20000
EnabledValue = "enabled"
DisabledValue = "disabled"
MergeValue = "merge"
MergeValue = "Merge"
IfNotPresentValue = "IfNotPresent"
IgnoreValue = "Ignore"
)

// KustomizationSpec defines the configuration to calculate the desired state
Expand Down
127 changes: 85 additions & 42 deletions docs/spec/v1/kustomizations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1044,6 +1044,79 @@ cd apps/my-app
kustomize create --autodetect --recursive
```

### Controlling the apply behavior of resources

To change the apply behaviour for specific Kubernetes resources, you can annotate them with:

| Annotation | Default | Values | Role |
|-------------------------------------|------------|----------------------------------------------------------------|-----------------|
| `kustomize.toolkit.fluxcd.io/ssa` | `Override` | - `Override`<br/>- `Merge`<br/>- `IfNotPresent`<br/>- `Ignore` | Apply policy |
| `kustomize.toolkit.fluxcd.io/force` | `Disabled` | - `Enabled`<br/>- `Disabled` | Recreate policy |
| `kustomize.toolkit.fluxcd.io/prune` | `Enabled` | - `Enabled`<br/>- `Disabled` | Delete policy |

**Note:** These annotations should be set in the Kubernetes YAML manifests included
in the Flux Kustomization source (Git, OCI, Bucket).

#### `kustomize.toolkit.fluxcd.io/ssa`

##### Override

The `Override` policy instructs the controller to reconcile the Kubernetes resources
with the desired state (YAML manifests) defined in the Flux source (Git, OCI, Bucket).

If you use `kubectl` to edit a Kubernetes resource managed by Flux, all changes will be
reverted when the controller reconciles a Flux Kustomization containing that resource.
In order to preserve fields added with `kubectl`, you have to specify
a field manager named `flux-client-side-apply` e.g.:

```sh
kubectl apply --field-manager=flux-client-side-apply
```

##### Merge

The `Merge` policy instructs the controller to preserve the fields added by other tools to the
Kubernetes resources managed by Flux.

The fields defined in the manifests applied by the controller will always be overridden,
the `Merge` policy works only for adding new fields that don’t overlap with the desired
state.

For lists fields which are atomic (e.g. `.spec.tolerations` in PodSpec), Kubernetes
doesn't allow different managers for such fields, therefore any changes to these
fields will be reverted. For more context, please see the Kubernetes enhancement document:
[555-server-side-apply](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/555-server-side-apply/README.md#lists).

##### IfNotPresent

The `IfNotPresent` policy instructs the controller to only apply the Kubernetes resources
if they are not present on the cluster.

This policy can be used for Kubernetes Secrets and ValidatingWebhookConfigurations managed by cert-manager,
where Flux creates the resources with fields that are later on mutated by other controllers.

##### Ignore

The `Ignore` policy instructs the controller to skip applying Kubernetes resources
even if they are included in a Flux source (Git, OCI, Bucket).

#### `kustomize.toolkit.fluxcd.io/force`

When set to `Enabled`, this policy instructs the controller to recreate the Kubernetes resources
with changes to immutable fields.

This policy can be used for Kubernetes Jobs to rerun them when their container image changes.

**Note:** Using this policy for StatefulSets may result in potential data loss.

#### `kustomize.toolkit.fluxcd.io/prune`

When set to `Disabled`, this policy instructs the controller to skip the deletion of
the Kubernetes resources subject to [garbage collection](#prune).

This policy can be used to protect sensitive resources such as Namespaces, PVCs and PVs
from accidental deletion.

### Role-based access control

By default, a Kustomization apply runs under the cluster admin account and can
Expand Down Expand Up @@ -1502,48 +1575,6 @@ Using `flux`:
flux reconcile kustomization <kustomization-name>
```

### Customizing reconciliation

You can configure the controller to ignore in-cluster resources by labelling or
annotating them with:

```yaml
kustomize.toolkit.fluxcd.io/reconcile: disabled
```

**Note:** When the `kustomize.toolkit.fluxcd.io/reconcile` annotation is set to
`disabled`, the controller will no longer apply changes from the source, nor
will it prune the resource. To resume reconciliation, set the annotation to
`enabled` in the source or remove it from the in-cluster object.

If you use `kubectl` to edit an object managed by Flux, all changes will be
reverted when the controller reconciles a Flux Kustomization containing that
object. In order to preserve fields added with `kubectl`, you have to specify
a field manager named `flux-client-side-apply` e.g.:

```sh
kubectl apply --field-manager=flux-client-side-apply
```

Another option is to annotate or label objects with:

```yaml
kustomize.toolkit.fluxcd.io/ssa: merge
```

**Note:** The fields defined in manifests will always be overridden, the above
procedure works only for adding new fields that don’t overlap with the desired
state.

For lists fields which are atomic (e.g. `.spec.tolerations` in PodSpec), Kubernetes
doesn't allow different managers for such fields, therefore any changes to these
fields will be undone, even if you specify a manager. For more context, please
see the Kubernetes enhancement document:
[555-server-side-apply](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/555-server-side-apply/README.md#lists).

To learn how to handle patching failures due to immutable field changes, refer
to [`.spec.force`](#force).

### Waiting for `Ready`

When a change is applied, it is possible to wait for the Kustomization to reach
Expand All @@ -1558,6 +1589,18 @@ kubectl wait kustomization/<kustomization-name> --for=condition=ready --timeout=
When you find yourself in a situation where you temporarily want to pause the
reconciliation of a Kustomization, you can suspend it using [`.spec.suspend`](#suspend).

To pause the reconciliation of a specific Kubernetes resource managed by a Flux Kustomization,
you can annotate or label the resource in-cluster with:

```yaml
kustomize.toolkit.fluxcd.io/reconcile: disabled
```

**Note:** When the `kustomize.toolkit.fluxcd.io/reconcile` annotation is set to
`disabled`, the controller will no longer apply changes, nor
will it prune the resource. To resume reconciliation, set the annotation to
`enabled` in the source or remove it from the in-cluster object.

#### Suspend a Kustomization

In your YAML declaration:
Expand Down
4 changes: 2 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ require (
github.com/fluxcd/pkg/http/fetch v0.5.2
github.com/fluxcd/pkg/kustomize v1.3.4
github.com/fluxcd/pkg/runtime v0.41.0
github.com/fluxcd/pkg/ssa v0.29.0
github.com/fluxcd/pkg/ssa v0.30.0
github.com/fluxcd/pkg/tar v0.2.0
github.com/fluxcd/pkg/testserver v0.4.0
github.com/fluxcd/source-controller/api v1.0.0
Expand Down Expand Up @@ -226,5 +226,5 @@ require (
k8s.io/kubectl v0.27.1 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/kustomize/kyaml v0.14.2 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.3.0 // indirect
)
8 changes: 4 additions & 4 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,8 @@ github.com/fluxcd/pkg/runtime v0.41.0 h1:hjWUwVRCKDuGEUhovWrygt/6PRry4p278yKuJNg
github.com/fluxcd/pkg/runtime v0.41.0/go.mod h1:1GN+nxoQ7LmSsLJwjH8JW8pA27tBSO+KLH43HpywCDM=
github.com/fluxcd/pkg/sourceignore v0.3.4 h1:0cfS2Pj7xp2qpaerMjYqOBr82LC+/mGHl6v6pRbi5hs=
github.com/fluxcd/pkg/sourceignore v0.3.4/go.mod h1:ejLx+/uIrPUgqVzMTR5JiWuUnzs+zTkoEf9gS92LqaE=
github.com/fluxcd/pkg/ssa v0.29.0 h1:s2M6507YlYRLsPuXuGhXez/EqA5LFLhI13TZe31sm10=
github.com/fluxcd/pkg/ssa v0.29.0/go.mod h1:wvmBGQC47669GqhOvi0Ec0HTMziqMSNkPIsyIPBtGTQ=
github.com/fluxcd/pkg/ssa v0.30.0 h1:SYf8EBXevbTNwdHoKqRuU00YdnmqqUuR5xcciRrIi0E=
github.com/fluxcd/pkg/ssa v0.30.0/go.mod h1:eUcni/amc2fM9rJ3ZR3oPeAW/ZYk3mOGO1TW9RFmAuI=
github.com/fluxcd/pkg/tar v0.2.0 h1:HEUHgONQYsJGeZZ4x6h5nQU9Aox1I4T3bOp1faWTqf8=
github.com/fluxcd/pkg/tar v0.2.0/go.mod h1:w0/TOC7kwBJhnSJn7TCABkc/I7ib1f2Yz6vOsbLBnhw=
github.com/fluxcd/pkg/testserver v0.4.0 h1:pDZ3gistqYhwlf3sAjn1Q8NzN4Qe6I1BEmHMHi46lMg=
Expand Down Expand Up @@ -737,7 +737,7 @@ sigs.k8s.io/kustomize/api v0.13.4 h1:E38Hfx0G9R9v7vRgKshviPotJQETG0S2gD3JdHLCAsI
sigs.k8s.io/kustomize/api v0.13.4/go.mod h1:Bkaavz5RKK6ZzP0zgPrB7QbpbBJKiHuD3BB0KujY7Ls=
sigs.k8s.io/kustomize/kyaml v0.14.2 h1:9WSwztbzwGszG1bZTziQUmVMrJccnyrLb5ZMKpJGvXw=
sigs.k8s.io/kustomize/kyaml v0.14.2/go.mod h1:AN1/IpawKilWD7V+YvQwRGUvuUOOWpjsHu6uHwonSF4=
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE=
sigs.k8s.io/structured-merge-diff/v4 v4.2.3/go.mod h1:qjx8mGObPmV2aSZepjQjbmb2ihdVs8cGKBraizNC69E=
sigs.k8s.io/structured-merge-diff/v4 v4.3.0 h1:UZbZAZfX0wV2zr7YZorDz6GXROfDFj6LvqCRm4VUVKk=
sigs.k8s.io/structured-merge-diff/v4 v4.3.0/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08=
sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo=
sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8=
4 changes: 4 additions & 0 deletions internal/controller/kustomization_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -658,6 +658,10 @@ func (r *KustomizationReconciler) apply(ctx context.Context,
applyOpts.Force = obj.Spec.Force
applyOpts.ExclusionSelector = map[string]string{
fmt.Sprintf("%s/reconcile", kustomizev1.GroupVersion.Group): kustomizev1.DisabledValue,
fmt.Sprintf("%s/ssa", kustomizev1.GroupVersion.Group): kustomizev1.IgnoreValue,
}
applyOpts.IfNotPresentSelector = map[string]string{
fmt.Sprintf("%s/ssa", kustomizev1.GroupVersion.Group): kustomizev1.IfNotPresentValue,
}
applyOpts.ForceSelector = map[string]string{
fmt.Sprintf("%s/force", kustomizev1.GroupVersion.Group): kustomizev1.EnabledValue,
Expand Down