Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement sharding for Flux controllers #96

Merged
merged 6 commits into from
Sep 16, 2024
Merged

Implement sharding for Flux controllers #96

merged 6 commits into from
Sep 16, 2024

Conversation

stefanprodan
Copy link
Member

@stefanprodan stefanprodan commented Sep 15, 2024

This PR allows configuring sharding and horizontal scaling for the Flux controllers.

Sharding configuration

The .spec.sharding field is optional and specifies the sharding configuration for the Flux controllers.

The .spec.sharding.key field is optional and specifies the sharding key label to use for the Flux controllers.
By default, the key is set to sharding.fluxcd.io/key.

The .spec.sharding.shards field is required and specifies the list of sharding values to use for the Flux controllers.

Example:

apiVersion: fluxcd.controlplane.io/v1
kind: FluxInstance
metadata:
  name: flux
  namespace: flux-system
spec:
  distribution:
    version: "2.3.x"
    registry: "ghcr.io/fluxcd"
    artifact: "oci://ghcr.io/controlplaneio-fluxcd/flux-operator-manifests"
  sharding:
    key: "sharding.fluxcd.io/key"
    shards:
      - "shard1"
      - "shard2"

For each shard, the operator will create a separate set of controllers, e.g.:

$ kubectl -n flux-system get deployments -l app.kubernetes.io/part-of=flux
NAME
source-controller
source-controller-shard1
source-controller-shard2
kustomize-controller
kustomize-controller-shard1
kustomize-controller-shard2
helm-controller
helm-controller-shard1
helm-controller-shard2

Note that only the source-controller, kustomize-controller and helm-controller controllers
support sharding.

To assign a resource to a specific shard, add the sharding.fluxcd.io/key label with the shard value,
e.g.: sharding.fluxcd.io/key: shard1.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
@stefanprodan stefanprodan added enhancement New feature or request area/api API related issues and pull requests labels Sep 15, 2024
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
@stefanprodan stefanprodan changed the title Implement sharding Implement sharding for Flux controllers Sep 16, 2024
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
@stefanprodan stefanprodan merged commit cfb54b3 into main Sep 16, 2024
1 check passed
@stefanprodan stefanprodan deleted the sharding branch September 16, 2024 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api API related issues and pull requests enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant