Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Annotations switch between master/slave does not work: no master #1144

Open
zifeo opened this issue Nov 26, 2024 · 2 comments
Open

Annotations switch between master/slave does not work: no master #1144

zifeo opened this issue Nov 26, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@zifeo
Copy link

zifeo commented Nov 26, 2024

What version of redis operator are you using?

redis-operator version: ghcr.io/ot-container-kit/redis-operator/redis-operator:v0.18.1

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (kubectl version)?

Client Version: v1.31.3
Kustomize Version: v5.4.2
Server Version: v1.30.3+rke2r1

What did you do?

redis-replication 0.16.4, clusterSize = 2

redis-sentinel 0.16.6, clusterSize = 3

correctly spawn 2 redis pods, one with redis-role=slave and one with redis-role=master. However after killing the master, 2 slaves are left.

operator logs

{"level":"error","ts":"2024-11-26T23:52:32Z","logger":"controllers.RedisSentinel","msg":"","Request.Namespace":"test","Request.Name":"redis","error":"no real master pod found","stacktrace":"github.com/OT-CONTAINER-KIT/redis-operator/pkg/k8sutils.getRedisReplicationMasterIP\n\t/workspace/pkg/k8sutils/redis-sentinel.go:349\ngithub.com/OT-CONTAINER-KIT/redis-operator/pkg/k8sutils.IsRedisReplicationReady\n\t/workspace/pkg/k8sutils/redis-replication.go:230\ngithub.com/OT-CONTAINER-KIT/redis-operator/pkg/controllers/redissentinel.(*RedisSentinelReconciler).Reconcile\n\t/workspace/pkg/controllers/redissentinel/redissentinel_controller.go:57\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.2/pkg/internal/controller/controller.go:227"}

all the sentinels have the same logs

1:X 26 Nov 2024 23:45:32.315 * Running mode=sentinel, port=26379.
1:X 26 Nov 2024 23:45:32.320 * Sentinel new configuration saved on disk
1:X 26 Nov 2024 23:45:32.320 # Sentinel ID is 0fb2f2583a21c2cdb08625fd8e0bdd7d3f459971
1:X 26 Nov 2024 23:45:32.320 # +monitor master myMaster 10.42.2.248 6379 quorum 2
1:X 26 Nov 2024 23:46:02.314 # +sdown master myMaster 10.42.2.248 6379

What did you expect to see?

successfully switching the master

What did you see instead?

no master

@zifeo zifeo added the bug Something isn't working label Nov 26, 2024
@momszx
Copy link

momszx commented Dec 4, 2024

this is help for me. But after pod is restarted same error
kubectl exec -it redis-sentinel-sentinel-1 -n ***** -- redis-cli -p 26379 -a redispass SENTINEL set myMaster auth-pass redispass

@momszx
Copy link

momszx commented Dec 4, 2024

Solution

apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisSentinel
metadata:
  name: redis-sentinel
  namespace: *
spec:
  clusterSize: 3
  podSecurityContext:
    runAsUser: 1000
    fsGroup: 1000
  redisSentinelConfig:
    redisReplicationName: redis-replication
    additionalSentinelConfig: addtlsentinel
  kubernetesConfig:
    image: quay.io/opstree/redis-sentinel:latest
    imagePullPolicy: IfNotPresent
    redisSecret:
      name: redis-secret
      key: password
    resources:
      requests:
        cpu: 101m
        memory: 128Mi
      limits:
        cpu: 101m
        memory: 128Mi
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: addtlsentinel
  namespace: *
data:
  redis-sentinel-additional.conf: |
    sentinel auth-pass myMaster redispass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants