Issues installing Redis Helm Chart in IBM Cloud

[Agusnez]

Which chart:

Redis Chart Version 6.4.4

Description

I cannot install Redis Helm Chart in IBM Cloud. What I see is that the slaves are failing.

Steps to reproduce the issue:

1. Write values.yaml
2. helm install --name shared-redis -f redis/values.yaml bitnami/redis --version 6.4.4

Describe the results you received:

And the slaves logs shows that the slave is not able to connect to the master I think...

 10:19:30.93 INFO  ==> ** Starting Redis **
1:C 16 Sep 10:19:30.935 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 16 Sep 10:19:30.935 # Redis version=4.0.14, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 16 Sep 10:19:30.935 # Configuration loaded
1:S 16 Sep 10:19:30.936 * Running mode=standalone, port=6379.
1:S 16 Sep 10:19:30.937 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:S 16 Sep 10:19:30.937 # Server initialized
1:S 16 Sep 10:19:30.937 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:S 16 Sep 10:19:30.937 * Ready to accept connections
1:S 16 Sep 10:19:30.937 * Connecting to MASTER shared-redis-master:6379
1:S 16 Sep 10:19:30.939 * MASTER <-> SLAVE sync started
1:S 16 Sep 10:20:31.076 # Timeout connecting to the MASTER...
1:S 16 Sep 10:20:31.076 * Connecting to MASTER shared-redis-master:6379
1:S 16 Sep 10:20:31.077 * MASTER <-> SLAVE sync started

Describe the results you expected:

I expect no errors. It's just the most simple installation.

Additional information you deem important (e.g. issue happens only occasionally):

The reason why I am running an older version is because I need Redis 4. We are currently performing a migration of our cluster. Also, the readiness probe failing is weird... I don't think it is because the password because I tried without it and got the same result.

Version of Helm and Kubernetes:

• Output of helm version:

Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}

• Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T12:36:28Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.10+IKS", GitCommit:"9b54f18231d2f8e39911df6f635e05fd0153826d", GitTreeState:"clean", BuildDate:"2019-07-11T14:17:49Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

• Additional: Output of helm install:

NAME:   shared-redis
LAST DEPLOYED: Mon Sep 16 11:26:01 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                 DATA  AGE
shared-redis         3     1s
shared-redis-health  3     1s

==> v1/NetworkPolicy
NAME          AGE
shared-redis  1s

==> v1/Pod(related)
NAME                                  READY  STATUS             RESTARTS  AGE
shared-redis-master-0                 0/1    Pending            0         1s
shared-redis-metrics-7b76fbd58-tjx7d  0/1    ContainerCreating  0         1s
shared-redis-slave-b65c89bdb-5ggsw    0/1    ContainerCreating  0         1s
shared-redis-slave-b65c89bdb-fhvlk    0/1    ContainerCreating  0         1s
shared-redis-slave-b65c89bdb-k28k4    0/1    ContainerCreating  0         1s

==> v1/Secret
NAME          TYPE    DATA  AGE
shared-redis  Opaque  1     1s

==> v1/Service
NAME                  TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)   AGE
shared-redis-master   ClusterIP  172.21.70.175   <none>       6379/TCP  1s
shared-redis-metrics  ClusterIP  172.21.221.123  <none>       9121/TCP  1s
shared-redis-slave    ClusterIP  172.21.135.192  <none>       6379/TCP  1s

==> v1beta1/Deployment
NAME                  READY  UP-TO-DATE  AVAILABLE  AGE
shared-redis-metrics  0/1    1           0          1s
shared-redis-slave    0/3    3           0          1s

==> v1beta2/StatefulSet
NAME                 READY  AGE
shared-redis-master  0/1    1s


NOTES:
** Please be patient while the chart is being deployed **
Redis can be accessed via port 6379 on the following DNS names from within your cluster:

shared-redis-master.default.svc.cluster.local for read/write operations
shared-redis-slave.default.svc.cluster.local for read-only operations


To get your password run:

    export REDIS_PASSWORD=$(kubectl get secret --namespace default shared-redis -o jsonpath="{.data.redis-password}" | base64 --decode)

To connect to your Redis server:

1. Run a Redis pod that you can use as a client:

   kubectl run --namespace default shared-redis-client --rm --tty -i --restart='Never' \
    --env REDIS_PASSWORD=$REDIS_PASSWORD \--labels="redis-client=true" \
   --image docker.io/bitnami/redis:4.0.14 -- bash

2. Connect using the Redis CLI:
   redis-cli -h shared-redis-master -a $REDIS_PASSWORD
   redis-cli -h shared-redis-slave -a $REDIS_PASSWORD


Note: Since NetworkPolicy is enabled, only pods with label
shared-redis-client=true"
will be able to connect to redis.

[rafariossaa]

Hi, @Agusnez
Have you tried with the version 7.1.1 of the chart? That is the latest for redis v4

...
stable/redis 8.0.1 5.0.5 Open source, advanced key-value store. It is often referr...
stable/redis 8.0.0 5.0.4 Open source, advanced key-value store. It is often referr...
stable/redis 7.1.1 4.0.14 Open source, advanced key-value store. It is often referr...
stable/redis 7.1.0 4.0.14 Open source, advanced key-value store. It is often referr...
stable/redis 7.0.3 4.0.14 Open source, advanced key-value store. It is often referr...
...

[Agusnez]

@rafariossaa Same thing but now new PVC for the slaves.

[rafariossaa]

Hi @Agusnez ,
Could you send the logs of the pods ?

[Agusnez]

Hello @rafariossaa. Yes. Here is the full content of the logs. Notice the timeout between master and slave and the consequent shutdown because of the readiness probe failing. Increasing this readiness probe time does not work.

/libredis.sh: line 211: REDIS_PASSWORD_FILE: unbound variable
 08:33:06.35 INFO  ==> ** Starting Redis **
1:C 19 Sep 08:33:06.353 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 19 Sep 08:33:06.353 # Redis version=4.0.14, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 19 Sep 08:33:06.354 # Configuration loaded
1:S 19 Sep 08:33:06.355 * Running mode=standalone, port=6379.
1:S 19 Sep 08:33:06.355 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:S 19 Sep 08:33:06.355 # Server initialized
1:S 19 Sep 08:33:06.355 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:S 19 Sep 08:33:06.355 * Ready to accept connections
1:S 19 Sep 08:33:06.356 * Connecting to MASTER shared-redis-master-0.shared-redis-headless.default.svc.cluster.local:6379
1:S 19 Sep 08:33:06.358 * MASTER <-> SLAVE sync started
1:S 19 Sep 08:34:07.491 # Timeout connecting to the MASTER...
1:S 19 Sep 08:34:07.491 * Connecting to MASTER shared-redis-master-0.shared-redis-headless.default.svc.cluster.local:6379
1:S 19 Sep 08:34:07.499 * MASTER <-> SLAVE sync started
1:S 19 Sep 08:35:08.635 # Timeout connecting to the MASTER...
1:S 19 Sep 08:35:08.635 * Connecting to MASTER shared-redis-master-0.shared-redis-headless.default.svc.cluster.local:6379
1:S 19 Sep 08:35:08.637 * MASTER <-> SLAVE sync started
1:signal-handler (1568882134) Received SIGTERM scheduling shutdown...
1:S 19 Sep 08:35:34.093 # User requested shutdown...
1:S 19 Sep 08:35:34.093 # Redis is now ready to exit, bye bye...

[Agusnez]

I think this chart does not work in IBM Cloud. I have been trying all possible things for over two weeks.

[juan131]

Hi @Agusnez

If your reason to use old versions of the chart (7.1.1 or 6.4.4) is using Redis 4, please note that you don't need to do that. You can override the image.tag instead.

$ helm repo update && helm search stable/redis
...
NAME           	CHART VERSION	APP VERSION	DESCRIPTION
stable/redis   	9.2.0        	5.0.5      	Open source, advanced key-value store. It is often referr...
...
$ helm install stable/redis --set image.tag=4.0 --name my-release
$ k describe pod my-release-redis-master-0
...
    Image:         docker.io/bitnami/redis:4.0
...

However, this setup is not currently tested (we're just testing the branch 5.0 in our pipeline) so it's likely to contain bugs.

[stale]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[stale]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

[alexellis]

Just curious, did Redis get deleted from the helm chart repo? https://github.com/bitnami/charts/tree/master/bitnami

[beltran-rubo]

Hi @alexellis,

Taking into account the stable deprecation timeline, we have been started the process of moving the Bitnami Helm Charts to the Bitnami repository, see helm/charts#20969 for more details.

The Helm Chart tgz continues existing in the stable repo but it would not get new updates. We also added more info and the upgrade process to start using the new repo in the NOTES.txt file helm/charts#21333 Any suggestions to improve the migration or documentation will be welcome, thanks.