-
-
Notifications
You must be signed in to change notification settings - Fork 276
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to initialize root password and custom user/password. #229
Comments
Hi @bluven, did you make sure that it's a new cluster? Once a cluster is initialized (has data in PVC) will not update the credentials anymore. The user creation and initialization are done by docker image entrypoint and it's run only when the server is started for the first time. There is an issue opened about this #75 |
@AMecea I'm pretty sure about that. Today I deployed a new k8s and create a new mysql cluster, the same problem happened. In fact this problem happended all the time, I never had root password and custom user/password initialized, I just thought it was designed that way until I read mysql-operator source code. I don't know how this happened. I tried percona images directly with docker, the initialization was successful. I'll try creating myqlcluster with my image with custom docker-entrypoint.sh to see what happened. |
@AMecea I guess i figured out what happened: When a pod bootstaps, it may restart several times. But in the docker-entrypoint.sh, there is datadir check, if datadir is already created, it will skip initialization. Here is the log of first time bootstrap:
This is all contents of first time bootstrap log, it aborted for no reason, the rest of initialization never happened. Then pod restarted, the initialization is just skipped. Does this happened in your enviroment? |
I never encountered that to restart the pod when it's in the This situation is possible and I think the operator should handle it somehow, but yet I'm not sure how, meanwhile let's debug your situation because this should happen very rarely and only at cluster bootstrap. Maybe more details will help here, what |
My k8s cluster has only one node, I deployed it by myself witout any cloud provider. k8s version:
It seems that the I did a lot of tests and found readiness probe failure truely have an impact. I found it take a lot of time(5m40s) to finish But the reason behind it is complex and is beyond my ability. I used a nfs-client-provisioner-1.2.1 to provide storage and calico to provide network. I guess these two caused slow initialization. But after I used only pod to test, the slow initialization didn't happen. Then I tried only statefulset, the slow initialization happened again. This is wierd. I tested two statefulsets with totally different config, this one works and the other one failed. |
I think it can be solved by using a custom percona image whose docker-entrypoint.sh initializes root and custom user/password in different if-else to promote error tolerance. I'd love to talk on slack, but we're in different timezone, most time when you go to work and I'm already at rent room which has a terriable network to access slack. |
I think initialization should be done in an init container, doing so, liveness probe will not kill the container anymore. |
This issue depends on percona/percona-docker#82 |
After a series of tests, I finllay found that the default option So this is basiclly envirioment problem. I guess maybe it's better to let user to set liveness probe and readiness probe, because different enviroment may have different initialization speed. It does'nt have to be specified in ClusterSpec, I think some mysql-operator options are enough. |
have the same problem running a 1 replica cluster on GKE node - g1-small (1 vCPU, 1.7 GB memory) |
Fixed by #342 |
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
I created a mysql cluster, but the generated mysql cluster didn't initialize root with specifyed password and custom user/password were neither initialized.
mysql-operator: 0.2.2
cluster.yaml
secret.yaml
Partial of mysql node env
The text was updated successfully, but these errors were encountered: