Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] kbcli connect mysql cluster select a follower #5953

Closed
JashBook opened this issue Nov 29, 2023 · 1 comment
Closed

[BUG] kbcli connect mysql cluster select a follower #5953

JashBook opened this issue Nov 29, 2023 · 1 comment
Assignees
@xuriwuyun
Labels
bug kind/bug Something isn't working
Milestone
Release 0.8.0

Comments

@JashBook
Copy link
Collaborator

Describe the bug
kbcli connect mysql cluster select a follower.
If leader is pod-0, kbcli connect is leader.
If leader is not pod-0, kbcli connect is follower.
Maybe kbcli only selected pod-0.
kbcli 0.8.0-alpha.9

To Reproduce
Steps to reproduce the behavior:

  1. create cluster
kbcli cluster create  mysql1 --termination-policy=WipeOut  --cluster-definition=apecloud-mysql --cluster-version=ac-mysql-8.0.30 --set cpu=100m,memory=0.5Gi,replicas=3,storage=1Gi  --namespace default
  1. See error
➜  ~ kbcli cluster connect mysql1      
Connect to instance mysql1-mysql-0: out of mysql1-mysql-0, mysql1-mysql-1, mysql1-mysql-2
Defaulted container "mysql" out of: mysql, metrics, vttablet, kb-checkrole, config-manager
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 40
Server version: 8.0.30 WeSQL Server - GPL, Release 5, Revision 4ca1eb8

Copyright (c) 2000, 2022, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select * from information_schema.wesql_cluster_health;
Empty set (0.00 sec)

mysql> use mydb; DROP TABLE IF EXISTS tmp_table; CREATE TABLE IF NOT EXISTS tmp_table (id INT PRIMARY KEY AUTO_INCREMENT, value VARCHAR(255)); 
Database changed
ERROR 7504 (HY000): The consensus follower is not allowed to to do current operation.
ERROR 7504 (HY000): The consensus follower is not allowed to to do current operation.
mysql>

describe cluster

kubectl describe cluster mysql1
Name:         mysql1
Namespace:    default
Labels:       clusterdefinition.kubeblocks.io/name=apecloud-mysql
              clusterversion.kubeblocks.io/name=ac-mysql-8.0.30
Annotations:  <none>
API Version:  apps.kubeblocks.io/v1alpha1
Kind:         Cluster
Metadata:
  Creation Timestamp:  2023-11-29T06:00:29Z
  Finalizers:
    cluster.kubeblocks.io/finalizer
  Resource Version:  531210
  UID:               de4e09a1-e0fd-4899-9305-a83cf78a71ff
Spec:
  Affinity:
    Pod Anti Affinity:     Preferred
    Tenancy:               SharedNode
  Cluster Definition Ref:  apecloud-mysql
  Cluster Version Ref:     ac-mysql-8.0.30
  Component Specs:
    Component Def Ref:  mysql
    Enabled Logs:
      error
      general
      slow
    Monitor:        false
    Name:           mysql
    No Create PDB:  false
    Replicas:       3
    Resources:
      Limits:
        Cpu:     100m
        Memory:  512Mi
      Requests:
        Cpu:               100m
        Memory:            512Mi
    Service Account Name:  kb-mysql1
    Volume Claim Templates:
      Name:  data
      Spec:
        Access Modes:
          ReadWriteOnce
        Resources:
          Requests:
            Storage:  1Gi
  Monitor:
  Resources:
    Cpu:     0
    Memory:  0
  Storage:
    Size:              0
  Termination Policy:  WipeOut
Status:
  Cluster Def Generation:  2
  Components:
    Mysql:
      Message:
        Pod/mysql1-mysql-1:  Role probe timeout, check whether the application is available
        Pod/mysql1-mysql-2:  Role probe timeout, check whether the application is available
      Phase:                 Running
      Pods Ready:            true
      Pods Ready Time:       2023-11-29T06:02:56Z
  Conditions:
    Last Transition Time:  2023-11-29T06:00:29Z
    Message:               The operator has started the provisioning of Cluster: mysql1
    Observed Generation:   1
    Reason:                PreCheckSucceed
    Status:                True
    Type:                  ProvisioningStarted
    Last Transition Time:  2023-11-29T06:00:29Z
    Message:               Successfully applied for resources
    Observed Generation:   1
    Reason:                ApplyResourcesSucceed
    Status:                True
    Type:                  ApplyResources
    Last Transition Time:  2023-11-29T06:02:56Z
    Message:               all pods of components are ready, waiting for the probe detection successful
    Reason:                AllReplicasReady
    Status:                True
    Type:                  ReplicasReady
    Last Transition Time:  2023-11-29T06:02:56Z
    Message:               Cluster: mysql1 is ready, current phase is Running
    Reason:                ClusterReady
    Status:                True
    Type:                  Ready
  Observed Generation:     1
  Phase:                   Running
Events:
  Type     Reason                    Age                  From                Message
  ----     ------                    ----                 ----                -------
  Normal   PreCheckSucceed           10m                  cluster-controller  The operator has started the provisioning of Cluster: mysql1
  Normal   ApplyResourcesSucceed     10m                  cluster-controller  Successfully applied for resources
  Normal   ComponentPhaseTransition  10m                  cluster-controller  component is Creating
  Normal   ComponentPhaseTransition  10m                  cluster-controller  component is Updating
  Normal   ComponentPhaseTransition  8m36s (x2 over 10m)  cluster-controller  component is Failed
  Warning  Failed                    8m36s (x2 over 10m)  cluster-controller  Cluster: mysql1 is Failed, check according to the components message
  Warning  Abnormal                  8m34s (x2 over 10m)  cluster-controller  Cluster: mysql1 is Abnormal, check according to the components message
  Normal   ComponentPhaseTransition  8m34s                cluster-controller  component is Abnormal
  Normal   ComponentPhaseTransition  8m31s                cluster-controller  component is Running
  Normal   AllReplicasReady          8m31s                cluster-controller  all pods of components are ready, waiting for the probe detection successful
  Normal   ClusterReady              8m31s                cluster-controller  Cluster: mysql1 is ready, current phase is Running
  Normal   Running                   8m31s                cluster-controller  Cluster: mysql1 is ready, current phase is Running

describe pod


➜  ~ kubectl describe pod mysql1-mysql-0
Name:         mysql1-mysql-0
Namespace:    default
Priority:     0
Node:         minikube/192.168.49.2
Start Time:   Wed, 29 Nov 2023 14:00:30 +0800
Labels:       app.kubernetes.io/component=mysql
              app.kubernetes.io/instance=mysql1
              app.kubernetes.io/managed-by=kubeblocks
              app.kubernetes.io/name=mysql
              app.kubernetes.io/version=mysql
              apps.kubeblocks.io/component-name=mysql
              controller-revision-hash=mysql1-mysql-58fbdcb66b
              kubeblocks.io/role=follower
              rsm.workloads.kubeblocks.io/access-mode=Readonly
              statefulset.kubernetes.io/pod-name=mysql1-mysql-0
Annotations:  apps.kubeblocks.io/component-replicas: 3
              apps.kubeblocks.io/last-role-snapshot-version: 2023-11-29T06:02:51.19652Z
Status:       Running
IP:           10.244.11.116
IPs:
  IP:           10.244.11.116
Controlled By:  StatefulSet/mysql1-mysql
Containers:
  mysql:
    Container ID:  docker://b9dff41885ea6f8f9ed3ebcdafc46b8ed9a7f72d0016563d0ea772a9b5b69764
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server:8.0.30-5.beta2.20230830.g4ca1eb8.13
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server@sha256:0f58f0cfac74cc502c7865c03ee53aead3bbb26fb9afb40bbef629fe47c0a3d6
    Ports:         3306/TCP, 13306/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /scripts/setup.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:01:00 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     100m
      memory:  512Mi
    Requests:
      cpu:     100m
      memory:  512Mi
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-0 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      SERVICE_PORT:              3306
      MYSQL_ROOT_HOST:           %
      MYSQL_ROOT_USER:           <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:       <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_DATABASE:            mydb
      MYSQL_USER:                u1
      MYSQL_PASSWORD:            u1
      CLUSTER_ID:                1
      CLUSTER_START_INDEX:       1
      REPLICATION_USER:          replicator
      REPLICATION_PASSWORD:      
      MYSQL_TEMPLATE_CONFIG:     
      MYSQL_CUSTOM_CONFIG:       
      MYSQL_DYNAMIC_CONFIG:      
      KB_EMBEDDED_WESQL:         1
    Mounts:
      /data/mysql from data (rw)
      /etc/annotations from annotations (rw)
      /opt/mysql from mysql-config (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-sxs9c (ro)
  metrics:
    Container ID:  docker://913589d4386f10e279d8d923d2b80d9f37be22fa43b6147a71a075095bd5c49d
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto@sha256:cbab349b90490807a8d5039bf01bc7e37334f20c98c7dd75bc7fc4cf9e5b10ee
    Port:          9104/TCP
    Host Port:     0/TCP
    Command:
      /scripts/agamotto.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:01:00 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-0 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      DB_TYPE:                   MySQL
      ENDPOINT:                  localhost:3306
      MYSQL_USER:                <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_PASSWORD:            <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
    Mounts:
      /data/mysql from data (rw)
      /opt/agamotto from agamotto-configuration (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-sxs9c (ro)
  vttablet:
    Container ID:  docker://c651cccbf4e63610918950800e1005fe6aa9152a659b52b14c42a5e2fadb2529
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale:0.1.1
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale@sha256:d391dde6e2f3f6a6056a4c41fd7e255f71ae94618b08e6afdd9527f6892a761b
    Ports:         15100/TCP, 16100/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /scripts/vttablet.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-0 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      CELL:                      zone1
      ETCD_SERVER:               $(KB_CLUSTER_NAME)-vtcontroller-headless
      ETCD_PORT:                 2379
      TOPOLOGY_FLAGS:            --topo_implementation etcd2 --topo_global_server_address $(ETCD_SERVER):$(ETCD_PORT) --topo_global_root /vitess/global
      VTTABLET_PORT:             15100
      VTTABLET_GRPC_PORT:        16100
      VTCTLD_HOST:               $(KB_CLUSTER_NAME)-vtcontroller-headless
      VTCTLD_WEB_PORT:           15000
      SERVICE_PORT:              $(VTTABLET_PORT)
      MYSQL_ROOT_USER:           <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:       <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
    Mounts:
      /conf from mysql-scale-config (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-sxs9c (ro)
      /vtdataroot from data (rw)
  kb-checkrole:
    Container ID:  docker://ddd0b0af8adad6bca706cfd9a540f41ffd71ffd7dce9617c7b43c92dfb0005db
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:b916d33dce53159198d6deffe15ceef56b92cd382e3357c4457c838735b52181
    Ports:         3501/TCP, 50001/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      lorry
      --port
      3501
      --config-path
      /config/lorry/components/
      --grpcport
      50001
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:      0
      memory:   0
    Readiness:  http-get http://:3501/v1.0/checkrole delay=0s timeout=1s period=1s #success=1 #failure=2
    Startup:    tcp-socket :3501 delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:                   mysql1-mysql-0 (v1:metadata.name)
      KB_POD_UID:                     (v1:metadata.uid)
      KB_NAMESPACE:                  default (v1:metadata.namespace)
      KB_SA_NAME:                     (v1:spec.serviceAccountName)
      KB_NODENAME:                    (v1:spec.nodeName)
      KB_HOST_IP:                     (v1:status.hostIP)
      KB_POD_IP:                      (v1:status.podIP)
      KB_POD_IPS:                     (v1:status.podIPs)
      KB_HOSTIP:                      (v1:status.hostIP)
      KB_PODIP:                       (v1:status.podIP)
      KB_PODIPS:                      (v1:status.podIPs)
      KB_CLUSTER_NAME:               mysql1
      KB_COMP_NAME:                  mysql
      KB_CLUSTER_COMP_NAME:          mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:      f78a71ff
      KB_POD_FQDN:                   $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      KB_SERVICE_USER:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PASSWORD:           <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PORT:               3306
      KB_DATA_PATH:                  /data/mysql
      KB_BUILTIN_HANDLER:            wesql
      KB_SERVICE_USER:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PASSWORD:           <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_USERNAME:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_PASSWORD:               <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_ACTION_SVC_LIST:        null
      KB_RSM_ROLE_UPDATE_MECHANISM:  DirectAPIServerEventUpdate
      KB_RSM_ROLE_PROBE_TIMEOUT:     1
      KB_SERVICE_CHARACTER_TYPE:     wesql
    Mounts:
      /data/mysql from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-sxs9c (ro)
  config-manager:
    Container ID:  docker://9c7f1e5af4ed9429fef35cb6005ef0334c414fe480300c1d864acc10610f415c
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:b916d33dce53159198d6deffe15ceef56b92cd382e3357c4457c838735b52181
    Port:          <none>
    Host Port:     <none>
    Command:
      env
    Args:
      PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$(TOOLS_PATH)
      /bin/reloader
      --log-level
      info
      --operator-update-enable
      --tcp
      9901
      --config
      /opt/config-manager/config-manager.yaml
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-0 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      KB_POD_NAME:               mysql1-mysql-0 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      CONFIG_MANAGER_POD_IP:      (v1:status.podIP)
      DB_TYPE:                   mysql
      MYSQL_USER:                <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_PASSWORD:            <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      DATA_SOURCE_NAME:          $(MYSQL_USER):$(MYSQL_PASSWORD)@(localhost:3306)/
      TOOLS_PATH:                /opt/kb-tools/reload/mysql-consensusset-config:/opt/config-manager
    Mounts:
      /conf from mysql-scale-config (rw)
      /opt/config-manager from config-manager-config (rw)
      /opt/kb-tools/reload/mysql-consensusset-config from cm-script-mysql-consensusset-config (rw)
      /opt/mysql from mysql-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-sxs9c (ro)
Events:
  Type     Reason                  Age                From                     Message
  ----     ------                  ----               ----                     -------
  Normal   Scheduled               11m                default-scheduler        Successfully assigned default/mysql1-mysql-0 to minikube
  Normal   SuccessfulAttachVolume  11m                attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-cf8028a0-d36b-4471-a38e-bd302a5296a5"
  Warning  Failed                  10m                kubelet                  Error: failed to sync configmap cache: timed out waiting for the condition
  Normal   Pulled                  10m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale:0.1.1" already present on machine
  Warning  Failed                  10m                kubelet                  Error: failed to sync secret cache: timed out waiting for the condition
  Normal   Started                 10m                kubelet                  Started container config-manager
  Normal   Pulled                  10m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9" already present on machine
  Normal   Created                 10m                kubelet                  Created container vttablet
  Normal   Started                 10m                kubelet                  Started container vttablet
  Normal   Pulled                  10m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9" already present on machine
  Normal   Created                 10m                kubelet                  Created container config-manager
  Normal   Started                 10m                kubelet                  Started container kb-checkrole
  Normal   Created                 10m                kubelet                  Created container kb-checkrole
  Normal   Pulled                  10m (x2 over 10m)  kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server:8.0.30-5.beta2.20230830.g4ca1eb8.13" already present on machine
  Normal   Created                 10m                kubelet                  Created container mysql
  Normal   Pulled                  10m (x2 over 10m)  kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1" already present on machine
  Normal   Started                 10m                kubelet                  Started container mysql
  Normal   Created                 10m                kubelet                  Created container metrics
  Normal   Started                 10m                kubelet                  Started container metrics
  Normal   checkRole               9m3s               lorry                    {"event":"Success","operation":"checkRole","originalRole":"","role":"Follower"}

➜  ~ kubectl describe pod mysql1-mysql-1
Name:         mysql1-mysql-1
Namespace:    default
Priority:     0
Node:         minikube/192.168.49.2
Start Time:   Wed, 29 Nov 2023 14:00:30 +0800
Labels:       app.kubernetes.io/component=mysql
              app.kubernetes.io/instance=mysql1
              app.kubernetes.io/managed-by=kubeblocks
              app.kubernetes.io/name=mysql
              app.kubernetes.io/version=mysql
              apps.kubeblocks.io/component-name=mysql
              controller-revision-hash=mysql1-mysql-58fbdcb66b
              kubeblocks.io/role=leader
              rsm.workloads.kubeblocks.io/access-mode=ReadWrite
              statefulset.kubernetes.io/pod-name=mysql1-mysql-1
Annotations:  apps.kubeblocks.io/component-replicas: 3
              apps.kubeblocks.io/last-role-snapshot-version: 2023-11-29T06:02:51.591686Z
Status:       Running
IP:           10.244.11.113
IPs:
  IP:           10.244.11.113
Controlled By:  StatefulSet/mysql1-mysql
Containers:
  mysql:
    Container ID:  docker://72e7a2198d47e88492da50ed481aacec8e8e36f1bd59abbfbaacdd9b6a0c1d66
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server:8.0.30-5.beta2.20230830.g4ca1eb8.13
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server@sha256:0f58f0cfac74cc502c7865c03ee53aead3bbb26fb9afb40bbef629fe47c0a3d6
    Ports:         3306/TCP, 13306/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /scripts/setup.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:01:00 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     100m
      memory:  512Mi
    Requests:
      cpu:     100m
      memory:  512Mi
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-1 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      SERVICE_PORT:              3306
      MYSQL_ROOT_HOST:           %
      MYSQL_ROOT_USER:           <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:       <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_DATABASE:            mydb
      MYSQL_USER:                u1
      MYSQL_PASSWORD:            u1
      CLUSTER_ID:                1
      CLUSTER_START_INDEX:       1
      REPLICATION_USER:          replicator
      REPLICATION_PASSWORD:      
      MYSQL_TEMPLATE_CONFIG:     
      MYSQL_CUSTOM_CONFIG:       
      MYSQL_DYNAMIC_CONFIG:      
      KB_EMBEDDED_WESQL:         1
    Mounts:
      /data/mysql from data (rw)
      /etc/annotations from annotations (rw)
      /opt/mysql from mysql-config (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nl77h (ro)
  metrics:
    Container ID:  docker://615cdb3a2b995b386b6332dc9be488de213f2b681b9d961178c7031c49ef5aeb
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto@sha256:cbab349b90490807a8d5039bf01bc7e37334f20c98c7dd75bc7fc4cf9e5b10ee
    Port:          9104/TCP
    Host Port:     0/TCP
    Command:
      /scripts/agamotto.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:01:00 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-1 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      DB_TYPE:                   MySQL
      ENDPOINT:                  localhost:3306
      MYSQL_USER:                <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_PASSWORD:            <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
    Mounts:
      /data/mysql from data (rw)
      /opt/agamotto from agamotto-configuration (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nl77h (ro)
  vttablet:
    Container ID:  docker://4f1e01f1acf78c9de7e1f58c95cbdf7de5fdaa2ff825cc1a84be4fe484c8cb69
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale:0.1.1
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale@sha256:d391dde6e2f3f6a6056a4c41fd7e255f71ae94618b08e6afdd9527f6892a761b
    Ports:         15100/TCP, 16100/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /scripts/vttablet.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-1 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      CELL:                      zone1
      ETCD_SERVER:               $(KB_CLUSTER_NAME)-vtcontroller-headless
      ETCD_PORT:                 2379
      TOPOLOGY_FLAGS:            --topo_implementation etcd2 --topo_global_server_address $(ETCD_SERVER):$(ETCD_PORT) --topo_global_root /vitess/global
      VTTABLET_PORT:             15100
      VTTABLET_GRPC_PORT:        16100
      VTCTLD_HOST:               $(KB_CLUSTER_NAME)-vtcontroller-headless
      VTCTLD_WEB_PORT:           15000
      SERVICE_PORT:              $(VTTABLET_PORT)
      MYSQL_ROOT_USER:           <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:       <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
    Mounts:
      /conf from mysql-scale-config (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nl77h (ro)
      /vtdataroot from data (rw)
  kb-checkrole:
    Container ID:  docker://51ce63ca6ff7c254c91308d3842e0843fae8623bcf5199b788b4f08889f02c7c
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:b916d33dce53159198d6deffe15ceef56b92cd382e3357c4457c838735b52181
    Ports:         3501/TCP, 50001/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      lorry
      --port
      3501
      --config-path
      /config/lorry/components/
      --grpcport
      50001
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:      0
      memory:   0
    Readiness:  http-get http://:3501/v1.0/checkrole delay=0s timeout=1s period=1s #success=1 #failure=2
    Startup:    tcp-socket :3501 delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:                   mysql1-mysql-1 (v1:metadata.name)
      KB_POD_UID:                     (v1:metadata.uid)
      KB_NAMESPACE:                  default (v1:metadata.namespace)
      KB_SA_NAME:                     (v1:spec.serviceAccountName)
      KB_NODENAME:                    (v1:spec.nodeName)
      KB_HOST_IP:                     (v1:status.hostIP)
      KB_POD_IP:                      (v1:status.podIP)
      KB_POD_IPS:                     (v1:status.podIPs)
      KB_HOSTIP:                      (v1:status.hostIP)
      KB_PODIP:                       (v1:status.podIP)
      KB_PODIPS:                      (v1:status.podIPs)
      KB_CLUSTER_NAME:               mysql1
      KB_COMP_NAME:                  mysql
      KB_CLUSTER_COMP_NAME:          mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:      f78a71ff
      KB_POD_FQDN:                   $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      KB_SERVICE_USER:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PASSWORD:           <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PORT:               3306
      KB_DATA_PATH:                  /data/mysql
      KB_BUILTIN_HANDLER:            wesql
      KB_SERVICE_USER:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PASSWORD:           <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_USERNAME:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_PASSWORD:               <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_ACTION_SVC_LIST:        null
      KB_RSM_ROLE_UPDATE_MECHANISM:  DirectAPIServerEventUpdate
      KB_RSM_ROLE_PROBE_TIMEOUT:     1
      KB_SERVICE_CHARACTER_TYPE:     wesql
    Mounts:
      /data/mysql from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nl77h (ro)
  config-manager:
    Container ID:  docker://1ca710fdb4a2a64404f13193cd287285937023b43578fc70600f94ce183822cb
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:b916d33dce53159198d6deffe15ceef56b92cd382e3357c4457c838735b52181
    Port:          <none>
    Host Port:     <none>
    Command:
      env
    Args:
      PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$(TOOLS_PATH)
      /bin/reloader
      --log-level
      info
      --operator-update-enable
      --tcp
      9901
      --config
      /opt/config-manager/config-manager.yaml
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-1 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      KB_POD_NAME:               mysql1-mysql-1 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      CONFIG_MANAGER_POD_IP:      (v1:status.podIP)
      DB_TYPE:                   mysql
      MYSQL_USER:                <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_PASSWORD:            <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      DATA_SOURCE_NAME:          $(MYSQL_USER):$(MYSQL_PASSWORD)@(localhost:3306)/
      TOOLS_PATH:                /opt/kb-tools/reload/mysql-consensusset-config:/opt/config-manager
    Mounts:
      /conf from mysql-scale-config (rw)
      /opt/config-manager from config-manager-config (rw)
      /opt/kb-tools/reload/mysql-consensusset-config from cm-script-mysql-consensusset-config (rw)
      /opt/mysql from mysql-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nl77h (ro)
Events:
  Type     Reason                  Age                From                     Message
  ----     ------                  ----               ----                     -------
  Normal   Scheduled               11m                default-scheduler        Successfully assigned default/mysql1-mysql-1 to minikube
  Normal   SuccessfulAttachVolume  11m                attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-7885ffc3-a9f7-4213-ad23-38da06abadca"
  Warning  Failed                  11m                kubelet                  Error: failed to sync configmap cache: timed out waiting for the condition
  Normal   Pulled                  10m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale:0.1.1" already present on machine
  Warning  Failed                  10m                kubelet                  Error: failed to sync secret cache: timed out waiting for the condition
  Normal   Started                 10m                kubelet                  Started container config-manager
  Normal   Pulled                  10m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9" already present on machine
  Normal   Created                 10m                kubelet                  Created container vttablet
  Normal   Started                 10m                kubelet                  Started container vttablet
  Normal   Pulled                  10m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9" already present on machine
  Normal   Created                 10m                kubelet                  Created container config-manager
  Normal   Started                 10m                kubelet                  Started container kb-checkrole
  Normal   Created                 10m                kubelet                  Created container kb-checkrole
  Normal   Pulled                  10m (x2 over 11m)  kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server:8.0.30-5.beta2.20230830.g4ca1eb8.13" already present on machine
  Normal   Created                 10m                kubelet                  Created container mysql
  Normal   Pulled                  10m (x2 over 11m)  kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1" already present on machine
  Normal   Started                 10m                kubelet                  Started container mysql
  Normal   Created                 10m                kubelet                  Created container metrics
  Normal   Started                 10m                kubelet                  Started container metrics
  Normal   checkRole               9m5s               lorry                    {"event":"Success","operation":"checkRole","originalRole":"","role":"Leader"}
➜  ~ 
➜  ~ 
➜  ~ kubectl describe pod mysql1-mysql-2
Name:         mysql1-mysql-2
Namespace:    default
Priority:     0
Node:         minikube/192.168.49.2
Start Time:   Wed, 29 Nov 2023 14:00:30 +0800
Labels:       app.kubernetes.io/component=mysql
              app.kubernetes.io/instance=mysql1
              app.kubernetes.io/managed-by=kubeblocks
              app.kubernetes.io/name=mysql
              app.kubernetes.io/version=mysql
              apps.kubeblocks.io/component-name=mysql
              controller-revision-hash=mysql1-mysql-58fbdcb66b
              kubeblocks.io/role=follower
              rsm.workloads.kubeblocks.io/access-mode=Readonly
              statefulset.kubernetes.io/pod-name=mysql1-mysql-2
Annotations:  apps.kubeblocks.io/component-replicas: 3
              apps.kubeblocks.io/last-role-snapshot-version: 2023-11-29T06:02:54.59503Z
Status:       Running
IP:           10.244.11.114
IPs:
  IP:           10.244.11.114
Controlled By:  StatefulSet/mysql1-mysql
Containers:
  mysql:
    Container ID:  docker://5e165447b4c373a6776f15b9cd205aba5daba0190ac364f68915b991cb9e49c4
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server:8.0.30-5.beta2.20230830.g4ca1eb8.13
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server@sha256:0f58f0cfac74cc502c7865c03ee53aead3bbb26fb9afb40bbef629fe47c0a3d6
    Ports:         3306/TCP, 13306/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /scripts/setup.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:01:00 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     100m
      memory:  512Mi
    Requests:
      cpu:     100m
      memory:  512Mi
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-2 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      SERVICE_PORT:              3306
      MYSQL_ROOT_HOST:           %
      MYSQL_ROOT_USER:           <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:       <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_DATABASE:            mydb
      MYSQL_USER:                u1
      MYSQL_PASSWORD:            u1
      CLUSTER_ID:                1
      CLUSTER_START_INDEX:       1
      REPLICATION_USER:          replicator
      REPLICATION_PASSWORD:      
      MYSQL_TEMPLATE_CONFIG:     
      MYSQL_CUSTOM_CONFIG:       
      MYSQL_DYNAMIC_CONFIG:      
      KB_EMBEDDED_WESQL:         1
    Mounts:
      /data/mysql from data (rw)
      /etc/annotations from annotations (rw)
      /opt/mysql from mysql-config (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ph6wk (ro)
  metrics:
    Container ID:  docker://81d1bdf8bda6ceb59c36b22529014fc1befe853600f960760d0a3271882262dd
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto@sha256:cbab349b90490807a8d5039bf01bc7e37334f20c98c7dd75bc7fc4cf9e5b10ee
    Port:          9104/TCP
    Host Port:     0/TCP
    Command:
      /scripts/agamotto.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:01:00 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-2 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      DB_TYPE:                   MySQL
      ENDPOINT:                  localhost:3306
      MYSQL_USER:                <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_PASSWORD:            <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
    Mounts:
      /data/mysql from data (rw)
      /opt/agamotto from agamotto-configuration (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ph6wk (ro)
  vttablet:
    Container ID:  docker://fd30d7f925e1acbb81a6c0c00ef2dd82dfaa7448bceea576813fa2d5f4893644
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale:0.1.1
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale@sha256:d391dde6e2f3f6a6056a4c41fd7e255f71ae94618b08e6afdd9527f6892a761b
    Ports:         15100/TCP, 16100/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /scripts/vttablet.sh
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-2 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      CELL:                      zone1
      ETCD_SERVER:               $(KB_CLUSTER_NAME)-vtcontroller-headless
      ETCD_PORT:                 2379
      TOPOLOGY_FLAGS:            --topo_implementation etcd2 --topo_global_server_address $(ETCD_SERVER):$(ETCD_PORT) --topo_global_root /vitess/global
      VTTABLET_PORT:             15100
      VTTABLET_GRPC_PORT:        16100
      VTCTLD_HOST:               $(KB_CLUSTER_NAME)-vtcontroller-headless
      VTCTLD_WEB_PORT:           15000
      SERVICE_PORT:              $(VTTABLET_PORT)
      MYSQL_ROOT_USER:           <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:       <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
    Mounts:
      /conf from mysql-scale-config (rw)
      /scripts from scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ph6wk (ro)
      /vtdataroot from data (rw)
  kb-checkrole:
    Container ID:  docker://f95f981dd078c6903fe8280db913bbce5fcb726355da8b20600f350ebbaa224a
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:b916d33dce53159198d6deffe15ceef56b92cd382e3357c4457c838735b52181
    Ports:         3501/TCP, 50001/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      lorry
      --port
      3501
      --config-path
      /config/lorry/components/
      --grpcport
      50001
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:      0
      memory:   0
    Readiness:  http-get http://:3501/v1.0/checkrole delay=0s timeout=1s period=1s #success=1 #failure=2
    Startup:    tcp-socket :3501 delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:                   mysql1-mysql-2 (v1:metadata.name)
      KB_POD_UID:                     (v1:metadata.uid)
      KB_NAMESPACE:                  default (v1:metadata.namespace)
      KB_SA_NAME:                     (v1:spec.serviceAccountName)
      KB_NODENAME:                    (v1:spec.nodeName)
      KB_HOST_IP:                     (v1:status.hostIP)
      KB_POD_IP:                      (v1:status.podIP)
      KB_POD_IPS:                     (v1:status.podIPs)
      KB_HOSTIP:                      (v1:status.hostIP)
      KB_PODIP:                       (v1:status.podIP)
      KB_PODIPS:                      (v1:status.podIPs)
      KB_CLUSTER_NAME:               mysql1
      KB_COMP_NAME:                  mysql
      KB_CLUSTER_COMP_NAME:          mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:      f78a71ff
      KB_POD_FQDN:                   $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      KB_SERVICE_USER:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PASSWORD:           <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PORT:               3306
      KB_DATA_PATH:                  /data/mysql
      KB_BUILTIN_HANDLER:            wesql
      KB_SERVICE_USER:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_SERVICE_PASSWORD:           <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_USERNAME:               <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_PASSWORD:               <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      KB_RSM_ACTION_SVC_LIST:        null
      KB_RSM_ROLE_UPDATE_MECHANISM:  DirectAPIServerEventUpdate
      KB_RSM_ROLE_PROBE_TIMEOUT:     1
      KB_SERVICE_CHARACTER_TYPE:     wesql
    Mounts:
      /data/mysql from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ph6wk (ro)
  config-manager:
    Container ID:  docker://60f22eb96840c8c1543b6fd3ac84bd63d930a9d823125ab4ee19b731248567e5
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9
    Image ID:      docker-pullable://infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:b916d33dce53159198d6deffe15ceef56b92cd382e3357c4457c838735b52181
    Port:          <none>
    Host Port:     <none>
    Command:
      env
    Args:
      PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$(TOOLS_PATH)
      /bin/reloader
      --log-level
      info
      --operator-update-enable
      --tcp
      9901
      --config
      /opt/config-manager/config-manager.yaml
    State:          Running
      Started:      Wed, 29 Nov 2023 14:00:58 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysql1-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:               mysql1-mysql-2 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      KB_POD_NAME:               mysql1-mysql-2 (v1:metadata.name)
      KB_POD_UID:                 (v1:metadata.uid)
      KB_NAMESPACE:              default (v1:metadata.namespace)
      KB_SA_NAME:                 (v1:spec.serviceAccountName)
      KB_NODENAME:                (v1:spec.nodeName)
      KB_HOST_IP:                 (v1:status.hostIP)
      KB_POD_IP:                  (v1:status.podIP)
      KB_POD_IPS:                 (v1:status.podIPs)
      KB_HOSTIP:                  (v1:status.hostIP)
      KB_PODIP:                   (v1:status.podIP)
      KB_PODIPS:                  (v1:status.podIPs)
      KB_CLUSTER_NAME:           mysql1
      KB_COMP_NAME:              mysql
      KB_CLUSTER_COMP_NAME:      mysql1-mysql
      KB_CLUSTER_UID_POSTFIX_8:  f78a71ff
      KB_POD_FQDN:               $(KB_POD_NAME).$(KB_CLUSTER_COMP_NAME)-headless.$(KB_NAMESPACE).svc
      CONFIG_MANAGER_POD_IP:      (v1:status.podIP)
      DB_TYPE:                   mysql
      MYSQL_USER:                <set to the key 'username' in secret 'mysql1-conn-credential'>  Optional: false
      MYSQL_PASSWORD:            <set to the key 'password' in secret 'mysql1-conn-credential'>  Optional: false
      DATA_SOURCE_NAME:          $(MYSQL_USER):$(MYSQL_PASSWORD)@(localhost:3306)/
      TOOLS_PATH:                /opt/kb-tools/reload/mysql-consensusset-config:/opt/config-manager
    Mounts:
      /conf from mysql-scale-config (rw)
      /opt/config-manager from config-manager-config (rw)
      /opt/kb-tools/reload/mysql-consensusset-config from cm-script-mysql-consensusset-config (rw)
      /opt/mysql from mysql-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ph6wk (ro)
Events:
  Type     Reason                  Age                From                     Message
  ----     ------                  ----               ----                     -------
  Normal   Scheduled               11m                default-scheduler        Successfully assigned default/mysql1-mysql-2 to minikube
  Normal   SuccessfulAttachVolume  11m                attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-daa6c8a5-2cf0-4ae5-825a-a4f5e7a6f2e0"
  Warning  Failed                  11m                kubelet                  Error: failed to sync configmap cache: timed out waiting for the condition
  Normal   Pulled                  11m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-scale:0.1.1" already present on machine
  Warning  Failed                  11m                kubelet                  Error: failed to sync secret cache: timed out waiting for the condition
  Normal   Started                 11m                kubelet                  Started container config-manager
  Normal   Pulled                  11m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9" already present on machine
  Normal   Created                 11m                kubelet                  Created container vttablet
  Normal   Started                 11m                kubelet                  Started container vttablet
  Normal   Pulled                  11m                kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.0-alpha.9" already present on machine
  Normal   Created                 11m                kubelet                  Created container config-manager
  Normal   Started                 11m                kubelet                  Started container kb-checkrole
  Normal   Created                 11m                kubelet                  Created container kb-checkrole
  Normal   Pulled                  11m (x2 over 11m)  kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/apecloud-mysql-server:8.0.30-5.beta2.20230830.g4ca1eb8.13" already present on machine
  Normal   Created                 11m                kubelet                  Created container mysql
  Normal   Pulled                  10m (x2 over 11m)  kubelet                  Container image "infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1" already present on machine
  Normal   Started                 10m                kubelet                  Started container mysql
  Normal   Created                 10m                kubelet                  Created container metrics
  Normal   Started                 10m                kubelet                  Started container metrics
  Normal   checkRole               9m5s               lorry                    {"event":"Success","operation":"checkRole","originalRole":"","role":"Follower"}
➜  ~

Expected behavior
kbcli connect mysql cluster select leader.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.
@JashBook JashBook added the kind/bug Something isn't working label Nov 29, 2023
@JashBook JashBook added this to the Release 0.8.0 milestone Nov 29, 2023
@xuriwuyun xuriwuyun linked a pull request Dec 14, 2023 that will close this issue
chore: improve Lorry performance #6109
Merged
@xuriwuyun xuriwuyun removed a link to a pull request Dec 14, 2023
chore: improve Lorry performance #6109
Merged
@xuriwuyun
Copy link
Contributor

image
connect as expected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xuriwuyun xuriwuyun

Labels
bug kind/bug Something isn't working
Projects
None yet
Milestone
Release 0.8.0
Development

No branches or pull requests
3 participants
@xuriwuyun @JashBook @apecloud-bot