controller-0: 11:06:45 INFO juju.worker.apicaller [78c956] "controller-0" successfully connected to "localhost:17070" controller-0: 11:06:45 INFO juju.worker.logforwarder config change - log forwarding not enabled controller-0: 11:06:45 INFO juju.worker.logger logger worker started controller-0: 11:06:45 INFO juju.worker.pruner.statushistory pruner config: max age: 336h0m0s, max collection size 5120M for foo6 (78c95646-7873-4e34-8022-882c1bd46b52) controller-0: 11:06:45 INFO juju.worker.pruner.action pruner config: max age: 336h0m0s, max collection size 5120M for foo6 (78c95646-7873-4e34-8022-882c1bd46b52) model-78c95646-7873-4e34-8022-882c1bd46b52: 11:06:48 INFO juju.worker.caasupgrader abort check blocked until version event received model-78c95646-7873-4e34-8022-882c1bd46b52: 11:06:48 INFO juju.worker.caasupgrader unblocking abort check model-78c95646-7873-4e34-8022-882c1bd46b52: 11:06:48 INFO juju.worker.muxhttpserver starting http server on [::]:17071 model-78c95646-7873-4e34-8022-882c1bd46b52: 11:06:48 INFO juju.worker.caasadmission ensuring model k8s webhook configurations controller-0: 11:06:52 INFO juju.worker.caasapplicationprovisioner.runner start "mysql-k8s" controller-0: 11:06:52 INFO juju.worker.caasapplicationprovisioner.mysql-k8s scaling application "mysql-k8s" to desired scale 3 controller-0: 11:06:52 INFO juju.worker.caasapplicationprovisioner.mysql-k8s scaling application "mysql-k8s" to desired scale 3 controller-0: 11:06:56 INFO juju.worker.caasapplicationprovisioner.runner start "mysql-test-app" controller-0: 11:06:58 INFO juju.worker.caasapplicationprovisioner.mysql-k8s scaling application "mysql-k8s" to desired scale 3 controller-0: 11:06:59 INFO juju.worker.caasapplicationprovisioner.mysql-test-app scaling application "mysql-test-app" to desired scale 1 controller-0: 11:06:59 INFO juju.worker.caasapplicationprovisioner.mysql-test-app scaling application "mysql-test-app" to desired scale 1 controller-0: 11:06:59 INFO juju.worker.caasapplicationprovisioner.mysql-test-app scaling application "mysql-test-app" to desired scale 1 unit-mysql-k8s-2: 11:07:04 INFO juju.cmd running containerAgent [3.1.7 0cd207d999fef1fc8b965c410e9f58fafe7ee335 gc go1.21.5] unit-mysql-k8s-2: 11:07:04 INFO juju.cmd.containeragent.unit start "unit" unit-mysql-k8s-2: 11:07:04 INFO juju.worker.upgradesteps upgrade steps for 3.1.7 have already been run. unit-mysql-k8s-1: 11:07:04 INFO juju.cmd running containerAgent [3.1.7 0cd207d999fef1fc8b965c410e9f58fafe7ee335 gc go1.21.5] unit-mysql-k8s-1: 11:07:04 INFO juju.cmd.containeragent.unit start "unit" unit-mysql-k8s-1: 11:07:04 INFO juju.worker.upgradesteps upgrade steps for 3.1.7 have already been run. unit-mysql-k8s-1: 11:07:04 INFO juju.worker.probehttpserver starting http server on 127.0.0.1:65301 unit-mysql-k8s-2: 11:07:04 INFO juju.worker.probehttpserver starting http server on 127.0.0.1:65301 unit-mysql-k8s-0: 11:07:04 INFO juju.cmd running containerAgent [3.1.7 0cd207d999fef1fc8b965c410e9f58fafe7ee335 gc go1.21.5] unit-mysql-k8s-0: 11:07:04 INFO juju.cmd.containeragent.unit start "unit" unit-mysql-k8s-0: 11:07:04 INFO juju.worker.upgradesteps upgrade steps for 3.1.7 have already been run. unit-mysql-k8s-0: 11:07:04 INFO juju.worker.probehttpserver starting http server on 127.0.0.1:65301 unit-mysql-k8s-2: 11:07:04 INFO juju.api connection established to "wss://controller-service.controller-micro.svc.cluster.local:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-k8s-1: 11:07:04 INFO juju.api connection established to "wss://controller-service.controller-micro.svc.cluster.local:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-k8s-2: 11:07:04 INFO juju.worker.apicaller [78c956] "unit-mysql-k8s-2" successfully connected to "controller-service.controller-micro.svc.cluster.local:17070" unit-mysql-k8s-1: 11:07:04 INFO juju.worker.apicaller [78c956] "unit-mysql-k8s-1" successfully connected to "controller-service.controller-micro.svc.cluster.local:17070" unit-mysql-k8s-0: 11:07:04 INFO juju.api cannot resolve "controller-service.controller-micro.svc.cluster.local": lookup controller-service.controller-micro.svc.cluster.local: operation was canceled unit-mysql-k8s-0: 11:07:04 INFO juju.api connection established to "wss://10.152.183.100:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-k8s-0: 11:07:04 INFO juju.worker.apicaller [78c956] "unit-mysql-k8s-0" successfully connected to "10.152.183.100:17070" unit-mysql-k8s-2: 11:07:04 INFO juju.worker.migrationminion migration phase is now: NONE unit-mysql-k8s-1: 11:07:04 INFO juju.api connection established to "wss://controller-service.controller-micro.svc.cluster.local:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-k8s-2: 11:07:04 INFO juju.worker.logger logger worker started unit-mysql-k8s-2: 11:07:04 INFO juju.worker.leadership mysql-k8s/2 promoted to leadership of mysql-k8s unit-mysql-k8s-2: 11:07:04 WARNING juju.worker.proxyupdater unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" unit-mysql-k8s-0: 11:07:04 INFO juju.worker.migrationminion migration phase is now: NONE unit-mysql-k8s-0: 11:07:04 INFO juju.worker.logger logger worker started unit-mysql-k8s-1: 11:07:04 INFO juju.worker.apicaller [78c956] "unit-mysql-k8s-1" successfully connected to "controller-service.controller-micro.svc.cluster.local:17070" unit-mysql-k8s-0: 11:07:04 INFO juju.worker.leadership mysql-k8s leadership for mysql-k8s/0 denied unit-mysql-k8s-0: 11:07:04 WARNING juju.worker.proxyupdater unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" unit-mysql-k8s-2: 11:07:04 INFO juju.worker.caasupgrader abort check blocked until version event received unit-mysql-k8s-2: 11:07:04 INFO juju.worker.caasupgrader unblocking abort check unit-mysql-k8s-1: 11:07:04 INFO juju.worker.migrationminion migration phase is now: NONE unit-mysql-k8s-0: 11:07:04 INFO juju.worker.caasupgrader abort check blocked until version event received unit-mysql-k8s-0: 11:07:04 INFO juju.worker.caasupgrader unblocking abort check unit-mysql-k8s-1: 11:07:04 INFO juju.worker.logger logger worker started unit-mysql-k8s-1: 11:07:04 INFO juju.worker.leadership mysql-k8s leadership for mysql-k8s/1 denied unit-mysql-k8s-1: 11:07:04 WARNING juju.worker.proxyupdater unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" unit-mysql-k8s-2: 11:07:04 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-mysql-k8s-2 unit-mysql-k8s-0: 11:07:04 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-mysql-k8s-0 unit-mysql-k8s-1: 11:07:04 INFO juju.worker.caasupgrader abort check blocked until version event received unit-mysql-k8s-1: 11:07:04 INFO juju.worker.caasupgrader unblocking abort check unit-mysql-k8s-1: 11:07:04 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-mysql-k8s-1 unit-mysql-k8s-2: 11:07:04 INFO juju.worker.uniter unit "mysql-k8s/2" started unit-mysql-k8s-0: 11:07:04 INFO juju.worker.uniter unit "mysql-k8s/0" started unit-mysql-k8s-0: 11:07:04 INFO juju.worker.uniter resuming charm install unit-mysql-k8s-0: 11:07:04 INFO juju.worker.uniter.charm downloading ch:amd64/jammy/mysql-k8s-113 from API server unit-mysql-k8s-2: 11:07:04 INFO juju.worker.uniter resuming charm install unit-mysql-k8s-0: 11:07:04 INFO juju.downloader downloading from ch:amd64/jammy/mysql-k8s-113 unit-mysql-k8s-2: 11:07:04 INFO juju.worker.uniter.charm downloading ch:amd64/jammy/mysql-k8s-113 from API server unit-mysql-k8s-2: 11:07:04 INFO juju.downloader downloading from ch:amd64/jammy/mysql-k8s-113 unit-mysql-k8s-1: 11:07:04 INFO juju.worker.uniter unit "mysql-k8s/1" started unit-mysql-k8s-1: 11:07:04 INFO juju.worker.uniter resuming charm install unit-mysql-k8s-1: 11:07:04 INFO juju.worker.uniter.charm downloading ch:amd64/jammy/mysql-k8s-113 from API server unit-mysql-k8s-1: 11:07:04 INFO juju.downloader downloading from ch:amd64/jammy/mysql-k8s-113 unit-mysql-k8s-0: 11:07:05 INFO juju.downloader download complete ("ch:amd64/jammy/mysql-k8s-113") unit-mysql-k8s-1: 11:07:05 INFO juju.downloader download complete ("ch:amd64/jammy/mysql-k8s-113") unit-mysql-k8s-2: 11:07:05 INFO juju.downloader download complete ("ch:amd64/jammy/mysql-k8s-113") unit-mysql-k8s-1: 11:07:05 INFO juju.downloader download verified ("ch:amd64/jammy/mysql-k8s-113") unit-mysql-k8s-2: 11:07:05 INFO juju.downloader download verified ("ch:amd64/jammy/mysql-k8s-113") unit-mysql-k8s-0: 11:07:05 INFO juju.downloader download verified ("ch:amd64/jammy/mysql-k8s-113") controller-0: 11:07:06 INFO juju.worker.caasapplicationprovisioner.mysql-test-app scaling application "mysql-test-app" to desired scale 1 unit-mysql-test-app-0: 11:07:07 INFO juju.cmd running containerAgent [3.1.7 0cd207d999fef1fc8b965c410e9f58fafe7ee335 gc go1.21.5] unit-mysql-test-app-0: 11:07:07 INFO juju.cmd.containeragent.unit start "unit" unit-mysql-test-app-0: 11:07:07 INFO juju.worker.upgradesteps upgrade steps for 3.1.7 have already been run. unit-mysql-test-app-0: 11:07:07 INFO juju.worker.probehttpserver starting http server on 127.0.0.1:65301 unit-mysql-test-app-0: 11:07:07 INFO juju.api cannot resolve "controller-service.controller-micro.svc.cluster.local": lookup controller-service.controller-micro.svc.cluster.local: operation was canceled unit-mysql-test-app-0: 11:07:07 INFO juju.api connection established to "wss://10.152.183.100:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-test-app-0: 11:07:07 INFO juju.worker.apicaller [78c956] "unit-mysql-test-app-0" successfully connected to "10.152.183.100:17070" unit-mysql-test-app-0: 11:07:07 INFO juju.api cannot resolve "controller-service.controller-micro.svc.cluster.local": lookup controller-service.controller-micro.svc.cluster.local: operation was canceled unit-mysql-test-app-0: 11:07:07 INFO juju.api connection established to "wss://10.152.183.100:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-test-app-0: 11:07:07 INFO juju.worker.apicaller [78c956] "unit-mysql-test-app-0" successfully connected to "10.152.183.100:17070" unit-mysql-test-app-0: 11:07:07 INFO juju.worker.migrationminion migration phase is now: NONE unit-mysql-test-app-0: 11:07:07 INFO juju.worker.logger logger worker started unit-mysql-test-app-0: 11:07:07 WARNING juju.worker.proxyupdater unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" unit-mysql-test-app-0: 11:07:07 INFO juju.worker.leadership mysql-test-app/0 promoted to leadership of mysql-test-app unit-mysql-test-app-0: 11:07:07 INFO juju.worker.caasupgrader abort check blocked until version event received unit-mysql-test-app-0: 11:07:07 INFO juju.worker.caasupgrader unblocking abort check unit-mysql-test-app-0: 11:07:07 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-mysql-test-app-0 unit-mysql-test-app-0: 11:07:07 INFO juju.worker.uniter unit "mysql-test-app/0" started unit-mysql-test-app-0: 11:07:07 INFO juju.worker.uniter resuming charm install unit-mysql-test-app-0: 11:07:07 INFO juju.worker.uniter.charm downloading ch:amd64/jammy/mysql-test-app-28 from API server unit-mysql-test-app-0: 11:07:07 INFO juju.downloader downloading from ch:amd64/jammy/mysql-test-app-28 unit-mysql-test-app-0: 11:07:08 INFO juju.downloader download complete ("ch:amd64/jammy/mysql-test-app-28") unit-mysql-test-app-0: 11:07:08 INFO juju.downloader download verified ("ch:amd64/jammy/mysql-test-app-28") unit-mysql-test-app-0: 11:07:14 INFO juju.worker.uniter hooks are retried true unit-mysql-test-app-0: 11:07:14 INFO juju.worker.uniter found queued "install" hook unit-mysql-test-app-0: 11:07:15 DEBUG unit.mysql-test-app/0.juju-log ops 2.8.0 up and running. unit-mysql-test-app-0: 11:07:15 INFO unit.mysql-test-app/0.juju-log Running legacy hooks/install. unit-mysql-test-app-0: 11:07:15 DEBUG unit.mysql-test-app/0.juju-log ops 2.8.0 up and running. unit-mysql-test-app-0: 11:07:15 DEBUG unit.mysql-test-app/0.juju-log Charm called itself via hooks/install. unit-mysql-test-app-0: 11:07:15 DEBUG unit.mysql-test-app/0.juju-log Legacy hooks/install exited with status 0. unit-mysql-test-app-0: 11:07:15 DEBUG unit.mysql-test-app/0.juju-log Using local storage: not a Kubernetes podspec charm unit-mysql-test-app-0: 11:07:15 DEBUG unit.mysql-test-app/0.juju-log Initializing SQLite local storage: /var/lib/juju/agents/unit-mysql-test-app-0/charm/.unit-state.db. unit-mysql-test-app-0: 11:07:15 DEBUG unit.mysql-test-app/0.juju-log Emitting Juju event install. unit-mysql-test-app-0: 11:07:16 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:07:16 DEBUG unit.mysql-test-app/0.juju-log application-peers:3: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:07:16 DEBUG unit.mysql-test-app/0.juju-log application-peers:3: Emitting Juju event application_peers_relation_created. unit-mysql-test-app-0: 11:07:16 INFO juju.worker.uniter.operation ran "application-peers-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:07:16 INFO juju.worker.uniter found queued "leader-elected" hook unit-mysql-test-app-0: 11:07:16 DEBUG unit.mysql-test-app/0.juju-log ops 2.8.0 up and running. unit-mysql-test-app-0: 11:07:16 DEBUG unit.mysql-test-app/0.juju-log Emitting Juju event leader_elected. unit-mysql-test-app-0: 11:07:16 INFO juju.worker.uniter.operation ran "leader-elected" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:07:17 DEBUG unit.mysql-test-app/0.juju-log ops 2.8.0 up and running. unit-mysql-test-app-0: 11:07:17 DEBUG unit.mysql-test-app/0.juju-log Emitting Juju event config_changed. unit-mysql-test-app-0: 11:07:17 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:07:17 INFO juju.worker.uniter found queued "start" hook unit-mysql-test-app-0: 11:07:17 DEBUG unit.mysql-test-app/0.juju-log ops 2.8.0 up and running. unit-mysql-test-app-0: 11:07:17 INFO unit.mysql-test-app/0.juju-log Running legacy hooks/start. unit-mysql-test-app-0: 11:07:17 DEBUG unit.mysql-test-app/0.juju-log ops 2.8.0 up and running. unit-mysql-test-app-0: 11:07:17 DEBUG unit.mysql-test-app/0.juju-log Charm called itself via hooks/start. unit-mysql-test-app-0: 11:07:17 DEBUG unit.mysql-test-app/0.juju-log Legacy hooks/start exited with status 0. unit-mysql-test-app-0: 11:07:17 DEBUG unit.mysql-test-app/0.juju-log Emitting Juju event start. unit-mysql-test-app-0: 11:07:18 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:19 INFO juju.worker.uniter hooks are retried true unit-mysql-k8s-1: 11:07:19 INFO juju.worker.uniter hooks are retried true unit-mysql-k8s-0: 11:07:19 INFO juju.worker.uniter hooks are retried true unit-mysql-k8s-2: 11:07:19 INFO juju.worker.uniter found queued "install" hook unit-mysql-k8s-1: 11:07:19 INFO juju.worker.uniter found queued "install" hook unit-mysql-k8s-0: 11:07:19 INFO juju.worker.uniter found queued "install" hook unit-mysql-k8s-1: 11:07:20 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:20 INFO unit.mysql-k8s/1.juju-log Running legacy hooks/install. unit-mysql-k8s-0: 11:07:20 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:20 INFO unit.mysql-k8s/0.juju-log Running legacy hooks/install. unit-mysql-k8s-2: 11:07:20 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:07:20 INFO unit.mysql-k8s/2.juju-log Running legacy hooks/install. unit-mysql-k8s-1: 11:07:21 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:21 DEBUG unit.mysql-k8s/1.juju-log Charm called itself via hooks/install. unit-mysql-k8s-0: 11:07:21 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:21 DEBUG unit.mysql-k8s/0.juju-log Charm called itself via hooks/install. unit-mysql-k8s-2: 11:07:21 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:07:21 DEBUG unit.mysql-k8s/2.juju-log Charm called itself via hooks/install. unit-mysql-k8s-1: 11:07:21 DEBUG unit.mysql-k8s/1.juju-log Legacy hooks/install exited with status 0. unit-mysql-k8s-1: 11:07:21 DEBUG unit.mysql-k8s/1.juju-log Using local storage: not a Kubernetes podspec charm unit-mysql-k8s-0: 11:07:21 DEBUG unit.mysql-k8s/0.juju-log Legacy hooks/install exited with status 0. unit-mysql-k8s-1: 11:07:21 DEBUG unit.mysql-k8s/1.juju-log Initializing SQLite local storage: /var/lib/juju/agents/unit-mysql-k8s-1/charm/.unit-state.db. unit-mysql-k8s-0: 11:07:21 DEBUG unit.mysql-k8s/0.juju-log Using local storage: not a Kubernetes podspec charm unit-mysql-k8s-0: 11:07:21 DEBUG unit.mysql-k8s/0.juju-log Initializing SQLite local storage: /var/lib/juju/agents/unit-mysql-k8s-0/charm/.unit-state.db. unit-mysql-k8s-1: 11:07:21 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event install. unit-mysql-k8s-2: 11:07:21 DEBUG unit.mysql-k8s/2.juju-log Legacy hooks/install exited with status 0. unit-mysql-k8s-0: 11:07:21 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event install. unit-mysql-k8s-2: 11:07:21 DEBUG unit.mysql-k8s/2.juju-log Using local storage: not a Kubernetes podspec charm unit-mysql-k8s-2: 11:07:21 DEBUG unit.mysql-k8s/2.juju-log Initializing SQLite local storage: /var/lib/juju/agents/unit-mysql-k8s-2/charm/.unit-state.db. unit-mysql-k8s-2: 11:07:21 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event install. unit-mysql-k8s-1: 11:07:21 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:21 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:21 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:22 DEBUG unit.mysql-k8s/1.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:22 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:22 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_created. unit-mysql-k8s-1: 11:07:22 DEBUG unit.mysql-k8s/1.juju-log restart:1: Emitting Juju event restart_relation_created. unit-mysql-k8s-2: 11:07:22 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:07:22 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_created. unit-mysql-k8s-1: 11:07:22 INFO juju.worker.uniter.operation ran "restart-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:22 INFO juju.worker.uniter.operation ran "database-peers-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:22 INFO juju.worker.uniter.operation ran "database-peers-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:23 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:23 DEBUG unit.mysql-k8s/0.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:07:23 DEBUG unit.mysql-k8s/2.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:23 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: Emitting Juju event upgrade_relation_created. unit-mysql-k8s-0: 11:07:23 DEBUG unit.mysql-k8s/0.juju-log restart:1: Emitting Juju event restart_relation_created. unit-mysql-k8s-2: 11:07:23 DEBUG unit.mysql-k8s/2.juju-log restart:1: Emitting Juju event restart_relation_created. unit-mysql-k8s-0: 11:07:23 INFO juju.worker.uniter.operation ran "restart-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:23 INFO juju.worker.uniter.operation ran "upgrade-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:23 INFO juju.worker.uniter.operation ran "restart-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:24 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:07:24 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:24 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:24 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Emitting Juju event upgrade_relation_created. unit-mysql-k8s-1: 11:07:24 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_created. unit-mysql-k8s-2: 11:07:24 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Emitting Juju event upgrade_relation_created. unit-mysql-k8s-2: 11:07:24 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Persisting dependencies to upgrade relation data... unit-mysql-k8s-1: 11:07:24 INFO juju.worker.uniter.operation ran "database-peers-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:24 INFO juju.worker.uniter.operation ran "upgrade-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:24 INFO juju.worker.uniter.operation ran "upgrade-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:24 INFO juju.worker.uniter found queued "leader-elected" hook unit-mysql-k8s-1: 11:07:25 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:25 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:25 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event leader_settings_changed. unit-mysql-k8s-1: 11:07:25 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event leader_settings_changed. unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event leader_elected. unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log Getting a secret when secret is not added in juju unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log Setting root-password unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log Setting server-config-password unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log Setting cluster-admin-password unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log Setting monitoring-password unit-mysql-k8s-2: 11:07:25 DEBUG unit.mysql-k8s/2.juju-log Setting backups-password unit-mysql-k8s-0: 11:07:25 INFO juju.worker.uniter.operation ran "leader-settings-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:25 INFO juju.worker.uniter.operation ran "leader-settings-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:26 INFO juju.worker.uniter.operation ran "leader-elected" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event mysql_pebble_ready. unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event mysql_pebble_ready. unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log Memory constrained by node allocable memory unit-mysql-k8s-0: 11:07:26 INFO unit.mysql-k8s/0.juju-log Setting up the logrotate configurations unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log Creating the logrotate config file unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log Memory constrained by node allocable memory unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log Writing the logrotate config file to the workload container unit-mysql-k8s-1: 11:07:26 INFO unit.mysql-k8s/1.juju-log Setting up the logrotate configurations unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log Creating the logrotate config file unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log Writing the logrotate config file to the workload container unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log Initializing instance unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log Data directory ownership: root:root unit-mysql-k8s-0: 11:07:26 DEBUG unit.mysql-k8s/0.juju-log Changing ownership to mysql:mysql unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log Initializing instance unit-mysql-k8s-2: 11:07:26 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log Data directory ownership: root:root unit-mysql-k8s-1: 11:07:26 DEBUG unit.mysql-k8s/1.juju-log Changing ownership to mysql:mysql unit-mysql-k8s-2: 11:07:26 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event mysql_pebble_ready. unit-mysql-k8s-2: 11:07:26 DEBUG unit.mysql-k8s/2.juju-log Memory constrained by node allocable memory unit-mysql-k8s-2: 11:07:26 INFO unit.mysql-k8s/2.juju-log Setting up the logrotate configurations unit-mysql-k8s-2: 11:07:26 DEBUG unit.mysql-k8s/2.juju-log Creating the logrotate config file unit-mysql-k8s-2: 11:07:26 DEBUG unit.mysql-k8s/2.juju-log Writing the logrotate config file to the workload container unit-mysql-k8s-2: 11:07:27 DEBUG unit.mysql-k8s/2.juju-log Initializing instance unit-mysql-k8s-2: 11:07:27 DEBUG unit.mysql-k8s/2.juju-log Data directory ownership: root:root unit-mysql-k8s-2: 11:07:27 DEBUG unit.mysql-k8s/2.juju-log Changing ownership to mysql:mysql unit-mysql-k8s-0: 11:07:36 DEBUG unit.mysql-k8s/0.juju-log Adding pebble layer unit-mysql-k8s-2: 11:07:36 DEBUG unit.mysql-k8s/2.juju-log Adding pebble layer unit-mysql-k8s-1: 11:07:36 DEBUG unit.mysql-k8s/1.juju-log Adding pebble layer unit-mysql-k8s-0: 11:07:37 DEBUG unit.mysql-k8s/0.juju-log Waiting for instance to be ready unit-mysql-k8s-2: 11:07:37 DEBUG unit.mysql-k8s/2.juju-log Waiting for instance to be ready unit-mysql-k8s-1: 11:07:37 DEBUG unit.mysql-k8s/1.juju-log Waiting for instance to be ready unit-mysql-k8s-0: 11:07:42 DEBUG unit.mysql-k8s/0.juju-log MySQL connection possible unit-mysql-k8s-0: 11:07:42 INFO unit.mysql-k8s/0.juju-log Configuring instance unit-mysql-k8s-0: 11:07:42 DEBUG unit.mysql-k8s/0.juju-log Configuring users unit-mysql-k8s-2: 11:07:42 DEBUG unit.mysql-k8s/2.juju-log MySQL connection possible unit-mysql-k8s-2: 11:07:42 INFO unit.mysql-k8s/2.juju-log Configuring instance unit-mysql-k8s-2: 11:07:42 DEBUG unit.mysql-k8s/2.juju-log Configuring users unit-mysql-k8s-0: 11:07:42 DEBUG unit.mysql-k8s/0.juju-log Configuring instance for InnoDB on mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-2: 11:07:42 DEBUG unit.mysql-k8s/2.juju-log Configuring instance for InnoDB on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-1: 11:07:42 DEBUG unit.mysql-k8s/1.juju-log MySQL connection possible unit-mysql-k8s-1: 11:07:42 INFO unit.mysql-k8s/1.juju-log Configuring instance unit-mysql-k8s-1: 11:07:42 DEBUG unit.mysql-k8s/1.juju-log Configuring users unit-mysql-k8s-1: 11:07:42 DEBUG unit.mysql-k8s/1.juju-log Configuring instance for InnoDB on mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-1: 11:07:48 DEBUG unit.mysql-k8s/1.juju-log MySQL connection possible unit-mysql-k8s-1: 11:07:48 DEBUG unit.mysql-k8s/1.juju-log Getting InnoDB version unit-mysql-k8s-1: 11:07:48 DEBUG unit.mysql-k8s/1.juju-log Checking existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-1: 11:07:48 DEBUG unit.mysql-k8s/1.juju-log Failed to confirm existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-1: 11:07:48 DEBUG unit.mysql-k8s/1.juju-log waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-1: 11:07:49 INFO juju.worker.uniter.operation ran "mysql-pebble-ready" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:49 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:49 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event database_storage_attached. unit-mysql-k8s-1: 11:07:50 INFO juju.worker.uniter.operation ran "database-storage-attached" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:50 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:50 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event config_changed. unit-mysql-k8s-1: 11:07:50 DEBUG unit.mysql-k8s/1.juju-log Memory constrained by node allocable memory unit-mysql-k8s-1: 11:07:51 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:51 INFO juju.worker.uniter found queued "start" hook unit-mysql-k8s-1: 11:07:51 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:51 INFO unit.mysql-k8s/1.juju-log Running legacy hooks/start. unit-mysql-k8s-1: 11:07:52 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:52 DEBUG unit.mysql-k8s/1.juju-log Charm called itself via hooks/start. unit-mysql-k8s-1: 11:07:52 DEBUG unit.mysql-k8s/1.juju-log Legacy hooks/start exited with status 0. unit-mysql-k8s-1: 11:07:52 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event start. unit-mysql-k8s-1: 11:07:52 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:52 DEBUG unit.mysql-k8s/0.juju-log MySQL connection possible unit-mysql-k8s-0: 11:07:52 DEBUG unit.mysql-k8s/0.juju-log Getting InnoDB version unit-mysql-k8s-2: 11:07:52 DEBUG unit.mysql-k8s/2.juju-log MySQL connection possible unit-mysql-k8s-2: 11:07:52 DEBUG unit.mysql-k8s/2.juju-log Getting InnoDB version unit-mysql-k8s-1: 11:07:53 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:53 DEBUG unit.mysql-k8s/0.juju-log Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-1: 11:07:53 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-2: 11:07:53 INFO unit.mysql-k8s/2.juju-log Creating cluster on the leader unit unit-mysql-k8s-2: 11:07:53 DEBUG unit.mysql-k8s/2.juju-log Creating a MySQL InnoDB cluster on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-1: 11:07:53 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Checking existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:07:53 DEBUG unit.mysql-k8s/0.juju-log Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:07:53 DEBUG unit.mysql-k8s/0.juju-log waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-1: 11:07:53 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-1: 11:07:53 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-0: 11:07:53 INFO juju.worker.uniter.operation ran "mysql-pebble-ready" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:53 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:54 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:54 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:54 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event database_storage_attached. unit-mysql-k8s-1: 11:07:54 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_joined. unit-mysql-k8s-0: 11:07:54 INFO juju.worker.uniter.operation ran "database-storage-attached" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:54 INFO juju.worker.uniter.operation ran "database-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:55 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:55 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:55 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event config_changed. unit-mysql-k8s-1: 11:07:55 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-1: 11:07:55 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Checking existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:07:55 DEBUG unit.mysql-k8s/0.juju-log Memory constrained by node allocable memory unit-mysql-k8s-2: 11:07:55 DEBUG unit.mysql-k8s/2.juju-log Creating cluster set name cluster-set-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-1: 11:07:55 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-1: 11:07:55 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-0: 11:07:55 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:55 INFO juju.worker.uniter found queued "start" hook unit-mysql-k8s-2: 11:07:55 DEBUG unit.mysql-k8s/2.juju-log Initializing the juju_units_operations table on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-1: 11:07:56 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:56 DEBUG unit.mysql-k8s/2.juju-log Checking existence of unit mysql-k8s-2 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:07:56 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:56 INFO unit.mysql-k8s/0.juju-log Running legacy hooks/start. unit-mysql-k8s-1: 11:07:56 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:56 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_joined. unit-mysql-k8s-2: 11:07:56 INFO juju.worker.uniter.operation ran "mysql-pebble-ready" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:56 INFO juju.worker.uniter.operation ran "database-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:07:57 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:57 DEBUG unit.mysql-k8s/0.juju-log Charm called itself via hooks/start. unit-mysql-k8s-0: 11:07:57 DEBUG unit.mysql-k8s/0.juju-log Legacy hooks/start exited with status 0. unit-mysql-k8s-0: 11:07:57 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event start. unit-mysql-k8s-2: 11:07:57 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:57 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:57 INFO unit.mysql-k8s/2.juju-log Starting the log rotate manager unit-mysql-k8s-2: 11:07:57 INFO unit.mysql-k8s/2.juju-log Started log rotate manager process with PID 673 unit-mysql-k8s-2: 11:07:57 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event database_storage_attached. unit-mysql-k8s-1: 11:07:57 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:07:57 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-1: 11:07:57 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Checking existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-2: 11:07:57 INFO juju.worker.uniter.operation ran "database-storage-attached" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:07:57 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-1 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-1: 11:07:57 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Getting cluster primary member's address from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-0: 11:07:58 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:58 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:07:58 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-2: 11:07:58 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:58 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:07:58 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-2: 11:07:58 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event config_changed. unit-mysql-k8s-2: 11:07:58 DEBUG unit.mysql-k8s/2.juju-log Memory constrained by node allocable memory unit-mysql-k8s-0: 11:07:58 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:58 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:07:58 INFO juju.worker.uniter found queued "start" hook unit-mysql-k8s-1: 11:07:58 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Attempting to acquire lock unit-add on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local for unit mysql-k8s-1 unit-mysql-k8s-1: 11:07:59 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Adding instance mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local/mysql-k8s-1 to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 with recovery method auto unit-mysql-k8s-0: 11:07:59 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:07:59 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_joined. unit-mysql-k8s-2: 11:07:59 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:07:59 INFO unit.mysql-k8s/2.juju-log Running legacy hooks/start. unit-mysql-k8s-1: 11:07:59 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Failed to add instance mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 with recovery method 'auto'. Trying method 'clone' unit-mysql-k8s-1: 11:07:59 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Adding instance mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local/mysql-k8s-1 to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 with recovery method clone unit-mysql-k8s-0: 11:07:59 INFO juju.worker.uniter.operation ran "database-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log Charm called itself via hooks/start. unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log Legacy hooks/start exited with status 0. unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event start. unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log Could not locate cos-tool at: "cos-tool-amd64" unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log Skipping injection of juju topology as label matchers unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log `cos-tool` unavailable. Leaving expression unchanged: mysql_up == 0 unit-mysql-k8s-2: 11:08:00 DEBUG unit.mysql-k8s/2.juju-log Reading alert rule from /var/lib/juju/agents/unit-mysql-k8s-2/charm/src/prometheus_alert_rules/metrics_alert_rules.yaml unit-mysql-k8s-0: 11:08:00 DEBUG unit.mysql-k8s/0.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:00 DEBUG unit.mysql-k8s/0.juju-log restart:1: Emitting Juju event restart_relation_joined. unit-mysql-k8s-2: 11:08:00 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:00 INFO juju.worker.uniter.operation ran "restart-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:01 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:01 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-0: 11:08:01 DEBUG unit.mysql-k8s/0.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:01 DEBUG unit.mysql-k8s/0.juju-log restart:1: Emitting Juju event restart_relation_changed. unit-mysql-k8s-0: 11:08:01 INFO juju.worker.uniter.operation ran "restart-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:02 DEBUG unit.mysql-k8s/0.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:02 DEBUG unit.mysql-k8s/0.juju-log restart:1: Emitting Juju event restart_relation_joined. unit-mysql-k8s-2: 11:08:02 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:02 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-k8s-2: 11:08:02 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-k8s-0: 11:08:02 INFO juju.worker.uniter.operation ran "restart-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:02 INFO unit.mysql-k8s/2.juju-log upgrade:2: Unit workload member-state is online with member-role primary unit-mysql-k8s-2: 11:08:03 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Getting cluster primary member's address from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-0: 11:08:03 DEBUG unit.mysql-k8s/0.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:03 DEBUG unit.mysql-k8s/0.juju-log restart:1: Emitting Juju event restart_relation_changed. unit-mysql-k8s-2: 11:08:03 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:03 INFO juju.worker.uniter.operation ran "restart-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:04 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:04 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:04 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:04 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-k8s-0: 11:08:04 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-k8s-0: 11:08:04 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:04 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-2: 11:08:04 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:04 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-0: 11:08:05 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:05 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:05 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_joined. unit-mysql-k8s-0: 11:08:05 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:05 INFO juju.worker.uniter.operation ran "database-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:05 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Emitting Juju event upgrade_relation_joined. unit-mysql-k8s-0: 11:08:06 INFO juju.worker.uniter.operation ran "upgrade-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:06 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:06 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:06 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:06 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:06 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-k8s-0: 11:08:06 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-k8s-0: 11:08:06 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:07 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:07 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-2: 11:08:07 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:07 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:07 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_joined. unit-mysql-k8s-2: 11:08:07 INFO juju.worker.uniter.operation ran "database-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:08 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:08 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Emitting Juju event upgrade_relation_joined. unit-mysql-k8s-2: 11:08:08 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:08 INFO juju.worker.uniter.operation ran "upgrade-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:08 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Emitting Juju event upgrade_relation_joined. unit-mysql-k8s-2: 11:08:08 INFO juju.worker.uniter.operation ran "upgrade-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:08 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:09 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-k8s-0: 11:08:09 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-k8s-0: 11:08:09 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-2: 11:08:09 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:09 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:09 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:09 DEBUG unit.mysql-k8s/0.juju-log upgrade:2: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-0: 11:08:09 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:09 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:10 DEBUG unit.mysql-k8s/2.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:10 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:10 DEBUG unit.mysql-k8s/2.juju-log restart:1: Emitting Juju event restart_relation_joined. unit-mysql-k8s-0: 11:08:10 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:10 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-2: 11:08:10 INFO juju.worker.uniter.operation ran "restart-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:10 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:10 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-0: 11:08:11 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:11 DEBUG unit.mysql-k8s/2.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:11 DEBUG unit.mysql-k8s/2.juju-log restart:1: Emitting Juju event restart_relation_changed. unit-mysql-k8s-2: 11:08:11 DEBUG unit.mysql-k8s/2.juju-log restart:1: Emitting custom event . unit-mysql-k8s-0: 11:08:11 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:11 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-2: 11:08:11 INFO juju.worker.uniter.operation ran "restart-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:11 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:12 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:12 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: waiting: unable to retrieve the cluster primary from peers unit-mysql-k8s-2: 11:08:12 DEBUG unit.mysql-k8s/2.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:12 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:12 DEBUG unit.mysql-k8s/2.juju-log restart:1: Emitting Juju event restart_relation_joined. unit-mysql-k8s-2: 11:08:12 INFO juju.worker.uniter.operation ran "restart-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:13 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:13 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_joined. controller-0: 11:08:13 INFO juju.worker.caasapplicationprovisioner.runner start "mysql-router-k8s" controller-0: 11:08:13 INFO juju.worker.caasapplicationprovisioner.mysql-router-k8s scaling application "mysql-router-k8s" to desired scale 3 unit-mysql-k8s-2: 11:08:13 DEBUG unit.mysql-k8s/2.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:13 INFO juju.worker.uniter.operation ran "database-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:13 DEBUG unit.mysql-k8s/2.juju-log restart:1: Emitting Juju event restart_relation_changed. unit-mysql-k8s-2: 11:08:13 DEBUG unit.mysql-k8s/2.juju-log restart:1: Emitting custom event . unit-mysql-k8s-2: 11:08:14 INFO juju.worker.uniter.operation ran "restart-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:14 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:14 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:14 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-2: 11:08:14 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:14 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:14 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Getting cluster primary member's address from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-2: 11:08:14 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-k8s-2: 11:08:14 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-k8s-2: 11:08:14 INFO unit.mysql-k8s/2.juju-log upgrade:2: Unit workload member-state is online with member-role primary unit-mysql-k8s-2: 11:08:15 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Getting cluster primary member's address from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-0: 11:08:15 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: waiting: cluster lock is held unit-mysql-k8s-2: 11:08:15 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:15 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:08:16 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:08:16 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_created. unit-mysql-k8s-2: 11:08:16 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:16 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:16 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:16 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:16 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-test-app-0: 11:08:16 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:16 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:16 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Getting cluster primary member's address from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-2: 11:08:16 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:17 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:17 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Emitting Juju event upgrade_relation_joined. unit-mysql-k8s-2: 11:08:18 INFO juju.worker.uniter.operation ran "upgrade-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:18 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: waiting: cluster lock is held unit-mysql-k8s-1: 11:08:18 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Releasing lock unit-add on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local for unit mysql-k8s-1 unit-mysql-k8s-0: 11:08:18 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:19 INFO juju.cmd running containerAgent [3.1.7 0cd207d999fef1fc8b965c410e9f58fafe7ee335 gc go1.21.5] unit-mysql-router-k8s-2: 11:08:19 INFO juju.cmd running containerAgent [3.1.7 0cd207d999fef1fc8b965c410e9f58fafe7ee335 gc go1.21.5] unit-mysql-router-k8s-2: 11:08:19 INFO juju.cmd.containeragent.unit start "unit" unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.upgradesteps upgrade steps for 3.1.7 have already been run. unit-mysql-router-k8s-1: 11:08:19 INFO juju.cmd.containeragent.unit start "unit" unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.upgradesteps upgrade steps for 3.1.7 have already been run. unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.probehttpserver starting http server on 127.0.0.1:65301 unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.probehttpserver starting http server on 127.0.0.1:65301 unit-mysql-router-k8s-0: 11:08:19 INFO juju.cmd running containerAgent [3.1.7 0cd207d999fef1fc8b965c410e9f58fafe7ee335 gc go1.21.5] unit-mysql-router-k8s-0: 11:08:19 INFO juju.cmd.containeragent.unit start "unit" unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.upgradesteps upgrade steps for 3.1.7 have already been run. unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.probehttpserver starting http server on 127.0.0.1:65301 unit-mysql-router-k8s-2: 11:08:19 INFO juju.api cannot resolve "controller-service.controller-micro.svc.cluster.local": lookup controller-service.controller-micro.svc.cluster.local: operation was canceled unit-mysql-router-k8s-2: 11:08:19 INFO juju.api connection established to "wss://10.152.183.100:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.apicaller [78c956] "unit-mysql-router-k8s-2" successfully connected to "10.152.183.100:17070" unit-mysql-router-k8s-1: 11:08:19 INFO juju.api cannot resolve "controller-service.controller-micro.svc.cluster.local": lookup controller-service.controller-micro.svc.cluster.local: operation was canceled unit-mysql-router-k8s-1: 11:08:19 INFO juju.api connection established to "wss://10.152.183.100:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.apicaller [78c956] "unit-mysql-router-k8s-1" successfully connected to "10.152.183.100:17070" unit-mysql-router-k8s-0: 11:08:19 INFO juju.api cannot resolve "controller-service.controller-micro.svc.cluster.local": lookup controller-service.controller-micro.svc.cluster.local: operation was canceled unit-mysql-router-k8s-0: 11:08:19 INFO juju.api connection established to "wss://10.152.183.100:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.apicaller [78c956] "unit-mysql-router-k8s-0" successfully connected to "10.152.183.100:17070" unit-mysql-k8s-1: 11:08:19 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Added instance mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local to cluster unit-mysql-router-k8s-2: 11:08:19 INFO juju.api connection established to "wss://controller-service.controller-micro.svc.cluster.local:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.apicaller [78c956] "unit-mysql-router-k8s-2" successfully connected to "controller-service.controller-micro.svc.cluster.local:17070" unit-mysql-router-k8s-1: 11:08:19 INFO juju.api connection established to "wss://controller-service.controller-micro.svc.cluster.local:17070/model/78c95646-7873-4e34-8022-882c1bd46b52/api" unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.migrationminion migration phase is now: NONE unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.logger logger worker started unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.apicaller [78c956] "unit-mysql-router-k8s-1" successfully connected to "controller-service.controller-micro.svc.cluster.local:17070" unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.leadership mysql-router-k8s/0 promoted to leadership of mysql-router-k8s unit-mysql-router-k8s-0: 11:08:19 WARNING juju.worker.proxyupdater unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.migrationminion migration phase is now: NONE unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.leadership mysql-router-k8s leadership for mysql-router-k8s/2 denied unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.caasupgrader abort check blocked until version event received unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.logger logger worker started unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.caasupgrader unblocking abort check unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.migrationminion migration phase is now: NONE unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.logger logger worker started unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.leadership mysql-router-k8s leadership for mysql-router-k8s/1 denied unit-mysql-router-k8s-2: 11:08:19 WARNING juju.worker.proxyupdater unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" unit-mysql-router-k8s-1: 11:08:19 WARNING juju.worker.proxyupdater unable to set snap core settings [proxy.http= proxy.https= proxy.store=]: exec: "snap": executable file not found in $PATH, output: "" unit-mysql-k8s-1: 11:08:19 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Instance mysql-k8s-1 is cluster member unit-mysql-router-k8s-0: 11:08:19 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-mysql-router-k8s-0 unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.caasupgrader abort check blocked until version event received unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.caasupgrader unblocking abort check unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.caasupgrader abort check blocked until version event received unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.caasupgrader unblocking abort check unit-mysql-router-k8s-2: 11:08:19 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-mysql-router-k8s-2 unit-mysql-router-k8s-1: 11:08:19 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-mysql-router-k8s-1 unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.uniter unit "mysql-router-k8s/0" started unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.uniter resuming charm install unit-mysql-router-k8s-0: 11:08:19 INFO juju.worker.uniter.charm downloading local:jammy/mysql-router-k8s-0 from API server unit-mysql-router-k8s-0: 11:08:19 INFO juju.downloader downloading from local:jammy/mysql-router-k8s-0 unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.uniter unit "mysql-router-k8s/2" started unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.uniter resuming charm install unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.uniter unit "mysql-router-k8s/1" started unit-mysql-router-k8s-2: 11:08:19 INFO juju.worker.uniter.charm downloading local:jammy/mysql-router-k8s-0 from API server unit-mysql-router-k8s-2: 11:08:19 INFO juju.downloader downloading from local:jammy/mysql-router-k8s-0 unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.uniter resuming charm install unit-mysql-router-k8s-1: 11:08:19 INFO juju.worker.uniter.charm downloading local:jammy/mysql-router-k8s-0 from API server unit-mysql-router-k8s-1: 11:08:19 INFO juju.downloader downloading from local:jammy/mysql-router-k8s-0 unit-mysql-router-k8s-0: 11:08:19 INFO juju.downloader download complete ("local:jammy/mysql-router-k8s-0") unit-mysql-router-k8s-2: 11:08:19 INFO juju.downloader download complete ("local:jammy/mysql-router-k8s-0") unit-mysql-router-k8s-1: 11:08:19 INFO juju.downloader download complete ("local:jammy/mysql-router-k8s-0") unit-mysql-router-k8s-0: 11:08:19 INFO juju.downloader download verified ("local:jammy/mysql-router-k8s-0") unit-mysql-router-k8s-2: 11:08:19 INFO juju.downloader download verified ("local:jammy/mysql-router-k8s-0") unit-mysql-k8s-2: 11:08:19 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-router-k8s-1: 11:08:19 INFO juju.downloader download verified ("local:jammy/mysql-router-k8s-0") unit-mysql-k8s-2: 11:08:19 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-k8s-1: 11:08:19 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:19 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-k8s-2: 11:08:19 INFO unit.mysql-k8s/2.juju-log upgrade:2: Unit workload member-state is online with member-role primary unit-mysql-k8s-0: 11:08:19 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:19 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_created. unit-mysql-k8s-2: 11:08:20 DEBUG unit.mysql-k8s/2.juju-log upgrade:2: Getting cluster primary member's address from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-0: 11:08:20 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:20 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:20 INFO unit.mysql-k8s/1.juju-log database:7: Starting the log rotate manager unit-mysql-k8s-1: 11:08:20 INFO unit.mysql-k8s/1.juju-log database:7: Started log rotate manager process with PID 1123 unit-mysql-k8s-1: 11:08:20 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_created. unit-mysql-k8s-2: 11:08:20 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:20 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:08:21 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:21 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:08:21 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 controller-0: 11:08:21 INFO juju.worker.caasapplicationprovisioner.mysql-router-k8s scaling application "mysql-router-k8s" to desired scale 3 unit-mysql-k8s-0: 11:08:21 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:08:21 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Getting cluster primary member's address from mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-1: 11:08:21 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:21 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:21 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-2: 11:08:21 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_created. unit-mysql-k8s-1: 11:08:21 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/1 unit-mysql-k8s-2: 11:08:22 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:22 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Labeling pod mysql-k8s-1 with label replicas unit-mysql-k8s-1: 11:08:22 INFO unit.mysql-k8s/1.juju-log database-peers:0: Kubernetes pod label replicas created unit-mysql-k8s-0: 11:08:22 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Attempting to acquire lock unit-add on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local for unit mysql-k8s-0 unit-mysql-k8s-2: 11:08:22 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:22 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:22 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-2: 11:08:22 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/2 unit-mysql-k8s-0: 11:08:22 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Adding instance mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local/mysql-k8s-0 to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 with recovery method auto unit-mysql-k8s-2: 11:08:23 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Labeling pod mysql-k8s-2 with label primary unit-mysql-k8s-2: 11:08:23 INFO unit.mysql-k8s/2.juju-log database-peers:0: Kubernetes pod label primary created unit-mysql-k8s-1: 11:08:23 DEBUG unit.mysql-k8s/1.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:08:23 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to add instance mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 with recovery method 'auto'. Trying method 'clone' unit-mysql-k8s-0: 11:08:23 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Adding instance mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local/mysql-k8s-0 to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 with recovery method clone unit-mysql-k8s-1: 11:08:23 DEBUG unit.mysql-k8s/1.juju-log restart:1: Emitting Juju event restart_relation_joined. unit-mysql-k8s-2: 11:08:23 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:23 INFO juju.worker.uniter.operation ran "restart-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:24 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:24 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-1: 11:08:24 DEBUG unit.mysql-k8s/1.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:24 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/2 unit-mysql-k8s-1: 11:08:24 DEBUG unit.mysql-k8s/1.juju-log restart:1: Emitting Juju event restart_relation_changed. unit-mysql-k8s-1: 11:08:25 INFO juju.worker.uniter.operation ran "restart-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:25 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Labeling pod mysql-k8s-2 with label primary unit-mysql-k8s-2: 11:08:25 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:25 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:25 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. controller-0: 11:08:26 INFO juju.worker.caasapplicationprovisioner.mysql-router-k8s scaling application "mysql-router-k8s" to desired scale 3 unit-mysql-k8s-1: 11:08:26 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/1 unit-mysql-k8s-1: 11:08:26 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Labeling pod mysql-k8s-1 with label replicas unit-mysql-router-k8s-1: 11:08:26 INFO juju.worker.uniter hooks are retried true unit-mysql-router-k8s-0: 11:08:26 INFO juju.worker.uniter hooks are retried true unit-mysql-router-k8s-2: 11:08:26 INFO juju.worker.uniter hooks are retried true unit-mysql-router-k8s-1: 11:08:26 INFO juju.worker.uniter found queued "install" hook unit-mysql-router-k8s-0: 11:08:26 INFO juju.worker.uniter found queued "install" hook unit-mysql-router-k8s-2: 11:08:26 INFO juju.worker.uniter found queued "install" hook unit-mysql-k8s-1: 11:08:26 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:27 DEBUG unit.mysql-k8s/1.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:27 DEBUG unit.mysql-k8s/1.juju-log restart:1: Emitting Juju event restart_relation_joined. unit-mysql-router-k8s-1: 11:08:27 DEBUG unit.mysql-router-k8s/1.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:27 INFO unit.mysql-router-k8s/1.juju-log Running legacy hooks/install. unit-mysql-router-k8s-0: 11:08:27 DEBUG unit.mysql-router-k8s/0.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:28 INFO unit.mysql-router-k8s/0.juju-log Running legacy hooks/install. unit-mysql-k8s-1: 11:08:28 INFO juju.worker.uniter.operation ran "restart-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:28 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:28 INFO unit.mysql-router-k8s/2.juju-log Running legacy hooks/install. unit-mysql-router-k8s-1: 11:08:28 DEBUG unit.mysql-router-k8s/1.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:28 DEBUG unit.mysql-router-k8s/1.juju-log Charm called itself via hooks/install. unit-mysql-router-k8s-1: 11:08:28 DEBUG unit.mysql-router-k8s/1.juju-log Legacy hooks/install exited with status 0. unit-mysql-router-k8s-1: 11:08:28 DEBUG unit.mysql-router-k8s/1.juju-log Using local storage: not a Kubernetes podspec charm unit-mysql-router-k8s-1: 11:08:28 DEBUG unit.mysql-router-k8s/1.juju-log Initializing SQLite local storage: /var/lib/juju/agents/unit-mysql-router-k8s-1/charm/.unit-state.db. unit-mysql-router-k8s-1: 11:08:28 DEBUG unit.mysql-router-k8s/1.juju-log Emitting Juju event install. unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log Charm called itself via hooks/install. unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log Legacy hooks/install exited with status 0. unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log Using local storage: not a Kubernetes podspec charm unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log Initializing SQLite local storage: /var/lib/juju/agents/unit-mysql-router-k8s-0/charm/.unit-state.db. unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log Emitting Juju event install. unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log Patching k8s service unit-mysql-router-k8s-2: 11:08:28 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:28 DEBUG unit.mysql-router-k8s/2.juju-log Charm called itself via hooks/install. unit-mysql-router-k8s-0: 11:08:28 DEBUG unit.mysql-router-k8s/0.juju-log Patched k8s service unit-mysql-k8s-1: 11:08:28 DEBUG unit.mysql-k8s/1.juju-log restart:1: ops 2.9.0 up and running. unit-mysql-router-k8s-1: 11:08:28 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:28 DEBUG unit.mysql-router-k8s/2.juju-log Legacy hooks/install exited with status 0. unit-mysql-router-k8s-2: 11:08:28 DEBUG unit.mysql-router-k8s/2.juju-log Using local storage: not a Kubernetes podspec charm unit-mysql-router-k8s-2: 11:08:28 DEBUG unit.mysql-router-k8s/2.juju-log Initializing SQLite local storage: /var/lib/juju/agents/unit-mysql-router-k8s-2/charm/.unit-state.db. unit-mysql-router-k8s-2: 11:08:28 DEBUG unit.mysql-router-k8s/2.juju-log Emitting Juju event install. unit-mysql-k8s-1: 11:08:28 DEBUG unit.mysql-k8s/1.juju-log restart:1: Emitting Juju event restart_relation_changed. unit-mysql-router-k8s-0: 11:08:29 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:08:29 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:08:29 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-2: 11:08:29 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:29 INFO juju.worker.uniter.operation ran "restart-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:08:29 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:29 DEBUG unit.mysql-router-k8s/1.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:29 DEBUG unit.mysql-router-k8s/1.juju-log database:6: Emitting Juju event database_relation_created. unit-mysql-router-k8s-1: 11:08:29 DEBUG unit.mysql-router-k8s/1.juju-log database:6: Peer relation not ready unit-mysql-test-app-0: 11:08:29 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:08:29 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-k8s-2: 11:08:29 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:29 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-1: 11:08:29 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:29 DEBUG unit.mysql-router-k8s/0.juju-log database:6: ops 2.6.0 up and running. unit-mysql-k8s-1: 11:08:29 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:08:29 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Emitting Juju event database_relation_created. unit-mysql-router-k8s-0: 11:08:29 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Peer relation not ready unit-mysql-k8s-1: 11:08:29 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-test-app-0: 11:08:29 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:29 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-router-k8s-2: 11:08:29 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:29 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_created. unit-mysql-k8s-1: 11:08:30 INFO unit.mysql-k8s/1.juju-log upgrade:2: Unit workload member-state is online with member-role secondary unit-mysql-k8s-2: 11:08:30 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:30 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:08:30 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:08:30 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-2: 11:08:30 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:30 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:30 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_created. unit-mysql-k8s-1: 11:08:30 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:08:30 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:30 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:08:30 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:08:30 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-0: 11:08:30 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:30 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_created. unit-mysql-k8s-2: 11:08:30 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:30 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:08:30 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:30 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_created. unit-mysql-router-k8s-2: 11:08:31 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Peer relation not ready unit-mysql-test-app-0: 11:08:31 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:31 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:31 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:31 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-test-app-0: 11:08:31 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-router-k8s-2: 11:08:31 INFO juju.worker.uniter.operation ran "backend-database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:31 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-test-app-0: 11:08:31 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-1: 11:08:31 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_created. unit-mysql-k8s-1: 11:08:31 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: Emitting Juju event upgrade_relation_joined. unit-mysql-test-app-0: 11:08:31 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:31 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_created. unit-mysql-k8s-1: 11:08:31 INFO juju.worker.uniter.operation ran "upgrade-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:08:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-test-app-0: 11:08:31 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:08:31 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:08:31 DEBUG unit.mysql-router-k8s/2.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:31 DEBUG unit.mysql-router-k8s/2.juju-log database:6: Emitting Juju event database_relation_created. unit-mysql-router-k8s-2: 11:08:31 DEBUG unit.mysql-router-k8s/2.juju-log database:6: Peer relation not ready unit-mysql-router-k8s-0: 11:08:32 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:32 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:08:32 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-test-app-0: 11:08:32 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_created. unit-mysql-router-k8s-2: 11:08:32 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-k8s-2: 11:08:32 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-k8s-1: 11:08:32 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:32 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Set unit status to WaitingStatus('') unit-mysql-k8s-1: 11:08:32 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_created. unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:32 INFO juju.worker.uniter.operation ran "backend-database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:32 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-k8s-1: 11:08:32 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:32 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_created. unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._partition=0, partition_=0 unit-mysql-router-k8s-0: 11:08:32 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Set app status to WaitingStatus('Waiting for mysql-k8s app on backend-database endpoint') unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:33 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:33 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:33 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Set unit status to WaitingStatus('') unit-mysql-k8s-2: 11:08:33 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:08:33 DEBUG unit.mysql-router-k8s/1.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:33 DEBUG unit.mysql-router-k8s/1.juju-log Emitting Juju event leader_settings_changed. unit-mysql-router-k8s-0: 11:08:33 INFO juju.worker.uniter.operation ran "backend-database-relation-created" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:33 INFO juju.worker.uniter found queued "leader-elected" hook unit-mysql-k8s-1: 11:08:33 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:33 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-k8s-2: 11:08:33 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:33 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-router-k8s-1: 11:08:33 INFO juju.worker.uniter.operation ran "leader-settings-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:33 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:33 DEBUG unit.mysql-router-k8s/2.juju-log Emitting Juju event leader_settings_changed. unit-mysql-k8s-1: 11:08:33 INFO unit.mysql-k8s/1.juju-log upgrade:2: Unit workload member-state is online with member-role secondary unit-mysql-router-k8s-2: 11:08:34 INFO juju.worker.uniter.operation ran "leader-settings-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Emitting Juju event leader_elected. unit-mysql-k8s-1: 11:08:34 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log Emitting Juju event mysql_router_pebble_ready. unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-k8s-2: 11:08:34 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._partition=0, partition_=0 unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-k8s-2: 11:08:34 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-k8s-2: 11:08:34 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting custom event . unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-k8s-2: 11:08:34 DEBUG unit.mysql-k8s/2.juju-log database:7: Updating pod labels unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:34 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:34 DEBUG unit.mysql-router-k8s/1.juju-log Set unit status to WaitingStatus('') unit-mysql-router-k8s-2: 11:08:34 DEBUG unit.mysql-router-k8s/2.juju-log Emitting Juju event mysql_router_pebble_ready. unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:34 DEBUG unit.mysql-router-k8s/2.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Set app status to WaitingStatus('Waiting for mysql-k8s app on backend-database endpoint') unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:34 DEBUG unit.mysql-router-k8s/2.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:34 DEBUG unit.mysql-router-k8s/2.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:34 DEBUG unit.mysql-router-k8s/0.juju-log Set unit status to WaitingStatus('') unit-mysql-k8s-1: 11:08:35 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-1: 11:08:35 INFO juju.worker.uniter.operation ran "mysql-router-pebble-ready" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:35 DEBUG unit.mysql-router-k8s/2.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:35 DEBUG unit.mysql-router-k8s/2.juju-log State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-2: 11:08:35 DEBUG unit.mysql-router-k8s/2.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-k8s-1: 11:08:35 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:08:35 DEBUG unit.mysql-router-k8s/2.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:35 DEBUG unit.mysql-router-k8s/2.juju-log Set unit status to WaitingStatus('') unit-mysql-router-k8s-0: 11:08:35 INFO juju.worker.uniter.operation ran "leader-elected" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:35 INFO unit.mysql-k8s/2.juju-log database:7: Kubernetes service mysql-k8s-primary created unit-mysql-k8s-2: 11:08:35 INFO unit.mysql-k8s/2.juju-log database:7: Kubernetes service mysql-k8s-replicas created unit-mysql-k8s-2: 11:08:35 DEBUG unit.mysql-k8s/2.juju-log database:7: Getting InnoDB version unit-mysql-router-k8s-2: 11:08:35 INFO juju.worker.uniter.operation ran "mysql-router-pebble-ready" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:35 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:35 DEBUG unit.mysql-router-k8s/1.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:35 DEBUG unit.mysql-router-k8s/1.juju-log Emitting Juju event config_changed. unit-mysql-k8s-2: 11:08:35 DEBUG unit.mysql-k8s/2.juju-log database:7: Kubernetes service endpoint ready unit-mysql-router-k8s-0: 11:08:35 DEBUG unit.mysql-router-k8s/0.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:35 DEBUG unit.mysql-router-k8s/0.juju-log Emitting Juju event mysql_router_pebble_ready. unit-mysql-router-k8s-0: 11:08:35 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:36 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:36 INFO juju.worker.uniter found queued "start" hook unit-mysql-router-k8s-2: 11:08:36 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:36 DEBUG unit.mysql-router-k8s/2.juju-log Emitting Juju event config_changed. unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._partition=0, partition_=0 unit-mysql-k8s-1: 11:08:36 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:36 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:36 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: Emitting Juju event upgrade_relation_joined. unit-mysql-router-k8s-2: 11:08:36 INFO juju.worker.uniter found queued "start" hook unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-k8s-2: 11:08:36 INFO unit.mysql-k8s/2.juju-log database:7: Created user for app mysql-router-k8s unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Set app status to WaitingStatus('Waiting for mysql-k8s app on backend-database endpoint') unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Required key='endpoints' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:36 DEBUG unit.mysql-router-k8s/1.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:36 INFO unit.mysql-router-k8s/1.juju-log Running legacy hooks/start. unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:36 DEBUG unit.mysql-router-k8s/0.juju-log Set unit status to WaitingStatus('') unit-mysql-k8s-1: 11:08:36 INFO juju.worker.uniter.operation ran "upgrade-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:36 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:36 INFO juju.worker.uniter.operation ran "mysql-router-pebble-ready" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:37 INFO unit.mysql-router-k8s/2.juju-log Running legacy hooks/start. unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log Charm called itself via hooks/start. unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log Legacy hooks/start exited with status 0. unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log Emitting Juju event start. unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-k8s-1: 11:08:37 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: ops 2.9.0 up and running. unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log Charm called itself via hooks/start. unit-mysql-k8s-1: 11:08:37 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: Emitting Juju event upgrade_relation_changed. unit-mysql-router-k8s-0: 11:08:37 DEBUG unit.mysql-router-k8s/0.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:37 DEBUG unit.mysql-router-k8s/0.juju-log Emitting Juju event config_changed. unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-k8s-1: 11:08:37 DEBUG unit.mysql-k8s/1.juju-log upgrade:2: upgrade-changed event handled before pre-checks, exiting... unit-mysql-router-k8s-1: 11:08:37 DEBUG unit.mysql-router-k8s/1.juju-log Set unit status to WaitingStatus('') unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log Legacy hooks/start exited with status 0. unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log Emitting Juju event start. unit-mysql-k8s-2: 11:08:37 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:37 INFO unit.mysql-k8s/1.juju-log upgrade:2: Unit workload member-state is online with member-role secondary unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-k8s-2: 11:08:37 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:37 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:37 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:37 INFO juju.worker.uniter found queued "start" hook unit-mysql-router-k8s-2: 11:08:37 DEBUG unit.mysql-router-k8s/2.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:38 DEBUG unit.mysql-router-k8s/2.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:38 DEBUG unit.mysql-router-k8s/2.juju-log Set unit status to WaitingStatus('') unit-mysql-k8s-1: 11:08:38 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:08:38 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:38 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:38 DEBUG unit.mysql-router-k8s/0.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:38 INFO unit.mysql-router-k8s/0.juju-log Running legacy hooks/start. unit-mysql-router-k8s-1: 11:08:38 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:38 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-1: 11:08:38 INFO unit.mysql-router-k8s/1.juju-log backend-database:7: database created at 2024-01-18 11:08:38.690156 unit-mysql-router-k8s-1: 11:08:38 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting custom event . unit-mysql-k8s-1: 11:08:38 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-2: 11:08:38 DEBUG unit.mysql-router-k8s/2.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:08:38 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-k8s-2: 11:08:38 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-1: 11:08:38 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-1: 11:08:38 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Enabling MySQL Router service unit-mysql-router-k8s-2: 11:08:38 DEBUG unit.mysql-router-k8s/2.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:08:39 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Getting MySQL Router user for unit_name='mysql-router-k8s/1' unit-mysql-k8s-1: 11:08:39 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-k8s-2: 11:08:39 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Charm called itself via hooks/start. unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Legacy hooks/start exited with status 0. unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Emitting Juju event start. unit-mysql-router-k8s-2: 11:08:39 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-1: 11:08:39 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-k8s-1: 11:08:39 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:08:39 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:08:39 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:08:39 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: No MySQL Router user found for unit_name='mysql-router-k8s/1' unit-mysql-router-k8s-1: 11:08:39 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Bootstrapping router tls=False, self._connection_info.host='mysql-k8s-primary.foo6.svc.cluster.local', self._connection_info.port='3306' unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-k8s-2: 11:08:39 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._partition=0, partition_=0 unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Set app status to WaitingStatus('Waiting for mysql-k8s app on backend-database endpoint') unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-2: 11:08:39 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:08:39 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:39 DEBUG unit.mysql-router-k8s/0.juju-log Set unit status to WaitingStatus('') unit-mysql-router-k8s-2: 11:08:40 INFO unit.mysql-router-k8s/2.juju-log backend-database:7: database created at 2024-01-18 11:08:40.003607 unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting custom event . unit-mysql-router-k8s-1: 11:08:40 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Bootstrapped router tls=False, self._connection_info.host='mysql-k8s-primary.foo6.svc.cluster.local', self._connection_info.port='3306' unit-mysql-router-k8s-1: 11:08:40 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Adding attributes='{"created_by_user": "relation-7", "router_id": "mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system", "created_by_juju_unit": "mysql-router-k8s/1"}' to username='mysql_router1_v52epoxqz97y' unit-mysql-k8s-1: 11:08:40 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:08:40 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-0: 11:08:40 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Enabling MySQL Router service unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Getting MySQL Router user for unit_name='mysql-router-k8s/2' unit-mysql-router-k8s-1: 11:08:40 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:08:40 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:08:40 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Added attributes='{"created_by_user": "relation-7", "router_id": "mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system", "created_by_juju_unit": "mysql-router-k8s/1"}' to username='mysql_router1_v52epoxqz97y' unit-mysql-k8s-1: 11:08:40 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: No MySQL Router user found for unit_name='mysql-router-k8s/2' unit-mysql-router-k8s-2: 11:08:40 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Bootstrapping router tls=False, self._connection_info.host='mysql-k8s-primary.foo6.svc.cluster.local', self._connection_info.port='3306' unit-mysql-router-k8s-0: 11:08:40 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:40 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-router-k8s-0: 11:08:40 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:40 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:40 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:40 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:40 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._partition=0, partition_=0 unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-k8s-1: 11:08:41 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-k8s-1: 11:08:41 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set app status to WaitingStatus('Waiting for mysql-k8s app on backend-database endpoint') unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:41 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set unit status to WaitingStatus('') unit-mysql-router-k8s-1: 11:08:41 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Creating logrotate config file unit-mysql-router-k8s-1: 11:08:41 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Created logrotate config file unit-mysql-router-k8s-1: 11:08:41 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Copying log rotate executor script to workload container unit-mysql-router-k8s-1: 11:08:41 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Copied log rotate executor to workload container unit-mysql-router-k8s-1: 11:08:41 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Starting the logrotate executor service unit-mysql-k8s-1: 11:08:41 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:41 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-k8s-1: 11:08:42 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Emitting custom event . unit-mysql-k8s-1: 11:08:42 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=False, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._partition=0, partition_=0 unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:08:42 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Started the logrotate executor service unit-mysql-router-k8s-1: 11:08:42 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Enabled MySQL Router service unit-mysql-router-k8s-1: 11:08:42 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Waiting until MySQL Router is ready unit-mysql-router-k8s-1: 11:08:42 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: MySQL Router is ready unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-1: 11:08:42 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-k8s-1: 11:08:42 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Set app status to WaitingStatus('Waiting for mysql-k8s app on backend-database endpoint') unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Required key='username' missing from databag for self._app_name='mysql-k8s' on self._endpoint_name='backend-database' unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:42 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Set unit status to WaitingStatus('') unit-mysql-router-k8s-0: 11:08:42 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:43 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-test-app-0: 11:08:43 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:08:43 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-k8s-1: 11:08:43 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-test-app-0: 11:08:43 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:08:43 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:08:43 INFO unit.mysql-router-k8s/0.juju-log backend-database:7: database created at 2024-01-18 11:08:43.514732 unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting custom event . unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Reconciling users event=, router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: State of reconcile users requested_users=[], self._shared_users=[] unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleting user if exists before creating user unit-mysql-router-k8s-0: 11:08:43 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleting username='relation-7-6' must_exist=False unit-mysql-k8s-1: 11:09:00 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:09:00 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:09:00 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-2: 11:09:00 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-router-k8s-2: 11:09:10 ERROR unit.mysql-router-k8s/2.juju-log backend-database:7: Uncaught exception while in charm code: Traceback (most recent call last): File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/./src/kubernetes_charm.py", line 185, in ops.main.main(KubernetesRouterCharm) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/main.py", line 441, in main _emit_charm_event(charm, dispatcher.event_name) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/main.py", line 149, in _emit_charm_event event_to_emit.emit(*args, **kwargs) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/framework.py", line 344, in emit framework._emit(event) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/framework.py", line 841, in _emit self._reemit(event_path) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/framework.py", line 930, in _reemit custom_handler(event) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/lib/charms/data_platform_libs/v0/data_interfaces.py", line 1958, in _on_relation_changed_event getattr(self.on, "database_created").emit( File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/framework.py", line 344, in emit framework._emit(event) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/framework.py", line 841, in _emit self._reemit(event_path) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/framework.py", line 930, in _reemit custom_handler(event) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/src/abstract_charm.py", line 254, in reconcile workload_.enable(tls=self._tls_certificate_saved, unit_name=self.unit.name) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/src/workload.py", line 248, in enable self._bootstrap_router(tls=tls) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/src/workload.py", line 193, in _bootstrap_router self._container.run_mysql_router(command, timeout=30) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/src/container.py", line 134, in run_mysql_router return self._run_command(args, timeout=timeout) File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/src/rock.py", line 163, in _run_command output, _ = process.wait_output() File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/pebble.py", line 1354, in wait_output exit_code: int = self._wait() File "/var/lib/juju/agents/unit-mysql-router-k8s-2/charm/venv/ops/pebble.py", line 1316, in _wait raise ChangeError(change.err, change) ops.pebble.ChangeError: cannot perform the following tasks: - exec command "mysqlrouter" (timed out after 30s: context deadline exceeded) ----- Logs from task 0 ----- 2024-01-18T11:09:10Z ERROR timed out after 30s: context deadline exceeded ----- unit-mysql-router-k8s-2: 11:09:10 ERROR juju.worker.uniter.operation hook "backend-database-relation-changed" (via hook dispatching script: dispatch) failed: exit status 1 unit-mysql-router-k8s-2: 11:09:10 INFO juju.worker.uniter awaiting error resolution for "relation-changed" hook unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted username='relation-7-6' must_exist=False unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted user if exists before creating user unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Creating database='continuous_writes_database' and username='relation-7-6' with attributes='{"created_by_user": "relation-7"}' unit-mysql-k8s-0: 11:09:13 ERROR unit.mysql-k8s/0.juju-log database-peers:0: Failed to add instance mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 on mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local Traceback (most recent call last): File "/var/lib/juju/agents/unit-mysql-k8s-0/charm/src/mysql_k8s_helpers.py", line 666, in _run_mysqlsh_script stdout, _ = process.wait_output() File "/var/lib/juju/agents/unit-mysql-k8s-0/charm/venv/ops/pebble.py", line 1441, in wait_output raise ExecError[AnyStr](self._command, exit_code, out_value, err_value) ops.pebble.ExecError: non-zero exit code 1 executing ['/usr/bin/mysqlsh', '--no-wizard', '--python', '--verbose=1', '-f', '/tmp/script.py', ';', 'rm', '/tmp/script.py'], stdout='* Waiting for server restart... \\ \r* Waiting for server restart... | \r* Waiting for server restart... / \r* Waiting for server restart... - \r* Waiting for server restart... \\ \r* Waiting for server restart... | \r* Waiting for server restart... / \r* Waiting for server restart... - \r* Waiting for server restart... \\ \r* Waiting for server restart... | \r* Waiting for server restart... ready \n', stderr='Cannot set LC_ALL to locale en_US.UTF-8: No such file or directory\nverbose: 2024-01-18T11:08:23Z: Loading startup files...\nverbose: 2024-01-18T11:08:23Z: Loading plugins...\nverbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local\nverbose: 2024-01-18T11:08:23Z: Shell.connect: tid=177: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local\nverbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000\nverbose: 2024-01-18T11:08:23Z: Dba.get_cluster: tid=178: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306\nverbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000\nverbose: 2024-01-18T11:08:23Z: Dba.get_cluster: tid=179: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306\nverbose: 2024-01-18T11:08:23Z: Group ' [truncated] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/var/lib/juju/agents/unit-mysql-k8s-0/charm/lib/charms/mysql/v0/mysql.py", line 1326, in add_instance_to_cluster self._run_mysqlsh_script("\n".join(connect_commands + add_instance_command)) File "/var/lib/juju/agents/unit-mysql-k8s-0/charm/src/mysql_k8s_helpers.py", line 669, in _run_mysqlsh_script raise MySQLClientError(e.stderr) charms.mysql.v0.mysql.MySQLClientError: Cannot set LC_ALL to locale en_US.UTF-8: No such file or directory verbose: 2024-01-18T11:08:23Z: Loading startup files... verbose: 2024-01-18T11:08:23Z: Loading plugins... verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local verbose: 2024-01-18T11:08:23Z: Shell.connect: tid=177: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:23Z: Dba.get_cluster: tid=178: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:23Z: Dba.get_cluster: tid=179: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:23Z: Group Replication 'group_name' value: d3a27e02-b5f1-11ee-be1f-0e739010502b verbose: 2024-01-18T11:08:23Z: Metadata 'group_name' value: d3a27e02-b5f1-11ee-be1f-0e739010502b verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:23Z: Dba.get_cluster: tid=180: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:23Z: Dba.get_cluster: tid=181: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:23Z: Cluster.add_instance: tid=182: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:23Z: Cluster.add_instance: tid=183: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:23Z: Cluster.add_instance: tid=184: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:23Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=185: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=187: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=188: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=42: CONNECTED: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local NOTE: The target instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' has not been pre-provisioned (GTID set is empty). The Shell is unable to decide whether incremental state recovery can correctly provision it. Clone based recovery selected through the recoveryMethod option verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=42: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 Validating instance configuration at mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306... verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=43: CONNECTED: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:24Z: Validating account clusteradmin@%... verbose: 2024-01-18T11:08:24Z: clusteradmin user has 1 accounts with wildcard or netmask and 0 without This instance reports its own address as mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:24Z: Validating InnoDB page size of instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306'. verbose: 2024-01-18T11:08:24Z: Checking if performance_schema is enabled on instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306'. verbose: 2024-01-18T11:08:24Z: Validating configuration of mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 (mycnf = ) Instance configuration is suitable. verbose: 2024-01-18T11:08:24Z: Validating if 'lower_case_table_names' variable has the same value on target instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' as it does on the cluster. verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=43: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:24Z: Validating if 'default_table_encryption' variable has the same value on target instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' as it does on the cluster. verbose: 2024-01-18T11:08:24Z: SSL mode used to configure the instance: 'REQUIRED' verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=190: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=44: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 NOTE: Group Replication will communicate with other members using 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306'. Use the localAddress option to override. verbose: 2024-01-18T11:08:24Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:24Z: Cluster.add_instance: tid=45: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 * Checking connectivity and SSL configuration... verbose: 2024-01-18T11:08:24Z: Checking connection from mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 to mysqlsh-lo.test@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 (localAddress) ssl_mode=REQUIRED auth_type=PASSWORD cert_issuer= cert_subject= verbose: 2024-01-18T11:08:24Z: Setting up async source for channel 'mysqlsh.test' of mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 to mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 (user 'mysqlsh-lo.test') verbose: 2024-01-18T11:08:24Z: Connection check mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 -> mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 io_state=OFF io_error=Fatal error: The replica I/O thread stops because source and replica have equal MySQL server ids; these ids must be different for replication to work (or the --replicate-same-server-id option must be used on replica but this does not always make sense; please check the manual before using it). (13117) at 2024-01-18 11:08:24.364453 verbose: 2024-01-18T11:08:24Z: Checking connection from mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 to mysqlsh.test@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 (report_host) ssl_mode=REQUIRED auth_type=PASSWORD cert_issuer= cert_subject= verbose: 2024-01-18T11:08:24Z: Setting up async source for channel 'mysqlsh.test' of mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 to mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 (user 'mysqlsh.test') verbose: 2024-01-18T11:08:24Z: Connection check mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 -> mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 io_state=OFF io_error=Source command COM_REGISTER_REPLICA failed: Access denied for user 'mysqlsh.test'@'%' (using password: YES) (Errno: 1045) (13120) at 2024-01-18 11:08:24.622708 verbose: 2024-01-18T11:08:24Z: Checking connection from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 to mysqlsh.test@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 (report_host) ssl_mode=REQUIRED auth_type=PASSWORD cert_issuer= cert_subject= verbose: 2024-01-18T11:08:24Z: Setting up async source for channel 'mysqlsh.test' of mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 to mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 (user 'mysqlsh.test') verbose: 2024-01-18T11:08:25Z: Connection check mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 -> mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 io_state=OFF io_error=Source command COM_REGISTER_REPLICA failed: Access denied for user 'mysqlsh.test'@'%' (using password: YES) (Errno: 1045) (13120) at 2024-01-18 11:08:24.879617 A new instance will be added to the InnoDB Cluster. Depending on the amount of data on the cluster this might take from a few seconds to several hours. verbose: 2024-01-18T11:08:25Z: Using Group Replication local address: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:25Z: Using Group Replication group seeds: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306,mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:25Z: Using Group Replication failover consistency: EVENTUAL verbose: 2024-01-18T11:08:25Z: Using Group Replication expel timeout: 5 Adding instance to the cluster... verbose: 2024-01-18T11:08:25Z: Validating if 'group_replication_gtid_assignment_block_size' variable has the same value on target instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' as it does on the cluster. verbose: 2024-01-18T11:08:25Z: Installing the clone plugin on instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306'. verbose: 2024-01-18T11:08:25Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:25Z: Cluster.add_instance: tid=201: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:25Z: Installing the clone plugin on instance 'mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306'. verbose: 2024-01-18T11:08:25Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:25Z: Cluster.add_instance: tid=46: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:25Z: Installing the clone plugin on instance 'mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306'. verbose: 2024-01-18T11:08:25Z: Creating recovery account 'mysql_innodb_cluster_530989616'@'%' for instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' verbose: 2024-01-18T11:08:25Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:25Z: Cluster.add_instance: tid=48: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:25Z: Joining 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' to cluster using account 'clusteradmin' to peer 'mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306'. verbose: 2024-01-18T11:08:25Z: Creating recovery account 'mysql_innodb_cluster_530989616'@'%' for instance 'mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' verbose: 2024-01-18T11:08:25Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:25Z: Cluster.add_instance: tid=49: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:25Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:25Z: Cluster.add_instance: tid=207: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:25Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:25Z: Cluster.add_instance: tid=50: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 Monitoring recovery process of the new cluster member. Press ^C to stop monitoring and let it continue in background. verbose: 2024-01-18T11:08:27Z: Waiting for GR recovery to start for mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306... verbose: 2024-01-18T11:08:27Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:27Z: Cluster.add_instance: tid=69: CONNECTED: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:27Z: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 has started verbose: 2024-01-18T11:08:27Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:27Z: Cluster.add_instance: tid=72: CONNECTED: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 Clone based state recovery is now in progress. NOTE: A server restart is expected to happen as part of the clone process. If the server does not support the RESTART command or does not come back after a while, you may need to manually start it back. * Waiting for clone to finish... NOTE: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is being cloned from mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 ** Stage DROP DATA: Completed ** Stage FILE COPY: Completed ** Stage PAGE COPY: Completed ** Stage REDO COPY: Completed verbose: 2024-01-18T11:08:31Z: Cluster.add_instance: tid=72: MySQL Error 1053 (08S01): Server shutdown in progress, SQL: SELECT *, end_time-begin_time as elapsed FROM performance_schema.clone_status WHERE begin_time >= '2024-01-18 11:08:25.197' ORDER BY id DESC LIMIT 1 NOTE: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is shutting down... verbose: 2024-01-18T11:08:31Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:32Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:33Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:34Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:35Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:36Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:37Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:38Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:39Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:40Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:41Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:08:41Z: Cluster.add_instance: tid=24: CONNECTED: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:08:41Z: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 has started * mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 has restarted, waiting for clone to finish... ** Stage FILE SYNC: Completed ** Stage RESTART: Completed * Clone process has finished: 74.34 MB transferred in about 1 second (~74.34 MB/s) verbose: 2024-01-18T11:08:41Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:42Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:43Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:44Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:45Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:46Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:47Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:48Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:49Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:50Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:51Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:52Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:53Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:54Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:55Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:56Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:57Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:58Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:08:59Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:00Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:01Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:02Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:03Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:04Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:05Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:06Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:07Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:08Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:09Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:10Z: Member mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 is in state OFFLINE, when recovery was expected (clone_state=Completed@2024-01-18 11:08:27.184). verbose: 2024-01-18T11:09:11Z: Cluster.add_instance: tid=42: MySQL Error 2013 (HY000): Lost connection to MySQL server during query, SQL: SELECT 1 verbose: 2024-01-18T11:09:11Z: Target connection to mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 lost: MySQL Error 2013 (HY000): Lost connection to MySQL server during query. Reconnecting... verbose: 2024-01-18T11:09:11Z: Cluster.add_instance: tid=25: CONNECTED: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:09:13Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:09:13Z: Cluster.add_instance: tid=271: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:09:13Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:09:13Z: Cluster.add_instance: tid=83: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:09:13Z: Cluster.add_instance: tid=25: MySQL Error 3021 (HY000): This operation cannot be performed with a running replica io thread; run STOP REPLICA IO_THREAD FOR CHANNEL 'group_replication_recovery' first., SQL: CHANGE REPLICATION SOURCE TO SOURCE_USER = /*(*/ 'mysql_innodb_cluster_530989616' /*)*/, SOURCE_PASSWORD = **** FOR CHANNEL 'group_replication_recovery' verbose: 2024-01-18T11:09:13Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:09:13Z: Cluster.add_instance: tid=276: CONNECTED: mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:09:13Z: Resetting recovery account credentials for 'mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' (with password). verbose: 2024-01-18T11:09:13Z: Connecting to MySQL at: mysql://clusteradmin@mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306?connect-timeout=5000 verbose: 2024-01-18T11:09:13Z: Cluster.add_instance: tid=84: CONNECTED: mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306 verbose: 2024-01-18T11:09:13Z: Resetting recovery account credentials for 'mysql-k8s-1.mysql-k8s-endpoints.foo6.svc.cluster.local:3306' (with password). ERROR: Cannot set replication user of channel 'group_replication_recovery' to 'mysql_innodb_cluster_530989616'. Error executing CHANGE SOURCE statement: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306: This operation cannot be performed with a running replica io thread; run STOP REPLICA IO_THREAD FOR CHANNEL 'group_replication_recovery' first. Traceback (most recent call last): File "", line 4, in RuntimeError: Cluster.add_instance: Cannot set replication user of channel 'group_replication_recovery' to 'mysql_innodb_cluster_530989616'. Error executing CHANGE SOURCE statement: mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local:3306: This operation cannot be performed with a running replica io thread; run STOP REPLICA IO_THREAD FOR CHANNEL 'group_replication_recovery' first. unit-mysql-k8s-0: 11:09:13 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Releasing lock unit-add on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local for unit mysql-k8s-0 unit-mysql-router-k8s-1: 11:09:13 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:09:13 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:13 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:13 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: cluster_set_router_ids=dict_keys(['mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:13 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-1: 11:09:13 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Created database='continuous_writes_database' and username='relation-7-6' with attributes='{"created_by_user": "relation-7"}' unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Setting databag self._id=6 self._database='continuous_writes_database', username='relation-7-6', router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Set databag self._id=6 self._database='continuous_writes_database', username='relation-7-6', router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Reconciled users event=, router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Enabling MySQL Router service unit-mysql-router-k8s-0: 11:09:13 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Getting MySQL Router user for unit_name='mysql-router-k8s/0' unit-mysql-k8s-0: 11:09:13 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Unable to add instance mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local to cluster. unit-mysql-router-k8s-0: 11:09:14 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:14 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:14 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-0: 11:09:14 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: No MySQL Router user found for unit_name='mysql-router-k8s/0' unit-mysql-router-k8s-0: 11:09:14 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Bootstrapping router tls=False, self._connection_info.host='mysql-k8s-primary.foo6.svc.cluster.local', self._connection_info.port='3306' unit-mysql-router-k8s-1: 11:09:14 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:09:14 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:14 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:14 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: cluster_set_router_ids=dict_keys(['mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:14 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Checked if router_id in cluster_set_router_ids=True unit-mysql-k8s-0: 11:09:14 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:14 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:14 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Set unit status to ActiveStatus('') unit-mysql-router-k8s-1: 11:09:14 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:14 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Bootstrapped router tls=False, self._connection_info.host='mysql-k8s-primary.foo6.svc.cluster.local', self._connection_info.port='3306' unit-mysql-router-k8s-0: 11:09:14 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Adding attributes='{"created_by_user": "relation-7", "router_id": "mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system", "created_by_juju_unit": "mysql-router-k8s/0"}' to username='mysql_router3_hpu5yr3ltulr' unit-mysql-k8s-0: 11:09:14 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:09:14 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:09:15 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Checking existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-router-k8s-0: 11:09:15 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:15 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:15 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Added attributes='{"created_by_user": "relation-7", "router_id": "mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system", "created_by_juju_unit": "mysql-router-k8s/0"}' to username='mysql_router3_hpu5yr3ltulr' unit-mysql-k8s-2: 11:09:15 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-1: 11:09:15 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:15 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-k8s-2: 11:09:15 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-k8s-1: 11:09:15 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:09:15 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Failed to confirm existence of unit mysql-k8s-0 in cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 unit-mysql-k8s-0: 11:09:15 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Getting cluster primary member's address from mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local unit-mysql-k8s-1: 11:09:15 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:09:15 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:09:15 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:09:15 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:15 INFO juju.worker.uniter awaiting error resolution for "relation-changed" hook unit-mysql-router-k8s-1: 11:09:16 DEBUG unit.mysql-router-k8s/1.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:16 DEBUG unit.mysql-router-k8s/1.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-0: 11:09:16 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Creating logrotate config file unit-mysql-router-k8s-0: 11:09:16 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Created logrotate config file unit-mysql-router-k8s-0: 11:09:16 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Copying log rotate executor script to workload container unit-mysql-router-k8s-0: 11:09:16 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Copied log rotate executor to workload container unit-mysql-router-k8s-0: 11:09:16 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Starting the logrotate executor service unit-mysql-router-k8s-1: 11:09:16 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-2: 11:09:16 INFO unit.mysql-router-k8s/2.juju-log backend-database:7: database created at 2024-01-18 11:09:16.505599 unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting custom event . unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Enabling MySQL Router service unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Getting MySQL Router user for unit_name='mysql-router-k8s/2' unit-mysql-router-k8s-1: 11:09:16 DEBUG unit.mysql-router-k8s/1.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:16 DEBUG unit.mysql-router-k8s/1.juju-log database:6: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-2: 11:09:16 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: No MySQL Router user found for unit_name='mysql-router-k8s/2' unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Bootstrapping router tls=False, self._connection_info.host='mysql-k8s-primary.foo6.svc.cluster.local', self._connection_info.port='3306' unit-mysql-router-k8s-1: 11:09:17 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Started the logrotate executor service unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Enabled MySQL Router service unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Waiting until MySQL Router is ready unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: MySQL Router is ready unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Checking if router_id='mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Bootstrapped router tls=False, self._connection_info.host='mysql-k8s-primary.foo6.svc.cluster.local', self._connection_info.port='3306' unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Adding attributes='{"created_by_user": "relation-7", "router_id": "mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system", "created_by_juju_unit": "mysql-router-k8s/2"}' to username='mysql_router3_2ztata4z6hzj' unit-mysql-router-k8s-1: 11:09:17 DEBUG unit.mysql-router-k8s/1.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:17 DEBUG unit.mysql-router-k8s/1.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:17 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Added attributes='{"created_by_user": "relation-7", "router_id": "mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system", "created_by_juju_unit": "mysql-router-k8s/2"}' to username='mysql_router3_2ztata4z6hzj' unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._partition=0, partition_=0 unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:17 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:09:17 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Set app status to ActiveStatus('') unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Checking if router_id='mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-1: 11:09:18 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:18 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_joined. unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:18 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Set unit status to ActiveStatus('') unit-mysql-router-k8s-1: 11:09:18 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:18 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Creating logrotate config file unit-mysql-router-k8s-2: 11:09:18 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Created logrotate config file unit-mysql-router-k8s-2: 11:09:18 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Copying log rotate executor script to workload container unit-mysql-router-k8s-2: 11:09:18 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Copied log rotate executor to workload container unit-mysql-router-k8s-2: 11:09:18 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Starting the logrotate executor service unit-mysql-router-k8s-0: 11:09:18 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:09:19 DEBUG unit.mysql-test-app/0.juju-log database:6: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:09:19 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-test-app-0: 11:09:19 INFO unit.mysql-test-app/0.juju-log database:6: database created at 2024-01-18 11:09:19.226271 unit-mysql-test-app-0: 11:09:19 DEBUG unit.mysql-test-app/0.juju-log database:6: Emitting custom event . unit-mysql-router-k8s-1: 11:09:19 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:19 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_changed. unit-mysql-router-k8s-0: 11:09:19 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:19 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-k8s-2: 11:09:19 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:09:19 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Attempting to acquire lock unit-add on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local for unit mysql-k8s-0 unit-mysql-k8s-2: 11:09:19 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:09:19 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:09:19 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-test-app-0: 11:09:19 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:19 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:09:19 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-k8s-0: 11:09:19 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Adding instance mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local/mysql-k8s-0 to cluster cluster-265b6141cc9a90bb9af1eee2b4367b24 with recovery method auto unit-mysql-router-k8s-2: 11:09:19 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Started the logrotate executor service unit-mysql-router-k8s-2: 11:09:19 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Enabled MySQL Router service unit-mysql-router-k8s-2: 11:09:19 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Waiting until MySQL Router is ready unit-mysql-router-k8s-2: 11:09:19 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: MySQL Router is ready unit-mysql-router-k8s-2: 11:09:19 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-test-app-0: 11:09:19 DEBUG unit.mysql-test-app/0.juju-log application-peers:3: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:09:19 DEBUG unit.mysql-test-app/0.juju-log application-peers:3: Emitting Juju event application_peers_relation_changed. unit-mysql-k8s-2: 11:09:19 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:09:20 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:20 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:20 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_joined. unit-mysql-test-app-0: 11:09:20 INFO unit.mysql-test-app/0.juju-log application-peers:3: Started continuous writes unit-mysql-test-app-0: 11:09:20 INFO unit.mysql-test-app/0.juju-log application-peers:3: Wrote random_value unit-mysql-router-k8s-0: 11:09:20 DEBUG unit.mysql-router-k8s/0.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:20 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-1: 11:09:20 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:09:20 INFO juju.worker.uniter.operation ran "application-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:20 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-test-app-0: 11:09:20 DEBUG unit.mysql-test-app/0.juju-log application-peers:3: ops 2.8.0 up and running. unit-mysql-test-app-0: 11:09:20 DEBUG unit.mysql-test-app/0.juju-log application-peers:3: Emitting Juju event application_peers_relation_changed. unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:20 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-1: 11:09:21 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:21 DEBUG unit.mysql-router-k8s/1.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_changed. unit-mysql-test-app-0: 11:09:21 INFO juju.worker.uniter.operation ran "application-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:21 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:21 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_joined. unit-mysql-router-k8s-2: 11:09:21 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:21 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:21 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:21 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:21 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-1: 11:09:21 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:21 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:21 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Set unit status to BlockedStatus('Router was manually removed from MySQL ClusterSet. Remove & re-deploy unit') unit-mysql-router-k8s-0: 11:09:21 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:21 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-router-k8s-0: 11:09:22 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:22 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_changed. unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:22 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:22 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-router-k8s-0: 11:09:22 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:09:22 DEBUG unit.mysql-k8s/1.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:09:22 DEBUG unit.mysql-k8s/2.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:09:22 DEBUG unit.mysql-k8s/1.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-k8s-2: 11:09:22 DEBUG unit.mysql-k8s/2.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:09:22 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-1: 11:09:22 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-k8s-1: 11:09:22 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:09:22 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:23 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:23 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_joined. unit-mysql-router-k8s-2: 11:09:23 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:23 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-0: 11:09:23 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:23 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:09:23 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:23 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:23 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:23 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-1: 11:09:23 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:23 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Set unit status to ActiveStatus('') unit-mysql-router-k8s-2: 11:09:23 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:23 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:23 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:23 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-k8s-0: 11:09:24 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Releasing lock unit-add on mysql-k8s-2.mysql-k8s-endpoints.foo6.svc.cluster.local for unit mysql-k8s-0 unit-mysql-router-k8s-2: 11:09:24 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:24 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-router-k8s-0: 11:09:24 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:24 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:24 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-k8s-0: 11:09:24 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Added instance mysql-k8s-0.mysql-k8s-endpoints.foo6.svc.cluster.local to cluster unit-mysql-k8s-0: 11:09:24 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Instance mysql-k8s-0 is cluster member unit-mysql-router-k8s-2: 11:09:24 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:24 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/0 unit-mysql-router-k8s-1: 11:09:24 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:24 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:24 DEBUG unit.mysql-router-k8s/0.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_changed. unit-mysql-router-k8s-2: 11:09:25 DEBUG unit.mysql-router-k8s/2.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:25 DEBUG unit.mysql-router-k8s/2.juju-log database:6: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-0: 11:09:25 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:25 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Labeling pod mysql-k8s-0 with label replicas unit-mysql-k8s-0: 11:09:25 INFO unit.mysql-k8s/0.juju-log database-peers:0: Kubernetes pod label replicas created unit-mysql-router-k8s-1: 11:09:25 DEBUG unit.mysql-router-k8s/1.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:25 DEBUG unit.mysql-router-k8s/1.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:09:25 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:25 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:25 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:25 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:25 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_joined. unit-mysql-router-k8s-2: 11:09:26 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:26 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_joined. unit-mysql-router-k8s-0: 11:09:26 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:26 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:26 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_joined. unit-mysql-k8s-2: 11:09:26 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-router-k8s-2: 11:09:26 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:09:26 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:09:26 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:09:26 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-1: 11:09:26 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:09:26 INFO unit.mysql-k8s/0.juju-log database-peers:0: Starting the log rotate manager unit-mysql-k8s-0: 11:09:26 INFO unit.mysql-k8s/0.juju-log database-peers:0: Started log rotate manager process with PID 2262 unit-mysql-k8s-0: 11:09:26 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-router-k8s-1: 11:09:26 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:09:26 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/2 unit-mysql-k8s-1: 11:09:26 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/1 unit-mysql-k8s-0: 11:09:26 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/0 unit-mysql-router-k8s-0: 11:09:26 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:26 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-router-k8s-2: 11:09:27 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:27 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-0: 11:09:27 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:27 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:27 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-k8s-2: 11:09:27 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Labeling pod mysql-k8s-2 with label primary unit-mysql-k8s-1: 11:09:27 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Labeling pod mysql-k8s-1 with label replicas unit-mysql-k8s-0: 11:09:27 DEBUG unit.mysql-k8s/0.juju-log database-peers:0: Labeling pod mysql-k8s-0 with label replicas unit-mysql-router-k8s-2: 11:09:27 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:27 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:27 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-1: 11:09:27 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-k8s-2: 11:09:27 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-1: 11:09:27 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:27 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:27 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:27 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-2: 11:09:28 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:28 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-0: 11:09:28 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:28 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:09:28 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:09:28 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: ops 2.9.0 up and running. unit-mysql-k8s-2: 11:09:28 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-k8s-0: 11:09:28 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-1: 11:09:28 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Emitting Juju event database_peers_relation_changed. unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:28 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-k8s-0: 11:09:28 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:09:29 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-k8s-2: 11:09:29 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/2 unit-mysql-router-k8s-1: 11:09:29 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Set unit status to ActiveStatus('') unit-mysql-k8s-1: 11:09:29 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Updating pod endpoint for mysql-k8s/1 unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-router-k8s-2: 11:09:29 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:29 DEBUG unit.mysql-router-k8s/2.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-k8s-0: 11:09:29 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:29 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Reconciling users event=, router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-2: 11:09:29 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: State of reconcile users requested_users=[], self._shared_users=[] unit-mysql-k8s-2: 11:09:29 DEBUG unit.mysql-k8s/2.juju-log database-peers:0: Labeling pod mysql-k8s-2 with label primary unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Reconciled users event=, router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-0: 11:09:29 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-k8s-1: 11:09:29 DEBUG unit.mysql-k8s/1.juju-log database-peers:0: Labeling pod mysql-k8s-1 with label replicas unit-mysql-router-k8s-1: 11:09:30 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-k8s-2: 11:09:30 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:30 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-k8s-1: 11:09:30 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:30 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-k8s-0: 11:09:30 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:30 DEBUG unit.mysql-router-k8s/2.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._partition=0, partition_=0 unit-mysql-router-k8s-2: 11:09:30 DEBUG unit.mysql-router-k8s/2.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:09:30 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set app status to ActiveStatus('') unit-mysql-router-k8s-0: 11:09:30 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-k8s-0: 11:09:30 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:30 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:30 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:30 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_joined. unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set unit status to ActiveStatus('') unit-mysql-router-k8s-2: 11:09:31 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:31 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_changed. unit-mysql-router-k8s-1: 11:09:31 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:31 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:09:31 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:31 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:09:31 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:31 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:31 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:31 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:31 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_joined. unit-mysql-router-k8s-2: 11:09:32 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:32 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_joined. unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:09:32 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:32 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:32 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-k8s-0: 11:09:32 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-1: 11:09:32 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:09:32 DEBUG unit.mysql-router-k8s/0.juju-log database:6: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:32 DEBUG unit.mysql-router-k8s/0.juju-log database:6: Emitting Juju event database_relation_changed. unit-mysql-k8s-0: 11:09:33 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:33 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:33 DEBUG unit.mysql-router-k8s/2.juju-log mysql-router-peers:4: Emitting Juju event mysql_router_peers_relation_changed. unit-mysql-router-k8s-1: 11:09:33 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:33 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:33 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-1: 11:09:33 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-1: 11:09:33 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-1: 11:09:33 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-1: 11:09:33 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:33 DEBUG unit.mysql-router-k8s/1.juju-log upgrade-version-a:5: Set unit status to ActiveStatus('') unit-mysql-router-k8s-2: 11:09:33 INFO juju.worker.uniter.operation ran "mysql-router-peers-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:33 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:33 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:09:33 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:33 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-k8s-0: 11:09:33 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-2: 11:09:33 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:33 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=True, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-k8s-0: 11:09:34 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Reconciling users event=, router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-1: 11:09:34 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-1: 11:09:34 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: State of reconcile users requested_users=[], self._shared_users=[] unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Reconciled users event=, router_read_write_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6446', router_read_only_endpoint='mysql-router-k8s.foo6.svc.cluster.local:6447' unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-1: 11:09:34 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:34 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-0: 11:09:34 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-k8s-0: 11:09:34 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._partition=0, partition_=0 unit-mysql-k8s-0: 11:09:35 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_joined. unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Setting self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set self._current_versions={'charm': '1.2+40a85e4', 'workload': '8.0.34'} in upgrade peer relation app databag unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-1: 11:09:35 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set app status to ActiveStatus('') unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-1: 11:09:35 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-k8s-0: 11:09:35 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:35 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=True unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-0: 11:09:35 DEBUG unit.mysql-router-k8s/0.juju-log upgrade-version-a:5: Set unit status to ActiveStatus('') unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:35 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Set unit status to BlockedStatus('Router was manually removed from MySQL ClusterSet. Remove & re-deploy unit') unit-mysql-k8s-0: 11:09:36 DEBUG unit.mysql-k8s/0.juju-log database:7: ops 2.9.0 up and running. unit-mysql-router-k8s-0: 11:09:36 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:36 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:36 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-1: 11:09:36 DEBUG unit.mysql-router-k8s/1.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-k8s-0: 11:09:36 DEBUG unit.mysql-k8s/0.juju-log database:7: Emitting Juju event database_relation_changed. unit-mysql-router-k8s-1: 11:09:36 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:09:36 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-0: 11:09:36 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:36 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_joined. unit-mysql-router-k8s-2: 11:09:36 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:36 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_joined. unit-mysql-router-k8s-0: 11:09:37 INFO juju.worker.uniter.operation ran "backend-database-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:37 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:37 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-0: 11:09:37 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:37 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-router-k8s-0: 11:09:37 DEBUG unit.mysql-router-k8s/0.juju-log backend-database:7: Emitting Juju event backend_database_relation_changed. unit-mysql-router-k8s-2: 11:09:37 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:37 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-2: 11:09:37 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-0: 11:09:38 INFO juju.worker.uniter.operation ran "backend-database-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:38 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:39 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:39 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:39 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:39 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:39 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:39 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:39 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Set unit status to BlockedStatus('Router was manually removed from MySQL ClusterSet. Remove & re-deploy unit') unit-mysql-router-k8s-2: 11:09:39 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:40 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:40 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_joined. unit-mysql-router-k8s-2: 11:09:40 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch) unit-mysql-router-k8s-2: 11:09:40 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Emitting Juju event upgrade_version_a_relation_changed. unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:41 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:09:42 DEBUG unit.mysql-router-k8s/2.juju-log upgrade-version-a:5: Set unit status to BlockedStatus('Router was manually removed from MySQL ClusterSet. Remove & re-deploy unit') unit-mysql-router-k8s-2: 11:09:42 INFO juju.worker.uniter.operation ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch) unit-mysql-k8s-0: 11:10:00 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:10:00 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:10:00 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:10:00 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-1: 11:10:00 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-2: 11:10:00 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-router-k8s-2: 11:10:56 DEBUG unit.mysql-router-k8s/2.juju-log ops 2.6.0 up and running. unit-mysql-router-k8s-2: 11:10:56 DEBUG unit.mysql-router-k8s/2.juju-log Emitting Juju event update_status. unit-mysql-router-k8s-2: 11:10:56 DEBUG unit.mysql-router-k8s/2.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:10:56 DEBUG unit.mysql-router-k8s/2.juju-log State of reconcile self._unit_lifecycle.authorized_leader=False, isinstance(workload_, workload.AuthenticatedWorkload)=True, workload_.container_ready=True, self._database_requires.is_relation_breaking(event)=False, self._upgrade.in_progress=False unit-mysql-router-k8s-2: 11:10:56 DEBUG unit.mysql-router-k8s/2.juju-log Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:10:57 DEBUG unit.mysql-router-k8s/2.juju-log Checking if router_id='mysql-router-k8s-2.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system' in cluster set unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_script.py') unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_error.json') unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log Deleted file self=_Path('/tmp/mysqlsh_output.json') unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log cluster_set_router_ids=dict_keys(['mysql-router-k8s-0.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system', 'mysql-router-k8s-1.mysql-router-k8s-endpoints.foo6.svc.cluster.local::system']) unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log Checked if router_id in cluster_set_router_ids=False unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log self._app_workload_version='mysql-router-k8s-5d95498478' self._unit_workload_versions={'mysql-router-k8s/0': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/1': 'mysql-router-k8s-5d95498478', 'mysql-router-k8s/2': 'mysql-router-k8s-5d95498478'} unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log Set unit status to BlockedStatus('Router was manually removed from MySQL ClusterSet. Remove & re-deploy unit') unit-mysql-router-k8s-2: 11:10:58 WARNING unit.mysql-router-k8s/2.juju-log No relation: certificates unit-mysql-router-k8s-2: 11:10:58 DEBUG unit.mysql-router-k8s/2.juju-log The previous update-status was fired by jhack. unit-mysql-k8s-2: 11:11:00 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:11:00 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:11:00 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:11:00 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-1: 11:11:00 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-0: 11:11:00 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-test-app-0: 11:11:26 DEBUG unit.mysql-test-app/0.juju-log ops 2.8.0 up and running. unit-mysql-test-app-0: 11:11:26 DEBUG unit.mysql-test-app/0.juju-log Emitting Juju event update_status. unit-mysql-test-app-0: 11:11:26 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch) unit-mysql-k8s-2: 11:12:00 DEBUG unit.mysql-k8s/2.juju-log ops 2.9.0 up and running. unit-mysql-k8s-0: 11:12:00 DEBUG unit.mysql-k8s/0.juju-log ops 2.9.0 up and running. unit-mysql-k8s-1: 11:12:00 DEBUG unit.mysql-k8s/1.juju-log ops 2.9.0 up and running. unit-mysql-k8s-2: 11:12:00 DEBUG unit.mysql-k8s/2.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-0: 11:12:00 DEBUG unit.mysql-k8s/0.juju-log Emitting Juju event rotate_mysql_logs. unit-mysql-k8s-1: 11:12:00 DEBUG unit.mysql-k8s/1.juju-log Emitting Juju event rotate_mysql_logs.