Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ctrl+C can't make tidb exit with 1M tables using infoschema v2 #54589

Closed
tiancaiamao opened this issue Jul 12, 2024 · 0 comments · Fixed by #54594
Closed

Ctrl+C can't make tidb exit with 1M tables using infoschema v2 #54589

tiancaiamao opened this issue Jul 12, 2024 · 0 comments · Fixed by #54594
Labels
component/statistics feature/developing the related feature is in development severity/moderate sig/planner SIG: Planner type/bug The issue is confirmed as a bug.

Comments

@tiancaiamao
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

I'm using @@global.tidb_schema_cache_size = '512MB' and there are 1M tables.
Also the configuration force-init-stats = false
Start TiDB and wait 4000 port ready.
Then Ctrl+C to exit tidb

2. What did you expect to see? (Required)

TiDB exit

3. What did you see instead (Required)

It takes 12m54s to exit!

[2024/07/11 16:25:27.006 +08:00] [INFO] [domain.go:1164] ["domain closed"] ["take time"=12m54.727120057s]
[2024/07/11 16:25:27.006 +08:00] [INFO] [gc_worker.go:223] [quit] [category="gc worker"] [uuid=642836439700005]

pd service discovery and tso dispatcher exit after domain close

[2024/07/11 16:25:27.006 +08:00] [INFO] [pd_service_discovery.go:550] ["[pd] exit member loop due to context canceled"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_dispatcher.go:210] ["[tso] stop fetching the pending tso requests due to context canceled"] [dc-location=global]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_client.go:147] ["[tso] exit tso dispatcher check loop"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_dispatcher.go:350] ["[tso] exit tso connection contexts updater"] [dc-location=global]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_dispatcher.go:171] ["[tso] exit tso dispatcher"] [dc-location=global]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_batch_controller.go:160] ["[pd] clear the tso batch controller"] [max-batch-size=10000] [best-batch-size=1] [collected-request-count=0] [pending-request-count=0]
[2024/07/11 16:25:27.006 +08:00] [INFO] [tso_dispatcher.go:137] ["[tso] exit tso deadline watcher"] [dc-location=global]
[2024/07/11 16:25:27.007 +08:00] [INFO] [resource_manager_client.go:290] ["[resource manager] exit resource token dispatcher"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_client.go:157] ["[tso] closing tso client"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_client.go:162] ["[tso] close tso client"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_batch_controller.go:160] ["[pd] clear the tso batch controller"] [max-batch-size=10000] [best-batch-size=1] [collected-request-count=0] [pending-request-count=0]
[2024/07/11 16:25:27.007 +08:00] [INFO] [tso_client.go:164] ["[tso] tso client is closed"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [pd_service_discovery.go:637] ["[pd] close pd service discovery client"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [client.go:347] ["[pd] http client closed"] [source=tikv-driver]

sql cpu collector and cgroup monitor exit after domain:

[2024/07/11 16:25:27.007 +08:00] [INFO] [cpu.go:98] ["sql cpu collector stopped"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [cgmon.go:77] ["cgroup monitor stopped"]
[2024/07/11 16:25:27.007 +08:00] [INFO] [cpuprofile.go:130] ["parallel cpu profiler stopped"]

gc worker print such log every minute, it's not exited after receiving the signal:

[2024/07/11 16:17:42.998 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche
[2024/07/11 16:18:42.995 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche
[2024/07/11 16:19:42.999 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche
[2024/07/11 16:20:42.999 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche
[2024/07/11 16:21:42.994 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche
[2024/07/11 16:22:42.999 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche
[2024/07/11 16:23:42.999 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche
[2024/07/11 16:24:42.999 +08:00] [WARN] [gc_worker.go:274] ["check leader"] [category="gc worker"] [error="[domain:8027]Information sche

The update stats worker exited immediately:

[2024/07/11 16:12:34.965 +08:00] [INFO] [domain.go:2490] ["updateStatsWorker exit preprocessing finished"]
[2024/07/11 16:12:34.965 +08:00] [INFO] [domain.go:2533] ["updateStatsWorker exited."]

4. What is your TiDB version? (Required)

master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/statistics feature/developing the related feature is in development severity/moderate sig/planner SIG: Planner type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants