Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util: refine expensive query log during bootstrap (#14181) #16144

Merged
merged 4 commits into from
Apr 13, 2020

Conversation

sre-bot
Copy link
Contributor

@sre-bot sre-bot commented Apr 8, 2020

cherry-pick #14181 to release-3.1


What problem does this PR solve?

Before this commit:

  1. modify the MemQuotaQuery to 32 in config.go
  2. compile and try to run the tidb-server binary
  3. tidb-server panic and log the following error messge
[2019/12/23 15:13:35.681 +08:00] [ERROR] [adapter.go:122] ["execute sql panic"] [sql="select HIGH_PRIORITY Host,User,Password,Create_priv, Super_priv, Execute_priv, Create_view_priv, Create_role_priv, Event_priv, Update_priv, Show_db_priv, Create_user_priv, Trigger_priv, Grant_priv, References_priv, Lock_tables_priv, Drop_role_priv, Select_priv, Insert_priv, Drop_priv, Process_priv, Alter_priv, Index_priv, Show_view_priv, Alter_routine_priv, Shutdown_priv, Delete_priv, Create_tmp_table_priv, Create_routine_priv,account_locked from mysql.user;"] [stack="github.com/pingcap/tidb/executor.(*recordSet).Next.func1\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/adapter.go:122\nruntime.gopanic\n\t/Users/xuhuaiyu/.gvm/gos/go1.13/src/runtime/panic.go:679\nruntime.panicmem\n\t/Users/xuhuaiyu/.gvm/gos/go1.13/src/runtime/panic.go:199\nruntime.sigpanic\n\t/Users/xuhuaiyu/.gvm/gos/go1.13/src/runtime/signal_unix.go:394\ngithub.com/pingcap/tidb/util/expensivequery.(*Handle).LogOnQueryExceedMemQuota\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/util/expensivequery/expensivequery.go:85\ngithub.com/pingcap/tidb/util/memory.(*LogOnExceed).Action\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/util/memory/action.go:64\ngithub.com/pingcap/tidb/util/memory.(*Tracker).Consume\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/util/memory/tracker.go:194\ngithub.com/pingcap/tidb/executor.(*ProjectionExec).prepare\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/projection.go:248\ngithub.com/pingcap/tidb/executor.(*ProjectionExec).parallelExecute\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/projection.go:199\ngithub.com/pingcap/tidb/executor.(*ProjectionExec).Next\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/projection.go:172\ngithub.com/pingcap/tidb/executor.Next\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/executor.go:212\ngithub.com/pingcap/tidb/executor.(*recordSet).Next\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/executor/adapter.go:125\ngithub.com/pingcap/tidb/privilege/privileges.(*MySQLPrivilege).loadTable\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/privilege/privileges/cache.go:522\ngithub.com/pingcap/tidb/privilege/privileges.(*MySQLPrivilege).LoadUserTable\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/privilege/privileges/cache.go:354\ngithub.com/pingcap/tidb/privilege/privileges.(*MySQLPrivilege).LoadAll\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/privilege/privileges/cache.go:269\ngithub.com/pingcap/tidb/privilege/privileges.(*Handle).Update\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/privilege/privileges/cache.go:1250\ngithub.com/pingcap/tidb/domain.(*Domain).LoadPrivilegeLoop\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/domain/domain.go:780\ngithub.com/pingcap/tidb/session.BootstrapSession\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/session/session.go:1616\nmain.createStoreAndDomain\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/tidb-server/main.go:247\nmain.main\n\t/Users/xuhuaiyu/Development/GOPATH/src/github.com/pingcap/tidb/tidb-server/main.go:185\nruntime.main\n\t/Users/xuhuaiyu/.gvm/gos/go1.13/src/runtime/proc.go:203"]
[2019/12/23 15:13:35.682 +08:00] [WARN] [cache.go:271] ["load mysql.user fail"] [error="runtime error: invalid memory address or nil pointer dereference"]
2019/12/23 15:13:35.682 terror.go:344: [fatal] [privilege:8049]mysql.user

What is changed and how it works?

Check whether expensiveQueryHandler.sm is nil before using it.

Check List

Tests

  • Manual test (add detailed scripts or steps below)
  1. modify the MemQuotaQuery to 32 in config.go
  2. compile and try to run the tidb-server binary
  3. everything works fine
[2019/12/23 16:22:44.091 +08:00] [INFO] [domain.go:369] ["full load and reset schema validator"]
[2019/12/23 16:22:44.095 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.097 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.099 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.100 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.101 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.102 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.102 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.103 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.103 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.104 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.104 +08:00] [INFO] [expensivequery.go:91] ["expensive_query during bootstrap phase"] [conn_id=0]
[2019/12/23 16:22:44.104 +08:00] [WARN] [server.go:270] ["secure connection is not enabled"]
[2019/12/23 16:22:44.104 +08:00] [INFO] [server.go:225] ["server is running MySQL protocol"] [addr=0.0.0.0:4000]
[2019/12/23 16:22:44.107 +08:00] [INFO] [domain.go:987] ["init stats info time"] ["take time"=4.011721ms]
[2019/12/23 16:22:44.108 +08:00] [INFO] [http_status.go:266] ["for status and metrics report"] ["listening on addr"=0.0.0.0:10080]


Code changes

  • Has exported function/method change

Side effects

N/A

Related changes

  • Need to cherry-pick to the release branch

Release note

N/A

Signed-off-by: sre-bot <sre-bot@pingcap.com>
@sre-bot
Copy link
Contributor Author

sre-bot commented Apr 8, 2020

/run-all-tests

@sre-bot sre-bot added sig/execution SIG execution type/3.1-cherry-pick type/enhancement The issue or PR belongs to an enhancement. labels Apr 8, 2020
@XuHuaiyu
Copy link
Contributor

XuHuaiyu commented Apr 9, 2020

/run-all-tests

Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@XuHuaiyu
Copy link
Contributor

/run-all-tests

Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason merged commit b8fbf56 into pingcap:release-3.1 Apr 13, 2020
@zz-jason zz-jason added status/all-tests-passed status/LGT2 Indicates that a PR has LGTM 2. labels Apr 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement. type/3.1-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants