statistics: use session pool when locking or unlocking table stats

[Rustin170506]

What problem does this PR solve?

Issue Number: ref #46351
Problem Summary:

What is changed and how it works?

• Fixed a typo: ...e/handletest/statslock/stats_lcok_test.go → ...e/handletest/statslock/stats_lock_test.go
• Removed incorrect comment. See: statistics: remove lockedTable cache #46555 (comment)
• Moved some code into query_lock.go. See: statistics: remove lockedTable cache #46555 (review)
• Used session pool to avoid getting mutex.
• Added unit tests for the lockstats packages

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[codecov]

Codecov Report

Merging #46611 (be8b49e) into master (19e888f) will decrease coverage by 0.6095%.
Report is 11 commits behind head on master.
The diff coverage is 68.7943%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #46611        +/-   ##
================================================
- Coverage   73.3655%   72.7560%   -0.6095%     
================================================
  Files          1315       1339        +24     
  Lines        395585     402706      +7121     
================================================
+ Hits         290223     292993      +2770     
- Misses        86902      91192      +4290     
- Partials      18460      18521        +61     

Flag	Coverage Δ
integration	25.4455% <18.6813%> (?)
unit	73.4348% <68.7943%> (+0.0692%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	54.0444% <ø> (ø)
parser	84.9689% <ø> (+0.0169%)	⬆️
br	48.1649% <ø> (-4.2948%)	⬇️

[Rustin170506]

1. Start the TiDB cluster: tiup playground nightly --tiflash 0 --db.binpath /Users/hi-rustin/GolandProjects/tidb/bin/tidb-server
2. Create table and analyze it:

create table t(a int, b int);
insert into t values (1,2), (3,4), (5,6), (7,8);
analyze table t;
show warnings;

3. Lock table: lock stats t;
4. Analyze it again: analyze table t;
5. Check the warnings: show warnings;

mysql> show warnings;
+---------+------+-----------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                 |
+---------+------+-----------------------------------------------------------------------------------------------------------------------------------------+
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t, reason to use this rate is "use min(1, 110000/4) as the sample-rate=1" |
| Warning | 1105 | skip analyze locked table: t                                                                                                            |
+---------+------+-----------------------------------------------------------------------------------------------------------------------------------------+

6. Check the mysql.stats_table_locked table:

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from stats_table_locked;
+----------+--------------+-------+--------------------+
| table_id | modify_count | count | version            |
+----------+--------------+-------+--------------------+
|       98 |            4 |     4 | 443863948666601483 |
+----------+--------------+-------+--------------------+
1 row in set (0.00 sec)

7. Try to lock it again: lock stats t;

mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> lock stats t;
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> show warnings;
+---------+------+-----------------------------------+
| Level   | Code | Message                           |
+---------+------+-----------------------------------+
| Warning | 1105 | skip locking locked table: test.t |
+---------+------+-----------------------------------+
1 row in set (0.00 sec)

8. Show locked stats: show stats_locked;.

mysql> show stats_locked;
+---------+------------+----------------+--------+
| Db_name | Table_name | Partition_name | Status |
+---------+------------+----------------+--------+
| test    | t          |                | locked |
+---------+------------+----------------+--------+
1 row in set (0.00 sec)

9. Unlock the table: unlock stats t;
10. Check the mysql.stats_table_locked table again:

mysql> select * from stats_table_locked;
Empty set (0.00 sec)

11. Show locked stats: show stats_locked;

mysql> show stats_locked;
Empty set (0.00 sec)

12. Unlock the table again and check warnings: unlock stats t;

mysql> unlock stats t;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> show warnings;
+---------+------+---------------------------------------+
| Level   | Code | Message                               |
+---------+------+---------------------------------------+
| Warning | 1105 | skip unlocking unlocked table: test.t |
+---------+------+---------------------------------------+
1 row in set (0.00 sec)

[Rustin170506]

self-check

[qw4990]

After this PR (in the next few PRs), can we thoroughly detach(decouple) StatsLock from StatsHandler? Specifically, can we remove all Handle.XXLockedXX methods (all methods in this file lock_stats_handler.go) and move them into a new separate package?

Logically, if we implement these XXLockedXX methods on Handler, we think the management of locked stats is part of the responsibility of Handler, although Handler is big enough and it already has too many responsibilities.
Another way is to let the Handler just as a consumer of locked stats info. It doesn't update locked stats info and only reads locked stats info from the storage and reacts to it accordingly.
I'm not sure whether this can simplify the design and implementation, what do you think of this? @hi-rustin
(I'm not sure whether I explain it clearly...

[Rustin170506]

I am not sure if this really helps. If we want to split the entry from the handle to stats locker. Then we need to put it in the domain or some other global structurations. I guess using the stats handle as an entry point for stats logic is fine. The problem with the handle is that we put too much business logic in the entry point.

Right now, the lock/unlock stats handler is pretty simple:

// AddLockedTables add locked tables id to store.
// - tids: table ids of which will be locked.
// - pids: partition ids of which will be locked.
// - tables: table names of which will be locked.
// Return the message of skipped tables and error.
func (h *Handle) AddLockedTables(tids []int64, pids []int64, tables []*ast.TableName) (string, error) {
	se, err := h.pool.Get()
	if err != nil {
		return "", errors.Trace(err)
	}
	defer h.pool.Put(se)

	exec := se.(sqlexec.RestrictedSQLExecutor)

	return lockstats.AddLockedTables(exec, tids, pids, tables)
}

// RemoveLockedTables remove tables from table locked array.
// - tids: table ids of which will be unlocked.
// - pids: partition ids of which will be unlocked.
// - tables: table names of which will be unlocked.
// Return the message of skipped tables and error.
func (h *Handle) RemoveLockedTables(tids []int64, pids []int64, tables []*ast.TableName) (string, error) {
	se, err := h.pool.Get()
	if err != nil {
		return "", errors.Trace(err)
	}
	defer h.pool.Put(se)

	exec := se.(sqlexec.RestrictedSQLExecutor)
	return lockstats.RemoveLockedTables(exec, tids, pids, tables)
}

// QueryTablesLockedStatuses query whether table is locked.
// Note: This function query locked tables from store, so please try to batch the query.
func (h *Handle) QueryTablesLockedStatuses(tableIDs ...int64) (map[int64]bool, error) {
	tableLocked, err := h.queryLockedTables()
	if err != nil {
		return nil, err
	}

	return lockstats.GetTablesLockedStatuses(tableLocked, tableIDs...), nil
}

// queryLockedTables query locked tables from store.
func (h *Handle) queryLockedTables() (map[int64]struct{}, error) {
	se, err := h.pool.Get()
	if err != nil {
		return nil, errors.Trace(err)
	}
	defer h.pool.Put(se)

	exec := se.(sqlexec.RestrictedSQLExecutor)
	return lockstats.QueryLockedTables(exec)
}

But I can try it recently to see if this helps. Do you have any suggestions on where to put the `statsLocker/statsUnlocker/statsQuerier'?

[qw4990]

And then in the feature, we can break the huge StatsHandler down to multiple smaller sub-components.
This makes our code easier to maintain (maintaining multiple smaller and simpler components is better than maintaining a big and complex component).
And these sub-components rely on the storage to synchronize information instead of relying on memory status, which is much safer.

[time-and-fate]

Looks like we can use const for these strings.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hawkingrei, time-and-fate

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hawkingrei,time-and-fate]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2023-09-04 03:38:36.11701143 +0000 UTC m=+2330280.666027412: ☑️ agreed by hawkingrei.
• 2023-09-05 10:10:36.104231343 +0000 UTC m=+2440200.653247315: ☑️ agreed by time-and-fate.