Skip to content

Commit

Permalink
lightning: calculate concurrency of schema import from region-concurr…
Browse files Browse the repository at this point in the history
  • Loading branch information
D3Hunter authored Dec 17, 2024
1 parent c8ba4e3 commit aec0fc5
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion lightning/pkg/importer/import.go
Original file line number Diff line number Diff line change
Expand Up @@ -593,7 +593,9 @@ func (rc *Controller) restoreSchema(ctx context.Context) error {
// we can handle the duplicated created with createIfNotExist statement
// and we will check the schema in TiDB is valid with the datafile in DataCheck later.
logger := log.FromContext(ctx)
concurrency := min(rc.cfg.App.RegionConcurrency, 8)
// the minimum 4 comes the fact that when connect to non-owner TiDB, the max
// QPS is 2 per connection due to polling every 500ms.
concurrency := max(2*rc.cfg.App.RegionConcurrency, 4)
// sql.DB is a connection pool, we set it to concurrency + 1(for job generator)
// to reuse connections, as we might call db.Conn/conn.Close many times.
// there's no API to get sql.DB.MaxIdleConns, so we revert to its default which is 2
Expand Down

0 comments on commit aec0fc5

Please sign in to comment.