Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: update ioutil func&pruning case #39798

Merged
merged 3 commits into from
Dec 10, 2022
Merged

fix: update ioutil func&pruning case #39798

merged 3 commits into from
Dec 10, 2022

Conversation

songzhibin97
Copy link
Contributor

What is changed and how it works?

utils/utils: ioutil obsolete method upgrade & case pruning

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • [x]mark No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Dec 10, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • hawkingrei
  • lance6716

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Dec 10, 2022
@ti-chi-bot
Copy link
Member

Welcome @songzhibin97!

It looks like this is your first PR to pingcap/tidb 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to pingcap/tidb. 😃

@sre-bot
Copy link
Contributor

sre-bot commented Dec 10, 2022

CLA assistant check
All committers have signed the CLA.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 10, 2022
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Dec 10, 2022
@lance6716
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 714f4d6

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 10, 2022
@hawkingrei
Copy link
Member

/run-unit-test

@ti-chi-bot ti-chi-bot merged commit 86fc600 into pingcap:master Dec 10, 2022
@sre-bot
Copy link
Contributor

sre-bot commented Dec 10, 2022

TiDB MergeCI notify

✅ Well Done! New fixed [2] after this pr merged.

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/common-test 🔴 failed 1, success 10, total 11 9 min 51 sec Existing failure
idc-jenkins-ci-tidb/tics-test ✅ all 1 tests passed 6 min 23 sec Fixed
idc-jenkins-ci-tidb/mybatis-test ✅ all 1 tests passed 3 min 52 sec Fixed
idc-jenkins-ci/integration-cdc-test 🟢 all 40 tests passed 20 min Existing passed
idc-jenkins-ci-tidb/integration-common-test 🟢 all 17 tests passed 12 min Existing passed
idc-jenkins-ci-tidb/integration-ddl-test 🟢 all 6 tests passed 10 min Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 5 min 30 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1 🟢 all 26 tests passed 4 min 32 sec Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test 🟢 all 1 tests passed 3 min 0 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed

ti-chi-bot added a commit that referenced this pull request Jan 12, 2023
* br: add retry for azblob read file (#39650)

close #39649

* copr: fix data race in the mppIterator (#39772)

close #39771

* testkit: set GOMAXPROCS in test (#39766)

* statistics: support download history stats from stats_history (#39701)

* *:  Support Failed-Login Tracking and Temporary Account Locking (#39322)

close #38938

* ddl: mark the writes from delete-only and drop them on merge (#39796)

close #39731

* fix: update ioutil func&pruning case (#39798)

* dumpling: meter network usage (#39774)

close #39674

* staleread: fix flaky test TestStaleReadProcessorWithExecutePreparedStmt (#39805)

close #39804

* domain: fix data race in the MockInfoCacheAndLoadInfoSchema (#39802)

close #39801

* *: upgrade rules_go (#39797)

* *: Add backfill job related tables and operations  (#39616)

close #37122

* ttl: implement ttl job schedule framework (#39472)

close #39471

* *: stable tiflash test in ddl (#39825)

close #39831

* planner: fix index prefix matching (#39377)

close #14581

* parser: add cast to array and member of support (#39820)

close #39821

* fix: update ioutil.readall (#39850)

* planner:  add three fields to statement summary table (#39828)

ref #39199

* ddl: fix unexpect fail when create expression index (#39822)

close #39784

* ttl: ouput TTL comment format when show create table (#39853)

close #39852

* dumpling: add enable paging for tidb server (#39791)

close #39790

* planner: support explain format='cost_trace' on model ver2 (#39840)

ref #35240

* ttl, variables: add system variables for ttl job schedule (#39727)

close #39725

* *: fix slow query nil pointer panic when get session for TTL (#39865)

close #39864

* ttl: Add metrics for TTL (#39849)

close #39848

* statistics: support gc outdated historical stats (#39788)

* planner: rename `general plan cache` to `non-prepared plan cache` (#39873)

* planner: rename 'tidb_enable_general_plan_cache' and 'tidb_general_plan_cache_size' to 'tidb_enable_non_prepared_plan_cache' and 'tidb_non_prepared_plan_cache_size'

* planner: binding from cluster statements summary (#39737)

ref #39199

* ttl: modify default value of `tidb_ttl_delete_batch_size` to 100 (#39878)

close #39877

* ddl: close lightning writers after the import is complete (#39879)

close #39800

* store/copr: handle region error from client (#39838)

ref #39361

* ttl: fix scan workers shrink (#39860)

close #39859

* docs: fix typos (#39870)

* domain: support dump sql meta in plan replayer (#39863)

close #39883

* SlowLog: Update `Wait_TS` field in slow log for every sql  (#39843)

close #39713

* session: fix a bug for InitDDLJobTables (#39861)

ref #39854

* ttl: reschedule scan tasks after update task state (#39891)

close #39890

* ttl: TTL support split scan ranges for some primary key with more than one columns (#39896)

close #39895

* *: add dependabot (#39229)

* expression: support `json_overlaps` function (#39867)

ref #39866

* executor, planner: `InitLogger` before `CreateMockStore` in tests to avoid data race (#39900)

close #39892

* statistics: add source column for stats_meta_hisotry table (#39835)

* ttl: fix ttl txn will not return error when rolling back (#39918)

close #39917

* expression: close recordset (#39899)

* br: make more sane variables of ticking timeout (#39887)

close #39620

* expression: support `member of` function (#39880)

ref #39866

* br: fix err overwrite (#39935)

close #39934

* ttl: add log in the whole lifecycle of a job (#39931)

close #39930

* ttl: use optimistic transaction with 0 retry limit for ttl (#39924)

close #39923

* ttl: periodically update state for a job in heartbeat (#39939)

close #39938

* parser: support syntax set binding status by sql digest  (#39515)

ref #39199

* *: lock row keys during merging back from temp index (#39936)

close #39929

* ttl: initialize notifyStateCh in ttl job manager (#39956)

close #39955

* domain, statistics: periodically update stats healthy distribution (#39739)

close #39349

* ttl: enable 1pc for ttl delete (#39958)

close #39957

* docs/design: format the name of 'Adding-Index-Acceleration' RFC. (#39952)

* *: fix auto analyze worker crash when killed by global memory limit (#39946)

close #39947

* doc: add a proposal for foreign key (#35982)

* domain: Support plan replayer continus capture (#39926)

* expression, json: fix cast json as string with shorter length (#39970)

close #39963

* sessionctx, executor: fix failure of reading MaxPreparedStmtCount (#39736)

close #39735

* resourcemanger: create cpu monitor  (#39962)

ref #39657

* *: prevent cursor read from being cancelled by GC (#39950)

close #39447

* dumpling: supports foreign key limitations (#39914)

close #39913

* build(deps): bump golang.org/x/net from 0.2.0 to 0.4.0 (#39996)

* planner: fix pessimistic-auto-commit not working for point plans (#39993)

close #39928

* fix: ebs volume restore failure since incorrect unfinished volumes (#39975)

close #39974

* planner, json: restore cast flag after substitute column in cast (#39997)

close #39806

* staleread, session: internal write request should be accepted with external ts (#39967)

close #39966

* *: revert 38302, forbid modify column on partition table (#39991)

ref #39915, ref #39922

* executor: close recordset again (#40010)

* ddl: retry prepare RPC when meets region error (#39834)

close #39836

* metrics: add EMA cpu usage (#39995)

close #40029

* executor: fix unstable TestMaxPreparedStmtCount (#40024)

ref #39735

* *: support sample for compressed files for adjustment (#39680)

ref #38514

* docs: fix 'enable_global_index' to 'enable-global-index' (#40009)

* expression: introduce `cast(... as ... array)` in expression index (#39992)

close #40045

* planner: fix the issue that NonPrep Cache cannot work with Explain well (#40044)

close #40039

* ddl: fix waiting for wrong schema version if TiDB server restart (#39985)

close #39981

* *: Fix issue 39999, used wrong column id list for checking partitions (#40003)

close #39999

* oomtest: add whitelist for oom test to fix flaky test (#40055)

close #40054

* *: extend skyline pruning diagnostic info and add extra warnings (#39894)

ref #39893

* ddl: add privilege check when alter table add foreign key (#40051)

close #40050

* planner: add more test cases for non-prep plan cache (#40060)

* sessionctx: disable gc aware memory track (#40019)

close #39971

* *: make auto-analyze killable by global memory limit (#39978)

ref #39971, close #39994

* cpu: fix ticker to avoid close early (#40036)

ref #40029

* build(deps): bump golang.org/x/time from 0.2.0 to 0.3.0 (#39912)

* ddl: use latest ts to read record for adding index (#40081)

close #40074

* executor: close recordset again (#40073)

* planner: add more test cases for MPP hints (#39933)

* planner: support set binding status by sql digest (#39517)

ref #39199

* ddl: check the limitation when creating multi-valued index  (#39818)

close #40086

* sysvar: allow modifying 'tidb_allow_remove_auto_inc' when SEM is ON (#40083)

close #38238

* *: fix issue of multi-schema change with foreign key (#40042)

close #40037

* planner: check the `ignore-plan-cache` hint in insert-stmt (#40080)

ref #39717, close #40079

* parser: support keep_order and no_keep_order hint (#39965)

ref #39964

* *: filter particularly errors when `truncateAsWarning` is true (#40078)

* metrics: add metrics for ema cpu metrics and GOGC (#40049)

close #40092

* server: avoid reusing cached stmt ctx on cursor read (#40023)

close #39998

* *: optimize mpp probe (#39932)

close #39686

* statistics: fix the sync load sql did'nt running internally (#40087)

close #39511

* server: support download plan replayer continues capture file (#40085)

* *: support foreign key check/cascade when execute replace into statement (#40069)

close #40068

* ddl: fix issue of add foreign key too slow in big table (#40112)

close #40111

* ddl: fix issue of partition table with foreign key and add more test case (#40122)

* parser: fix cast function restore (#40129)

close #40130

* ddl: set context correctly in the setDDLLabelForDiagnosis (#40090)

close #40099

* planner: move code of generating IndexMergePath to a separated file (#40134)

* statistics: fix incorrect datetime value when loading stats (#40084)

close #39336

* ttl: escape parameters for finish job sql (#40118)

close #40115

* ddl: prohibit change name of partitioning column (#40147)

close #40135

* lightning: disable foreign key checks (#40032)

close #40027

* br: recursive split batch create table when restore failure on batch create table (#39829)

close #39029

* store/copr: don't reset concurrency of analyze req to 2 (#40168)

close #40162

* build(deps): bump golang.org/x/oauth2 from 0.2.0 to 0.3.0 (#40140)

* domain: revise plan replayer process log (#40126)

* expression: impl cast as array function (#40076)

close #40128

* session: fix tidb_enable_gc_aware_memory_track after upgrade (#40173)

ref #39971, close #40174

* restore: Fix prealloc (#40176)

close #40177

* metrics: remove Keep Alive OPM metric (#40142)

close #40143

* util: fix data race in the cpu test (#40154)

close #40125

* executor: move some test into writetest  (#40097)

* planner: fix an unstable explain test case  (#40199)

ref #35240

* ddl: Refine the error message to compatible with MySQL when drop a partition table partition key column (#38740)

close #38739

* executor: TiFlash supports stale read (#40048)

close #40047

* ddl: ignore internal http client goroutine in leak tests (#40190)

close #40183

* ddl: add more foreign key test case (#40052)

close #40189

* *: add support for -initialize-sql-file on first bootstrap (#35625)

close #35624

* *: fix PointGet will return an stale value when `tidb_enable_plan_replayer_capture` is set (#40197)

close #40194

* planner: support converting `json_member_of` to IndexMerge to access MVIndex (#40175)

ref #40191

* planner: record reasons when skipping Plan Cache (#40210)

* session: Do not run telemetry loops when it's disabled in config (#40156)

close #40155

* docs/design: REORGANIZE PARTITION design (#38246)

ref #15000, ref #15002

* ddl: Disable RENAME COLUMN for partitioning columns | tidb-test=pr/2054 (#40151)

close #40150

* dumpling: add foreign key e2e test (#40133)

close #40227

* ttl: make the TTL compatible with dumpling, lightning and BR  (#40216)

close #40215

* ttl, domain: setup a customized session pool with stats collector (#40171)

close #40109

* ddl: Support flashback cluster with ddl history (#40209)

ref #40026

* lightning: add foreign key test (#40103)

close #40226

* planner: refine planner code for disaggregated tiflash mode (#39813)

close #39814

* docs: update tidb roadmap based on v6.5 (#40170)

* planner: remove the unnecessary skip-plan-cache flag in StmtCtx (#40235)

* ttl: fix change status sql argument (#40234)

close #40231

* planner: update the plan cache strategy when expressions with parameters affect null-check (#40218)

close #38205, close #40093

* go.mod: upgrade badger to 20221229114011 (#40236)

close #39237

* executor: write multi-valued index (#40172)

close #40207

* expression: implement `unhex` function pushdown to tiflash (#39898)

ref #5112, ref pingcap/tiflash#5112

* planner: remove the unnecessary skip-plan-cache flag in StmtCtx (#40246)

* restore: set config value to default when failed to get config from tikv (#36051)

close #36053

* ddl: check default value again in updateColumnDefaultValue (#40230)

close #40164

* docs: update roadmap (#40244)

* *: remove the support of the old ddl framework (#39684)

ref #39824

* lightning: check peers write stall when switch-mode is disabled (#40228)

close #40163

* ddl, parser: make generated column and expression index same as MySQL (#39888)

close #39826

* session: make plan replayer capture support prepared stmt (#40167)

close #40161

* expression: implement `regexp_replace` function pushdown (#40267)

close #40266

* planner: supply more test for binding from history (#40242)

ref #39199

* ttl: disable ttl job when recover/flashback table/database/cluster (#40268)

close #40265

* admin: impl admin check index for mv index (#40270)

close #40272

* planner: refactor plan-cache UseCache flag (#40256)

ref #36598

* test: fix data race in cast as array (#40277)

close #40276

* planner: support converting `json_overlaps/contains` to IndexMerge to access MVIndex (#40195)

ref #40191

* autoid_service: add unit test for the package (#40193)

* executor: special handling is required when an "auto id out of range" error occurs in `insert ignore into ... on on duplicate ...` (#39847)

close #38950

* util: gorotinue pool (#39872)

close #38039

* metrics: add metrics for plan replayer and historical stats (#40271)

* sysvar: fix circular dependency in rebuildSysVarCache leading to deadlock (#40283)

close #40240

* ddl: fix flaky test TestGlobalVariablesOnFlashback (#40291)

close #40287

* server, tidb-server: kill auto analyze when gracefully shutting down (#40284)

close #40038

* planner: better coordination between the ignore_plan_cache() binding and plan-cache (#40280)

close #40225

* store/copr: fix build batchCop in disaggregated tiflash mode (#40008)

close #40035

* ddl: persist index info after changing backfill state (#40229)

close #40217

* doc: update flashback doc (#40031)

ref #40026

* ddl, parser: Implement the write-reorg state split task related functions, and the related interfaces of backfill worker (#39982)

close #37123

* planner: support pushing down predicates to memory tables in prepared mode (#40262)

close #39605

* planner: open the partial order prop push down for LIST PARTITION (#40290)

close #40273

* br: fix bootstrap domain (#40307)

close #40281

* br: add errch buf for checkpoint (#40166)

close #40165

* planner: skip plan-cache for prepared queries with `INT in (Decimals...)` (#40312)

close #40224

* statistics: support historical stats dump partition table (#40310)

* *: update badger (#40331)

* expression: enlarge timeout for test (#40332)

* resourcemanger: add cpu scheduler (#39886)

close #39657

* util: use go-deadlock to find deadlock (#40288)

close #40293

* *: support `curdate()` as column's default value  | tidb-test=pr/2057 (#40326)

close #38356

* ttl: don't schedule ttl job when EnableTTLJob is off (#40336)

close #40335

* *: add warnings to slow log and slow query table (#39884)

ref #39893

* planner,infoschema,executor: Add tiflash fine grained shuffle support for hash join and aggregation (#40121)

close #40123

* ddl: fill in original default for extra writable columns in batch insert (#40198)

close #40192

* ddltest: update goleak and enable race (#40357)

close #40358

* planner: let the planner can select IndexMerge on MVIndex without hint (#40347)

ref #40191

* makefile: enable deadlock in the with-real-tikv (#40359)

close #40293

* planner: support more types to use IndexMerge to access MVIndex (#40343)

ref #40191

* log-backup: should set default value `gc.ratio-threshold` after pitr finished (#40141)

close #40185

* domain: let plan replayer dump support recording stats mem status (#40334)

* docs: add TiDB Forum link and adjust link order (#39868)

* ddl: support online create multi-valued index (#40304)

close #40337

* ttl: add `ttl_job_interval` attribute and remove `tidb_ttl_job_interval` variable (#40033)

close #40028

* *: remove the support of the `amending transaction` (#39714)

close #40381

* executor: reset the related session vars for both INSERT and REPLACE (#40354)

close #40351

* *: fix static pruning partition table in disaggregated tiflash mode (#40238)

close #40239

* planner: fix typo (#40367)

* session: fix data race in the LazyTxn.LockKeys (#40350)

close #40355

* build(deps): bump golang.org/x/crypto from 0.1.0 to 0.5.0 (#40379)

* executor: fix data race at the ShowExec (#39817)

close #39816, close #40295

* planner: fix panic during starting tidb-server if creating global binding for partition table (#40402)

* fix

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* *: Supports create/alter/drop resource group (#40188)

ref #38825

* session: fix deadlock when init domain failed (#40409)

close #40408

* executor: support insert ignore/duplicate replace into with unique multi-valued index (#40369)

close #40207

* br: support reset_tiflash after ebs restoration (#40124)

close #40208

* ttl: add ttl goroutine exit log (#40416)

close #40415

* planner, executor: split the range for unsigned pk of partition table when limit is used (#40313)

close #40309

* planner: add nil check when handling empty charset in `setUnionFlen` (#40429)

close #40285

* planner: support keep_order and no_keep_order hint (#40089)

ref #39964

* planner: unify the behavior of prepare/execute limit to mysql (#40360)

ref #40219

* gpool: register gpool into resource manager (#40410)

ref #40412

* ttl: Add `CommandClient` to trigger TTL job manually (#40346)

close #40345

* ttl: make ttl's group in granfana collapse by default (#40444)

* br: reset cloud_admin and root after ebs restoration (#39986)

close #40418

* ddl: support displaying sub-job reorg type in admin show ddl (#40387)

close #40386

* planner: support use IndexMerge to access composite MVIndex (#40426)

ref #40191

* planner: add test cases for keep_order and no_keep_order hints (#40321)

ref #39964

* statistics: change the upper bound of the out-of-range estimation to modify count | tidb-test=pr/2012 (#39011)

close #39008

* docs: update CI build badages (#40324)

close #40325

* planner: update the non-prep cacheable checker (#40446)

close #40296

* expression, planner: exclude concat_ws from aggresive folding constant (#38383)

close #36888

* keyspace: introduce keyspace conf and etcd path (#40269)

ref #40425

* ddl: skip deleting the unmatched index key on merge (#40465)

close #40464

* Merge pingcap/master and pingcap/feature/reorganize-partition

* Updated CODEOWNERS for the feature branch

* ci: add document about some task that will be triggered manually (#40459)

* errno, store: fix ErrPDServerTimeout arguments (#40468)

close #40439

* ttl: fix the TTL job reports error when primary key contains a column with type `ENUM` (#40457)

close #40456

* planner: support using DNF to construct IndexMerge to access MVIndex (#40471)

* fixup

* fixup

* fixup

* executor: add partition table testcase for historical stats (#40453)

* add partition table testcase

* add partition table testcase

* fix lint

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* ttl: forbid creating/altering a table with TTL options when pk contains float/double column (#40487)

* ttl: forbid create/alter a table with TTL options when pk contains float/double column

* format

* update

* update

* update

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* planner: make MVIndex be compatible with the `use/force index` hint (#40495)

* fixup

* fixup

* fixup

* fixup

* lightning: fix codecPDClient (#40496)

Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>

Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>

* ddl: let concurrent truncate on the same table depend on the previous one (#40501)

* done

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

* refine test

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* statistics: fix estimation error when ranges are too many and modify count is large (#40472)

* fix

* fmt

* bazel update

* update test result

Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* executor: fix issue of KEY_COLUMN_USAGE table return wrong result about foreign key column (#40491)

Signed-off-by: crazycs520 <crazycs520@gmail.com>

Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* sysvar: add switch for plan replayer capture using historical stats (#40492)

* add switch

* fix

* fix

* fix

Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* *: upgrade go1.19.5 (#40483)

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* br: need init MDL variable when GetDomain (#40512)

Signed-off-by: joccau <zak.zhao@pingcap.com>

Signed-off-by: joccau <zak.zhao@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* ddl: avoid commit conflicts when updating/delete from mysql.tidb_ddl_reorg. (#38738)

* Added test case

* ddl fix #38669.

The issue was that mysql.tidb_ddl_reorg table was updated by an
inner transaction after the outer transaction started,
which then made a commit conflict in the outer transaction,
when it deleted the same row.

* Fixed typo in comment

* Added test case for #24427

* Disabled tests for CI testing

* Revert "Disabled tests for CI testing"

This reverts commit 17c28f3.

* Revert "Revert "Disabled tests for CI testing""

This reverts commit 65c84d9.

* removed test skips

* Clean up the tidb_ddl_reorg entry after DDL is completed

* Use a cleanup job afterwards instead.

* Fixed test

* Moved cleanup before asyncNotify

* More detailed test failure log

* Refined test error message

* Injecting timoeut to get stack traces from CI

* Updated Debug Dump on timeout

* Delete mulitple entries in tidb_ddl_reorg if needed

* Linting

* Linting

* Added CI debug logs

* Linting + CI debugs

* fixed CI debug

* Try to cleanup also if job.State == synced

* check for non-error of runErr instead of error...

* Use a new session, instead of reusing worker.sess

* Also handle case when job == nil

* Removed CI debug logs

* Misssed change session from w.sess to newly created sess

* Improved TestConcurrentDDLSwitch and added CI debug logs

* Always cleaning up all orphan mysql.tidb_ddl_reorg entries

* linting

* Also cleanup if job is nil

* Updated TestModifyColumnReorgInfo + CI debug logs

* more CI debug

* refactored the cleanupDDLReorgHandle code

* Added missing cleanup in handleDDLJobQueue

* Removed debug panic

* Code cleanup

* Test updates

* Debug cleanup

* Cleaned up test after removal of old non-concurrent DDL code merge

* Linting

* always wrap changes to tidb_ddl_reorg in an own transaction

+ fixed some typos

* Minimum fix

* Always update reorg meta, not only on error

* Issue is here :)

* Fixed newReorgHandler

* Wrapped more tidb_ddl_reorg changes into separate transactions

* linting

* Removed updateDDLReorgStartHandle

* cleanups

* Made runInTxn a method on *session, instead of normal function

* Update test

* Final touches

* Removed duplicate test

* CleanupDDLReorgHandles should only be called from HandleJobDone.

* Variable rename

* Renamed 'delete' variabel name

* Updated test

* small revert

* Removed timeout debugging code

* Simplified the cleanup to only start a new txn and not a new session

* Reverted the change of GetDDLInfo

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

* Post merge fix

Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Signed-off-by: joccau <zak.zhao@pingcap.com>
Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com>
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Song Gao <disxiaofei@163.com>
Co-authored-by: lastincisor <64511824+lastincisor@users.noreply.github.com>
Co-authored-by: tangenta <tangenta@126.com>
Co-authored-by: Bin <49082129+songzhibin97@users.noreply.github.com>
Co-authored-by: Obliviate <756541536@qq.com>
Co-authored-by: Lynn <zimu_xia@126.com>
Co-authored-by: YangKeao <yangkeao@chunibyo.icu>
Co-authored-by: Daniël van Eeden <git@myname.nl>
Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com>
Co-authored-by: fzzf678 <108643977+fzzf678@users.noreply.github.com>
Co-authored-by: Hangjie Mo <mohangjie1995@gmail.com>
Co-authored-by: 王超 <cclcwangchao@hotmail.com>
Co-authored-by: Chunzhu Li <lichunzhu@stu.xjtu.edu.cn>
Co-authored-by: Yuanjia Zhang <zhangyuanjia@pingcap.com>
Co-authored-by: you06 <you1474600@gmail.com>
Co-authored-by: raymonder jin <rayjingithub@gmail.com>
Co-authored-by: TonsnakeLin <87681388+TonsnakeLin@users.noreply.github.com>
Co-authored-by: Zhou Kunqin <25057648+time-and-fate@users.noreply.github.com>
Co-authored-by: 山岚 <36239017+YuJuncen@users.noreply.github.com>
Co-authored-by: Yifan Xu <30385241+xuyifangreeneyes@users.noreply.github.com>
Co-authored-by: bb7133 <bb7133@gmail.com>
Co-authored-by: Xiaoju Wu <wuxiaoju@pingcap.com>
Co-authored-by: crazycs <crazycs520@gmail.com>
Co-authored-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: fengou1 <85682690+fengou1@users.noreply.github.com>
Co-authored-by: wjHuang <huangwenjun1997@gmail.com>
Co-authored-by: mychoxin <mychoxin@gmail.com>
Co-authored-by: Chengpeng Yan <41809508+Reminiscent@users.noreply.github.com>
Co-authored-by: CbcWestwolf <1004626265@qq.com>
Co-authored-by: hackersean <hackersean@users.noreply.github.com>
Co-authored-by: Yujie Xia <xiayjchn@gmail.com>
Co-authored-by: jiyfhust <jiyfhust@163.com>
Co-authored-by: hehechen <chentongli@pingcap.com>
Co-authored-by: Morgan Tocker <mtocker@squareup.com>
Co-authored-by: Xiaoguang Sun <sunxiaoguang@users.noreply.github.com>
Co-authored-by: guo-shaoge <shaoge1994@163.com>
Co-authored-by: Ran <huangran.alex@gmail.com>
Co-authored-by: baishen <baishen2009@gmail.com>
Co-authored-by: lance6716 <lance6716@gmail.com>
Co-authored-by: xzhangxian1008 <xzhangxian@foxmail.com>
Co-authored-by: tiancaiamao <tiancaiamao@gmail.com>
Co-authored-by: Jk Xu <54522439+Dousir9@users.noreply.github.com>
Co-authored-by: djshow832 <zhangming@pingcap.com>
Co-authored-by: Yiding Cui <winoros@gmail.com>
Co-authored-by: yibin <huyibin@pingcap.com>
Co-authored-by: Zak Zhao <57036248+joccau@users.noreply.github.com>
Co-authored-by: luzizhuo <496521310@qq.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: ShuNing <nolouch@gmail.com>
Co-authored-by: 3pointer <luancheng@pingcap.com>
Co-authored-by: wuhuizuo <wuhuizuo@126.com>
Co-authored-by: ystaticy <y_static_y@sina.com>
Co-authored-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants