DM/Openapi: ignore can not connect error when force delete task #5364

WizardXiao · 2022-05-09T08:20:16Z

What problem does this PR solve?

Issue Number: close #5363

What is changed and how it works?

add check when delete task.

Check List

Tests

Unit test
Integration test

Release note

None

ti-chi-bot · 2022-05-09T08:20:17Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

Ehco1996
buchuitoudegou

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

WizardXiao · 2022-05-09T08:27:40Z

/run-all-tests

WizardXiao · 2022-05-09T09:22:25Z

/cc @Ehco1996

Ehco1996

rest LGTM

Ehco1996 · 2022-05-10T07:53:45Z

dm/dm/master/openapi_controller.go

+		if err == nil {
+			return true
+		}
+		if force && strings.Contains(err.Error(), "connect: connection refused") {


how about use
errors.Is(err, syscall.ECONNREFUSED)

i tried errors.Is(err, syscall.ECONNREFUSED) and run inegration test, but it can't ignore connection refused. I still don't know the root cause, i will find it out later

i think this is because this error is wrapped by terror

tiflow/dm/pkg/conn/basedb.go

Line 117 in e915545

return nil, terror.DBErrorAdapt(err, terror.ErrDBDriverError)

ok not change here

Ehco1996

LGTM

WizardXiao · 2022-05-12T01:43:28Z

please help me take a look, tanks. @GMHDBJD @dsdashun

buchuitoudegou · 2022-05-12T08:17:54Z

dm/dm/master/openapi_controller.go

 	}
 	metaSchema := *task.MetaSchema
 	err = s.removeMetaData(ctx, taskName, metaSchema, toDBCfg)
 	if err != nil {
-		return terror.Annotate(err, "while removing metadata")
+		if !ignoreCannotConnectError(err) {


I understand forcing to delete tasks shouldn't have returned errors. But what if the user restarts the same task again but the metadata still exists?

i make a discussion note here.
when user recreates the same task, and the downstream recovers again, it will still have the metadata, then it need remove metadata when start task or manually delete metadata.
after disscuss with @sunzhaoyang, this is rare, we can just print warning log. but it still has risk，and most of openapi's interface has no warning message when it can run successfully. so i will pose a #5410 to note this.

buchuitoudegou

LGTM but probably result in #5409

codecov-commenter · 2022-05-12T10:38:54Z

Codecov Report

Merging #5364 (d4e6208) into master (687e248) will decrease coverage by 0.0898%.
The diff coverage is 52.6616%.

Flag	Coverage Δ
cdc	`61.0650% <52.6616%> (+0.4664%)`	⬆️
dm	`52.1201% <ø> (-0.3495%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #5364        +/-   ##
================================================
- Coverage   56.1240%   56.0341%   -0.0899%     
================================================
  Files           522        528         +6     
  Lines         65325      69438      +4113     
================================================
+ Hits          36663      38909      +2246     
- Misses        25094      26819      +1725     
- Partials       3568       3710       +142

WizardXiao · 2022-05-12T12:30:25Z

/merge

ti-chi-bot · 2022-05-12T12:30:28Z

This pull request has been accepted and is ready to merge.

Commit hash: e51ecb0

WizardXiao · 2022-05-12T12:51:11Z

/run-dm-integration-test

…rce-remove-meta

into force-remove-meta

WizardXiao · 2022-05-12T14:52:37Z

/merge

ti-chi-bot · 2022-05-12T14:52:41Z

This pull request has been accepted and is ready to merge.

Commit hash: 19fe4fa

WizardXiao · 2022-05-12T14:55:01Z

/run-all-tests

WizardXiao · 2022-05-12T23:38:16Z

/run-dm-integration-test

WizardXiao · 2022-05-13T00:14:26Z

/run-dm-integration-test

WizardXiao · 2022-05-13T00:38:29Z

/run-leak-test

commit-message: init

429f9aa

ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 9, 2022

WizardXiao added area/dm Issues or PRs related to DM. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 9, 2022

commit-message: fix make check

b4173d6

ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 9, 2022

ti-chi-bot requested a review from Ehco1996 May 9, 2022 09:22

Ehco1996 reviewed May 10, 2022

View reviewed changes

Ehco1996 approved these changes May 11, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 11, 2022

buchuitoudegou reviewed May 12, 2022

View reviewed changes

buchuitoudegou mentioned this pull request May 12, 2022

OpenAPI: forcing to delete task may result in metadata inconsistency #5409

Open

buchuitoudegou approved these changes May 12, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 12, 2022

WizardXiao and others added 2 commits May 12, 2022 18:21

commit-message: merge master

2bf424b

Merge branch 'master' into force-remove-meta

d4e6208

Merge branch 'master' into force-remove-meta

e51ecb0

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 12, 2022

WizardXiao added 2 commits May 12, 2022 22:36

Merge branch 'master' of https://github.com/WizardXiao/tiflow into fo…

5d257e9

…rce-remove-meta

Merge branch 'force-remove-meta' of https://github.com/WizardXiao/tiflow

19fe4fa

into force-remove-meta

ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label May 12, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 12, 2022

ti-chi-bot merged commit 08928bf into pingcap:master May 13, 2022

WizardXiao mentioned this pull request May 17, 2022

Tracking issue for DM WebUI v3 #5442

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DM/Openapi: ignore can not connect error when force delete task #5364

DM/Openapi: ignore can not connect error when force delete task #5364

WizardXiao commented May 9, 2022

ti-chi-bot commented May 9, 2022 •

edited

Loading

WizardXiao commented May 9, 2022

WizardXiao commented May 9, 2022

Ehco1996 left a comment

Ehco1996 May 10, 2022

WizardXiao May 11, 2022 •

edited

Loading

Ehco1996 May 11, 2022

Ehco1996 left a comment

WizardXiao commented May 12, 2022

buchuitoudegou May 12, 2022

WizardXiao May 12, 2022 •

edited

Loading

buchuitoudegou left a comment

codecov-commenter commented May 12, 2022 •

edited

Loading

WizardXiao commented May 12, 2022

ti-chi-bot commented May 12, 2022

WizardXiao commented May 12, 2022

WizardXiao commented May 12, 2022

ti-chi-bot commented May 12, 2022

WizardXiao commented May 12, 2022

WizardXiao commented May 12, 2022

WizardXiao commented May 13, 2022

WizardXiao commented May 13, 2022

DM/Openapi: ignore can not connect error when force delete task #5364

DM/Openapi: ignore can not connect error when force delete task #5364

Conversation

WizardXiao commented May 9, 2022

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented May 9, 2022 • edited Loading

WizardXiao commented May 9, 2022

WizardXiao commented May 9, 2022

Ehco1996 left a comment

Choose a reason for hiding this comment

Ehco1996 May 10, 2022

Choose a reason for hiding this comment

WizardXiao May 11, 2022 • edited Loading

Choose a reason for hiding this comment

Ehco1996 May 11, 2022

Choose a reason for hiding this comment

Ehco1996 left a comment

Choose a reason for hiding this comment

WizardXiao commented May 12, 2022

buchuitoudegou May 12, 2022

Choose a reason for hiding this comment

WizardXiao May 12, 2022 • edited Loading

Choose a reason for hiding this comment

buchuitoudegou left a comment

Choose a reason for hiding this comment

codecov-commenter commented May 12, 2022 • edited Loading

Codecov Report

WizardXiao commented May 12, 2022

ti-chi-bot commented May 12, 2022

WizardXiao commented May 12, 2022

WizardXiao commented May 12, 2022

ti-chi-bot commented May 12, 2022

WizardXiao commented May 12, 2022

WizardXiao commented May 12, 2022

WizardXiao commented May 13, 2022

WizardXiao commented May 13, 2022

ti-chi-bot commented May 9, 2022 •

edited

Loading

WizardXiao May 11, 2022 •

edited

Loading

WizardXiao May 12, 2022 •

edited

Loading

codecov-commenter commented May 12, 2022 •

edited

Loading