dmctl(dm): add config task-template #3991

Ehco1996 · 2021-12-20T09:45:21Z

What problem does this PR solve?

Issue Number: ref #3583

What is changed and how it works?

add new cmd dmctl config task-template

➜ dmctl config task-template  -h
show task template which is created by WebUI with task config format

Usage:
  dmctl config task-template [task-name] [flags]

Flags:
  -h, --help   help for task-template

fix templates openapi path from api/v1/task/templates to api/v1/tasks/templates

we don't exporse this cmd to public user , beascause it's just a workaround for user who user WebUI and want to get task config template that created by openapi but want to dump with v1 format

Check List

Tests

Unit test
Integration test

Code changes

Side effects

Possible performance regression

Related changes

Need to update the documentation

Release note

`None`.

ti-chi-bot · 2021-12-20T09:45:22Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

lance6716
okJiang

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Ehco1996 · 2021-12-21T07:00:16Z

will add dmctl it test after #3656 merged

dm/dm/ctl/master/config.go

lance6716 · 2021-12-21T14:58:54Z

it's get-config task, nothing to do with template

Ehco1996 · 2021-12-21T23:22:22Z

it's get-config task, nothing to do with template

but we already have get-confing task cmd, so we need another alias for this cmd , do you have some suggestions?

lance6716 · 2021-12-22T02:12:24Z

it's get-config task, nothing to do with template

but we already have get-confing task cmd, so we need another alias for this cmd , do you have some suggestions?

the old get-config task can be replaced by this logic 🤔

Ehco1996 · 2021-12-22T02:18:19Z

the old get-config task can be replaced by this logic 🤔

you mean when user useget-config task we try get config from subtask in scheduler first, if there is no task we fallback to openapi.Task in etcd ?

I'm worried if there are two tasks with the same name: test, one in the scheduler and the other in etcd 😂

lance6716 · 2021-12-23T08:10:50Z

subtask is generated by task. Currently get-config task will merge subtasks to task but some information is lost. If we can save original task I think it's always better. @GMHDBJD PTAL

GMHDBJD · 2021-12-23T08:51:39Z

Prefer template <task|master|worker> <name>

Ehco1996 · 2021-12-23T08:58:00Z

/hold wait change api path to templates

Ehco1996 · 2022-01-05T03:19:08Z

/run-dm-integration-tests

codecov-commenter · 2022-01-05T04:25:42Z

Codecov Report

Merging #3991 (69c8a71) into master (ab62e6b) will decrease coverage by 0.2517%.
The diff coverage is 44.7849%.

Flag	Coverage Δ
cdc	`59.1007% <55.5926%> (-0.5152%)`	⬇️
dm	`52.1050% <40.2387%> (+0.0395%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #3991        +/-   ##
================================================
- Coverage   55.5183%   55.2665%   -0.2518%     
================================================
  Files           495        497         +2     
  Lines         61151      61321       +170     
================================================
- Hits          33950      33890        -60     
- Misses        23789      24021       +232     
+ Partials       3412       3410         -2

Ehco1996 · 2022-01-12T02:02:25Z

could you please take another look for this pr ? thanks 🧡🧡🧡

/cc @GMHDBJD @okJiang @WizardXiao

okJiang

LGTM

okJiang · 2022-01-17T02:31:00Z

dm/tests/openapi/run.sh

+	# delete source success and clean data for other test
+	openapi_source_check "delete_source_with_force_success" "mysql-01"
+	openapi_source_check "delete_source_with_force_success" "mysql-02"
+	openapi_source_check "list_source_success" 0
+	openapi_task_check "get_task_list" 0


how about wrap these to one function

added in d5e9530 and 0f3dd42

Ehco1996 · 2022-01-17T03:45:26Z

/run-dm-integration-tests

Ehco1996 · 2022-01-17T05:32:22Z

/run-dm-integration-tests

okJiang · 2022-01-17T09:27:02Z

/run-all-tests

Ehco1996 · 2022-01-17T23:38:54Z

/merge

ti-chi-bot · 2022-01-17T23:38:57Z

This pull request has been accepted and is ready to merge.

Commit hash: 3553334

Ehco1996 · 2022-01-18T00:06:40Z

/run-leak-test

ref pingcap#3583

Ehco1996 added the area/dm Issues or PRs related to DM. label Dec 21, 2021

Ehco1996 changed the title ~~dmctl(dm): support OpenAPI task config in get-config command~~ dmctl(dm): get-config command support --template flag Dec 21, 2021

ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Dec 21, 2021

Ehco1996 added 2 commits December 21, 2021 14:46

savework

8f37e18

save work

d6ea738

ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 21, 2021

Ehco1996 commented Dec 21, 2021

View reviewed changes

dm/dm/ctl/master/config.go Outdated Show resolved Hide resolved

Ehco1996 marked this pull request as ready for review December 21, 2021 07:04

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 21, 2021

ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 23, 2021

Ehco1996 added 2 commits January 5, 2022 10:21

Merge branch 'master' into openapi-config-template-dmctl

ffae708

fix for merge

127795c

fix url and add it test

49bf6d3

ti-chi-bot removed the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jan 5, 2022

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 10, 2022

Ehco1996 added 2 commits January 11, 2022 08:50

change func name

aeab0ae

Merge branch 'master' into openapi-config-template-dmctl

a21d5f4

ti-chi-bot requested review from GMHDBJD, okJiang and WizardXiao January 12, 2022 02:02

Ehco1996 added 3 commits January 12, 2022 17:15

Merge branch 'master' into openapi-config-template-dmctl

cd0cbc3

Merge branch 'master' into openapi-config-template-dmctl

1279fcb

Merge branch 'master' into openapi-config-template-dmctl

2bd5daa

okJiang approved these changes Jan 17, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jan 17, 2022

address comment

d5e9530

Ehco1996 added 2 commits January 17, 2022 13:31

fix test

0f3dd42

Merge branch 'master' into openapi-config-template-dmctl

69c8a71

Merge branch 'master' into openapi-config-template-dmctl

5896209

Merge branch 'master' into openapi-config-template-dmctl

3553334

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jan 17, 2022

ti-chi-bot merged commit 4bfa080 into pingcap:master Jan 18, 2022

buchuitoudegou pushed a commit to buchuitoudegou/tiflow that referenced this pull request Jan 18, 2022

dmctl(dm): add config task-template (pingcap#3991)

1e89b51

ref pingcap#3583

buchuitoudegou pushed a commit to buchuitoudegou/tiflow that referenced this pull request Jan 18, 2022

dmctl(dm): add config task-template (pingcap#3991)

de1e946

ref pingcap#3583

buchuitoudegou pushed a commit to buchuitoudegou/tiflow that referenced this pull request Jan 19, 2022

dmctl(dm): add config task-template (pingcap#3991)

bf0d6a2

ref pingcap#3583

zhaoxinyu pushed a commit to zhaoxinyu/ticdc that referenced this pull request Jan 20, 2022

dmctl(dm): add config task-template (pingcap#3991)

f532a04

ref pingcap#3583

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dmctl(dm): add config task-template #3991

dmctl(dm): add config task-template #3991

Ehco1996 commented Dec 20, 2021 •

edited

Loading

ti-chi-bot commented Dec 20, 2021 •

edited

Loading

Ehco1996 commented Dec 21, 2021

lance6716 commented Dec 21, 2021

Ehco1996 commented Dec 21, 2021

lance6716 commented Dec 22, 2021

Ehco1996 commented Dec 22, 2021

lance6716 commented Dec 23, 2021

GMHDBJD commented Dec 23, 2021

Ehco1996 commented Dec 23, 2021

Ehco1996 commented Jan 5, 2022

codecov-commenter commented Jan 5, 2022 •

edited

Loading

Ehco1996 commented Jan 12, 2022

okJiang left a comment

okJiang Jan 17, 2022

Ehco1996 Jan 17, 2022

Ehco1996 commented Jan 17, 2022

Ehco1996 commented Jan 17, 2022

okJiang commented Jan 17, 2022

Ehco1996 commented Jan 17, 2022

ti-chi-bot commented Jan 17, 2022

Ehco1996 commented Jan 18, 2022

dmctl(dm): add config task-template #3991

dmctl(dm): add config task-template #3991

Conversation

Ehco1996 commented Dec 20, 2021 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Dec 20, 2021 • edited Loading

Ehco1996 commented Dec 21, 2021

lance6716 commented Dec 21, 2021

Ehco1996 commented Dec 21, 2021

lance6716 commented Dec 22, 2021

Ehco1996 commented Dec 22, 2021

lance6716 commented Dec 23, 2021

GMHDBJD commented Dec 23, 2021

Ehco1996 commented Dec 23, 2021

Ehco1996 commented Jan 5, 2022

codecov-commenter commented Jan 5, 2022 • edited Loading

Codecov Report

Ehco1996 commented Jan 12, 2022

okJiang left a comment

Choose a reason for hiding this comment

okJiang Jan 17, 2022

Choose a reason for hiding this comment

Ehco1996 Jan 17, 2022

Choose a reason for hiding this comment

Ehco1996 commented Jan 17, 2022

Ehco1996 commented Jan 17, 2022

okJiang commented Jan 17, 2022

Ehco1996 commented Jan 17, 2022

ti-chi-bot commented Jan 17, 2022

Ehco1996 commented Jan 18, 2022

Ehco1996 commented Dec 20, 2021 •

edited

Loading

ti-chi-bot commented Dec 20, 2021 •

edited

Loading

codecov-commenter commented Jan 5, 2022 •

edited

Loading