Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TiKV CDC Development Tasks #48

Closed
26 of 28 tasks
pingyu opened this issue Feb 14, 2022 · 2 comments · Fixed by #115, #116 or #132
Closed
26 of 28 tasks

TiKV CDC Development Tasks #48

pingyu opened this issue Feb 14, 2022 · 2 comments · Fixed by #115, #116 or #132
Assignees
@pingyu @haojinming @zeminzhou
Labels
component/cdc

Comments

@pingyu
Copy link
Collaborator

pingyu commented Feb 14, 2022 •
edited
Loading

TiKV CDC Development Tasks

Related issue: tikv/tikv#11745
Related RFC: tikv/rfcs#86

Tasks

  • Core data structure in ETCD
    • raw changefeed
    • raw task (a part of a changefeed, for one processor)
    • ETCD path planning
  • Owner
    • New changefeed for RawKV
    • Delete table changefeed
  • Scheduler
    • New scheduler for Key Span
    • Delete table scheduler
  • Capture
    • Support rawkv changefeed
    • Delete support of table changefeed
  • Processor
    • New processor for Key Span
  • Sink
    • TiKV Sink
  • Command line
    • cdc cli
  • Others
    • Upgrade dependencies
  • TiUP support
  • Metrics
    • New Grafana board for TiKV-CDC
    • Determine metrics (ref to TiCDC metrics)
  • Tests
    • Integration tests
    • E2E tests

Development Guide

  1. Choose a task you are interested in.
  2. File a ISSUE for the task.
  3. Fix the ISSUE by PR (also refer to CONTRIBUTING guide).
  4. Get the PR merged.
@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity.

This was referenced Mar 22, 2022
Closed
Closed
Merged
@pingyu pingyu mentioned this issue Apr 18, 2022
Merged
pingyu added a commit that referenced this issue Apr 18, 2022
@zeminzhou @pingyu @zz-jason
[to #48] modify TiDB CDC to TiKV CDC (#74)
3b82376 
* modify TiCDC to TiKV CDC

Signed-off-by: zeminzhou <zeminzhou_@outlook.com>

Co-authored-by: Ping Yu <yuping@pingcap.com>
Co-authored-by: Jian Zhang <zjsariel@gmail.com>
@zeminzhou zeminzhou mentioned this issue Apr 18, 2022
Merged
zz-jason pushed a commit that referenced this issue Apr 19, 2022
@pingyu
[to #48] Create actions for CDC (#86)
aa54859
@pingyu pingyu mentioned this issue Apr 19, 2022
Merged
This was referenced Apr 20, 2022
Merged
Merged
zz-jason pushed a commit that referenced this issue Apr 21, 2022
@zeminzhou
[to #48] remove mount metircs (#87)
73e6870 
Co-authored-by: zeminzhou <zhouzemin@ping.com>
zz-jason pushed a commit that referenced this issue Apr 21, 2022
@pingyu
[to #48] Fix CI of client-test (#88)
5451620
zz-jason added a commit that referenced this issue Apr 21, 2022
@zeminzhou @zz-jason
[to #48] remove commented codes (#89)
48a3c74 
Co-authored-by: zeminzhou <zhouzemin@ping.com>
Co-authored-by: Jian Zhang <zjsariel@gmail.com>
@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity.

@zeminzhou zeminzhou mentioned this issue Apr 26, 2022
Closed
This was referenced Jun 9, 2022
Merged
Merged
@pingyu pingyu reopened this Jun 13, 2022
pingyu added a commit that referenced this issue Jun 15, 2022
@zeminzhou @pingyu
[to #48] try to fix scheduler problem (#90)
7aefe87 
* try fix scheduler problem

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* optimization

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* fix ut

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* fix ut

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* remove commented codes

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* fix ut for owner

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* add ut for processor

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* fix ut

Signed-off-by: zeminzhou <zhouzemin@ping.com>

* fix ut

Signed-off-by: zeminzhou <zhouzemin@pingcap.com>

* remove spaceline

Signed-off-by: zeminzhou <zhouzemin@pingcap.com>

* int64 -> uint64

Signed-off-by: zeminzhou <zhouzemin@pingcap.com>

* .

Signed-off-by: zeminzhou <zhouzemin@pingcap.com>

* .

Signed-off-by: zeminzhou <zhouzemin@pingcap.com>

Co-authored-by: zeminzhou <zhouzemin@ping.com>
Co-authored-by: Ping Yu <yuping@pingcap.com>
@pingyu pingyu reopened this Jun 16, 2022
This was referenced Jun 19, 2022
Merged
Merged
pingyu added a commit that referenced this issue Jun 22, 2022
@pingyu
[to #48] cdc: Refine codes (#130)
d87a853 
* wip

Signed-off-by: pingyu <yuping@pingcap.com>

* bug fix

Signed-off-by: pingyu <yuping@pingcap.com>

* fix

Signed-off-by: pingyu <yuping@pingcap.com>

* add TRACE tag

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu pingyu mentioned this issue Jun 22, 2022
Merged
pingyu added a commit that referenced this issue Jun 23, 2022
@pingyu
[to #48] Refine TiKV sink (#131)
82eca32 
* wip

Signed-off-by: pingyu <yuping@pingcap.com>

* add unit test

Signed-off-by: pingyu <yuping@pingcap.com>

* fix misspell

Signed-off-by: pingyu <yuping@pingcap.com>

* refine test code

Signed-off-by: pingyu <yuping@pingcap.com>

* byte limit

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu pingyu mentioned this issue Jun 23, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pingyu pingyu

@haojinming haojinming

@zeminzhou zeminzhou

Labels
component/cdc
Projects
None yet
Milestone
No milestone
3 participants
@pingyu @haojinming @zeminzhou